Sign In to Follow Application
View All Documents & Correspondence

"Low Cost Intelligent Colorimeter Using Color Leds"

Abstract: The invention relates to a very low cost colorimeter that is conceived using color LEDs, microcontroller and artificial neural networks. LEDs of the three primary colors Red, Green and Blue are used to measure the primary components of the reflected light from a printed patch under a bright white illuminant. The LEDs are interfaced with a microcontroller (PIC18F4550) to measure the reflected: light intensity falling on each sensor LED. Depending upon the wavelengths reflected from the printed patch produced by the three LEDs, photocurrents of varying magnitude are produced. The microcontroller interface transforms this into another form of representation - the time it takes the photocurrent to discharge the internal capacitor. Various techniques to map this data from sensor readings to actual color values are considered. Then an artificial neural network is trained with the data (comprising a 256 color palette) right after the data has been normalized in the 0.0-1.0 range. Once the desired degree of accuracy has been obtained in the training of the network, the updated weights are then saved and can be used to map sensor readings to actual color values.

Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence

Patent Information

Application #
Filing Date
14 August 2015
Publication Number
37/2015
Publication Type
INA
Invention Field
PHYSICS
Status
Email
Parent Application
Patent Number
Legal Status
Grant Date
2019-09-05
Renewal Date

Applicants

1. DR. SIDDHARTHA BHATTACHARYYA
RCC INSTITUTE OF INFOMATION TECHNOLOGY CANAL SOUTH ROAD, BELIAGHATA, KOLKATA - 700015, WEST BENGAL

Inventors

1. DR. SIDDHARTHA BHATTACHARYYA
RCC INSTITUTE OF INFOMATION TECHNOLOGY CANAL SOUTH ROAD, BELIAGHATA, KOLKATA - 700015, WEST BENGAL
2. DR. SANKHA SUBHRA MUKHERJEE
NO. 2, R. N. TAGORE ROAD, (BASAK BAGAN), P.O.-SODEPUR, DIST.-24 PARGANAS (N), KOLKATA - 700110
3. PROF. (DR). PARAMARTHA DUTTA
DEPARTMENT OF COMPUTER & SYSTEM SCIENCE, VISVA BHATRATI, SANTINIKETAN, WEST BENGAL
4. PROF. (DR). SUSANTA CHAKRABORTY
DEPARTMENT OF COMPUTER SCIENCE AND TECHNOLOGY IIEST, SHIBPUR HOWRAH WEST BENGAL

Specification

DESC:FIELD OF THE INVENTION:
The present invention is relating to recognize primary colour components present in a printed coloured patch. More specifically, the present invention is directed to a low cost colorimeter for recognizing the primary colour components of the printed coloured patch using light emitting diodes (LED) under a bright white illumination.

BACKGROUND ART.:
With the advent of technology, different types of colorimeter have been developed for colour component measurement. There some of colorimeter device or spectrometer device have been reported which are directed to recognize the colours of different samples. E.g.

US patent 4797000 describes a comparative colorimeter for use in the field which simultaneously compares the colour densities of two liquid samples and designates the degree of difference. The apparatus includes two major subsystems, optical and electronic. The optical subsystem is designed to provide identical light beams through both the sample and standard solutions and to minimize the effect which imperfect sample tubes have on the output. The electronic subsystem includes a log conversion and differential amplifier circuit for generating a difference signal representative of the difference between the optical densities of the sample and standard solutions. The difference signal is evaluated by a comparative circuit consisting of a resistive chain having a plurality of node voltages and hex inverters connected to the node voltages. The hex inverters drive a series of LED indicator lights to designate the relative degree of difference between the optical densities of the sample and standard. A correction circuit is provided for referencing the difference signal to the threshold voltage of the hex inverters and for compensating for initial differences in the optical paths of the sample and standard solutions.

US patent 7046347 discloses a colorimeter instrument useful in providing light transmission and scatter measurements through a test sample comprising an array of multiple light-emitting diodes each having a different wavelength and equally spaced around a tubular sample holder facing inwardly. Photodiode light sensors that detect light transmitted through the sample is mounted to the tubular sample holder at a position directly opposite each light-emitting diode wherein selective activation of the LED's and the associated photo-diode the light transmission through a solution is ascertained which subsequently processed in computer to plot the transmission and absorption on a graph for each of the different wavelengths.

CN 200989824 discloses a colour measurement instrument comprising of a shell, a light source arranged inside the shell, a sensor, a control circuit, a data processor and an output cell arranged on the shell. The light source comprises six LED lights, which have three colours that are red, white and blue, and the number of each colour light is two. The six LED lights distributes around the sensor, forming a circle and laying in an equally spaced way, irradiating samples in 45 degrees. The LED lights emit light which impinge on the sample and the reflected light obtained by the sensor is used to obtain an electrical signal which sent to microcontroller (MCU) for processing and obtaining colour parameters.

WO 2002/093116 discloses a handheld, portable colour measuring device for measuring the primary colours of red, green and blue in a colour target. The colour measuring device includes an elongated color measuring probe housing having a battery powered white LED light source and a light pipe centered inside the probe housing to captures the light reflected off the colour target and projects the captured light onto a 3 color sensor wherein the light signal is amplified and converted to a digital signal using an A/D converter for further processing and measurement of the colours.

The above state of art colorimeter devices includes separate sensing device to recognize colour of the sample and none of the above state of art colorimeter device, individually or collectively attempts to create a low cost and simplified colorimeter to recognize the primary colour components of a printed coloured patch by simply using sensitivity colour LEDs to the spectrum of electromagnetic radiation which is equal or shorter in wavelength than the wavelength(s) that the LED emits.

Accordingly there was a long felt need to have a simplified colorimeter device to recognize the primary colour components of a printed coloured patch by involving simple colour LEDs which will be free of any heavy maintenance.

OBJECTIVE OF THE INVENTION:
It is thus the basic object of the present invention is to develop a colorimeter device to recognize the primary colour components of different resolution in a printed coloured patch.


Another object of the present invention is to develop a colorimeter device to recognize the primary colour components of different resolution in a printed coloured patch which would be portable, easy to use and low cost.

Another object of the present invention is to develop a colorimeter device to recognize the primary colour components in a printed coloured patch which would be adapted to involve simple colour LEDs having sensitivity to the spectrum of electromagnetic radiation which is equal or shorter in wavelength than the wavelength(s) that the LED emits for recognizing the primary colour components.

Another object of the present invention is to develop a colorimeter device which would be adapted to predict unknown colors and detect detecting any adulteration of colours.

SUMMARY OF THE INVENTION:
Thus according to the basic aspect of the present invention there is provided a LED based colorimeter device comprising
light emitting diode (LED) light source to illuminate the colour for detection and recognition of primary colours;
light emitting diode (LED) photo-sensor for sensing primary colour components present in the light reflected from the illuminated colour;
microcontroller operatively connected to said LED light source and said LED photo-sensor and adapted to measure sensor readings corresponding to the LED based photo-sensor’s output;
artificial neural network enabled for processing said sensor readings to map with actual color values for detecting or recognizing primary colour components in the illuminated colour.

According to another aspect in the present LED based colorimeter, the artificial neural network is configured to map the sensor reading into actual colour values by involving elimination of any multiple mapping of the sensor readings corresponding to the LED based photo-sensor’s output.

According to another aspect in the present LED based colorimeter, the artificial neural network include computing module embodied within the microcontroller itself or as a separate computing platform having the operative communication with said microcontroller.

According to another aspect in the present LED based colorimeter, the LED light source and the LED photo-sensor are housed in a housing preferably having black covering to eliminate effect ambient lighting and reduce secondary reflections.

According to another aspect in the present LED based colorimeter, the illuminated colour comprises a colour printed patch and LED of said LED photo-sensor in reverse bias generates photocurrent output under exposure of the reflected light from the printed patch having equal or shorter in wavelength than the wavelength(s) that the LED emits and involving linear relationship to the intensity of light falling upon it.

According to another aspect, the present LED based colorimeter comprises operative interfacing means to transform the photocurrent output into time taken by the photocurrent to discharge internal junction capacitance of the said reverse biased LEDs.

According to yet another aspect, the present LED based colorimeter includes
said LED light source comprising of white LED configured for emitting white light to illuminate the printed colour patch;
said LED photo-sensor comprising of LEDs of primary colours configured for sensing RED, GREEN and BLUE light component reflected from the illuminated printed patch.

According to another aspect in the present LED based colorimeter, the LEDs of primary colours includes
atleast one RED light emitting LED acting as the photo-sensor for all three RED, GREEN and BLUE light components;
atleast one GREEN light emitting LED as acting as the photo-sensor for GREEN and BLUE light components;
atleast one BLUE light emitting LED acting as the photo-sensor for only BLUE light component.

According to another aspect in the present LED based colorimeter, the interfacing means includes
tri-state buffers in output mode corresponding to connecting pins of the microcontroller and the LED photo-sensor and high state in the connecting pins to reverse bias the LEDs of the LED photo-sensor and enabling current flows out of the tri-state buffer and charges junction capacitance of the LEDs;
said tri-state buffers in input mode after charging the junction capacitance to enable the LEDs acts as current source and discharging the junction capacitance through said current source.

According to yet another aspect, the present LED based colorimeter comprises corresponding bit at LAT register of the microcontroller set to 0 to disconnect internal pull-up resistances for facilitating discharging of the junction capacitance through the current source.

According to another aspect in the present LED based colorimeter, the microcontroller includes timer means to measures the sensor readings by counting time takes by each LED to pull input pin LOW indicating the time to discharge the junction capacitance and shorter time indicates larger photocurrent and higher light intensity.

According to yet another aspect in the present LED based colorimeter, the microcontroller and the LEDs are incorporated in same PCB to eliminate effects due to varying parasitic capacitance.

According to yet another aspect, the present LED based colorimeter includes
said LED light source comprising of LEDs of primary colours configured to illuminate the printed colour patch with the RED, GREEN and BLUE light having wavelengths widely spread on the visible spectrum;
said LED photo-sensor comprising of white LED or infra red LED configured for sensing primary colour components reflected from the illuminated printed patch.

According to another aspect in the present LED based colorimeter, the LEDs of primary colours of the LED light source includes
atleast one RED light emitting LED;
atleast one GREEN light emitting LED;
atleast one BLUE light emitting LED.

According to another aspect in the present LED based colorimeter, the interfacing means facilitates activation of the Red, Green and Blue LEDs in sequential manner and measuring sensor readings corresponding to the LED based photo-sensor’s output.

According to another aspect in the present LED based colorimeter, the interfacing means includes the microcontroller with five parallel ports and a ICSP module for clocking.

According to another aspect in the present LED based colorimeter, the microcontroller generates normalized sensor reading by repeatedly reverse biasing the LEDs corresponding to RED, GREEN and BLUE light and averaging all the sensor readings in normalization a scale of 0-255;

said normalized sensor reading fed into neural network computing module embodied in the microcontroller for further processing and obtaining the colour value in RGB format.

According to another aspect in the present LED based colorimeter, the microcontroller generates sensor reading string by repeatedly reverse biasing the LEDs corresponding to RED, GREEN and BLUE light and formalizing all the sensor readings in an output string;

said sensor reading string fed into the computing platform of said neural network based computing module for further processing and obtaining the colour value in RGB format.

According to another aspect in the present LED based colorimeter, the neural network based computing module processes the sensor readings and generates the colour value in RGB format involving

dataset for a new network architecture comprising of 8 bit colour palette having 256 colours in it;

receiving scaled values of the Red (X), Green (Y), Blue (Z) sensor readings corresponding to their respective values from 24 bit color palette/255;

approximating function for Red (? (X, Y, Z)), Green (f (X, Y, Z)) and Blue(? (X, Y, Z) ) components from the scaled values of the Red (X), Green (Y), Blue (Z) sensor readings by involving the new network architecture configured to classify at least 256 colors correctly and encode around 256 bits of information in its weights;

predicting colour value for a input scaled sensor readings by involving the approximated function for Red, Green and Blue and thus predicting primary colour components in printed coloured patch corresponds to said input scaled sensor readings.

According to another aspect in the present LED based colorimeter, the approximation 3D function for Red (? (X, Y, Z)), Green (f (X, Y, Z)) and Blue(? (X, Y, Z) ) components includes

initially approximating the function for Blue(? (X, Y, Z) ) component in the network which is unique valued in the X, Y, Z space;

approximating the function for Green (f (X, Y, Z)) component by extending to 4th dimension function G= f (X, Y, Z, B ) with the inclusion of the B component determined by previous network for approximating the Blue(? (X, Y, Z) ) component to make the extended function unique valued in the X, Y, Z space eliminating one to many mappings of the Green (f (X, Y, Z)) component its X, Y, Z domain occurring due to sensitivity of the GREEN LED for both GREEN and BLUE spectrum;

approximating the function for Red (f (X, Y, Z)) component by extending to 5th dimension function R= f (X, Y, Z, B, G ) with the inclusion of the B and G components determined by previous networks to make the extended function unique valued in the X, Y, Z space eliminating one to many mappings of the Red (f (X, Y, Z)) component its X, Y, Z domain occurring due to sensitivity of the RED LED for entire RGB spectrum.

According to another aspect in the present LED based colorimeter, the microcontroller is in operative communication with a display system preferably LCD display through a display controller for displaying the recognized colour components.

According to another aspect in the present LED based colorimeter, the microcontroller is disposed in operative communication with a keyboard interface to receive operative inter instruction.

BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS:
Figure 1 shows formation different colours by combining three primary colours.
Figure 2 shows light reflection from a surface which absorbs blue light when illuminated by white light.
Figure 3 shows spectral response of an IR LED.
Figure 3a shows photocurrent Ip generating equivalent circuit of LED as photodiode in accordance with the present invention.
Figure 4 shows a preferred embodiment of trans-impedance amplifier for interfacing LED sensors with microcontroller of the present colorimeter device.
Figure 5 shows a preferred embodiment of cascaded amplifier based interfacing device of the present colorimeter device.
Figure 6a shows a preferred embodiment of the type I digital interfacing device configuration the present colorimeter device.
Figure 6b shows a preferred embodiment of the type II digital interfacing device configuration the present colorimeter device.
Figure 7 a & b shows block representation of the present colorimeter device.
Figure 8 shows generic circuit representation of an I/O port of the microcontroller associated with the present colorimeter device.
Figure 9 shows block diagram of USB interface of the microcontroller associated with the present colorimeter device.
Figures 10 & 11 show interface of LCD display with the mocrocontroler for displaying the results and PCB design of the LCD .
Figure 12 shows keyboard interfacing with the microcontroller for controlling operation of the present device.
Figure 13a & 13b illustrates operation of the present colorimeter device.
Figure 14 shows cascaded architecture of the neural network architecture in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION WITH REFERENCE TO THE ACCOMPANYING DRAWINGS:
As stated herein before, the present invention discloses a novel low cost colorimeter device comprising of LEDs of three primary colours i.e. Red, Green and Blue, a microcontroller and a neural network based computing module for detecting or recognizing the primary colour components in a printed colour patch on a paper or a substrate under a bright white illuminant. The computing module may be embodied within the microcontroller itself or in a separate computing platform such as PC, Laptop having the operative communication with said microcontroller.
It is well known in the art that, colours can be added or combined together to form other colours. White is an example of this which is a combination of the seven distinct colours, red, orange, yellow, green, blue, indigo and violet. The combination of three different wavelengths such as red, green and blue (RGB) can also form white light on the condition that the three different wavelengths must be widely spread on the visible spectrum. These three colours are called primary colours. Figure 1 shows how these primary colours can be combined to form other colours.
Similarly, the colour can also be subtracted to form other colours. Colour subtraction is a result of pigments on surfaces. These pigments are chemicals which have the ability to absorb one or more frequencies. Figure 2 shows the result of a surface which absorbs blue light when illuminated by white light. From the figure 2 and figure 1 it is logical that yellow paints absorb blue light. Other examples are magenta and cyan paints absorbing green and red lights respectively. When combining yellow, magenta and cyan, black colour is produced. This is known as a subtractive set. The printed patch follows the subtractive colour model wherein, if the printed patch is illuminated with white light then some of its components are absorbed and the other wavelengths are reflected back to sensors. Thus by detecting components in the reflected light primary colour component of the printed patch can be detected or recognized.
In the present colorimeter device, the LEDs are used as photodiodes for sensing colour component present in the light reflected from the printed patch. The LEDs are sensitive to the spectrum of electromagnetic radiation which is equal or shorter in wavelength than the wavelength(s) that the LEDs emit. Thus, when photons of energy (E=h?) greater than equal to the minimum required energy (E0=h?0) impinges on a LED, an electron-hole pair is formed in the process (photoelectric effect). Hence, the RED light emitting LED as a sensor will be sensitive to all the three RED, GREEN and BLUE components; the GREEN light emitting LED as a sensor will be sensitive to the GREEN and BLUE components; and the BLUE light emitting LED as a sensor will be sensitive to only the blue component.
The resulting photocurrent in a LED sensor, under exposure of the light having equal or shorter in wavelength than the wavelength(s) that the LED emits, has an approximately linear relationship to the intensity of light (in the sensitive region) falling upon it. The spectral response of an IR LED is shown in the accompanying figure 3. For the Red Green and Blue sensors the peak shifts increasingly to the left. The LED equivalent circuit under reverse bias is shown in the accompanying figure 3a. The microcontroller interface transforms the LED output i.e. the photocurrent into another form of representation which is the time it takes the photocurrent to discharge the internal junction capacitor.
In a preferred embodiment of the present colorimeter device, the LED sensors are positioned in a housing preferably a black photographic film casing to eliminate the effect of the ambient lighting and reduce the secondary reflections. The LED sensors are interfaced with the microcontroller though an interfacing device. The interfacing device may include any one from trans-impedance amplifier, cascaded amplifier or digital interface.
In trans-impedance amplifier based interfacing device, the photocurrent of the LED sensor (in the range of nA) is converted into voltage using the trans-impedance amplifier. The output voltage of trans-impedance amplifier is interfaced to the microcontroller’s analog to digital converter (ADC). The accompanying figure 4 shows a preferred embodiment of such trans-impedance amplifier. The accompanying figure 5 shows a preferred embodiment of cascaded amplifier based interfacing device.
The digital interfacing device of the present colorimeter device includes two different configurations.
In first digital interfacing device configuration, the LEDs operate as tri-state buffers. In a tri-state buffer operation, when the tri-state buffer's control bit is active, its input is transferred to the output and when the tri-state buffer's control bit is not active the output of the device is a high-impedance or, equivalently, nothing. In the present type I digital interfacing device configuration for measuring colour in the reflected light by involving the LED photo-sensor, the LEDs which are directly connected with the microcontroller are first reversed biased making corresponding connecting pins with the microcontroller high and putting their corresponding output driver tri-state buffers in output mode. Hence, current flows out of the tri-state buffer and charges junction capacitance of the diode.
Next the output driver of the first LED is disconnected by setting the corresponding tri-state buffer's control bit to 0 thus making it input or equivalently the output of the device is then a high-impedance or, equivalently, nothing. Internal pull-up resistances are then disconnected using setting the corresponding bit at the LAT register to 0. In absence of an input/charging source, under exposure of the reflected light from the illuminated patch, the LED which acts as current source allows the junction capacitance to discharge through the current source. The microcontroller’s timer counts the time it takes the LED to pull the input pin LOW which is measure of the time the photocurrent takes to discharge the junction capacitance. The higher the light intensity, the larger is the photocurrent and the shorter is the time for the 1?0 transition. Figure 6a shows such a type I digital interfacing device configuration.
These digital interfacing devices are highly sensitive to any parasitic capacitance noise. The configuration of the 20 line flat interface cable induces minor changes by changing the capacitance. So the flat cable has to be in a very constant and stable configuration. However this is easily eliminated by incorporating the sensor and microcontroller in the same PCB. It will eliminate the effects due to varying parasitic capacitance.
In an alternative embodiment, the type II digital interfacing device configuration is used as interface between the LEDs and the microcontroller. These type II digital interfacing devices are very similar to the first digital interfacing device except that the Red, Green and Blue LEDs are the illuminants and a white/ infra-red LED is used for sensing. The microcontroller with the help of the interfacing device activates the Red, Green and Blue LEDs in a sequential manner and read the white/ infra-red LED output for each one. This gives the amount of RED, Green and Blue components being reflected by the printed patch. Here as the sensor is the same the variance in R, G and B readings and the noise is very less. The second digital interfacing device configuration preferably includes the microcontroller PIC18F4550 with five parallel ports and an ICSP module for clocking. The accompanying figure 6b shows a type II digital interfacing device configuration.
The illuminant(s) is/are controlled by the microcontroller. When the measurement is requested the microcontroller switches on the illuminants one by one. Current limiting resistors are there to limit the port current.
The microcontroller is the most important part of the present colorimeter device. It performs anyone or more from (i) Measuring sensors’ transition time (ii) Scaling the sensor readings (iii) Communicating with associated computing device such as PC via USB transceiver (optional) (iv) Calculating the results if the computing module is embodied in the microcontroller instead of the computing platform) (v) Controlling the LCD display (vi) Formatting the result for display and (vii) Displaying the result.
The present colorimeter device can be housed on a development board which can interface with the external components or interface boards. The board is designed to be compatible with it using 20 pin flat conductors. The Figure 7a shows the block representation of the present colorimeter device with the type I digital configuration device (2) and the embaded computing module (3) within the microcontroller (1). Similarly the Figure 7b shows the block representation of the present colorimeter device with the type II digital configuration device (2) and the embaded computing module (3) within the microcontroller (1).
The microcontroller of the present colorimeter device preferably includes PIC18F4550 with 5 parallel ports. The number of input or output (I/O) ports and port pins varies depending on port. Figure 8 shows the generic circuit representation of an I/O port of the present microcontroller. The present microcontroller includes PORT A, PORT B, PORT C, PORT D, and PORT E. The pins of a port are labeled as RPn, where P is the port letter and n is the port bit number. For example, PORT A pins are labeled RA0 to RA7, PORT B pins are labeled RB0 to RB7, and so on.
Working with a port includes: A. Setting port direction, B. Setting an output value, C. Reading an input value and D. Setting an output value and then read back the output value. In the PIC18F series, a latch register (e.g., LATA for PORT A) is introduced to the I/O ports to hold the actual value sent to a port pin. Reading from the port reads the latched value, which is not affected by any external device.
The TIMER (4) module is used for timing purposes. It interrupts the USB bus every 3.3 ms so that the endpoint polling does not go to sleep mode. The TIMER (4) module incorporates the following features: i. Software selectable operation as a timer or counter in both 8-bit and 16-bit modes, ii. Readable and writable registers, iii. Dedicated 8-bit, software programmable prescaler, iv. Selectable clock source (internal or external), v. Edge select for external clock, vi, Interrupt on overflow. The T0CON register controls all the aspects of the module’s operation, including the prescale selection. It is both readable and writable.

The PIC18F4550 implements a USB 2.0 full speed device interface through its user interface (5). Figure 9 shows the block diagram.

An LCD display controller (8) is interfaced with the mocrocontroler which is used to display the results. The interface is a 4-bit mode of the HD 44870 LCD controller as shown in Figures 10 and 11. A PS2 keyboard (6) is also interfaced for selection of menu and commencing measurement. The PS2 keyboard connection is shown in Figure 12.

A serial EEPROM is also attached to the I2C bus of the microcontroller. This 64Kb EEPROM is used to store and retrieve the network weights corresponding to processing of the sensory data by the neural netwok based computing module.

The colour component recognization operation involving the neural network based computing module associted with the present colorimeter device which either embodied in the microcontroller or in the associate computing platform can be classified in two modes viz. Embedded Mode and PC interface Mode. These modes are selectable in runtime by an initial menu and keyboard. A very simplified flowchart of the menu display is shown in the accompanying figure 13.

The Mode 1 as shown in the figure 13a illustrates the operation in the Embedded mode with the type II digital interfacing device. In Mode 1, first the sensor connecting pins, TRISTATE bits and LATCH bits for RED, GREEN and BLUE LED sensors are defined. After receiving, the measurement command, if character entered from keyboard is found to be equal to Carriage Return, Count=1; Rx=By=Gz=0 are initialized. Then, the RED, GREEN and BLUE LED sensors are reverse biased for taking readings. After completing the reading Rx, By, Gz, the count value are increased and if it is found that the count value after the increment is less than the predefined value then the procedure of reverse biasing the RED, GREEN and BLUE LED sensors for reading Rx, By, Gz and increment of the count value is repeated.

When the count value exceeds the predefined value, average of Rx, By and Gz over all the readings are calculated and normalized on a scale of 0-255. The normalized data is then forwarded into neural network based computing module for further processing and obtaining the colour value in RGB format. The obtained result is then displayed on the LCD display.

The Mode 2 as shown in the figure 13b illustrates the operation in the PC interface Mode with the type II digital interfacing device. In the PC interface Mode, the PC or the Laptop or the equivalent computing platform includes an operating software written in .NET framework which can reads the data from the LED sensors and interprets the data and displays the result on screen and also sends it back to the MCU for LCD display.

In Mode 2, first the sensor connecting pins, TRISTATE bits and LATCH bits for RED, GREEN and BLUE LED sensors are defined and PC user interface device is polled for input.

After receiving, the measurement command from keyboard, Count=1; Rx=By=Gz=0 are initialized. Then, the RED, GREEN and BLUE LED sensors are reverse biased for taking readings. After completing the reading Rx, By, Gz, the count value are increased and if it is found that the count value after the increment is less than the predefined value then the procedure of reverse biasing the RED, GREEN and BLUE LED sensors for reading Rx, By, Gz and increment of the count value is repeated.

When the count value exceeds the predefined value, average of Rx, By and Gz over all the readings are calculated and formalized to obtain an output string. The output string is then sent to the associated PC or the Laptop or the equivalent computing platform via HID_Write which is synchronously read by the associated PC or the Laptop or the equivalent computing platform via HID_Read. The result obtained by processing using neural network is then displayed on the LCD display.

Herein, the PC software written in .NET framework works as the HID host and connects to the device and can perform the following tasks
Data Acquisition: It instructs the microcontroller to read the data and acquires the result string from the microcontroller.
Data Formatting: It scale and process the input data.
Interpret the Result: It interprets the sensor readings by involving the neural network and identify the colors.
Acquire training data: The software can sequentially collect all the training data and by instructing the user to place the sensor on each patch one by one
Train the Artificial Neural Network: After acquiring the total data set it can train an ANN using the Back propagation Algorithm to interpret the dataset. Several networks may be trained for various types of paper (Glossy, Matte, Plain etc.).
Load/Save training profile: It can save new training profiles for new types of printing mediums and also load them appropriately for interpretation of results.

In the present invention, the Data Acquisition includes preparation of training dataset comprising of 8bit colour palette having 256 colours in it. These colours are printed on non-glossy papers.

The LED sensors are placed on each patch one by one and the readings are recorded on the computing module by the PC software. The data is then scaled and a scaled chart is generated with SCALED_RED, SCALED_GREEN, and SCALED_BLUE values which correspond to the respective values from the 24 bit color palette/255. Each originally can take values from 0-255. When divided by 255 the data is transformed in the 0.0 – 1.0 range for each of them. This transforms is required for the neural network. When the output is obtained from the network each component is multiplied back by 255 to obtain the original color in 24-bit format. Scaled values of the X (Red), Y (Green), Z (Blue) sensor readings are obtained as following. Here the min value and range for each is obtained from processing the data.

SCALED_X= (X-Xmin)/Xrange
SCALED_Y= (Y-Ymin)/Yrange
SCALED_Z= (Z-Zmin)/Zrange

The SCALED_X, SCALED_Y, and SCALED_Z are the inputs of the network. The SCALED_RED, SCALED_GREEN, and SCALED_BLUE are the outputs.

Now, if the SPD of the illuminant be given by the function(?), the reflectance of the patch be given by R(?) and the relative spectral sensitivity of the LED s be specified as S_R (?),S_G (?),S_B (?) for the Red , Green and Blue LEDs respectively. Then the integration of these three multiplied factors would effectively give a quantity namely the activation of each of the LEDs. This discharge time of the junction Capacitor is effectively a function of this quantity.
A_R=?_k¦?I(?_k )R(?_k ) S_R (?_k ) ?
A_G=?_k¦?I(?_k )R(?_k ) S_G (?_k)? f(x)=a_0+?_(n=1)^8¦(a_n cos??npx/L?+b_n sin??npx/L? )
A_B=?_k¦?I(?_k )R(?_k ) S_B (?_k)?
T_R=f_R (A_R)
T_G=f_G (A_G )
T_B=f_B (A_B)
The values of A_R,A_G,A_B are scaled in 0% to 100%. 0 % is result of the measurements of black patch while 100% is the result of the measurements of white patch.
The first problem is to establish the relation between the discharge times and the activations of the LEDs. To do that the illuminant I(?)is defined as
I(?)=n?i(?) where n is a scaling factor of intensity ranging from 0...1.
The computation of CIE XYZ values from the I(?)*R(?) are as follows
X=N?_k¦?I(?_k )R(?_k ) x ¯(?_k)?
Y=N?_k¦?I(?_k )R(?_k ) y ¯(?_k)?
Z=N?_k¦?I(?_k )R(?_k ) z ¯(?_k)?
Here N is the normalization factor calculated by
N=100/(?_k¦?I(?_k ) y ¯(?_k)?)
Using the standard x ¯,y ¯ ¸ and z ¯ functions at 10nm intervals the N comes out to be 39.3918118126159.

It is well known that, the neural networks are very robust and useful for pattern classification, prediction, function approximation and a myriad of other tasks. The input data from the sensors can be used train a feed-forward neural network. This neural network can then be used to classify the input data in real-time to some color value. This system is very robust in the sense that it not only can classify the trained data correctly but also can predict the color of an unknown input. A good architecture and training set can give very good accuracy in the approximation of an unknown input. So once the optimal weights have been found for the training set the network can be then used to recognize colors from the entire color gamut. The accuracy is only limited by the architecture and the threshold Mean-Square error used in training. Moreover, it can scale very well.

In the present invention the neural network based computing module involve a 3D function approximation of each component of Red=? (X, Y, Z), Green=f (X, Y, Z), and Blue=? (X, Y, Z) for sensory data interpretation, where X,Y, Z are the input values of the Red, Green and Blue LED sensors respectively.

Now due to the readings by involving the Red, Green and Blue LEDs, the f and ? functions have a few one to many mappings from its X, Y, Z domain to their respective ranges. The reason for it is that the RED sensor is sensitive to the whole RGB spectrum and the GREEN sensor is sensitive to the G and B spectrum. So for a specific value of (X, Y, Z) the functions may have two or more values. This leads to problems with training the network at such points. If no unique valued solution exists the network averages the output at those points and this leads to major misclassifications in those data points. The ? (Red) function is the most ill-behaved followed by the f (green) function.

In the present computing module, the above problem is solved by an unique mappings which eliminates existence on the sensor readings corresponding to one LED output in the output of other LEDs. The unique mapping technique involves addition of an extra dimension to separate the data points in that nD+m space, where m is the number of extra dimensions added to the make the problem separable.

The B=? (X, Y, Z) is unique valued in the X, Y, Z ? B space. Different architectures were trained in parallel and after some time the one with the best learning performance was selected for further learning and refinement under an adaptive training procedure. The first phase is competition among the various architectures. The next phase is the refinement of the winning architecture. Among the various architectures tested the B component was best approximated by the following network:

Table III: Network architecture for training B component
Layer Number of Neurons
Input 3
Hidden 1 12
Hidden 2 10
Hidden 3 6
Output 1

Next the domain of the f function can be extended in the 4th dimension which is just the B component determined by the previous network. This makes the mapping unique valued and solves the problem. Hence, G= f (X, Y, Z, B). Among the architectures tested the G component was best approximated by the following network:

Table IV: Network architecture for training G component
Layer Number of Neurons
Input 4
Hidden 1 15
Hidden 2 10
Hidden 3 8
Hidden 4 6
Output 1

Next the domain of R is extended in the 5th dimension with the G component also along with the X, Y, Z and B inputs from previous networks. Hence R=? (X, Y, Z, B, G). Among the architectures tested the R component was best approximated by the following network:

Table V: Network architecture for training R component
Layer Number of Neurons
Input 5
Hidden 1 15
Hidden 2 8
Output 1

This leads to a cascaded network architecture as shown in Figure 14.

It is thus the principle advantages achieved by the present invention may be summarized as hereunder
a) The colour measurement method and system are cost effective.
b) The technology is adapted to meet the demand for portability.
c) It is economic and involves simplified construction and maintenance, thereby can ensure easy determination of colour spectra.

Different embodiments of the invention are possible to achieve the best method of performance and to obtain the effective colorimeter. It will be understood that the invention may be carried out into practice by skilled persons with many modifications, variations and adaptations without departing from its spirit or exceeding the scope of claims in describing the invention for the purpose of illustration.
,CLAIMS:WE CLAIM:
1. LED based colorimeter device comprising
light emitting diode (LED) light source to illuminate the colour for detection and recognition of primary colours;
light emitting diode (LED) photo-sensor for sensing primary colour components present in the light reflected from the illuminated colour;
microcontroller operatively connected to said LED light source and said LED photo-sensor and adapted to measure sensor readings corresponding to the LED based photo-sensor’s output;
artificial neural network enabled for processing said sensor readings to map with actual color values for detecting or recognizing primary colour components in the illuminated colour.

2. LED based colorimeter as claimed in claim 1, wherein the artificial neural network is configured to map the sensor reading into actual colour values by involving elimination of any multiple mapping of the sensor readings corresponding to the LED based photo-sensor’s output.

3. LED based colorimeter as claimed in anyone of claim 1 or 2, wherein said artificial neural network include computing module embodied within the microcontroller itself or as a separate computing platform having the operative communication with said microcontroller.

4. LED based colorimeter as claimed in anyone of claims 1 to 3, wherein the LED light source and the LED photo-sensor are housed in a housing preferably having black covering to eliminate effect ambient lighting and reduce secondary reflections.

5. LED based colorimeter as claimed in anyone of claims 1 to 4, wherein said illuminated colour comprises a colour printed patch and LED of said LED photo-sensor in reverse bias generates photocurrent output under exposure of the reflected light from the printed patch having equal or shorter in wavelength than the wavelength(s) that the LED emits and involving linear relationship to the intensity of light falling upon it.

6. LED based colorimeter as claimed in anyone of claims 1 to 5, comprising operative interfacing means to transform the photocurrent output into time taken by the photocurrent to discharge internal junction capacitance of the said reverse biased LEDs.

7. LED based colorimeter as claimed in anyone of claims 1 to 6, includes
said LED light source comprising of white LED configured for emitting white light to illuminate the printed colour patch;
said LED photo-sensor comprising of LEDs of primary colours configured for sensing RED, GREEN and BLUE light component reflected from the illuminated printed patch.

8. LED based colorimeter as claimed in anyone of claims 1 to 7, wherein the LEDs of primary colours includes
atleast one RED light emitting LED acting as the photo-sensor for all three RED, GREEN and BLUE light components;
atleast one GREEN light emitting LED as acting as the photo-sensor for GREEN and BLUE light components;
atleast one BLUE light emitting LED acting as the photo-sensor for only BLUE light component.

9. LED based colorimeter as claimed in anyone of claims 1 to 8, comprising interfacing means including
tri-state buffers in output mode corresponding to connecting pins of the microcontroller and the LED photo-sensor and high state in the connecting pins to reverse bias the LEDs of the LED photo-sensor and enabling current flows out of the tri-state buffer and charges junction capacitance of the LEDs;
said tri-state buffers in input mode after charging the junction capacitance to enable the LEDs acts as current source and discharging the junction capacitance through said current source.

10. LED based colorimeter as claimed in anyone of claims 1 to 9, comprising corresponding bit at LAT register of the microcontroller set to 0 to disconnect internal pull-up resistances for facilitating discharging of the junction capacitance through the current source.

11. LED based colorimeter as claimed in anyone of claims 1 to 10, wherein the microcontroller includes timer means to measures the sensor readings by counting time takes by each LED to pull input pin LOW indicating the time to discharge the junction capacitance and shorter time indicates larger photocurrent and higher light intensity.

12. LED based colorimeter as claimed in anyone of claims 1 to 11, wherein the microcontroller and the LEDs are incorporated in same PCB to eliminate effects due to varying parasitic capacitance.

13. LED based colorimeter as claimed in anyone of claims 1 to 6, includes
said LED light source comprising of LEDs of primary colours configured to illuminate the printed colour patch with the RED, GREEN and BLUE light having wavelengths widely spread on the visible spectrum;
said LED photo-sensor comprising of white LED or infra red LED configured for sensing primary colour components reflected from the illuminated printed patch.

14. LED based colorimeter as claimed in claim 13, wherein the LEDs of primary colours of the LED light source includes
atleast one RED light emitting LED;
atleast one GREEN light emitting LED;
atleast one BLUE light emitting LED.

15. LED based colorimeter as claimed in anyone of claim 13 or 14, wherein the interfacing means facilitates activation of the Red, Green and Blue LEDs in sequential manner and measuring sensor readings corresponding to the LED based photo-sensor’s output.

16. LED based colorimeter as claimed in anyone of claims 13 to 15, wherein the interfacing means includes the microcontroller with five parallel ports and a ICSP module for clocking.

17. LED based colorimeter as claimed in anyone of claims 13 to 16, the microcontroller generates normalized sensor reading by repeatedly reverse biasing the LEDs corresponding to RED, GREEN and BLUE light components and averaging all the sensor readings in normalization a scale of 0-255;

said normalized sensor reading fed into neural network computing module embodied in the microcontroller for further processing and obtaining the colour value in RGB format.

18. LED based colorimeter as claimed in anyone of claims 13 to 17, the microcontroller generates sensor reading string by repeatedly reverse biasing the LEDs corresponding to RED, GREEN and BLUE light component and formalizing all the sensor readings in an output string;

said sensor reading string fed into the computing platform of said neural network based computing module for further processing and obtaining the colour value in RGB format.

19. LED based colorimeter as claimed in anyone of claims 1 to 18, wherein the neural network based computing module processes the sensor readings and generates the colour value in RGB format involving

dataset for a new network architecture comprising of 8 bit colour palette having 256 colours in it;

receiving scaled values of the Red (X), Green (Y), Blue (Z) sensor readings corresponding to their respective values from 24 bit color palette/255;

approximating function for Red (? (X, Y, Z)), Green (f (X, Y, Z)) and Blue(? (X, Y, Z) ) components from the scaled values of the Red (X), Green (Y), Blue (Z) sensor readings by involving the new network architecture configured to classify at least 256 colors correctly and encode around 256 bits of information in its weights;

predicting colour value for a input scaled sensor readings by involving the approximated function for Red, Green and Blue and thus predicting primary colour components in printed coloured patch corresponds to said input scaled sensor readings.

20. LED based colorimeter as claimed in claim 19, wherein the approximation 3D function for Red (? (X, Y, Z)), Green (f (X, Y, Z)) and Blue(? (X, Y, Z) ) components includes

initially approximating the function for Blue(? (X, Y, Z) ) component in the network which is unique valued in the X, Y, Z space;

approximating the function for Green (f (X, Y, Z)) component by extending to 4th dimension function G= f (X, Y, Z, B ) with the inclusion of the B component determined by previous network for approximating the Blue(? (X, Y, Z) ) component to make the extended function unique valued in the X, Y, Z space eliminating one to many mappings of the Green (f (X, Y, Z)) component its X, Y, Z domain occurring due to sensitivity of the GREEN LED for both GREEN and BLUE spectrum;

approximating the function for Red (f (X, Y, Z)) component by extending to 5th dimension function R= f (X, Y, Z, B, G ) with the inclusion of the B and G components determined by previous networks to make the extended function unique valued in the X, Y, Z space eliminating one to many mappings of the Red (f (X, Y, Z)) component its X, Y, Z domain occurring due to sensitivity of the RED LED for entire RGB spectrum.

21. LED based colorimeter as claimed in anyone of claims 1 to 20, wherein the microcontroller is in operative communication with a display system preferably LCD display through a display controller for displaying the recognized colour components.

22. LED based colorimeter as claimed in anyone of claims 1 to 21, wherein the microcontroller is disposed in operative communication with a keyboard interface to receive operative inter instruction.

Documents

Application Documents

# Name Date
1 886-KOL-2015-(14-08-2015)-SPECIFICATION.pdf 2015-08-14
1 886-KOL-2015-RELEVANT DOCUMENTS [15-05-2023(online)].pdf 2023-05-15
2 886-KOL-2015-(14-08-2015)-PA.pdf 2015-08-14
2 886-KOL-2015-RELEVANT DOCUMENTS [05-07-2022(online)].pdf 2022-07-05
3 886-KOL-2015-Annexure [18-12-2021(online)].pdf 2021-12-18
3 886-KOL-2015-(14-08-2015)-FORM-9.pdf 2015-08-14
4 886-KOL-2015-Response to office action [18-12-2021(online)].pdf 2021-12-18
4 886-KOL-2015-(14-08-2015)-FORM-3.pdf 2015-08-14
5 886-KOL-2015-RELEVANT DOCUMENTS [04-06-2021(online)].pdf 2021-06-04
5 886-KOL-2015-(14-08-2015)-FORM-2.pdf 2015-08-14
6 886-KOL-2015-PROOF OF ALTERATION [01-06-2021(online)].pdf 2021-06-01
6 886-KOL-2015-(14-08-2015)-FORM-18.pdf 2015-08-14
7 886-KOL-2015-IntimationOfGrant05-09-2019.pdf 2019-09-05
7 886-KOL-2015-(14-08-2015)-FORM-1.pdf 2015-08-14
8 886-KOL-2015-PatentCertificate05-09-2019.pdf 2019-09-05
8 886-KOL-2015-(14-08-2015)-DRAWINGS.pdf 2015-08-14
9 886-KOL-2015-(14-08-2015)-DESCRIPTION (COMPLETE).pdf 2015-08-14
9 886-KOL-2015-Written submissions and relevant documents (MANDATORY) [03-09-2019(online)].pdf 2019-09-03
10 886-KOL-2015-(14-08-2015)-CORRESPONDENCE.pdf 2015-08-14
10 886-KOL-2015-ExtendedHearingNoticeLetter_21-08-2019.pdf 2019-08-21
11 886-KOL-2015-(14-08-2015)-CLAIMS.pdf 2015-08-14
11 886-KOL-2015-HearingNoticeLetter20-08-2019.pdf 2019-08-20
12 886-KOL-2015-(14-08-2015)-ABSTRACT.pdf 2015-08-14
12 886-KOL-2015-Correspondence to notify the Controller (Mandatory) [12-08-2019(online)].pdf 2019-08-12
13 886-KOL-2015-(08-07-2016)-OTHERS.pdf 2016-07-08
13 886-KOL-2015-ABSTRACT [05-06-2019(online)].pdf 2019-06-05
14 886-KOL-2015-(08-07-2016)-FORM-5.pdf 2016-07-08
14 886-KOL-2015-CLAIMS [05-06-2019(online)].pdf 2019-06-05
15 886-KOL-2015-(08-07-2016)-FORM-13.pdf 2016-07-08
15 886-KOL-2015-COMPLETE SPECIFICATION [05-06-2019(online)].pdf 2019-06-05
16 886-KOL-2015-(08-07-2016)-FORM-1.pdf 2016-07-08
16 886-KOL-2015-FER_SER_REPLY [05-06-2019(online)].pdf 2019-06-05
17 886-KOL-2015-FORM-8 [05-06-2019(online)].pdf 2019-06-05
17 886-KOL-2015-(08-07-2016)-CORRESPONDENCE.pdf 2016-07-08
18 886-KOL-2015-OTHERS [05-06-2019(online)].pdf 2019-06-05
18 Other Patent Document [10-08-2016(online)].pdf 2016-08-10
19 886-KOL-2015-PETITION UNDER RULE 137 [05-06-2019(online)].pdf 2019-06-05
19 Other Document [10-08-2016(online)].pdf_10.pdf 2016-08-10
20 886-KOL-2015-FER.pdf 2018-12-13
20 Other Document [10-08-2016(online)].pdf 2016-08-10
21 886-KOL-2015-FORM 18A [29-11-2018(online)].pdf 2018-11-29
21 Marked Copy [10-08-2016(online)].pdf 2016-08-10
22 Form 26 [10-08-2016(online)].pdf 2016-08-10
22 Form 3 [28-12-2016(online)].pdf 2016-12-28
23 Assignment [09-09-2016(online)].pdf 2016-09-09
23 Form 13 [10-08-2016(online)].pdf_9.pdf 2016-08-10
24 Form 13 [10-08-2016(online)].pdf 2016-08-10
24 Assignment [12-08-2016(online)].pdf 2016-08-12
25 CERTIFIED COPIES TRANSMISSION TO IB [12-08-2016(online)].pdf 2016-08-12
25 Description(Complete) [10-08-2016(online)].pdf 2016-08-10
26 Description(Complete) [12-08-2016(online)].pdf 2016-08-12
26 Drawing [12-08-2016(online)].pdf 2016-08-12
27 Description(Complete) [12-08-2016(online)].pdf 2016-08-12
27 Drawing [12-08-2016(online)].pdf 2016-08-12
28 CERTIFIED COPIES TRANSMISSION TO IB [12-08-2016(online)].pdf 2016-08-12
28 Description(Complete) [10-08-2016(online)].pdf 2016-08-10
29 Assignment [12-08-2016(online)].pdf 2016-08-12
29 Form 13 [10-08-2016(online)].pdf 2016-08-10
30 Assignment [09-09-2016(online)].pdf 2016-09-09
30 Form 13 [10-08-2016(online)].pdf_9.pdf 2016-08-10
31 Form 26 [10-08-2016(online)].pdf 2016-08-10
31 Form 3 [28-12-2016(online)].pdf 2016-12-28
32 886-KOL-2015-FORM 18A [29-11-2018(online)].pdf 2018-11-29
32 Marked Copy [10-08-2016(online)].pdf 2016-08-10
33 886-KOL-2015-FER.pdf 2018-12-13
33 Other Document [10-08-2016(online)].pdf 2016-08-10
34 886-KOL-2015-PETITION UNDER RULE 137 [05-06-2019(online)].pdf 2019-06-05
34 Other Document [10-08-2016(online)].pdf_10.pdf 2016-08-10
35 886-KOL-2015-OTHERS [05-06-2019(online)].pdf 2019-06-05
35 Other Patent Document [10-08-2016(online)].pdf 2016-08-10
36 886-KOL-2015-FORM-8 [05-06-2019(online)].pdf 2019-06-05
36 886-KOL-2015-(08-07-2016)-CORRESPONDENCE.pdf 2016-07-08
37 886-KOL-2015-(08-07-2016)-FORM-1.pdf 2016-07-08
37 886-KOL-2015-FER_SER_REPLY [05-06-2019(online)].pdf 2019-06-05
38 886-KOL-2015-(08-07-2016)-FORM-13.pdf 2016-07-08
38 886-KOL-2015-COMPLETE SPECIFICATION [05-06-2019(online)].pdf 2019-06-05
39 886-KOL-2015-(08-07-2016)-FORM-5.pdf 2016-07-08
39 886-KOL-2015-CLAIMS [05-06-2019(online)].pdf 2019-06-05
40 886-KOL-2015-(08-07-2016)-OTHERS.pdf 2016-07-08
40 886-KOL-2015-ABSTRACT [05-06-2019(online)].pdf 2019-06-05
41 886-KOL-2015-(14-08-2015)-ABSTRACT.pdf 2015-08-14
41 886-KOL-2015-Correspondence to notify the Controller (Mandatory) [12-08-2019(online)].pdf 2019-08-12
42 886-KOL-2015-(14-08-2015)-CLAIMS.pdf 2015-08-14
42 886-KOL-2015-HearingNoticeLetter20-08-2019.pdf 2019-08-20
43 886-KOL-2015-(14-08-2015)-CORRESPONDENCE.pdf 2015-08-14
43 886-KOL-2015-ExtendedHearingNoticeLetter_21-08-2019.pdf 2019-08-21
44 886-KOL-2015-(14-08-2015)-DESCRIPTION (COMPLETE).pdf 2015-08-14
44 886-KOL-2015-Written submissions and relevant documents (MANDATORY) [03-09-2019(online)].pdf 2019-09-03
45 886-KOL-2015-(14-08-2015)-DRAWINGS.pdf 2015-08-14
45 886-KOL-2015-PatentCertificate05-09-2019.pdf 2019-09-05
46 886-KOL-2015-IntimationOfGrant05-09-2019.pdf 2019-09-05
46 886-KOL-2015-(14-08-2015)-FORM-1.pdf 2015-08-14
47 886-KOL-2015-PROOF OF ALTERATION [01-06-2021(online)].pdf 2021-06-01
47 886-KOL-2015-(14-08-2015)-FORM-18.pdf 2015-08-14
48 886-KOL-2015-RELEVANT DOCUMENTS [04-06-2021(online)].pdf 2021-06-04
48 886-KOL-2015-(14-08-2015)-FORM-2.pdf 2015-08-14
49 886-KOL-2015-Response to office action [18-12-2021(online)].pdf 2021-12-18
49 886-KOL-2015-(14-08-2015)-FORM-3.pdf 2015-08-14
50 886-KOL-2015-Annexure [18-12-2021(online)].pdf 2021-12-18
50 886-KOL-2015-(14-08-2015)-FORM-9.pdf 2015-08-14
51 886-KOL-2015-(14-08-2015)-PA.pdf 2015-08-14
51 886-KOL-2015-RELEVANT DOCUMENTS [05-07-2022(online)].pdf 2022-07-05
52 886-KOL-2015-(14-08-2015)-SPECIFICATION.pdf 2015-08-14
52 886-KOL-2015-RELEVANT DOCUMENTS [15-05-2023(online)].pdf 2023-05-15

Search Strategy

1 search_strategy_886_09-01-2018.pdf

ERegister / Renewals

3rd: 04 Dec 2019

From 14/08/2017 - To 14/08/2018

4th: 04 Dec 2019

From 14/08/2018 - To 14/08/2019

5th: 04 Dec 2019

From 14/08/2019 - To 14/08/2020

6th: 10 Aug 2020

From 14/08/2020 - To 14/08/2021

7th: 10 Aug 2020

From 14/08/2021 - To 14/08/2022

8th: 10 Aug 2020

From 14/08/2022 - To 14/08/2023

9th: 10 Aug 2020

From 14/08/2023 - To 14/08/2024

10th: 10 Aug 2020

From 14/08/2024 - To 14/08/2025

11th: 05 Aug 2025

From 14/08/2025 - To 14/08/2026

12th: 05 Aug 2025

From 14/08/2026 - To 14/08/2027

13th: 05 Aug 2025

From 14/08/2027 - To 14/08/2028

14th: 05 Aug 2025

From 14/08/2028 - To 14/08/2029

15th: 05 Aug 2025

From 14/08/2029 - To 14/08/2030