PL239789B1 - Digital-optical system and method of recognizing labels with superimposed data projection - Google Patents

Description

PL 239 789 B1

Description of the invention

The present invention relates to a digital-optical method for recognizing labels with superimposed data projection. In this specification, the term "label" includes any print or other representation of a graphic pattern on the flat or curved surfaces of products.

There are two-dimensional, matrix, square graphic codes (QR Codes) developed by the Japanese company Denso-Wave in 1994. The structure of the code makes it possible to place and read it on objects that move quickly in relation to the scanner, which is usually a suitably programmed portable device. Using these codes, numerous new systemic applications are patented as disclosed in patents e.g. US10366315B1, US10375060B1. Another popular solution is the use of barcodes on product labels. The main application of the bar code is automatic product identification. These codes, like QR codes, are standardized, and on their basis new methods used in the art are patented, as disclosed in patents, e.g. US10380580B2 and US10212306B1.

So far, graphic codes, scanned, imaged and recognized by means of electronic mobile devices such as smartphones, used specially placed on product labels, do not create an impression on the user, as if the digital content associated with a given graphic code was part of a recognized label. This is due to the fact that in known solutions the information read from these codes is not technically directly displayed, in almost real time, in relation to the entire label being pictured.

The object of the invention was to provide a label recognition method with superimposed data projection which gives the user the impression that the projection of additional data (e.g., consumer useful information) is part of a recognizable label.

The essence of the invention is a digital-optical method of recognizing labels with superimposed data projection, which consists in the fact that a digital camera with an automatic focusing function records a digital image containing a label under lighting with an intensity of at least 200 lux, which is then numerically parameterized by a microcontroller and the the values are compared with the numerical values of the parameterized digital pattern of the graphic motif mapped on the label. If there is a match between the parameters of the pattern and the registered graphic motif, the data is projected on the display screen in the part of the recognized graphic motif that covers this graphic motif, with the background of the recorded image unchanged.

Thanks to the developed solution, the display shows the recorded image, where, in place of its characteristic graphic motif, graphic data is projected almost in real time, e.g. information on product properties. This gives the user the impression that the displayed data is part of a recognizable label. This technique enables the production of visually appealing product labels whose distinctive graphic motifs will be recognized and replaced with the predicted data projection.

In an embodiment, the invention is illustrated in the drawing which explains the method of recognizing labels with superimposed data projection.

Example of implementation

The digital-optical label recognition system 1 with superimposed data projection in the exemplary embodiment consisted of a label 1 placed on the surface 2 of the bottle, a digital camera 3 with an autofocus function with a resolution of 10 megapixels and a sampling rate of 30 frames per second, managed by a type 4 microcontroller Arm Cortex M4 (TXZ 4 series), which was combined with 5 external memory and 6 display screen with FulIHD 1920 x 1080 pixels resolution (AMOLED display). This system is characterized in that after the camera 3 has recorded and the microcontroller 4 recognizes the digital image 7, a part of the display screen 6 is separated for the projection of data independent of the recorded image. A separate part 9 of the display 6 covers the recognized graphic motif 8, with the background 10 of the recorded digital image 7 unchanged on the rest of the display 6. The displayed data, in an exemplary embodiment, related to the health properties of the labeled product. In another embodiment, the system may be an integral part of known mobile devices such as smartphone, iPhone.

The method in the embodiment was carried out in the following steps:

• a digital camera 3 with an auto focus function records a digital image 7 containing a graphic pattern (8) in a 200 lux lighting condition, • numerical parameterization of the recorded images 7 through a microcontroller 4, • comparing the numerical values of the parameterized image 7 with the values parameterized digital pattern 11 of the graphic pattern 8, • when the compared parameters of pattern 11 and the registered graphic pattern 8 match, the data is projected on the display screen 6 in part 9 of the recognized graphic pattern 8, which covers this graphic pattern 8, with an unchanged background 10 recorded on an ongoing basis image 7.

The parameterization of the digital pattern 11 of the graphic motif 8 in the example implementation consisted in:

• working out a heat map of the pattern 11 image - stage A;

• then, to calculate the checksum for each area of the heat map that was more detailed than its surroundings - stage B;

• determining the check sum of the whole image in the form of a checksum matrix - step C;

• averaging the color values of all areas - stage D.

The essence of the previously indicated stages (A, B, C and D) is in the example implementation, in turn:

• Stage A

In this step, the individual pixels of the image are compared along with the adjacent pixels. Each gradient of color change or intensity of individual pixels is marked with a value from 0 to 1024, where 0 is no change, and 1024 is a 100% change of at least one red, green or blue channel (R, G or B). The percentage of change on individual channels is summed up, but only until it reaches 100%. On the basis of such a map, it is possible to distinguish areas that differ from the surroundings or other, unassociated areas, e.g. an image of two different objects standing next to each other.

• Stages B and C

For the calculation of the checksum in the example embodiment, only those areas whose gradient is at least 512 were taken. For each area, an average color, area aspect ratio, area size (expressed in square pixels) and gradients between individual area pixels are calculated. Such a set of information about a given area is stored in binary form, and the checksums of all areas form a matrix which is the checksum of the entire graphic.

• Stage D

The averaging of the area colors consisted in summing the red channel values of all pixels and dividing the sum by the number of pixels. Thus, the average value of the red channel (R) was obtained. The same process should be repeated for the green (G) and blue (B) channels. As a result, we get the values of all three channels, i.e. the average color of the area.

The determined parameters of the digital pattern 11 of the graphic motif 8 together with the data to be projected in the display part 6 were in the external memory 5.

The parameterization of the image 7 and its recognition in comparison with the parameterized digital pattern 11 took place, in an exemplary embodiment, in the following steps:

1. Directing the camera 3 at the label 1 in such a way as to record the entire graphic pattern 8 together with the information on the focus and brightness of the lighting at the time of recording the image 7.

2. Calculation of the checksum, similarly to the calculation of the checksum of digital pattern 11, however, taking into account the following changes:

• after working out the heat map of the image 7, computing the checksums of the areas takes place in the sequence from the center of the image, spiraling to its edges;

• if, after calculating the checksum for the areas covering 50% of the image, no area corresponding to the checksum of the area from the digital pattern 11 with the tolerance of 15% was found, further calculation of the checksum for this image is not continued and the microcontroller 4 starts processing the next saved in image memory;

Claims (1)

In the case of detecting in the first 50% of the image of the area corresponding to the check sum of the area from the graphic of the digital pattern 11 in digital form with a tolerance of 15%, the processor returns information about the position of the recognized area in relation to the center of the recorded image and in relation to the center of the graphic of the label in the form of digital; • when comparing successive captured images 7, instead of 50%, 90% of the image size is considered, and the tolerance is increased to 25%. If no similarity is detected, these values return to the original values for subsequent images; • at the same time, when the microcontroller 4 is comparing consecutive images recorded by the camera 7, these are transferred to the screen for the user to follow the area captured by the camera 3; • at the moment when the microcontroller recognizes the image 7 with the digital pattern 11, the screen showing the image from the camera begins to display the digital content assigned to the recognized label. The displayed content (data projection) is in size and position 9 relative to the location of the detected graphic pattern 8 in the scanned image and the target position of the digital content assumed for the given label graphic in digital form. This gives the user the impression that the digital content is part of a recognized label. In an embodiment of the method according to the invention, the use of an exemplary method of recognizing digital images is presented, which does not exclude the use of other image analysis methods that fulfill the same object of the invention. Patent claim 1. A digital-optical label recognition method with superimposed data projection, characterized in that the digital camera (3) with an autofocus function captures a digital image (7) containing the label (1) under an illumination of at least 200 lux, which then the microcontroller (4) is numerically parameterized, and the determined values are compared with the numerical values of the parameterized digital pattern (11) of the graphic pattern (8) mapped on the label (1), provided that if there is a match between the parameters of the pattern (11) and the registered graphic pattern ( 8), the data is projected on the display screen (6) in the part (9) of the recognized graphic pattern (8) that covers this graphic pattern (8), with the background (10) of the continuously recorded image (7) unchanged.