RU97199U1 - SYSTEM, MOBILE DEVICE AND READING DEVICE FOR TRANSFER OF TEXT INFORMATION USING GRAPHIC IMAGES - Google Patents

Abstract

 1. A system for transmitting textual information using graphic images, comprising a mobile device having a graphic image display device, and a reader having a means for reading graphic images, characterized in that the mobile device contains:! a device for generating graphic images, configured to generate graphic images consisting of graphic elements of at least two colors, and the device for generating graphic images generates each graphic image based on a portion of the transmitted text information so as to enable transmission of information from using a sequence of generated graphic images; ! a device for sequential transmission of graphic images, configured to sequentially transmit graphic images generated by the device for generating graphic images to a display device; ! moreover, the reading device comprises an image processing device configured to process a sequence of read images and extract text information encoded by a sequence of read images. ! 2. The system according to claim 1, characterized in that the graphic image contains from 3 to 180 graphic elements. ! 3. The system according to claim 2, characterized in that the graphic image contains not more than 80 graphic elements. ! 4. The system according to claim 2, characterized in that the graphic image contains 20 graphic elements. ! 5. The system according to claim 2, characterized in

Description

The technical field to which the utility model relates.

This utility model relates to optical information transmission systems and can be used in optical information transmission and reading systems used in such areas as, for example, mobile electronic payment systems, access control systems for people in various organizations and enterprises, systems for providing subscriber systems mobile communications access to information, goods and services, etc.

State of the art

There are a large number of ways to transmit information visually and read using a device with a camera. One of the most popular methods is using a two-dimensional graphic code (it can also be designated as a two-dimensional bar code). The essence of the method is to determine the position of a two-dimensional graphic code in space using special graphic elements and the subsequent decoding of the data embedded in the image. Typically, information is encoded in a variety of black and white pixels. The black square means bit “1” in the transmitted block of information, the white square means bit “0”. Two-dimensional graphic codes are significantly superior to one-dimensional codes, for example, a bar code, in information capacity.

One of the most popular two-dimensional graphic codes for today is the QR-Code (US Patent US 005726435 A). A QR-code is a two-dimensional code, including: cells, each of which contains binary-encoded data and at least two positioning symbols located at predetermined positions in the matrix. Positioning symbols are used to determine the position of the graphic code in space, regardless of the position of the reading device. QR-code allows you to transfer about 2 kilobytes of data.

Two-dimensional graphic codes were invented a long time ago and were originally printed only on paper. With the popularization of mobile phones, a need arose for the rapid transfer of information using the screen of a mobile phone. For example, with the development of the mobile marketing industry, the mobile phone has often been used as a carrier of electronic coupons. To account for the use of electronic coupons, instant transfer of information from a mobile phone to a seller’s device is required. For these purposes, the technology of graphic codes began to be used in mobile phones, however, the reading of existing graphic codes from mobile phones of buyers is not as effective as from paper. It is worth considering the fact that most of the screens of mobile phones are small, so in order to recognize the small details of the graphic code, a high-definition camera or favorable reading conditions are required. Such circumstances as the trembling hand of the buyer holding the phone, as well as glare on the glass can significantly increase the reading time. The reason for these problems is the high degree of complexity of existing technologies for displaying graphic code. On the one hand, you can put a minimal amount of information in a graphic code, and then it can be easily read even from a small screen of a mobile phone. However, this approach is not applicable, for example, for mobile payment systems, in which both the speed of reading information for a quick purchase and the security of personal information of a client, which requires the transmission of a long identification key (from 128 bits), are equally important.

Another disadvantage is the high cost of equipment for reading graphic codes with a large amount of information, since a camera is required that is resistant to movements of the user's hand and reader's hand.

Utility Model Disclosure

This utility model is aimed at creating a system and devices for transmitting textual information using graphic images through a screen (display device) of a mobile device that does not have the noted drawbacks inherent in the prior art. The objective of the utility model is to propose such a system and devices in which the requirements for equipment for reading information from the screen of a mobile device are reduced. An additional objective of the utility model is to ensure quick reading of information from the screen of a mobile device, as well as providing the ability to vary the amount of data transmitted with a slight increase in the time required to read data from the screen of a mobile device.

The specified technical result is achieved using a system for transmitting text information using graphic images containing a mobile device having a display device for graphic images, and a reading device having means for reading graphic images. The mobile device included in such a system comprises: a device for generating graphic images, configured to generate graphic images consisting of graphic elements of at least two colors, the device for generating graphic images generates each graphic image based on a portion of the transmitted text information in such a way as to enable the transmission of information using a sequence of generated graphic images; and a device for sequential transmission of graphic images, configured to sequentially transmit graphic images generated by the device for generating graphic images to a display device. At the same time, a reader included in such a system comprises an image processing device configured to process a sequence of read images and extract text information encoded by a sequence of read images.

Depending on the size of the display device of the mobile device included in the system, the graphic image may contain from 3 to 180 graphic elements, and in a preferred embodiment, the graphic image contains no more than 80 graphic elements, preferably 20 graphic elements. In one embodiment, the graphic elements have the same shape and the same size, and in order to enable re-reading, a cyclic repetition of the sequence of graphic images can also be realized. It is preferable that the graphic image contains service graphic elements for determining the orientation of the graphic image relative to the reader and / or the number of graphic images in the displayed sequence of graphic images, in one embodiment, at least some of the service graphic elements have a color, different from the color of the elements used to transmit text information and / or a form different from the shape of the elements used soy for the transmission of textual information.

A mobile device included in the system, in one embodiment, contains a timer, and the device for sequential transmission of graphic images is configured to transmit graphic images to the display device at time intervals measured by the timer. Moreover, the time intervals measured by the timer preferably have an equal or greater value compared to the reciprocal of the number of frames per unit time performed by the reader. A mobile device included in the system is one of the following devices: a mobile phone, a smartphone, a communicator, a PDA, or it can be any mobile device that has a screen (display device) on which images can change.

The means for reading a graphic image, which is part of the reader, which is part of the proposed system, may be a camera or a video camera, and the reader may be one of the following devices: mobile phone, smartphone, communicator, PDA.

In another aspect, the objective of the present utility model is solved by a mobile device for transmitting textual information using graphic images to a reader having a graphic image display device. Such a mobile device also: a device for generating graphic images, configured to generate graphic images consisting of graphic elements of at least two colors, and each graphic image is generated based on a portion of the transmitted text information so as to transmit information using a sequence generated graphic images; and a device for sequential transmission of graphic images, configured to sequentially transmit graphic images generated by the device for generating graphic images to a display device.

The proposed mobile device may include a timer, and the device for sequential transmission of graphic images can be configured to transmit graphic images to the display device at intervals measured by the timer. The proposed mobile device included in the system is one of the following devices: a mobile phone, smartphone, communicator, PDA, or may be any mobile device having a display device on which images can be changed.

Depending on the size of the display device, the graphic image generated by the proposed mobile device may contain from 3 to 180 graphic elements, at the same time preferably 20 graphic elements. In one embodiment, the graphic elements of the generated image have the same shape and the same size, and the sequence of graphic images can be cyclically repeated.

The graphic image generated by the proposed mobile device may comprise service graphic elements for determining the orientation of the graphic image with respect to the reader and / or the number of graphic images in the displayed sequence of graphic images, at least some service graphic elements may have a color different from the color elements used to transmit textual information and / or a form different from the form of the element Used to transmit textual information.

In an additional aspect, the objective of the present utility model is solved by a reader having means for reading graphic images from a display device of a mobile device. Such a reading device comprises an image processing device configured to process a sequence of read images and extract text information encoded by a sequence of read images.

In a preferred embodiment, the means for reading a graphic image in the proposed reader is a camera or video camera, and the reader is one of the following devices: mobile phone, smartphone, communicator, PDA. The proposed reader may be designed to process graphic images containing graphic elements having the same shape and the same size. The present reader may be configured to process a cyclically repeated sequence of graphic images.

In a preferred embodiment, the reader is intended to process a graphic image containing service graphic elements designed to determine the orientation of the graphic image relative to the reader and / or the number of graphic images in the displayed sequence of graphic images, at least some service graphic elements may have a color different from the color of the elements used to transmit text information and / or shape, o which differs from the form of the elements used to transmit textual information.

Thanks to the proposed system, a mobile device and a reader for transmitting textual information using graphic images, it is possible to reduce the requirements for the reading device, in particular, the resolution and / or focusing system of the means for reading graphic images, due to the increase in the size of the graphic elements that make up graphic images generated on the basis of transmitted text information. Due to the reading of the sequence of generated graphic images with enlarged graphic elements, it is possible to transmit the same or greater amount of information as when reading a single graphic image with a large number of small graphic elements, at the same time, allowing for quick reading of information from the screen of a mobile device , slightly increasing the time or even reducing the time required to read data from the screen of a mobile device. Since the reading of graphic images occurs at a speed of, for example, 10, 15, 25, 30, or 60 frames per second, during the presentation of a mobile device to a reader having a cheap camera with a low resolution, a sequence of graphic images can be read with the same transmission or even more textual information than when reading a single graphic image with a large number of graphic elements using a more expensive camera with high resolution. In addition, thanks to this utility model, it is also possible to vary the amount of transmitted data with a slight increase in the time required to read data from the screen of a mobile device.

Brief Description of the Drawings

Figure 1 presents a block diagram of a system for transmitting text information using graphic images.

Figure 2 illustrates the process of converting and transmitting information from one system component to another.

Figure 3 shows the sequence of actions performed by the device for generating graphic images.

Figure 4 shows the sequence of images obtained as a result of the actions shown in figure 3.

Figure 5 shows a block diagram of the algorithm of the device for serial transmission of graphic images.

Figure 6 shows a block diagram of the algorithm of the image processing device.

Utility Model Implementation

The proposed system, mobile device and reader are designed to transmit text information using graphic images. Textual information is understood to mean alphanumeric, hieroglyphic, and any other information whose elements have predetermined or standard meanings. In some embodiments, the text information elements can be encoded, for example, using ASCII, KOI-8 codes, etc. The mobile device that is part of the system can be one of the following devices: mobile phone, smartphone, communicator, PDA, and for reading a graphic image, it can be a camera or a video camera, and a reader can be one of the following devices: mobile phone, smartphone, communicator, PDA.

As shown in FIG. 1, a system for transmitting information through a screen of a mobile device comprises a mobile device M1, a device M2 for generating graphic images, a device M3 for sequential transmission of graphic images, a timer M4, and a display device M5, which may also be referred to as a screen display or monitor. The timer M4 in the mobile device M1 may not be available. The system also includes a reader C1, comprising means for reading graphic images in the form of a camera C2, as well as a device for processing images C3. The mobile device and the reader are connected by continuously receiving images from the camera C2, which shows the screen of the mobile device M1.

Figure 2 shows how the conversion and transmission of information from one system component to another occurs. Figure 2 shows that the first transmitted information is presented in the form of a final sequence of bytes 1, presumably located in the memory of the mobile device. Then the sequence of bytes, using the device M2 for generating graphic images, is represented as a final sequence of graphic images 2-1 ... 2-N. The device MZ for sequential transmission of graphic images sequentially transmits to the display device M5 a sequence of received graphic images 2-1 ... 2-N with some time delay between two consecutive images, moreover, the time delay in one embodiment can be set by timer M4, and in another embodiment, it can determined by the duration of the processes of generation and / or transmission of graphic images. Further, the camera C2 of the reading device C1 continuously produces a long sequence of images 3-1, 3-2, ..., and the reading device C1 is positioned in space relative to the mobile device M1 so that among the elements of the sequence 3-1, 3-2, ... the final subsequence 4-1 ... 4-N of N images, each of which shows the display device M5, the image 4-1 received at the time when the image 2-1 was shown on the display device M5, the image 4-2 received at the moment when the image 2-2 was shown on the display device M5, and so on, the 4-N image was obtained at the time when the 2-N image was displayed on the display device M5. Then, the image processing apparatus C3 processes the sequence 3-1, 3-2, ... until the indicated final subsequence 4-1 ... 4-N is highlighted. Next, the device for processing images from the existing sequence 4-1 ... 4-N restores the original sequence of images 2-1 ... 2-N. Next, the device for processing images from the sequence of images 2-1 ... 2-N restores the sequence of bytes 5, exactly or with some error determined by the conditions for the transfer of graphic images corresponding to the transmitted sequence of bytes 1.

Figure 3 shows the sequence of actions performed by one of the possible algorithms, which follows the device M2 for generating graphic images. At the input, the device M2 receives the final sequence of bytes 1. At the output, the device M2 produces a sequence of graphic images 2-1, 2-2, 2-3, in which the sequence of bytes 1 is encoded, as well as some overhead information.

At the input of the algorithm, there is the following sequence of bytes: A1 = 20, A2 = 30, A3 = 40.

At step 6, these numbers are presented in binary notation: A1 = 00010100 A2 = 00011110 A3 = 00101000.

At step 7, a blank for image 2-1 is generated. The blank is a 4 by 5 table. The upper left, upper right and lower left cells of the table are painted in red (in the figures the red color is represented by hatching). This is subsequently necessary to determine the orientation of the image on the screen of the mobile device M1 in space relative to the camera of the device C1 when receiving images from it. 17 cells of the table remain unpainted.

At step 8, the unfilled cells of the image table 2-1 are sequentially painted from left to right in accordance with the binary representation of byte A1. Cells corresponding to bit 0 are painted over in white, cells corresponding to bit 1 are painted over in black.

At step 9, the unfilled cells of the image table 2-1 are sequentially painted from left to right in accordance with the binary representation of byte A2. Cells corresponding to bit 0 are painted over in white, cells corresponding to bit 1 are painted over in black. The right bottom cell of the table is not used and is painted over in white.

At step 10, image 2-2 is generated by analogy with steps 7-9. Since the initial data set is limited to three bytes, the remaining free cells in the table are painted over in white.

At step 11, the generation of the image 2-3, containing service information necessary for the correct recognition of the sequence of images by the device C3 for image processing, in particular, the number of images in the displayed sequence. One or more of the images could be lost, and the information from image 2-3 will make it clear that this has happened. For example: if it turned out that the “service image” indicates that a total of 10 images should be shown, but in reality the camera and / or image processing device recorded only 8 photos that depict different from each other pictures from the sequence 2-1 ... 2 -N. This can happen if some task unexpectedly starts on the reader, because of which the camera “missed” a few pictures. In such a situation, the sequence of images will be cyclically repeated until the remaining 2 images are captured by the camera. The blank for image 2-3 is done the same as for the previous images - a table is displayed and three squares are painted over in red. By analogy with steps 7 and 9, the following two numbers are encoded into image 2-3: 3 and 170. The number 3 indicates the number of images in the sequence 2-1 ... 2-N, and the number 170 is a service number and indicates that in this image, the number of images in the sequence is encoded.

Figure 4 shows the sequence of images obtained as a result of the actions shown in figure 3. In General, depending on the size of the display device of the mobile device included in the system, the graphic image may contain from 3 to 180 graphic elements, and in the preferred embodiment, the graphic image contains not more than 80 graphic elements, preferably 20 graphic elements. In one embodiment, the graphic elements have the same shape and the same size, and to enable re-reading, a cyclic repeat of the sequence of graphic images can also be implemented. Preferably, the graphic image comprises service graphic elements for determining the orientation of the graphic image with respect to the reader and / or the number of graphic images in the displayed sequence of graphic images, however, images may be provided in which such service elements are absent. If there are service elements in one of the options, at least some service graphic elements have a color different from the color of elements used to transmit text information, however, all service elements can have the same colors compared to elements used to transmit text information . Some service graphic elements may have a shape different from the shape of the elements used to transmit text information, however, in the described case, their shape is the same.

Figure 5 shows a block diagram of the algorithm of the device MOH for serial transmission of graphic images and its relationship with the timer M4.

At step 12, the counter of the number of transmitted images K is set to zero.

At step 13, the 2-K image is transmitted to the display device.

At step 14, the counter K is incremented by one.

At step 15, the timer is accessed. The timer stops the MZ device for a certain amount of time. The required amount of delay time of the timer M4, necessary for the correct reception of images by camera C2 of device C1, can be calculated by the following formula:

T = 1 / (FPS / N)

The following parameters participate in the formula: T - the required timer delay time in fractions of a second, FPS - the number of frames per second (characteristic of the C2 camera) and N - the number of images in the sequence 2-1 ... 2-N. The formula shows that for device C1, the choice of a camera with a high FPS value is more preferable. The duration of the demonstration of the image may be more than the calculated value of the delay timer.

At step 16, a check is made to see if the last image in sequence has been transmitted to the display device. If not, the algorithm proceeds to step 13; if so, the algorithm returns to step 12.

The algorithm stops its work at the request of the user of the mobile device.

Figure 6 shows a block diagram of the algorithm of the device SZ image processing corresponding to the encoding algorithm for a sequence of bytes, an example of which is described in figure 3. At the input, the algorithm receives a long sequence of images 3-1, 3-2, ... from camera C2 of device C1. At the output, the algorithm produces a sequence of bytes A1 ... AM encoded in the received images. For ease of description, this algorithm assumes that in the original sequence of bytes there are no consecutive identical pairs of bytes, the sequence does not end with zero and does not contain the number 170.

At step 17, the variable N denoting the number of images in the sequence 2-1 ... 2-N is set to -1 (the value has not yet been determined). The counter of the number of byte pairs recognized in the current cycle is set to 0.

At step 18, the next in order image from the camera is checked for red shapes in it. For this, the image processing device sequentially passes through all the pixels of the image until a color close to red appears (color is considered close, for which the following relations are true for the parameters of the RGB color components: R> 150, G <100, B <100). If such a pixel is found, the algorithm scans all neighboring pixels until the outlines of the red figure in the image are determined. It is assumed that the red figure found in the image is one of the three red cells in one of the 2-1 ... 2-N images. Further, the image scan continues to the end of the data array, while the pixels included in the first red figure are not taken into account. Thus, after step 18, several red shapes are highlighted in the image. If not found on one red piece, the algorithm returns to step 18.

In step 19, the red shapes found in the previous step are compared in volume. If you manage to isolate the three figures that are close to each other in volume, as well as all the other red figures found in the image, differ in volume from the three indicated by at least 50%, it is believed that three red squares are found from one of the images 2-1 ... 2 -N. In this case, the algorithm proceeds to step 20. If three triples with the specified property were not found, the algorithm returns to step 18.

At step 20, two bytes encoded in the current image are detected. To do this, a search is made for the points of the centers of mass of the three red figures found in step 19. The points found form a parallelogram. Knowing the coordinates of the parallelogram, as well as the number of cells on each side in the table in the images 2-1 ... 2-N, the algorithm can determine the coordinates of the centers of all cells with information on the resulting image. Further, analyzing the color of the pixel at the centers of these cells, the algorithm obtains a sequence of 16 bits that were encoded in one of the 2-1 ... 2-N images.

At step 21, a check is made to see if the second encoded byte is the number 170 (overhead). If so, the algorithm proceeds to step 22. If the value is not equal to 170 and the last two bytes of the sequence of bytes recognized by the current time differ from the pair of encoded bytes obtained from the current image, then this pair of bytes is entered at the end of the sequence of recognized bytes A1 ... AM, as well as to the counter K, the number of bytes recognized in the current cycle of bytes is added to one. Next, the algorithm returns to step 18.

At step 22, when the algorithm detects that encoded overhead information is present on the image (service number 170 and number N), it checks whether all pairs of bytes in the sequence are found (K = N-1). If this check gives a positive result, the algorithm ends and the result of its work is a sequence of bytes A1 ... AM, consisting of 2 * (N-1) bytes. If the equality is not satisfied, the variable N is assigned the value encoded in the first byte, the value K is reset, the not found sequence of bytes A1 ... AM is erased, and the algorithm returns to step 18.

The description of the device is made with the division into functional blocks, however, such a division is not essential for the implementation of devices that can be performed both hardware and software, or in a combination of hardware and software.

The present utility model relates both to a system for transmitting text information using graphic images in general, and to a mobile device and a reader included in the system separately.

Claims (25)

1. A system for transmitting textual information using graphic images, comprising a mobile device having a display device for graphic images, and a reader having means for reading graphic images, characterized in that the mobile device contains: a device for generating graphic images, configured to generate graphic images consisting of graphic elements of at least two colors, and the device for generating graphic images generates each graphic image based on a portion of the transmitted text information so as to enable transmission of information from using a sequence of generated graphic images; a device for sequential transmission of graphic images, configured to sequentially transmit graphic images generated by the device for generating graphic images to a display device; moreover, the reading device comprises an image processing device configured to process a sequence of read images and extract text information encoded by a sequence of read images. 2. The system according to claim 1, characterized in that the graphic image contains from 3 to 180 graphic elements. 3. The system according to claim 2, characterized in that the graphic image contains not more than 80 graphic elements. 4. The system according to claim 2, characterized in that the graphic image contains 20 graphic elements. 5. The system according to claim 2, characterized in that the graphic elements have the same shape and the same size. 6. The system according to claim 1, characterized in that the sequence of graphic images is cyclically repeated. 7. The system according to claim 1, characterized in that the graphic image contains service graphic elements designed to determine the orientation of the graphic image relative to the reader and / or the number of graphic images in the displayed sequence of graphic images. 8. The system according to claim 7, characterized in that at least some service graphic elements have a color different from the color of the elements used to transmit text information and / or a shape different from the shape of the elements used to transmit text information. 9. The system according to claim 1, characterized in that the mobile device contains a timer, and the device for sequential transmission of graphic images is configured to transmit graphic images to the display device at intervals measured by the timer. 10. The system according to claim 9, characterized in that the time intervals measured by the timer have an equal or greater value compared to the reciprocal of the number of frames per unit time performed by the reader. 11. The system according to claim 1, characterized in that the mobile device is one of the following devices: a mobile phone, smartphone, communicator, PDA, and the means for reading a graphic image is a camera or video camera, and the reader is one of the following devices: mobile phone, smartphone, communicator, PDA. 12. A mobile device for transmitting text information using graphic images to a reader having a display device for graphic images and characterized in that it contains: a device for generating graphic images, configured to generate graphic images consisting of graphic elements of at least two colors, each graphic image being generated based on a portion of the transmitted text information so as to transmit information using a sequence of generated graphic images; a device for sequential transmission of graphic images, configured to sequentially transmit graphic images generated by the device for generating graphic images to a display device. 13. The mobile device according to item 12, characterized in that it contains a timer, and the device for sequential transmission of graphic images is configured to transmit graphic images to the display device at intervals measured by the timer. 14. The mobile device according to item 12, characterized in that it is one of the following devices: mobile phone, smartphone, communicator, PDA. 15. The mobile device according to item 12, wherein the graphic image contains from 3 to 180 graphic elements, preferably 20 graphic elements. 16. The mobile device according to clause 15, wherein the graphic elements have the same shape and the same size. 17. The mobile device according to item 12, characterized in that the sequence of graphic images is cyclically repeated. 18. The mobile device according to item 12, wherein the graphic image contains service graphic elements for determining the orientation of the graphic image relative to the reader and / or the number of graphic images in the displayed sequence of graphic images. 19. The mobile device according to p. 18, characterized in that at least some service graphic elements have a color different from the color of the elements used to transmit text information and / or a shape different from the shape of the elements used to transmit text information . 20. A reading device having means for reading graphic images from a display device of a mobile device and characterized in that it comprises an image processing device configured to process a sequence of read images and extract text information encoded by a sequence of read images. 21. The reader according to claim 20, characterized in that the means for reading the graphic image is a camera or a video camera, and the reader is one of the following devices: mobile phone, smartphone, communicator, PDA. 22. The reader device according to claim 20, characterized in that the graphic image contains graphic elements having the same shape and the same size. 23. The reader device according to claim 20, characterized in that the sequence of graphic images is cyclically repeated. 24. The reader according to claim 20, wherein the graphic image contains service graphic elements for determining the orientation of the graphic image relative to the reader and / or the number of graphic images in the displayed sequence of graphic images. 25. The reader device according to paragraph 24, wherein at least some service graphic elements have a color different from the color of the elements used to transmit text information and / or a shape different from the shape of the elements used to transmit text information .