JP3817691B2 - Communication method and communication system using screen - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a communication method and a communication system using a screen for displaying arbitrary data on a screen and communicating by reading the data drawn on the screen with a reading device.
[0002]
[Prior art]
Conventionally, various interfaces have been used to communicate with devices, which has affected the price of the devices and caused compatibility problems. Conventionally, various devices have screens, and communication methods using images have been proposed, but these methods display symbols such as barcodes and are recognized by reading devices such as laser scanners and cameras. It is a method to make it.
[0003]
[Problems to be solved by the invention]
The conventional communication method has a problem in that the amount of data is limited by the resolution of the screen and the reading device, and the amount of data that can be sent by a normal screen and reading device is limited. In addition, the method using the scanning line has a problem that it cannot be used on a display screen having a large amount of afterglow such as liquid crystal and having a different scanning method. Furthermore, since it is impossible to transmit data of an arbitrary length, there is a limit to the amount of data that can be transmitted even though the amount of data to be transmitted varies depending on the application. The system used may not be realized. In addition, it is possible to superimpose other visible images on the code, or to include a visible image in the code, so that human visual identification and information transmission are possible, but it is also impossible to overlay visible images. There were many problems.
[0004]
[Means for Solving the Problems]
In the present invention, it is possible to communicate data of an arbitrary length by continuously displaying barcodes and patterns displayed on various display screens. In addition, by using the preamble and model resolution identification signal to inform the reader of the afterglow characteristics, resolution, and redrawing speed of the screen, communication is performed in the fastest state on each screen regardless of the screen model. It is also possible. The timing of screen redrawing is notified to the reading device by a synchronization marker or a frame number marker, or the reading device can be recognized by detecting a change in the screen, and the conventional problems are solved. It is. Furthermore, according to the present invention, any terminal equipped with a screen, for example, a mobile phone, a portable terminal, a notebook computer, a desktop computer, a video game machine, a television, etc. can be used to improve each of the above problems. Length data can be transmitted.
[0005]
The present invention notifies the start point of data transmission by the preamble display method, starts data transmission, encodes the transmission data, displays a synchronization marker and a frame number marker, and then performs imaging processing. The image obtained in this way is transmitted, the communication condition is recognized, the image is received, the screen position / angle is detected, the correction process is performed, the parameter determination process is performed, and the screen redrawing timing is set. This is a communication method using a screen characterized in that information is read while avoiding data errors, the read information is decoded to be decoded data, and the decoded data is read . A second aspect of the present invention is the communication method using a screen according to the first aspect, wherein the data is transmitted from a terminal or a server.
[0006]
The invention according to claim 3 is a technique for recognizing the screen regardless of the orientation of the screen by processing the received image, and by using a marker whose state does not change at a specific location. The reader is made to recognize the orientation of the screen. The marker can be recognized by the reading device, and the orientation (up / down / left / right) of the screen can be determined from the position. With respect to the inclination of the screen, coordinates are transformed using affine transformation, rotation transformation, or position vector calculation.
[0007]
According to a fourth aspect of the present invention, there is provided a communication method using a screen according to the first aspect, wherein the communication condition is recognized using a model resolution identification signal .
[0008]
According to a fifth aspect of the present invention, in the method according to the first aspect, the screen redrawing timing is performed by causing a reading device to recognize a synchronization marker .
[0009]
According to a sixth aspect of the present invention, in the method of the first aspect, the screen redrawing timing is performed by causing the reading device to recognize a frame number marker on the screen, and is a communication method using a screen. .
[0010]
A seventh aspect of the invention is a communication method using a screen characterized in that in the method of the first aspect, a screen communication method using a difference in monochrome gradation and a screen communication method using color signals are used.
[0011]
The invention according to claim 8 is the communication method using a screen according to claim 1, characterized in that an error correction code due to a read error is used to avoid a data error.
[0012]
A ninth aspect of the present invention is the communication method using a screen according to the first aspect of the present invention, wherein interleaving for diffusing local reading defects is used.
[0013]
According to the tenth aspect of the present invention, the data to be transmitted or the data obtained from the server is interleaved, pre-processed and encoded, converted into an image, converted into an image, and the output is displayed on the screen. Input to the display terminal, transmit the image displayed on the screen, correct the received image obtained from the transmission with respect to the screen position and angle, then perform parameter determination processing, and read the processed information with a reader This is a communication system using a screen characterized by sequentially combining the processes of decoding this and obtaining decoded data.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
In the first embodiment of the present invention, communication is performed by continuously displaying an image similar to a two-dimensional barcode on a screen of a terminal possessed by a user and reading the image by a reading device including a camera or the like.
[0020]
The drawing on the screen in the present invention is based on a program, various screen drawing changes such as GIF and PNG images, and video signal display.
[0021]
The present invention performs communication on a screen. The screen is composed of blocks composed of pixels. The block is composed of an arbitrary number of dots, and communication can be performed at an appropriate speed for each model even when there is a difference in performance depending on the model.
[0022]
The start of communication is indicated by the preamble. The preamble frame includes a specific signal and indicates the start of communication. There can be any number of preamble frames. When the frame number is displayed in each frame, the preamble is not essential, and reading can be started in the middle of communication. By including information in this preamble, the reading device can know the start point of communication.
[0023]
The position of the screen is indicated by a position marker. The position marker indicates the position, orientation, inclination, etc. of the screen in order to read data from the screen.
[0024]
Screen redrawing speed depends on device performance. For this reason, a block or a frame number for synchronization is provided to prevent the influence. Data is transmitted to the receiver side by changing the pattern drawn by the block. A specific block (for example, one on the upper left of the screen) is always turned on and off alternately, and this is used as a signal for a synchronization marker that changes the screen, or a block indicating a frame number is displayed. Send code in other blocks. It is also possible to automatically synchronize by detecting a change in the image on the reading device side.
[0025]
The number of divided blocks in the above needs to have a degree of freedom because the response speed and resolution of the screen are different. The number of blocks (number of bits per frame) used by the screen for signal transmission is notified to the receiver at the preamble stage, or is transmitted using a model resolution identification signal. The drawing speed (time until the frame is changed) is transmitted using a synchronization marker or a frame number marker, or the reading device automatically synchronizes.
[0026]
Considering the screen of the i-mode terminal of the mobile phone as a reference, the number of blocks divided is 300 blocks if the resolution is 120 × 160 dots, assuming 8 × 8 dots as one block. If all the blocks are used for code transmission, a 300-bit signal can be transmitted in one frame, and if a synchronization block is provided, it becomes 299 bits.
[0027]
If the response speed of the screen is slow including the rise and afterglow processing and is 200 ms, since 5 frames can be transmitted per second, transmission of about 1500 bits per second is possible.
[0028]
Next, depending on the performance of the screen, some have a response speed of 50 ms or less, and it seems that transmission can be performed at a higher speed. Also, the number of blocks per frame can be increased if the accuracy of the reader is high.
[0029]
The high-speed communication in the above will be described. In the ideal state of communication by the screen, the number of bits that can be transmitted in one frame is the number of pixels of the screen. A screen having a resolution of 120 × 160 dots would be 19.2 kilobits. If drawing is performed at a response speed of about 50 ms, 20 frames can be transmitted per second.
[0030]
In the case of a color screen, it is possible to transmit a signal for each color, and even for a monochrome terminal, it is possible to divide the transmission signal according to the difference in pixel luminance. In this case, the number of transmission bits per frame can be further increased. The ideal theoretical value is 256 gradations (8 bits) in monochrome, and 65536 colors (16 bits) in the case of color. The number of bits that can be transmitted per frame is 120 x 160 dots, respectively. 120 × 160 × 8 = 153,600 bits = 154 kilobits, 120 × 160 × 16 = 307, 200 bits = 307 kilobits.
[0031]
In the communication, it is desirable to use an error correction code and interleaving for error compensation and ensuring reliability.
[0032]
By overlaying or partially displaying a visible image on the display screen, a symbol or design that can be visually understood by a person different from the data read by the reading device can be displayed.
[0033]
【Example】
An embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a flowchart of a reading process in the embodiment. As shown in FIG. 1, data 10 is displayed on the display screen 13 as a sequence of an arbitrary number of images, that is, frames by an encoding process 11 and an imaging process 12. The encoding process 11 performs preprocessing such as adding an error correction signal to the data 10 and applying interleaving. The imaging process 12 converts the data processed by the encoding process 11 into an image. The reading device 21 converts this screen into an electric signal by the image pickup device 14, and detects the position, orientation, and inclination of the display screen by the screen position angle detection processing 15. Further, the parameter determination process 16 detects the screen resolution, display performance, display frame redrawing, preamble frame, frame number, and the like. The data reading 17 reads the screen, and the decoding process 18 performs the reverse process of the encoding process 11 to restore the data 19.
[0034]
FIG. 2 is an example in which an image is generated by a server 22 different from the terminal in the embodiment. The terminal 20 displays an image already sent from the server 22.
[0035]
FIG. 3 shows the configuration of one frame in the same embodiment. The frame 36 includes a position marker 30 for determining a position, a model identification signal 31, a resolution identification signal 32, a frame number marker 33, a direction identification marker 34, and encoded data 35. The position, orientation, and inclination of the screen 14 are detected by the position marker 30 and the direction identification marker 34, and the screen performance and resolution are detected by the model identification signal 31 and the resolution identification signal 32. The frame number is identified by a frame number marker 33. These are used in the process shown in FIG. Instead of the frame number marker 33, blinking of a synchronization marker may be used. Further, the model identification signal 31 and the resolution identification signal 32 may be communicated by independent preamble images without being included in the frame 36. The encoded data 35 may use halftones or colors according to the performance of the screen 13 or the image pickup device 14.
[0036]
If necessary, it is possible to superimpose or include some visible image on a part or the whole of the communication screen. FIG. 4 shows a frame 40 with a visible image on which the visible images are superimposed. Visible images can be used for advertising. As shown in FIG. 5, the base image 40a and the character 40b of “KEIO UNIV” are overlapped.
[0037]
When error correction or the like is included in the encoding process 11, the influence of this image on reading can be reduced. Further, it may be displayed independently on a part of the screen without being superimposed on the frame 36 or may be included in the encoded data 35.
[0038]
【The invention's effect】
It is possible to perform communication by displaying data of an arbitrary length on the screen and reading the data drawn on the screen with a reading device. This communication is used to exchange data such as tickets, admission tickets, exchange tickets for products, electronic keys for doors, coupons, etc., or exchange tickets for tickets, entrance tickets, products, etc., electronic keys for doors, coupons, etc. It can be used for various purposes such as transmission of product identification information, transmission of line monitoring information in factories, transmission of location information of street terminals, transmission of information from TV programs to other devices, etc. .
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a communication device in the present invention.
FIG. 2 is a block diagram in which the configuration of the display device is similarly changed.
FIG. 3 is a view showing a configuration of a frame to be displayed.
FIG. 4 is a view showing an example of a frame in which visible images are superimposed.
FIG. 5 is also an explanatory diagram of FIG. 4;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 Data to transmit 11 Encoding process 12 Process to image the encoded data 13 Display screen 14 Imager 15 using screen as electrical signal Screen position angle detection process 16 Screen parameter determination process 17 Data reading process from image 18 Data decoding process 19 Transmitted data 20 Terminal 21 Reading device 22 Server 30 Position marker 31 Model identification signal 32 Resolution identification signal 33 Frame number marker 34 Direction identification marker 35 Encoded data 36 Code frame 40 Overlaid with visible image flame

Claims (10)

An image obtained by informing the data transmission start point by the preamble display method, starting data transmission, encoding the transmission data, displaying a synchronization marker and a frame number marker, and then imaging After receiving the image after detecting the communication conditions, detecting the screen position and angle, correcting this, determining the parameter and timing the redrawing of the screen, this information is a data error A communication method using a screen, characterized in that the read information is read, the read information is decoded to be decoded data, and the decoded data is read . 2. The method according to claim 1, wherein the data is transmitted from a terminal or a server. The method according to claim 1, wherein the position angle of the received screen is detected and corrected to allow the reading device to recognize the correct screen position regardless of the orientation of the received screen. Communication method using the selected screen. Communication method using the method of claim 1, wherein the screen communications conditions were characterized by causing recognized using a model resolution identification signal. Communication method using the method of claim 1, wherein the screen and wherein the timing of changing draw the screen is carried out by recognizing the read-device synchronization marker. The method of claim 1, wherein, the timing of changing drawing screen, communication method using a screen and characterized in that by recognizing the read-device frame number marker screen.   2. The communication method using a screen according to claim 1, characterized by a screen communication method based on a difference in monochrome gradation and a screen communication method using color signals. The method of claim 1, wherein the communication method avoids data error using a screen characterized by using the Rue error correction code by the read error.   2. A communication method using a screen according to claim 1, wherein interleaving for diffusing local reading defects is used. The data to be transmitted or the data obtained from the server is interleaved, preprocessed and encoded , converted into an image and converted into an image, and the output is input to a screen display terminal. sending processing an image of the screen display, to correct for screen position and angle of the received image obtained from the transmission, then the parameter determination process, reads the processed information with reader, and decoding it, A communication system using a screen characterized by sequentially combining the processes for obtaining decoded data.

Images