JP2003189020A - Communication method using screen and communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform a communication of data having a variable data length by processing the data after receiving as a picture image which is composed of graphic data and symbol data and is transmitted. <P>SOLUTION: The communication is performed through steps of displaying given data as the picture image, transmitting the picture image, processing the data after receiving the picture image, and reading the data by a reading device. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a communication method and communication using a screen for the purpose of displaying arbitrary data on the screen and reading the data drawn on the screen with a reading device for communication. Regarding the system.

[0002]

2. Description of the Related Art Conventionally, in order to communicate with a device,
Various interfaces have been used to affect the price of the device and cause compatibility problems. Conventionally, various devices have a screen as a device and a communication method using an image has been proposed, but those methods display a symbol such as a barcode and recognize it by a reading device such as a laser scanner or a camera. It is a method to let.

[0003]

In the above-mentioned conventional communication method, the amount of data is limited by the resolution of the screen or reading device, and the amount of data that can be sent by a normal screen or reading device is limited. There was a problem. Also,
The method using the scanning line has a problem that it cannot be used on a display screen having a large amount of afterglow such as liquid crystal and different scanning methods. Furthermore, since it is impossible to send data of arbitrary length, there is a limit to the amount of data that can be sent, although the amount of data to be sent differs depending on the application, so a large amount of data can be sent. In some cases, the system used cannot be realized. In addition, by superimposing another visible image on the code, or by including the visible image in the code, it is possible to visually identify and transmit information, but it is also impossible to include the visible image. There were many problems.

[0004]

According to the present invention, it is possible to communicate data of arbitrary length by continuously displaying bar codes and patterns displayed on various display screens. In addition, by communicating the afterglow characteristics, resolution, rewriting speed, etc. of the screen to the reading device side using the preamble and model resolution identification signal, communication is performed in the fastest state on each screen regardless of the model of the screen. It is also possible. The conventional problems such as the fact that the timing of screen rewriting can be recognized by notifying the reading device with a synchronization marker or a frame number marker or by the change in the screen being detected by the reading device have been solved. Of. Further, according to the present invention, any terminal provided with a screen, for example, a mobile phone, a portable terminal, a notebook computer, a desktop computer, a video game machine, a television, or the like is used to improve the above-mentioned problems and Length data can be sent.

According to the present invention, data is displayed on a screen as an image or a symbol, a part or the whole of the screen is displayed once or redrawn a plurality of times, converted into data by a reading device, read, and the data is read. It is a communication method using a screen characterized by communication.

The invention according to claim 2 is a method of processing the received image so that the screen can be recognized regardless of the orientation of the screen, and a marker whose state does not change is used at a specific location. By doing so, the reading device is made to recognize the orientation of the screen. Let the reader recognize the marker, and from that position,
It is possible to determine the orientation of the screen (up, down, left and right). Regarding the tilt of the screen, an affine transformation (aff
The coordinates are converted by using an inine transformation), a rotation conversion, or a position vector calculation.

According to a third aspect of the present invention, the screen can be recognized regardless of the inclination of the screen, and by using a marker whose state does not change at a specific position,
It is characterized in that the reading device is made to recognize the orientation of the screen.

According to a fourth aspect of the present invention, the preamble display method during low-speed communication is to display a frame including some information on the screen to indicate the start of communication.

According to the fifth aspect of the invention, the model resolution identification signal is a signal for transmitting the performance, resolution, data transfer rate, etc. of the screen to the reading device, and the information necessary for communication is transmitted on the screen.

According to a sixth aspect of the invention, the synchronization marker is
The feature is that the reader is notified of the timing at which the screen changes. By using the synchronization marker, the fluctuation in the drawing timing of the screen during communication is absorbed and the drawing timing is captured.

The invention according to claim 7 is characterized in that the frame number marker informs the reading device of the screen number. The frame number is used to determine the temporal position of the communication data.

According to an eighth aspect of the invention, the reading device detects the change in the state of the screen by a calculation method such as a difference, and then reads.

The visible image of the invention according to claim 9 makes it possible to display advertisements and the like on the screen.

According to a tenth aspect of the present invention, a screen communication method based on a difference in monochrome gradation is characterized in that a transmission signal is divided according to a difference in liquid crystal luminance. For example, 256
If gradation display is possible, ideally one block can transmit a signal of 8 bits. Furthermore, the screen communication method using the color-specific signals is characterized in that the color-specific signals are transmitted when the screen supports color display.
For example, if 65536 colors can be displayed, ideally, one block can transmit a signal of 16 bits.

An eleventh aspect of the present invention is a method capable of performing communication including an error correction code so that an error does not occur even if the screen data is missed.

The invention according to the twelfth aspect is a method capable of performing communication by using interleaving in which errors are dispersed in the time or space direction so that errors are not locally concentrated.

According to a thirteenth aspect of the present invention, the data to be transmitted is encoded, the image obtained by the image processing is transmitted, the received image is corrected with respect to the screen position / angle, and then the parameter determining process is performed. After that, the information is read, the composite processing is performed, and the transmission data is obtained, which is a communication system using a screen. The correction of the image reception is to correct the image in a certain state so that the image can be read correctly regardless of the form received.

According to a fourteenth aspect of the present invention, the data obtained from the server is coded, the data is imaged, the output is input to the terminal of the screen display, and the image of the screen display is transmitted. A communication system using a screen, characterized in that a received image is corrected with respect to a screen position / angle, then parameter determination processing is performed, information is read, and composite processing is performed to obtain transmission data. The above-mentioned invention is a case of processing the data obtained from the server, which is image-processed and then connected to the terminal of the display screen.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION In the first embodiment of the present invention, an image similar to a two-dimensional bar code is continuously displayed on the screen of a terminal possessed by a user and read by a reading device including a camera or the like. Is to communicate with.

The drawing on the screen in the present invention is various by a program, redrawing of the screen by a GIF or PNG image, display of a video signal and the like.

The present invention communicates on a screen. The screen is composed of blocks composed of pixels. The block is composed of an arbitrary number of dots,
Even if there is a difference in performance between models, it is possible to communicate with each model at an appropriate speed.

The communication is marked with a starting point by a preamble. The preamble frame contains a specific signal and signals the start of communication. There may be any number of preamble frames. When the frame number is displayed in each frame, the preamble is not essential and the reading can be started in the middle of communication. By including information in this preamble, the reading device can know the starting point of communication.

The position of the screen is indicated by a position marker. Position markers are used to read data from the screen.
Indicate the position, orientation and inclination of the screen.

The screen redrawing speed depends on the performance of the device. For this reason, a block or frame number for synchronization is provided to prevent the above influence. The data is transmitted to the receiver side by changing the pattern drawn by the block. A specific block (for example, one at the upper left of the screen) is always turned on and off alternately, and this is used as a signal of a synchronization marker for changing the screen, or a block showing a frame number is displayed. Send the code in the other blocks. It is also possible to synchronize automatically by detecting a change in the image on the reading device side.

The number of blocks divided in the above is required to have a degree of freedom because the response speed and resolution of the screen are different. The number of blocks (the number of bits per frame) used by the screen for signal transmission is notified to the receiver side at the preamble stage or transmitted by a model resolution identification signal. The drawing speed (the time until the frame is redrawn) is transmitted by using the synchronization marker or the frame number marker, or the reading device automatically synchronizes.

When the screen of an i-mode terminal of a mobile phone is taken as a reference, the number of divisions of the block is 300 blocks if the resolution of 120 × 160 dots is set as 1 block of 8 × 8 dots. become. If all blocks are used for code transmission, it is possible to transmit a 300-bit signal in one frame, and if a block for synchronization is provided, it will be 299 bits.

If the response speed of the screen is slow, including the rising and afterglow processing, and is 200 ms, 5 frames can be transmitted per second, so that about 1500 bits per second can be transmitted.

Next, depending on the performance of the screen, some screens have a response speed of 50 ms or less, and it seems that transmission can be performed at higher speed. Also, the number of blocks per frame could be increased if the accuracy of the reading device is high.

The high-speed communication in the above will be described.
In an ideal state of screen communication, the number of bits that can be transmitted in one frame is the number of screen pixels. 120 x 16
If the screen has a resolution of 0 dots, it will be 19.2 kilobits. Further, if drawing is performed at a response speed of about 50 ms, 20 frames can be transmitted per second.

In the case of a color screen, it is possible to transmit a signal for each color, and even in a monochrome terminal, it is possible to divide the transmission signal according to the difference in pixel brightness. In this case, the number of transmission bits per frame can be further increased. Assuming that the ideal theoretical value is 256 gradations (8 bits) for monochrome and 65536 colors (16 bits) for color, the number of bits that can be transmitted per frame is 12
For a screen of 0x160 dots, 120x1 each
60 × 8 = 153,600 bits = 154 kilobits,
120 × 160 × 16 = 307, 200 bits = 307
It will be a kilobit.

In the above communication, it is desirable to use an error correction code and interleaving for error compensation and reliability assurance.

By superimposing a visible image on the display screen or displaying it partially, it is possible to display a symbol or design that is different from the data read by the reading device and can be visually understood by a human.

[0033]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a flowchart of the reading process in the embodiment. As shown in FIG. 1, data 1 is displayed on the display screen 13.
0 is displayed by the encoding process 11 and the imaging process 12 as an arbitrary number of images, that is, a series of frames. The encoding process 11 performs preprocessing such as adding an error correction signal to the data 10 and applying interleaving. The imaging process 12 converts the data processed by the encoding process 11 into an image. The reading device 21 converts this screen into an electric signal by the image pickup device 14, and the screen position angle detection processing 15
Detects the position, orientation, and tilt of the display screen. Further, the parameter determination process 16 is performed by the screen resolution, display performance,
Detects redrawing of display frame, preamble frame, frame number, etc. The data reading 17 reads the screen, and the decoding process 18 performs the reverse process of the encoding process 11 to restore the data 19.

FIG. 2 shows an example in which an image is generated by the server 22 different from the terminal in the embodiment. Terminal 2
At 0, the image already sent from the server 22 is displayed.

FIG. 3 shows the structure of one frame in the embodiment. The frame 36 includes a position marker 30 for determining a position, a model identification signal 31, and a resolution identification signal 3
2, a frame number marker 33, a direction identification marker 34, and encoded data 35. The position, orientation, and inclination of the screen 14 are detected by the position marker 30 and the direction identification marker 34, and the screen performance and resolution are model identification signal 31 and resolution identification signal 3.
Detect with 2. Frame number is frame number marker 3
Identify with 3. These are used in the processing shown in FIG. Blinking of the synchronization marker may be used instead of the frame number marker 33. The model identification signal 31 and the resolution identification signal 32 may be communicated by independent preamble images without being included in the frame 36. Encoded data 35 is screen 1
Alternatively, halftones or colors may be used depending on the performance of the image pickup device 3 and the image pickup device 14.

If necessary, it is possible to overlay or include some visible image on a part or the whole of the communication screen. FIG. 4 shows a frame 40 with a visible image on which a visible image is superimposed. The visible image can be used for advertisements. As shown in FIG. 5, the base image 40a and “KEIO UN
Overlay the character 40b of "IV".

When error correction is included in the encoding process 11, the influence of this image on reading can be reduced. Alternatively, the coded data 35 may be displayed independently on a part of the screen without being overlapped with the frame 36.
May be included in.

[0038]

EFFECTS OF THE INVENTION It is possible to perform communication by displaying data of an arbitrary length on a screen and reading the data drawn on the screen with a reading device. This communication is used for authentication of tickets, admission tickets, vouchers for products, electronic keys for doors, coupons, etc., or data transmission for tickets, admission tickets, vouchers for products, electronic keys for doors, coupons, etc. It can be used for various purposes such as use, transmission of product identification information, transmission of line monitoring information in factories, transmission of location information of street terminals, transmission of information from TV programs to other devices. .

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a communication device in the present invention.

FIG. 2 is a block diagram in which the configuration of the display device is also changed.

FIG. 3 is a diagram showing a configuration of a frame to be displayed similarly.

FIG. 4 is an exemplary view of a frame in which visible images are also superimposed.

5 is an explanatory diagram of FIG.

[Explanation of symbols]

10 data to send 11 Encoding process 12 Processing to image encoded data 13 Display screen 14 Imager with screen as electrical signal 15 Screen position angle detection processing 16 screen parameter determination process 17 Data reading process from images 18 Process for decrypting data 19 transmitted data 20 terminals 21 Reader 22 servers 30 position markers 31 Model identification signal 32 resolution identification signal 33 Frame number marker 34 Direction identification marker 35 encoded data 36 code frames 40 Frames with superimposed visible images

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Kohei Awatani             2-14-10 Tana, Nerima-ku, Tokyo (72) Inventor Hiroshi Shigeno             6-3-35 Hiyoshi, Kohoku Ward, Yokohama City, Kanagawa Prefecture             502 F-term (reference) 5B035 AA00 BB03 BB14 BC00                 5B072 CC08 CC21 DD21 DD25 MM02                 5C062 AA05 AB17 AB38 AC42 AE07                       AF00 BA04

Claims (14)

[Claims] 1. Data is displayed on a screen as an image or a symbol, and a part or the whole of the screen is displayed once or redrawn a plurality of times, converted into data by a reading device, and this is read for data communication. A communication method using a screen characterized by: 2. The method according to claim 1, wherein by processing the received screen, regardless of the screen orientation.
A communication method using a screen, which is characterized by causing a reading device to recognize a correct screen. 3. The communication method using a screen according to claim 1, wherein the reading device is made to recognize a correct screen regardless of the inclination of the screen with respect to the reading device. 4. The communication method using a screen according to claim 1, wherein a data transmission start point is notified by a preamble display method. 5. The communication method using a screen according to claim 1, wherein the communication condition is recognized using a model resolution identification signal. 6. The communication method using a screen according to claim 1, wherein a synchronization marker is displayed on the screen, and the reading device is made to recognize the timing of rewriting the screen. 7. The communication method using a screen according to claim 1, wherein a frame number marker is displayed on the screen and a reading device is made to recognize the timing of rewriting the screen. 8. The communication method using a screen according to claim 1, wherein the reading device recognizes the timing of redrawing the screen from the state of the screen. 9. The communication method using a screen according to claim 1, wherein a visible image unrelated to the data can be displayed on the screen together with the data. 10. The communication method using a screen according to claim 1, wherein the screen communication method is based on a difference in monochrome gradation and the screen communication method is based on a color signal. 11. A communication method using a screen according to claim 1, wherein an error correction code for avoiding a data error due to a read error is used. 12. A communication method using a screen according to claim 1, wherein interleaving is used to diffuse local read defects. 13. Data to be transmitted is encoded, an image obtained by image-forming this is transmitted, the received image is corrected for screen position / angle, and then parameter determination processing is performed, and then information is provided. A communication system using a screen, which is characterized in that it is read, combined, and converted into transmission data. 14. The data obtained from the server is encoded, the image is subjected to image processing, the output thereof is input to a terminal of screen display, the image of the screen display is transmitted, and the received image is displayed at the screen position / A communication system using a screen, characterized in that the angle is corrected, and then parameter determination processing is performed, information is read, composite processing is performed, and transmission data is obtained.