WO2015115294A1

WO2015115294A1 - Transmission device, reception device, and information processing system

Info

Publication number: WO2015115294A1
Application number: PCT/JP2015/051642
Authority: WO
Inventors: 孝介八木
Original assignee: 三菱電機株式会社
Priority date: 2014-01-31
Filing date: 2015-01-22
Publication date: 2015-08-06
Also published as: JP5976240B2; JPWO2015115294A1

Abstract

A system for transferring data by having a reception device receive code images displayed by a transmission device, wherein at the transmission device, a series of data is divided into multiple data portions, and each of the data portions obtained by the division is converted into a code image and output. Identification images are added at timing other than code image switching timing. Strings of the generated code images are connected together and transmitted in the form of a moving picture. At the reception device, images obtained by image capture are accumulated, the accumulated images are read in sequence, and the code images that appear simultaneously with the identification images are decoded. Transfer failures due to inappropriate image switching timing and image capture timing can be reduced.

Description

Transmission device, reception device, and information processing system

The present invention relates to a transmission device, a reception device, and an information processing system, and more particularly to transmission of information performed by imaging a code image displayed on the transmission device with the reception device.

As mobile devices become more sophisticated, information has been exchanged through mobile phone lines and wireless networks. On the other hand, there are situations where it should be avoided to use radio waves such as in hospitals, airplanes, or crowded places. In some cases, the network may not be used due to device incompatibility or complicated network settings, and there may be cases where the network is not used due to resistance to the network or concerns. Bar code or two-dimensional code reading is often used as a simple information acquisition means that does not use a network.
The QR code (registered trademark) that is often used at present can transfer approximately 2000 characters with one code, but the amount of data that can be transferred is not sufficient for transferring photographs and graphic information.

Therefore, as shown in Patent Documents 1 to 3, a method of increasing the amount of transfer data by color multiplexing or time multiplexing is considered.

In the visible light wireless communication system disclosed in Patent Document 1, a data transmission device converts data to be transmitted into a code, generates a frame including cells in which the code is two-dimensionally arranged, and uses the frame as a display screen. it's shown. The data receiving apparatus captures the displayed frame, and returns the code of the two-dimensional array corresponding to the cells constituting the captured frame to the data before conversion. That is, the data receiving apparatus captures the displayed frame and returns the code of the two-dimensional array to the data before conversion. In such processing, a method of inserting a correction cell into a cell constituting a frame, or a method of inserting a correction frame made up of correction cells, reduces the influence of the characteristics of background light or variations thereof, or an optical device. This makes it possible to accurately decode data by reducing the influence of the characteristics or variations thereof.

In the communication method disclosed in Patent Document 2, binary data that the data transmission side wants to transfer is converted into a concentric color component code whose color changes with time, and is displayed on a monitor as a moving image. The data receiving side continuously photographs concentric color component codes with the camera of the camera-equipped mobile terminal device, and extracts binary data from the color information of each track.

In the information processing apparatus disclosed in Patent Document 3, the first information processing apparatus generates and sequentially displays a plurality of first graphic images (for example, two-dimensional codes). In addition, the first information processing apparatus detects second graphic images that are sequentially displayed in other information processing apparatuses, and acquires input data represented by the second graphic image. The detection area for detecting the second graphic image is formed in a part of the display area for displaying the first graphic image.

Japanese Patent No. 4749856 (paragraphs 0033 to 0036, FIGS. 1 and 2) JP 2007-13786 (paragraphs 0032 to 0044, FIG. 8) Japanese Patent No. 4400355 (paragraphs 0088 to 0099, FIG. 7)

The display screen refresh rate and camera scan rate may vary depending on the device. Even if the refresh rate is the same, the phases are not always the same during actual use. For this reason, the screen in the middle of the display rewriting may be shot and the code may not be read accurately. Therefore, in the methods shown in Patent Documents 1 to 3, transfer may fail, and a more reliable transfer method is desired.

Also, if some data is damaged due to camera shake or lack of shooting period, all data will be invalid. Therefore, a mechanism having higher robustness, that is, a mechanism capable of partially extracting or rereading data when data is damaged is required.

In the methods of Patent Documents 1 to 3, the user needs to hold the camera for a certain period of time. However, when the user does not know the shooting period or the shooting timing, the timing of holding over is not appropriate, and the data may not be read accurately.

Therefore, an object of the present invention is to efficiently transfer information from a transmission device to a reception device, and to enable a user to easily read data.

A transmission device according to one aspect of the present invention includes:
A transmission data storage unit for storing a series of data;
The series of data stored in the transmission data storage unit is divided into a plurality of data parts, each of the data parts obtained by the division is converted into a code image, and each data part generated by the conversion A code generator that sequentially outputs code images corresponding to
An identification image generator for generating and outputting an identification image;
A combining unit that combines the code image generated by the code generation unit with the identification image output from the identification image generation unit to generate a combined image;
An image connecting unit that connects the composite images generated by the combining unit to create a moving image corresponding to the series of data;
A video playback unit that plays back and outputs the video created by the image connection unit;
A display unit that displays the video played by the video playback unit on a screen;
In the moving image output from the moving image reproducing unit, the code images are sequentially switched, and the identification image appears at a timing other than the switching timing.

A transmission device according to another aspect of the present invention includes:
A transmission data storage unit for storing a series of data;
The series of data stored in the transmission data storage unit is divided into a plurality of data parts, each of the data parts obtained by the division is converted into a code image, and each data part generated by the conversion A code generator that sequentially outputs code images corresponding to
A first image connecting unit that connects the code images generated by the code generating unit to create a moving image;
A first moving image reproduction unit that reproduces and outputs the moving image created by the first image connection unit;
An identification image generator for generating and outputting an identification image;
A second image connecting unit that connects the identification images generated by the identification image generating unit to create a moving image;
A second moving image reproduction unit that reproduces and outputs the moving image created by the second image connecting unit;
A synthesis unit that synthesizes the video created by the first video playback unit and the video created by the second video playback unit to generate and output a composite video;
A display unit that displays the synthesized video generated by the synthesis unit on a screen;
In the synthesized video output from the synthesizing unit, the code images are sequentially switched, and the identification image appears at a timing other than the timing of the switching.

A transmission device according to yet another aspect of the present invention provides:
A transmission data storage unit for storing a series of data;
The series of data stored in the transmission data storage unit is divided into a plurality of data parts, each of the data parts obtained by the division is converted into a code image, and each data part generated by the conversion A code generator that sequentially outputs code images corresponding to
An image connecting unit that connects the code images generated by the code generating unit to create a moving image;
A video playback unit that plays back and outputs the video created by the image connection unit;
A display unit for displaying the video played by the video playback unit on a screen;
The moving image displayed on the display unit includes a light emitting unit that emits light at a timing other than a timing at which the code image is switched.

A receiving apparatus according to one aspect of the present invention includes:
An imaging unit that sequentially switches the code image, captures a screen displaying a video in which the identification image appears at a timing other than the switching timing, and sequentially acquires the image of the screen;
A video buffer for storing images sequentially acquired by the imaging unit;
A reading unit for sequentially reading out images stored in the video buffer;
An identification image detection unit for sequentially detecting the identification images in images sequentially read by the reading unit;
A code decoding unit that decodes the code image that appears at the same timing as the identification image detected by the identification image detection unit, and sequentially acquires corresponding data portions;
And a reception data storage unit that sequentially writes data portions sequentially acquired by the code decoding unit and connects the sequentially written data portions to form a series of data.

A receiving apparatus according to another aspect of the present invention includes:
A screen that displays code images that are switched in sequence,
The light from the light emitting unit that is arranged adjacent to the screen and emits light at a timing other than the switching timing is captured, and the image displayed on the screen and the light image from the light emitting unit are sequentially acquired. An imaging unit;
A video buffer for accumulating images sequentially acquired by the imaging unit;
A reading unit for sequentially reading out images stored in the video buffer;
A detection unit for detecting an image portion by light from the light emitting unit in images sequentially read by the reading unit;
A code decoding unit that decodes the code image that appears at the same timing as the image portion by the light sequentially detected by the detection unit, and sequentially obtains the corresponding data portion;
And a reception data storage unit that sequentially writes data portions sequentially acquired by the code decoding unit and connects the sequentially written data portions to form a series of data.

According to the present invention, it is possible to improve the transfer efficiency when transferring information from the transmission device to the reception device with simple handling of the reception device.

1 is an overview diagram schematically showing a configuration of an information processing system according to a first embodiment of the present invention. FIG. 3 is a block diagram schematically showing a configuration of a transmission apparatus in the first embodiment. 6 is a schematic diagram for explaining synthesis of a code image and a mark indicating timing according to Embodiment 1. FIG. (A) to (C) are timings showing the display timing of the timing mark assigned to the green channel, the first code assigned to the red channel, and the second code assigned to the blue channel in the first embodiment. It is a chart. FIG. 3 is a block diagram schematically showing a configuration of a receiving apparatus in the first embodiment. It is a figure which shows an example of the dimension of each part of the image of a timing mark. (A) to (C) are timing charts showing an example of timing mark and code image display timing by the display unit of the transmission device and an imaging frame period of the imaging unit of the reception device in the first embodiment. . 6 is a flowchart illustrating an operation of displaying a video based on transfer data in the transmission device according to the first embodiment. It is a flowchart which shows the detail of step S12 of FIG. 4 is a flowchart illustrating an operation of acquiring and storing transfer data from a captured image in the receiving device according to the first embodiment. (A) to (M) are schematic views showing modifications of the timing mark in the first embodiment. (A) And (B) is a figure which shows the example of a display image and a captured image in case the frame period of the display in a display part and the frame period of the imaging in an imaging part have shifted | deviated. (A)-(D) are figures which show the change of the display image within a display frame period. (A)-(C) are figures which show the change of the display image within a display frame period. (A)-(C) are figures which show the change of the display image within a display frame period. It is a block diagram which shows roughly the structure of the transmitter in Embodiment 2 of this invention. It is a block diagram which shows roughly the structure of the transmitter in Embodiment 3 of this invention. FIG. 10 is a block diagram schematically showing a configuration of a receiving apparatus in a third embodiment. It is a block diagram which shows roughly the structure of the transmitter in Embodiment 4 of this invention. FIG. 10 is a block diagram schematically showing a configuration of a receiving apparatus in a fourth embodiment. It is a block diagram which shows roughly the structure of the transmitter in Embodiment 5 of this invention. FIG. 10 is a block diagram schematically showing a configuration of a receiving apparatus in a fifth embodiment. It is the schematic which shows the layout of a timing mark and a code image used in Embodiment 5 and 6 of this invention. (A) to (C) are schematic diagrams illustrating temporal changes in video displayed on the display unit of the transmission device in the fifth and sixth embodiments. It is a block diagram which shows roughly the structure of the transmitter in Embodiment 6 of this invention. FIG. 20 is a block diagram schematically showing a configuration of a receiving apparatus in a sixth embodiment. It is a block diagram which shows roughly the structure of the transmitter in Embodiment 7 of this invention. FIG. 20 is a block diagram schematically showing a configuration of a receiving apparatus in a seventh embodiment. It is a figure which shows an example of the sequence of QR Code which attached | subjected the different kind of timing mark in Embodiment 8 of this invention. FIG. 20 is a block diagram schematically showing a configuration of a transmission apparatus in an eighth embodiment. FIG. 20 is a block diagram schematically showing a configuration of a receiving apparatus in an eighth embodiment.

FIG. 1 shows an example of an information processing system 1 according to the present invention.
An information processing system 1 illustrated in FIG. 1 includes a transmission device 100 and a reception device 200.
The information processing system 1 transfers the data file D ₀ from the transmission device 100 to the reception device 200. The transmission device 100 displays an image on the display screen 131. Then, the receiving device 200 captures the image. Data file D ₀ to be transferred, the document files, image files, video files, may be either a voice file or executable file. The audio file or the execution file is converted into image data such as a QR code, for example. That is, any data is converted into image data such as a QR code.

In the present invention, the transmission device 100, the data file D ₀ to be transferred is divided into a plurality of data portions are converted into the data portion of one or more code image D _3, are sequentially displayed on the display screen 131 The
Receiving apparatus 200 sequentially captures the code image _{D 3} to be displayed on the display screen 131 of the transmitting apparatus 100. Then, the receiving apparatus 200 restores the data portion by decoding a code image D _3. Receiving apparatus 200, by connecting the recovered data to restore the original data file D _0.

Code image D ₃ above, for example, two-dimensional bar code such as a QR code, or what is called a one-dimensional bar code.
The display frame rate of the transmission apparatus 100 and the imaging frame rate of the reception apparatus 200 may be the same or different from each other.

Embodiment 1 FIG.
<Configuration of Transmitting Device 100>
FIG. 2 is a block diagram schematically showing a configuration of transmitting apparatus 100 in the first embodiment.
The transmission apparatus 100 illustrated in FIG. 2 includes a mark generation unit 106, a data processing unit 110, a synthesis unit 122, and a display unit 130. Further, the transmission device 100 can include a transmission data storage unit 102, a reference clock generation unit 104, a video buffer 126, and a moving image reproduction unit 128.

Transmission data storage unit 102 stores a plurality of data files D _0.
The “data file” is a piece of data, for example. Incidentally, when the capacity of the data file D ₀ is large, for example, it is sent to the receiving apparatus 200 from the transmitting apparatus 100 the data of the data file D ₀ by using a plurality of QR code or the like.
The transmission data storage unit 102 gives the stored transmission data D ₁ to the data processing unit 110. The transmission data D ₁ is, for example, a data file D ₀ stored in the transmission data storage unit 102. Further, the transmission data D ₁ are, for example, the greater the capacity of the data file D ₀ stored in the transmission data storage unit 102 may divide the data file D ₀ in volume suitable for transmission to the data processing unit 110 It has been done.
The reference clock generation unit 104 generates a clock signal CL. Then, the reference clock generation unit 104 provides the clock signal CL to the mark generation unit 106 and the data processing unit 110.

Mark generation unit 106, in synchronization with the generated clock signal CL at the reference clock generator 104, the mark showing the timing (timing mark) to produce a D _2. Further, the mark generating unit 106 includes an image storage unit 107 for storing the timing mark _{D 2.}

Timing mark _{D 2} is an identification image. In other words, the timing mark D ₂ in the image if it is displayed, for example, is recognized as an image, such as a QR code being displayed at the same time is intended to include the appropriate information. Conversely, if the timing mark D ₂ is not visible in the image, for example, it is recognized as an image, such as a QR code being displayed at the same time are those which do not contain adequate information. “Not including appropriate information” means that data cannot be correctly decoded.

Image storage unit 107 stores an image plane including timing mark D _2.
The image storage unit 107 for frame not including the timing mark D _2, and stores the image plane which does not include a timing mark D _2.

Timing mark D ₂ is synthesized with the code image D ₃ as described later. Timing mark D ₂ is used as an identification image for the code image D ₃ combined by the combining unit 122 to identify whether or not suitable for decoding. For this reason, the mark generation unit 106 is also referred to as an “identification image generation unit”. Here, “compositing” refers to handling different data as one image data. For example, it is to represent a code image D ₃ and the timing mark D ₂ as an image in one frame.

Timing mark D ₂ is, for example, an image composed of pixels having the pixel and the minimum value takes the maximum value of the gradation values (pixel values) of possible values. When the gradation value is expressed by 8 bits, the maximum value is “255” and the minimum value is “0”.
Figure 3 is a schematic view for explaining a combination of the code image D ₃ and the timing mark D _2. Code image D ₃ is, for example, an image representing the QR code or the like. That is, a coded data file D _0. Code image _{D 3} is produced from the transmission data _{D 1.}
For example, as shown in FIG. 3, the timing mark D < _b > ₂ is configured by a rectangular area (shown by cross-hatching) composed of pixels whose gradation values have the above maximum values. In other words, the timing mark D ₂ shown in FIG. 3 is a rectangular image of the maximum tone value. In that case, the gradation value of the pixel outside the above region becomes the minimum value. In Figure 3, a timing mark D ₂ it is composed of only the data of green (G).
Also, the code image D ₃ is composed of data of the data and blue red (R) (B). In the example shown in FIG. 3, the code image D ₃ of the green timing marks D ₂ and red (G) (R) and blue (B) is synthesized as an image of one frame.

As shown in FIG. 3, when the code image D ₃ is indicated by color data, the red (R) data (code image D ₃ ) and the blue (B) data (code image D ₃ ) are different from each other. can do. That is, two pieces of data can be sent with one image (display frame). Thus, it is possible to speed up the transfer of data file D _0.

The data processing unit 110 selects one of the plurality of data files D ₀ stored in the transmission data storage unit 102. The data processing unit 110 divides the data file D ₀ that was selected plurality of data portions. That is, the data processing unit 110 divides the data easily accepts data file D ₀ selected volume. Then, the data processing unit 110 receives the divided data (transmission data D ₁ ).

The data processing unit 110 converts each divided data portion into two graphic codes.
In FIG. 3, each divided data portion is converted into a graphic code displayed in red (R) and a graphic code displayed in blue (B). The data processing unit 110 generates a code image _{D 3} from the graphic code. Code image D ₃ is obtained by converting so that the image when drawing a graph code.

Code image D ₃ is, for example, an image composed of pixels having a pixel and a minimum value taking the maximum value of the values that can take the tone values (pixel values).

The data processing unit 110 includes a code generation unit 116. The code generation unit 116 may include a first code generation unit 116A and a second code generation unit 116B.
The data processing unit 110 can include a reading unit 112 and a data dividing unit 114.

The reading unit 112 divides the data file D ₀ stored in the transmission data storage unit 102 into a plurality of data portions (herein referred to as “shots”). The divided data portion (shot) is obtained by dividing the data file D ₀ into a capacity suitable for transmission to the data processing unit 110 as described above. That is, one data portion divided into a capacity suitable for transmission is called a “shot”. Transmission data _{D 1} is one shot. Hereinafter, in order to simplify the description, the data for the image displayed in the one-shot period _Td shown in FIG. 4A will be described as “one shot”. That is, when two QR codes or the like are displayed separately for color images as described in FIG. 3, one shot is displayed as an image of two QR codes or the like. Further, when one QR code or the like is displayed without being divided into color images as described in FIG. 6, one shot is displayed as an image such as one QR code.
The reading unit 112 sequentially sends a plurality of shots (shot data D ₄ ) generated by dividing the data file D ₀ to the data dividing unit 114. Further, the reading unit 112 sends a synchronization signal SY described later to the mark generation unit 106.
The data dividing unit 114 further divides the data D _{4 of} each shot read by the reading unit 112 into two data parts (herein referred to as “data unit D ₅ ”). Here, “two data portions” are different data portions displayed for each color channel. In the first embodiment, the first data unit D ₅₁ displayed in the red channel and the second data unit D ₅₂ displayed in the blue channel. The data dividing unit 114 gives the first data portion (first data unit D ₅₁ ) to the first code generating unit 116A. The data dividing unit 114 gives the second data portion (second data unit D ₅₂ ) to the second code generating unit 116B.

The first code generating unit 116A converts the first data unit _{D 51} supplied from the data dividing unit 114, the corresponding code image (first code _{image) D 31.} That is, the first data unit D ₅₁ is converted into a graphic code, and this graphic code is further converted into a corresponding code image (first code image) D ₃₁ . The first code generating unit 116A, the code image (first code _image) to the _{D 31.} Incidentally, actually it outputs an image plane including a first code image D _31. However, in FIG. 2, simplified to describe the output of the first code generating unit 116A as the first code image _{D 31.}

The second code generating unit 116B converts the second data unit _{D 52} supplied from the data dividing unit 114, the corresponding code image (second code _{image) D 32.} That is, the second data unit D ₅₂ is converted into a graphic code, and the graphic code is further converted into a code image (second code image) D ₃₂ corresponding thereto. The second code generating unit 116B outputs the code image (first code _{image) D 32.} Incidentally, actually it outputs an image plane including a first code image D _31. However, in FIG. 2, simplified to describe the output of the first code generating unit 116A as the first code image _{D 31.}

Data unit D ₅ is the data of the chunk numbers. On the other hand, the code image D ₃ is in the form of data that can be displayed as an image. Code image D _3, for example, is represented by bitmap data. Conversion to code image data units D ₅ referred to as "drawing." The rules for conversion of each data unit D ₅₁ , D ₅₂ into graphic codes are determined in advance. The rules for converting the data units D ₅₁ and D ₅₂ into graphic codes are shared by the transmission device 100 and the reception device 200.

Examples of the first code image _{D 31} and the second code image _{D 32} is shown in FIG. For example, images of red (R) is the first code image _{D 31.} For example, an image of blue (B) is the second code image _{D 32.}
In the example of FIG. 3, the first code image D ₃₁ and the second code image D ₃₂ are formed in the same region as the timing mark D ₂ in the frame. Accordingly, the first code image D ₃₁ , the second code image D ₃₂ and the timing mark D ₂ are formed so as to overlap each other.

A plurality of shot data D ₄ read by the reading unit 112 are sequentially input to the data dividing unit 114. First data unit D ₅₁ supplied from the data dividing unit 114 in the first code generating unit 116A is switched in association with the shot data D ₄ input to the data dividing unit 114 is switched. Therefore, the code image D ₃₁ outputted from the first code generating unit 116A is switched each time the switches are available for the data D ₄ that is read by the reading unit 112. Similarly, the code image D ₃₂ outputted from the second code generating unit 116B is switched each time the switches are available for the data D ₄ that is read by the reading unit 112.

Both the output of the first code image D ₃₁ by the first code generation unit 116A and the output of the second code image D ₃₂ by the second code generation unit 116B are performed in synchronization with the clock signal CL from the reference clock generation unit 104. Is called. The output of the timing marks D ₂ by the mark generation unit 106 is performed in synchronization with the clock signal CL. The generation of the timing mark _{D 2} in the mark generating unit 106, the code generating unit 116A, to synchronize with the output of the code image _D _{31, D 32} in 116B, the read unit 112 each time the read data _{D 4} of a shot, A synchronization signal SY indicating this is output. Mark generation unit 106 receives the (signal indicating to read the data D ₄ of a shot) above synchronizing signal SY from the reading unit 112 determines the timing of the output of the timing mark D _2.

The combining unit 122 assigns the timing mark D ₂ , the first code image D _31, and the second code image D ₃₂ to independent color channels. Then, the synthesizing unit 122 synthesizes the timing mark D ₂ , the first code image D _31, and the second code image D ₃₂ to generate a synthesized image (color image) D ₆₁ . The timing mark D2 is output from the mark generation unit 106. The first code image _{D 31} is outputted from the first code generating unit 116A. Second code image _{D 32} is outputted from the second code generating unit 116B.
Synthesis by combining unit 122 includes an image plane comprising a timing mark D _2, and an image plane including a first code image D _31, is performed by superimposing an image plane including a second code image D _32. “Plain” is like a transparent plate on which an image is pasted. Generally, an image is composed of a plurality of screens (planes) called planes. There are three planes that express red, green, and blue in the plane, and colors are expressed by combining them. An image plane expressed by a plurality of planes is called a frame.

As the color channel, for example, three primary color channels of red (R), blue (B), and green (G) light are employed. A “color channel” is information obtained by separating colors by components. Specifically, there are a red channel, a green channel, a blue channel, and the like. Then, the composition unit 122 assigns a timing mark _{D 2} of FIG. 3 in green (G) channel, a first code image _{D 31} assigned to the red (R) channel, the second code image _{D 32} blue (B ) Assign to the channel and compose. For example, the synthesizing unit 122 synthesizes the tone values of the timing mark D ₂ , the first code image D _31, and the second code image D ₃₂ as the tone values of the respective color channels.

In the composite image D _61, the maximum value of the timing mark D ₂ has a maximum value of the gradation values of the green channel (green component value). In FIG. 3, the timing mark D < _b > ₂ is shown in a portion indicated by cross hatching. The maximum value of the first code image D ₃₁ has a maximum value of the gradation values of the red channel (red component values). In Figure 3, the maximum value of the first code image D ₃₁ is a portion which is shown in black. The maximum value of the second code image D ₃₂ has a maximum value of the gradation values of the blue channel (blue component value). In Figure 3, the maximum value of the second code image D ₃₂ is a portion which is shown in black.

The synthesizer 122 synthesizes the timing mark D ₂ , the first code image D _31, and the second code image D ₃₂ to generate an image for one frame displayed on the display unit 130.

The video buffer 126 stores the composite image D ₆₁ generated by the combining unit 122. The video buffer 126 couples the combined image _{D 61} that has accumulated. In other words, the video buffer 126 connects a plurality of synthesized one-frame images so that the display unit 130 can display them as “moving images”. Connecting a plurality of images for one frame is called “connection”.
Video buffer 126, the synthesized images are sequentially connected to the generating a combined image D _62.
An “image” is a still image displayed in one frame. A “video” is an image in which several frames of images are connected and displayed continuously. That is, when the “image” of each frame is different, the “video” is a moving image.
The image for one frame combined by the combining unit 122 is the same image. Therefore, the viewer even display unit 130 displays the combined image D ₆₂ recognizes the one shown the same image.

The video buffer 126 connects the accumulated composite video _D62 to each other. As described above, the composite video _D62 is image data obtained by combining several frames of the same image. The composite video D ₆₂ combined with the video buffer 126 is of different images. For this reason, as will be described later, when an image displayed on the display unit 130 is captured by the receiving device 200, the receiving device 200 may capture two different images. Thus, the composite video D ₆₃ obtained as a result of the connection of the composite video D ₆₂ is a video in which the content of the image is switched. As will be described later, for example, in FIGS. 4A to 4C, the composite video D ₆₃ obtained as a result of the concatenation of the composite video D ₆₂ is a video whose contents are switched at a cycle of 4 frames.

Accumulation of the image in the video buffer 126 is continued until the conversion of all data (data portion) of the data file D ₀ to be transferred into the code image D ₃ is completed. That is, when the capacity of the data file D ₀ is large, for example, to create a combined image D ₆₃ formed by coupling a plurality of QR codes. In the example illustrated in FIG. 3, a composite video D ₆₃ obtained by connecting two types of composite video D ₆₂ can transmit four types of QR codes. State accumulation of image in the image buffer 126 at the time ended, one data file timing mark D ₂ that is added code image D ₃ and to a series of frames constituting an image corresponding to the D ₀ is stored It becomes.
As described above, the video buffer 126 functions as a connecting unit that connects the composite image D ₆₁ to generate the composite video D ₆₂ . In addition, the video buffer 126 functions as a connecting unit that generates a composite video D ₆₃ (moving image) by connecting the composite video D ₆₂ . Incidentally, the step of connecting the composite image D ₆₁ generates a combined image D _62, may be synthesized unit 122 performs.

After the conversion of all data of one data file D ₀ to the code image D ₃ is completed, the moving image reproducing unit 128 stores the data (the image including the code image D ₃ and the timing mark D ₂₎ stored in the video buffer 126. A sequence of frames) is sequentially read out for each frame and reproduced as a moving image. Here, the “sequence of image frames” refers to the composite video D ₆₃ .

The display unit 130 displays a color image according to the output of the moving image playback unit 128.
In Figure 3, in the image displayed on the display unit 130 at the timing mark D ₂ green, first code image D ₃₁ is red, the second code image D ₃₂ are respectively displayed in blue.

Hereinafter, for example, it is assumed that the display unit 130 is configured by a liquid crystal display device that performs display by a dot sequential scanning method or a line sequential scanning method and a progressive display method.

<Display processing of transmitting apparatus 100>
As described above, the timing marks D ₂ is generated in synchronization with the clock signal CL. The output of the first code image D _{31 and} the output of the second code image D ₃₂ are both performed in synchronization with the clock signal CL from the reference clock generation unit 104. Therefore, the first code image _{D 31} and the second code image _{D 32} is switched simultaneously. It carried out as appears timing mark D ₂ at the timing just before and except immediately after the timing of the changeover. For example, so that appears timing mark D ₂ in switched immediately before and after the non-Framed frame. That is, immediately before and frame after the switching of the frame timing mark D ₂ is not synthesized.

4A to 4C are timing charts showing display timings of the timing mark D ₂ , the first code image D ₃₁ , and the second code image D ₃₂ . FIG. 4 (A) represents a timing mark _{D 2.} FIG. 4 (B) represents the first code image _{D 31.} FIG. 4 (C) represents a second code image _{D 32.} Figure 4 (A) ~ FIG 4 (C) shows in an example of the combined image _{D 63,} the switching of the code image _{D 3,} and the period in which the timing mark _{D 2} appears. In FIG. 4A to FIG. 4C, the vertical axis indicates whether or not each signal is output. A signal is output at “H”, and no signal is output at “L”. In FIGS. 4A to 4C, the horizontal axis represents time. 4A to 4C, a dotted line in the vertical direction indicates a break of a frame period of display by the display unit 130. The symbol T _t represents one frame period (one frame period) at the display frame rate.

In the example shown in FIGS. 4A to 4C, a period (one-shot period) T _d in which the composite video D ₆₂ generated from one-shot data D ₄ is displayed is four frame periods T. _t . The contents of the code image D ₃ to be displayed for each 4-frame period is switched. Of the one-shot period _{T d,} the first frame period _{T t1} of, does not display timing mark _{D 2.} Code images D ₃₁ and D ₃₂ are displayed in a period T _t1 of the first one frame in one shot period T _d . That is, of the one-shot period _{T d,} the period _{T t1} of the first frame, without displaying the timing mark _{D 2,} of the one-shot period _{T d,} the period _{T t1} of the first frame, the code Only the images D ₃₁ and D ₃₂ are displayed. The period _{T e} of the next two frames, the timing mark _{D 2} and the code image _D _{31, D 32} is displayed together. The period _{T t2} of the last 1 frame is not displayed timing mark _{D 2,} only the code image _D _{31, D 32} is displayed. Thus, the timing marks D ₂ is the switching of the immediately following frame of the code image D ₃ (the first frame period of each shot), it does not appear in the frame immediately before the changeover (last frame of each shot) . Timing mark D ₂ appears at their other frames (second and third frame periods of each shot).

In switched immediately before and immediately after the frame of the code image D ₃ (the fourth frame and the first frame), the contents of the code image D ₃ is sometimes disturbed part. Meanwhile, in the second frame and the third frame after a certain time from the switched contents of the code image D ₃ it is estimated to be stable. Therefore, in the invention according to the first embodiment, using the code image D ₃ of the second frame and the third frame to the decoding in the receiving device 200. Then, in order to distinguish the code image D ₃ of the second frame and the third frame from the code image D ₃ of the first frame and the fourth frame, it is denoted by the timing mark D ₂ in the transmitting apparatus 100. Then, the receiving apparatus 200, and whether the determination is a code image D ₃ of the first frame and the fourth frame or a code image D ₃ of the second frame and third frame based on the timing mark D ₂ .

As described above, the second frame and the third frame are used for decoding by the receiving apparatus 200. Therefore, these frames are called “body frames”. The front and rear frames (first frame and second frame) of the body frame are referred to as front and rear “guard frames”, respectively. The period in which the body frame appears is called the body period. A period in which the guard frame appears is called a guard period. In FIG. 4 (A), the body period is a period _{T e} of the two frames. The guard period is the first one frame period T _t1 and the last one frame period T _t2 .

<Configuration of receiving apparatus 200>
FIG. 5 is a block diagram schematically showing a configuration of receiving apparatus 200 in the first embodiment. The receiving apparatus 200 illustrated in FIG. 5 includes an imaging unit 202, a mark recognition unit 210, an image cutout unit 220, and a code decoding unit 230. In addition, the receiving device 200 can include a video buffer 204, a reading unit 206, a separating unit 208, a combining unit 242, and a received data storage unit 244.

The imaging unit 202 images the display screen 131 of the display unit 130 of the transmission device 100. Then, the imaging unit 202 generates video data D ₇ representing red, green, and blue component values of the video displayed on the display screen 131. That is, the imaging unit 202 generates red component video data, green component video data, and blue component video data of the video (composite video D ₆₃ ) displayed on the display screen 131. The video data D ₇ generated by the imaging unit 202 is a series of frame data corresponding to the composite video D ₆₃ . The video data D ₇ of each frame generated by the imaging unit 202 includes data representing a red component value, a green component value, and a blue component value.
Data representing the red component value, the green component value, and the blue component value is generated by, for example, a photoelectric conversion element including a red color filter, a green color filter, and a blue color filter, respectively.

Note that an image displayed on the display unit 130 of the transmission device 100 is formed on the imaging surface of the imaging unit 202. For example, the distance to the display screen 131 of the display unit 130 is known and the display screen 131 of the display unit 130 is focused. Alternatively, it is assumed that the imaging unit 202 has an autofocus function.

The video buffer 204 stores video data D ₇ composed of a series of frames output from the imaging unit 202. The accumulation of the video data D ₇ is performed by imaging all the frames (all frames of the composite video D ₆₃ ) constituting the video corresponding to one data file D ₀ displayed on the display unit 130 of the transmission device 100. Continue until the end. At the time of storing the image data D ₇ is finished, the state where data of a series of frames constituting the data file D ₀ (all frames of the composite video D ₆₃₎ is stored.

The reading unit 206 sequentially reads out and outputs a series of frames of data D ₈ constituting a video corresponding to one data file D ₀ stored in the video buffer 204. That is, the reading unit 206 outputs one data of the data file _{D 0} as data _{D 8} of the frame is read from one frame by the video buffer 204.

The separation unit 208 converts the data D _{8 of} each frame read by the reading unit 206 into green component value data D _9G (green channel data) and red component value data D _9R (red channel data). ) And blue component value data D _9B (blue channel data). The separation unit 208 gives the green component value data D _9G to the mark recognition unit 210. The separation unit 208 supplies the image component 220 with red component value data D _9R and blue component value data D _9B .

The mark recognizing unit 210 detects the timing mark D ₂ from the green component image (green component data D _9G ) given from the separating unit 208. Mark recognition unit 210 functions as a detection unit for detecting a timing mark D _2.
Mark recognition unit 210 recognizes that the timing of the timing mark D ₂ is detected, a timing that can read the code image D ₃ correctly. That is, the mark recognition unit 210 recognizes detects the timing mark _{D 2,} and read the code image _{D 3} correctly.

The “timing” here may be, for example, a timing for each frame.
Moreover, the timing for each line of the display screen may be used. Here, “line” means a column of pixels extending in the main scanning line direction in the dot sequential scanning method, and means a line in the line sequential scanning method.

Mark recognition unit 210 at the timing of the timing mark D ₂ is detected, it instructs the cutout image in the image clipping unit 220. Mark recognition unit 210 provides the image clipping signal _{D 10} of the image clipping unit 220.

Image clipping unit 220, in response to an instruction from the mark recognition unit 210 (cut signal D ₁₀ of the _image), cut out the code image D ₃ from a video of a predetermined color component. For example, in the first embodiment, the code image D ₃ is represented in the red component data D _9R and the blue component value data D _9B . The red component data D _9R and the blue component value data D _9B representing the code image D ₃ are images of predetermined color components. That is, the image clipping unit 220, at the timing when the mark recognition unit 210 detects the timing mark D _2, cuts out the code image D ₃ from a video color components determined in advance. The image cutout unit 220 includes a first cutout unit 222A and a second cutout unit 222B.
The first cutout portion 222A receives the data _{D 9R} red components from the mark recognition unit 210. The first cutout unit 222A cuts out the first code image from the red component data (video) _D9R given from the separation unit 208 at the timing instructed by the mark recognition unit 210.
The second cutout portion 222B receives the data _{D 9B} of the blue component from the mark recognition unit 210. The second cutout unit 222B cuts out the second code image from the blue component data (video) _D9B given from the separation unit 208 at the timing instructed by the mark recognition unit 210.

Code decoder 230 decodes the code image D ₃ cut out by the image clipping unit 220 to restore the original data. That is, the code decoder 230, from the code image D _3, and generates a graphic code corresponding thereto. The code decoding unit 230 further converts the graphic code into data corresponding thereto. As a result, the code decoding unit 230 acquires data (data file D ₀ ) represented by the captured code image D ₃ . The code decoding unit 230 includes a first decoding unit 232A and a second decoding unit 232B.

The first decoding unit 232A receives the first code image _{D 11R} from the first cutout portion 222A. The first decryption unit 232A decrypts the first code image _{D 11R} cut out by the first cutout portion 222A, restores the decoded data _{D 12R} of the corresponding color units. That is, the first decoding unit 232A generates a first graphic code corresponding to the first code image _D11R . Here, the decoded data _{D12R in} units of color is data for one frame. Then, the first decoding unit 232A converts the generated first graphic code into decoded data _D12R in color units corresponding thereto. This restoration, the first data unit _{D 51} entered if there is no error, the first code generating unit 116A to the process from the generation of the code image _{D 31} in the first code generating unit 116A to decoding in the first decoding unit 232A Will produce the same.

Second decoding unit 232B receives the second code image _{D 11B} from the second cutout portion 222B. Second decoding unit 232B decodes the second code image _{D 11B} cut out in the second cutout portion 222B, restores the decoded data _{D 12B} of the corresponding color units. That is, the second decoding unit 232B generates a second graphic code corresponding to the second code image _{D 11B.} Here, the decoded data _{D12B in} units of color is data for one frame. Then, the second decoding unit 232B converts the generated second graphic code into decoded data _D12B in color units corresponding thereto. If there is no error in the process from the generation of the code image D ₃₂ in the second code generation unit 116B to the decoding in the second decoding unit 232B by this restoration, the second data unit D ₅₂ input to the second code generation unit 116B. Will produce the same.

For such conversion, the first decoding unit 232A and the second decoding unit 232B are configured to generate the code images D ₃₁ and D ₃₂ in the first code generation unit 116A and the second code generation unit 116B of the transmission device 100. The conversion rules used in the above are applied to the decoding of the decoded data D _12R and D _12B .

The synthesizing unit 242 combines the decoded data D _12R and D _12B in units of colors output from the first decoding unit 232A and the second decoding unit 232B. The synthesizing unit 242 synthesizes the decoded data D _12R and D _12B in units of colors and restores the transfer data D ₁₃ for one shot.
The reception data storage unit 244 stores the transfer data D ₁₃ for one shot (one frame) restored by the combining unit 242. Transfer data D ₁₃ to be stored may be connected to the transfer data D ₁₃ to which it was previously stored in the same data file D _0.
By the above process is repeated, the received data storage unit 244, one of all the data in the data file D ₀ is stored.

<Imaging Process of Receiving Device 200>
For simplicity in the following, it is assumed for determining for each frame will be described only the code image D ₃ in the frame overall timing mark D ₂ has been correctly detected as being used in decoding. In the following, the operation of the receiving apparatus 200 will be described using an example of reading the image shown in FIG.

It is possible by keeping shared by the previously transmitting apparatus 100 the rules relating to the shape of the timing mark D ₂ and the reception apparatus 200 to correctly determine the overall timing mark D _2.
This point will be described below with reference to an example of FIG. Figure 6 is a diagram showing an example of the dimensions of each part of the timing mark D ₂ of the image. In FIG. 3, but showing the image of the timing mark D ₂ to an image of a color component, in FIG. 6 represent the image of the timing mark D ₂ to an image of all color components. Also, it represents one of the code image D ₁₁ to an image of all color components. Therefore, the image and the code image D ₁₁ of the timing mark D _2, for example, is displayed on the display unit 130 as a black image. When a black image is used as illustrated in FIG. 6, the transmission apparatus 100 may have only one code generation unit 116. Further, the image cutout unit 220 of the receiving apparatus 200 only needs to have one cutout unit 222.

In the example of FIG. 6, the timing mark D <b> ₂ is shown by two mutually parallel black belt-shaped areas St _a and St _b located on both sides of the code area Acd. The widths of the regions St _a and St _b are equal to each other and have the dimension Wst. The interval Sep between the regions St _a and St _b is 6 times the width Wst of the region. Further, the height direction of the band-shaped regions St _a and St _b coincides with the sub-scanning direction of the dot sequential scanning method or the scanning direction (direction orthogonal to the line) of the line sequential scanning method. Furthermore, the height Hgh of the regions St _a and St _b is four times the width Wst. Acceptable if the image to be displayed has such a dimensional relationship, the mark recognition unit 210, the timing marks D ₂ in the image obtained by the photographing, whether they have the same dimensional relationships as the (difference Or not). Thus, the receiving apparatus 200 can determine whether the whole of the timing mark D ₂ has been correctly acquired. For example, in an image obtained by photographing of the receiving device 200, strip-shaped area St _a, is shorter than 4 times the height of the St _b is the width, mark recognition unit 210, the overall timing mark D ₂ is correct shooting It can be determined that it has not been done.

Figure 7 (A), FIG. 7 (B) and FIG. 7 (C) showing a process of the image clipping unit 220 at the timing when the mark recognition unit 210 as described above detects the timing mark D ₂ performs the clipping of the image . FIG. 7A shows the same thing as FIG. FIG. 7 (B) shows what corresponds to FIG. 4 (B) and FIG. 4 (C). That is, FIG. 7 (A) represents a timing mark _{D 2.} FIG. 7 (B) represents a code image _{D 3.} That is, in the description of FIGS. 7A to 7C, description will be made using the image of FIG. Therefore, the code image D ₃ (code area Acd) and the timing mark D ₂ (areas St _a , St _b ) are displayed in the data of each color component. FIG. 7C shows a frame shot by the imaging unit 202. In FIG. 7C, a reference symbol P _n is attached to each frame photographed by the imaging unit 202. “N” represents a frame number, which is indicated by 1 to 14 in FIG.
The dotted line in the vertical direction in FIG. 7A indicates a break of the frame period T _t of the display by the display unit 130 as in FIGS. 4A to 4C. On the other hand, a vertical dotted line in FIGS. 7B and 7C indicates a break of the frame period _Tr of the imaging by the imaging unit 202. Accordingly, the interval _Tr between the vertical lines in FIGS. 7B and 7C represents the length of the imaging frame period (cycle).

In the example shown in FIGS. 7A to 7C, the period of the frames taken by the imaging unit 202 (frame period T _{r of} imaging) is the frame period of display by the display unit 130 (frame period T _{t of} display). It is assumed that the length is 0.9 times as large as.
In the period of the _third frame P ₃ in the imaging frame period _Tr of FIG. 7C, the timing mark D ₂₀₁ is displayed over the entire display period. Therefore, the code image _{D 301} of the corresponding period of the frame _{P 3} is cut out. Similarly, in the seventh frame P ₇ and the eighth frame P ₈ , the timing mark D ₂₀₂ is displayed over all the display periods. For this reason, the code image D _{302 in} the period of the corresponding frames P ₇ and P ₈ is cut out. Similarly, in the 12-th frame _{P 12,} over all of the display periods, a timing mark _{D 203} are displayed. For this reason, the first code image D _{313 in} the period of the corresponding frame P ₁₂ is cut out.

As in the above example, when the imaging frame period T _r (cycle) is different from the display frame period T _t (cycle), it is constant which frame image of each display frame is cut out. do not do. That is, it is not constant which display frame period Tt image is cut out.
In order to ensure that any code image D ₃₀₁ , D ₃₀₂ , D ₃₀₃ is read out reliably, in each synthesized video D ₆₂ , code images D ₃₀₁ , D ₃₀₂ , D ₂₀₃ with timing marks D ₂₀₁ , D ₂₀₂ , D ₂₀₃ are added. It is desirable that the number of frames (body frames) in which D ₃₀₃ is displayed be two or more. When the frame rate (1 / T _r ) of the imaging frame is lower than the frame rate (1 / T _t ) of the display frame, it is desirable to increase the number of body frames in each composite video D ₆₂ . Incidentally, composite video D ₆₂ is for synthesizing unit 122 is created by connecting a plurality of synthetic images D61. Composite video _{D 62} is an image for displaying a single code image _{D 3.}

As in the above example of the code image D _302, if the code image D ₁₁ of the frame Pn of the plurality of captured were cut out, the decoded decode data of the color unit obtained by decoding the code image D ₁₁ For D ₁₂ (data representing the pixel value of each pixel), for example, a process of taking an average thereof can be performed. In addition, it is possible to perform processing such as selecting either one (for example, the top one).

<Display processing method of transmitting apparatus 100>
Next, with reference to FIGS. 8 and 9, it is performed in the transmitting apparatus 100, the procedure of the processing for displaying an image based on the transmission data D _1. FIG. 8 is a flowchart showing an operation of displaying video based on transfer data in the transmission apparatus 100. FIG. 9 is a flowchart showing details of step S12.
The process shown in FIG. 8 is started in transmission apparatus 100 based on a predetermined program. For example, in the process shown in FIG. 8, a transmission program for transmitting data to the receiving device 200 is activated, and one of the plurality of data files D ₀ stored in the transmission data storage unit 102 is selected. Will start when.

When the process of FIG. 8 is started, in step S11, the read unit 112 divides the data file D ₀ that was selected plurality of data portions (shots).
Next, in step S12, one of the plurality of data parts generated by the division in step S11 is selected, and the selected data part (data for one shot) is processed. “Processing” here refers to processing performed by the data dividing unit 114, the mark generation unit 106, the code generation unit 116, and the synthesis unit 122 described above.
The processing performed by the data dividing unit 114, the mark generating unit 106, the code generating unit 116, and the combining unit 122 will be described with reference to FIG.

In step S13, the read unit 112, it is determined whether or not finished reading of all the data portions of the data file D ₀ that is selected. Determining that the reading of all the data portions ended data file D ₀ of the read unit 112 is, for example, sequentially reads out the data of one shot from the data file D _0, it is when it reaches the end of the data file D ₀ . Alternatively, the determination that all the data portions of the data file D ₀ of the reading unit 112 have been read is performed by dividing the data file D ₀ and temporarily storing it in a memory or the like, for example. The data that has been read out is erased, and all the data in the temporary storage location is lost. Note that erasing data can be replaced by moving the data or adding a processed mark to the data.
If it is determined in step S13 that reading of all data portions has not been completed (YES in S13), the process returns to step S11 to select the next data portion, and thereafter the processing in steps S12 and S13 is performed. repeat.
By repeating such processing, a series of frame data constituting the video is accumulated in the video buffer 126.

In step S13, when the processing for all the data portions is determined over, the process proceeds to step S14, moving image playback unit 128 performs the output of data D ₆₃ stored in the video buffer 126. Here, the “data portion” is data for one shot. That is, the moving image reproducing unit 128 sequentially displays the data D ₆₃ of a series of frames constituting the video from the video buffer 126 on the display unit 130 as a moving image.

FIG. 9 shows details of step S12 of FIG.
First, in step S21, the data dividing unit 114, a single (one shot data) of a plurality of data portions generated by the division of the data file D ₀ in step S11, data of the first color channel (first 1 data unit D ₅₁ ) and second color channel data (second data unit D ₅₂ ). By this division, the data units D ₅₁ and D ₅₂ of each color channel become data having a data amount that can be expressed by one graphic code.
As the plurality of color channels, for example, a red (R) channel and a blue (B) channel are used. Data dividing unit 114 provides the first data unit _{D 51} to the first code generating unit 116A. Then, the data dividing unit 114 provides a second data unit _{D 52} to the second code generating unit 116B.

In step S22, the first code generation unit 116A converts the first data unit D ₅₁ given from the data division unit 114 into the first code image D ₃₁ , and outputs an image plane including the first code image D _31. .
On the other hand, in step S23, the second code generating unit 116B converts the second data unit _{D 52} supplied from the data dividing unit 114 to the second code image _{D 32,} the image plane containing the second code image _{D 32} Output.

In step S24, the combining unit 122 includes an image plane comprising a first code image _{D 31} generated in step S22, and an image plane comprising a second code image _{D 32} generated in step S23, each red channel and Assign to the blue channel and combine to generate a color image. That is, in the step S24, the combining unit 122 assigns image plane including a first code image _{D 31} generated in step S22 to the red channel. Then, the composition unit 122 assigns image plane comprising a second code image D ₃₂ generated in step S23 to the blue channel. Further, the combining unit 122 assigns image plane which does not include a timing mark D ₂ stored in the image storage unit 107 in the green channel. Then, the composition unit 122, an image plane including a first code image D ₃₁ assigned to the red channel, the timing mark D assigned to the image plane and the green channel comprises a second code image D ₃₂ assigned to the blue channel _An image plane not including ₂ is synthesized to generate a color image.

Next, in step S25, the color image created in step S24 is repeated a predetermined number of times, for example, once, an image for one frame is generated, sent to the video buffer 126, and written into the video buffer 126. . That is, a color image obtained by combining the image plane of the first code image D ₃₁ , the image plane of the second code image D ₃₂ , and the image plane not including the timing mark D ₂ written in the video buffer 126 is shown in FIG. This corresponds to the image of the first one frame period T _t1 shown in A). Here, the “predetermined number of times” is one in the example of FIG.

In parallel with the processing of steps S21 ~ S25, in step S26, performs image generation timing mark _{D 2.}

For this process, for example, mark generation unit 106 previously generates an image plane containing the advance timing mark D _2. For example, the mark generator 106, allowed to store the image plane including timing mark D ₂ in the image storage unit 107.
In step S < _b > 26, the mark generation unit 106 reads out an image plane including the timing mark D < _b > ₂ stored in the image storage unit 107 and gives it to the synthesis unit 122. Also, for frame not including the timing mark D _2, the mark generator 106, allowed to store the image plane which does not include a timing mark D ₂ in the image storage unit 107.

The next step S25 and step S26, in step S27, the combining unit 122 includes an image plane comprising a first code image _{D 31} generated in step S22, the second code image _{D 32} generated in Step S23 including an image plane and an image plane including timing mark D ₂ generated in step S26, each red channel synthesizes assigned to the blue channel and the green channel, the image D ₃ of the timing mark D ₂ with code A color image (synthesized image D ₆₁ ) is generated. That is, the combining unit 122 assigns image plane including a first code image _{D 31} in the red channel. Combining unit 122 assigns image plane comprising a second code image _{D 32} in the blue channel. Combining unit 122 assigns image plane including timing mark D ₂ in the green channel. The combining unit 122 combines the color channels. Then, the composition unit 122 generates a composite image _{D 61.}

Next, in step S28, the image created in step S27 is repeated a predetermined number of times, for example, twice, and two frames of video are generated, sent to the video buffer 126, and written into the video buffer 126. These two frames are written as a frame subsequent to one frame written in step S25. That is, the color image obtained by combining the image plane of the first code image D ₃₁ , the image plane of the second code image D ₃₂ , and the image plane of the timing mark D ₂ written in the video buffer 126 is shown in FIG. It corresponds to the image of the period T _e of the two frames indicated by. Here, the “specified number of times” is two in the example of FIG.

Next, in step S29, it erases the timing mark _{D 2.} Since this process, the mark generator 106 previously generates an image plane that does not include the pre-timing mark D _2. For example, the mark generation unit 106 stores an image plane that does not include the timing mark D ₂ in the image storage unit 107. In step S < _b > 29, the mark generation unit 106 reads an image plane that does not include the timing mark D < _b > ₂ stored in the image storage unit 107, and provides the image plane to the synthesis unit 122.

Next, in step S30, the combining unit 122 includes an image plane comprising a first code image _{D 31} generated in step S22, an image plane comprising a second code image _{D 32} generated in step S23, step S29 in generated, respectively red channel and an image plane in which the timing mark D ₂ is erased, synthesized by assigning the blue channel, and green channel, free of timing marks D _2, first code image D ₃₁ and a second generating a color image including a code image D _32. That is, in the step S30, the combining unit 122, an image plane of the first code image _{D 31,} the image plane where the image plane and the timing marks _{D 2} has been erased in the second code image _{D 32} synthesizes.
Also in synthesis in the step S24, it is also possible to synthesize an image plane in which the timing mark D ₂ is erased. That is, it is also possible to synthesize an image plane that does not include a timing mark D ₂ stored in the image storage unit 107.

Next, in step S31, the image created in step S30 is repeated a predetermined number of times, for example, once to generate an image for one frame. Then, the generated image for one frame is sent to the video buffer 126 and written into the video buffer 126.
The image for one frame is written as a frame following the video for two frames written in step S28. In other words, the color image obtained by combining the image plane of the first code image D ₃₁ , the image plane of the second code image D ₃₂ , and the image plane from which the timing mark D ₂ is erased is written in the video buffer 126 here. This corresponds to the image of the last one frame period T _t2 shown in (A). Here, the “predetermined number of times” is one in the example of FIG. The process in step S30 is the same as the process in step S25.

There are several methods for storing the composite image D _{61 on} the video buffer 126. Here, the video buffer 126 has, for example, a management area including the number of frames or reference information to actual image data, and a buffer area in which image data is actually stored.

When linking frames, the end address of the image stored in the current buffer area is read from the management area information. Then, the data of the frame to be added is copied after the stored image. Thereafter, the data in the management area is updated by adding one frame.

When the composite video D ₆₃ written in the video buffer 126 is read and displayed as described above, a plurality of shots constituting the data file D ₀ are each displayed over a period of 4 frames.
As described above, one shot is image data of one QR code, for example. Also, when the capacity of the data file D ₀ is large, the data is sent by the image data of a plurality of QR codes. That is, when the capacity of the data file D ₀ is large, the data file D ₀ data is sent by the image data of a plurality of QR codes, each QR code is displayed over 4-frame period.
The display content changes as follows during the display of one shot. In the first frame period _{T t1,} only the first code image _{D 31} and the second code image _{D 32} is displayed.
In a period _{T e} of the two frames that follows, the first code image _{D 31,} the second code image _{D 32} and timing marks _{D 2} is displayed. In the end of one frame period _{T t2} subsequent, only the first code image _{D 31} and the second code image _{D 32} is displayed.

<Imaging Processing Method of Receiving Device 200>
Next, with reference to FIG. 10, a procedure of processing for acquiring and storing the transfer data D ₁₃ from the captured video data D ₇ performed in the receiving apparatus 200 will be described.

10, in the receiving apparatus 200 acquires transfer data D ₁₃ from the video data D ₇ imaged is a flowchart showing the operation of storing. The process shown in FIG. 10 is started in receiving apparatus 200 based on a predetermined program. For example, the process illustrated in FIG. 10 is started when a reception program for receiving data (synthesized video D ₆₃ ) transmitted from the transmission device 100 is activated.

First, in step S41, the imaging unit 202 captures a moving image. That is, the image (synthesized image D ₆₃ ) displayed on the display unit 130 is imaged by the imaging unit 202.
Next, in step S42, the sequence of frames constituting the video data D ₇ obtained by the imaging, will be stored sequentially in the video buffer 204. After the end of recording, reading the frames constituting the video data D ₇ stored in the video buffer 204 sequentially performs processing to be described later.

First, in step S43, the reading unit 206 selects one of the frames from the video buffer 204 and reads the image data of the selected frame.
Next, in step S44, the separation unit 208, a frame of data _{D 8} read in step S43, the separated data _{D 9} color component values (step S44). Then, mark recognition unit 210 detects the timing mark _{D 2} (step S45). In the case where the timing mark _{D 2} is not detected, if there are remaining frames in the video buffer 204, reading unit 206 repeats from 1 frame read from the video buffer 204 (step S43). Reading unit 206 performs a frame read (step S43) from the video buffer 204 to the timing mark _{D 2} is detected.

That is, in step S44, the separation unit 208, the data _{D 8} of one frame read in step S43, the separated data for each color channel. In the first embodiment, three primary colors of red (R), blue (B), and green (G) are adopted as color channels. Thus, the separation unit 208, the data D ₈ of the frame, the image data of the red channel (image data of the red component), (image data of the blue component) image data of the blue channel and the image data of the green channel (green component Image data). Then, the separation unit 208 provides the red component image data to the first cutout unit 222A, the blue component image data to the second cutout unit 222B, and the green component image data to the mark recognition unit 210.

In step S45, the mark recognition unit 210 determines whether the timing mark D ₂ is detected. That is, the mark recognition unit 210, and determines whether contains timing marks D ₂ to the image represented by the data D _9G of the separated green component value at step S44. In the case where the image represented by the data _{D 9G} of the separated green component value in step S44 does not include a timing mark _{D 2} (NO in step S45), the process proceeds to step S57.

When the timing marks _{D 2} is detected (YES at step S45), in step S47, S49, the image clipping unit 220 is included in each color channel code image (color component values of the data _D _{9R, D 9B)} is Cut out the area. In steps SS48 and S50, the code decoding unit 230 executes code decoding.

That is, in step S47, the first cutout unit 222A cuts out the first code image _D11R from the image represented by the red component value data _D9R separated in step S44. In step S48, the first decoding unit 232A decodes the first code image _D11R included in the image cut out in step S47. As a result, the decoded decoded data D _12R in color units corresponding to the first data unit D ₅₁ is extracted.

At step S49, the second cutout portion 222B from the image represented by the data _{D 9B} of the separated blue component value at step S44, cut a second code image _{D 11B.} In step S50, the second decoding unit 232B decodes the second code image _D11B included in the image cut out in step S49. As a result, the decoded decoded data D _12B in color units corresponding to the second data unit D ₅₂ is extracted.

When the decoding in step S48 and step S50 is completed, in step S51, the combining unit 242 combines the decoded data D _12R and D _{12B in} units of colors.
In step S52, the received data storage unit 244, the decoded decode data _D 12R color _unit, stores the transfer data _{D 13} obtained as a result of synthesis of _{D 12B.}

That is, in step S51, the synthesizer 242 decodes the color-unit decoded decoded data D _12R (first data unit D ₅₁ ) extracted in step S48 and the color-unit decoded decoded data D extracted in step S48. _12B (second data unit D ₅₂ ) is combined to restore the transfer data D ₁₃ .
In step S52, the combining unit 242 sends the transfer data _{D 13} that is restored to the received data storage unit 244. The received data storage unit 244 stores the transfer data _{D 13} received.

In the synthesis of the decoded data D _12R and D _12B in units of color in the synthesis unit 242, for example, assuming that the red channel is the preceding data D _12R and the blue channel is the succeeding data D _12B , The red channel data D _12R acquired this time is added, and then the blue channel data D _12B is added and stored.

After step S52, the process proceeds to step S53, where it is determined whether there are any remaining frames in the video buffer 204. If there are no remaining frames (NO in step S50), the process is terminated. Any remaining frame (YES at step S50), in step S54, the reading unit 206 further reads out data _{D 8} for one frame from the video buffer 204.

Next, in step S55, the separation unit 208, the data _{D 8} of the frame read in step S54, the image of each color channel (data _D 9G color component _values, D _{9R, D 9B)} to separate.

In step S56, the mark recognition unit 210, (or erased) Timing mark D ₂ is either detected to determine whether. In step S56, the timing mark D ₂ which has been detected to verify no longer detected. For example, the mark recognition unit 210, the timing marks D ₂ from the data D _9G of the separated green component value in step S55, it is determined whether or not it is erased. That is, the last one frame period T _t2 shown in FIG. 7A is detected. Because if the green is left timing mark _{D 2} data _{D 9G} component values (YES in S56), the code image _D _11R, is _{D 11B} not yet changed, the

separation unit

208, 1 frame read Minute data is discarded, and the process returns to step S53.
If there are no remaining timing mark _{D 2} (NO in S56) the data _{D 9G} green component value, control is passed to step S57.

In step S57, it is determined whether there are any remaining frames in the video buffer 204. If there are remaining frames, the process returns to step S43. If there are no remaining frames, the process ends.
In this way, by performing processing until the video data D _{7 in the} video buffer 204 is exhausted, the receiving device 200 can restore the transferred data (transfer data D ₁₃ ).

The above processing, of the one shot constituting one frame (data D _{8 frames),} if the timing mark D ₂ was detected in the plurality of frames, the first frame timing mark D ₂ is detected Only the included code images D _11R and D _11B are decoded.

Since the transmission device 100 and the reception device 200 in the first embodiment are configured as described above, as illustrated in FIG. 1, the display unit 130 of the transmission device 100 and the imaging unit 202 of the reception device 200. by facing the door, can be automatically using the code image are sequentially displayed on the display unit 130 (the combined image D _61), efficiently capture the transmission data D ₁ to the receiving device 200.

Further, in the first embodiment, by realizing means for automatically reading, the amount of information represented by each code is reduced, and each point constituting the code image is increased. Can be displayed. In other words, if an attempt is made to transfer with one QR code within a limited size screen, a high-density QR code is obtained. And it is necessary to make each cell small. Here, the “cell” indicates the smallest monochrome pattern. As a result, blurring or blurring when taking a QR code is likely to occur. By dividing and transmitting sequentially to a plurality of QR codes, the density of one QR code can be lowered. When this QR code is displayed large on the screen, each cell can be enlarged. If the cell is large, the occurrence of blurring or blurring when taking a QR code is suppressed. The probability of successful decoding of the QR code increases. For this reason, it is possible to improve resistance to defocusing and resistance to camera shake during shooting. In other words, the tolerance at the time of shooting with respect to defocus and camera shake is improved. Then, when the transmission device 100 is held up toward the reception device 200, the workability of the user can be improved.

As in the first embodiment, by using a plurality of color channels, when independently of the code image D ₃ sends to the timing mark D _2, for example, the timing mark D ₂ is at least drawing code image D ₃ It can be represented as an image that occupies the entire range (code region). The timing mark D ₂ can be expressed as a uniform image. “Uniform image” is, for example, an image represented by the maximum gradation value in the entire rectangular area (area shown by cross-hatching) shown in FIG. In the example shown in FIG. 3, the drawing range of the code images D ₃₁ and D _{32 and} the range occupied by the timing mark D ₂ are the same. Due to the scanning system relationship between the display unit 130 of the transmission device 100 and the imaging unit 202 of the reception device 200, a part of the captured frame is displayed at the moment when the screen of the display unit 130 is switched. An incomplete video such as a mixture of previous and subsequent images may be taken. For this reason, when the timing mark D ₂ is represented as a uniform image including the entire code images D ₃₁ and D ₃₂ , and the entire timing mark D ₂ is read without being lost, the code images D ₃₁ and D ₃₂ can be regarded as being read correctly.

In the above example, the period (body period) in which the code images D ₃₁ and D ₃₂ with the timing mark D ₂ are continuously displayed is a two-frame period. A period (guard period) during which the code images D ₃₁ and D ₃₂ without the timing mark D ₂ are displayed before and after the body period is one frame period. However, the length of the body period (number of frames) and the length of the guard period (number of frames) should be determined according to various conditions, and the invention of Embodiment 1 is not limited to the above example.

Timing mark D ₂ is not limited to those described above. Further, if there is room in the timing to check chipping timing mark D ₂ (overall timing mark D ₂ is a confirmation of whether or captured) it becomes unnecessary. Then, further, the degree of freedom relating to the determination of the shape of the timing mark D ₂ is higher. The timing margin increases as the imaging frame rate increases with respect to the display frame rate. Also, the larger the number of guard frames and the number of body frames, the larger. Further, for example, the timing mark D ₂ are marks fills the entire screen in a certain color, it marks fill a uniform pattern, a circular mark, the mark of multiple circles, a further square around the square marks or square It is possible to adopt a mark with an enclosed edge.

In the case of using a plurality of timing marks D _2, it can have different meanings in different timing mark D _2. In other words, it is possible to add a timing mark D ₂ which differs depending on data portion to the data file D _0. For example, a plurality of timing marks D _2, it can be shown a series of delimited data, termination, etc. of the start or data of the data.

Figure 11 (A) ~ FIG 11 (M) show an example of a different timing mark _{D 2} of the combination they have different meanings.
For example, as a timing mark D _2, it can be shown that the leading data part of the data file D ₀ by a circular timing mark D _2A shown in Figure 11 (A). The timing mark D _2B depicting one wheel on the outside of the circular shown in FIG. 11 (B), it can be shown that the data portion of the intermediate (non-head and tail) of the data file D _0. The timing mark D _2C depicting two wheels on the outside of the circular shown in FIG. 11 (C), it is possible to indicate the end of the data portion of the data file D _0. Here, circular depicted in FIG. 11 (A) ~ FIG 11 (C) is preferably displayed on the position corresponding to the code image D _3. That is, the circular depicted in FIG. 11 (A) ~ FIG 11 (C) is preferably encoded image D ₃ is displayed in the area to be displayed.

Further, as the timing mark D _2, it can be shown that the leading data part of the data file D ₀ by rectangular timing mark D _2D shown in FIG. 11 (D). The timing mark D _2E depicting one frame outside the rectangle shown in FIG. 11 (E), it can be shown that the data portion of the intermediate (non-head and tail) of the data file D _0. The timing mark D _2F depicting two frames on the outside of the rectangle shown in FIG. 11 (F), can be shown to be the last data portion of the data file D _0. Here, rectangle depicted in FIG. 11 (D) ~ FIG 11 (F), drawn with a size corresponding to the code image D _3, it is desirable to be displayed in the corresponding position in the code image. That is, the rectangle depicted in FIG. 11 (D) ~ FIG 11 (F) is preferably encoded image D ₃ is displayed in the area to be displayed.

Furthermore, the data file D _0, a plurality of portions (referred to herein as "segment") will be described which can be separated into. “Segment” is a set of one or more “shots”. Even if an error occurs in one shot, the file transfer is divided so as not to fail as a whole. That is, the “segment” is divided so that the file transfer can be said to be “partially successful”. When applied to signage, etc., the same data is repeatedly displayed. Even if a part of shots becomes an error in the first round, if the transfer including the shot is successful in the second round, the file transfer can be successful as a whole. Considering camera shake at the time of photographing, it is difficult to photograph a plurality of shots necessary for one transfer without error. The “segment” is a mechanism for relieving such an error.
The rectangular timing mark D _2G shown in FIG. 11 (G), it can be shown that the leading data part of the data file D _0.
On the left side of the rectangle shown in FIG. 11 (H), the timing mark D _2H arranged one smaller rectangle than the rectangle, that the data portion of the intermediate (non-head and tail) of the data file D ₀ Can be shown.
On the left side of the rectangle shown in FIG. 11 (I), a smaller rectangle than the rectangle by two arranged the timing mark D _2I, can be shown to be the last data portion of the data file D _0.
A timing mark _D2J in which three rectangles smaller than the rectangle are arranged on the left side of the rectangle shown in FIG. 11 (J) can indicate the data portion at the beginning or end of each segment.
“The data portion at the beginning or end of each segment” means, for example, the first shot or the tenth shot of one segment when one segment is composed of ten shots.
Here, large rectangle depicted in FIG. 11 (G) ~ FIG 11 (J), drawn with a size corresponding to the code image D _3, it may be displayed at a position corresponding to the code image D ₃ desirable. In other words, large rectangle depicted in FIG. 11 (G) ~ FIG 11 (J) is preferably encoded image D ₃ is displayed in the area to be displayed.

Incidentally, as a timing mark D _2, it can be shown that the leading data part of the data file D ₀ by rectangular timing mark D _2K shown in FIG. 11 (K).
The timing mark D _2E depicting each one rectangular sides of the rectangle shown in Figure 11 (L), it can be shown that the data portion of the intermediate (non-head and tail) of the data file D ₀ .
The timing mark D _2M depicting each of the two rectangles on each side of the rectangle shown in Fig. 11 (M), can also be shown to be the last data portion of the data file D _0.
Here, large rectangle depicted in FIG. 11 (K) ~ FIG 11 (M), drawn with a size corresponding to the code image D _3, may be displayed at a position corresponding to the code image D ₃ desirable. In other words, large rectangle depicted in FIG. 11 (K) ~ FIG 11 (M), it is desirable that the code image D ₃ is displayed in the area to be displayed.

If these as, for example, it can be given a position of each data portion in the data file D ₀ from the read unit 112 to the mark generating unit 106 (e.g., the beginning, in the middle or end) information indicating. Also, information indicating the head, middle or end of the segment to which each data part belongs can be given. Based on these such information, mark generation unit 106 may generate a corresponding timing mark D _2.

As described above, in the first embodiment, by converting the entire data file D ₀ in the code image D _3, subsequently, displaying the code image D ₃ as a moving image. As a result, a video for transfer can be efficiently displayed even on a terminal having a low calculation capability.

On the other hand, the video imaged by the imaging unit 202 is stored in the video buffer 204 as moving image data. Then, after taking a picture, the decoding process is taken out the video data D ₇ frame by frame from the video buffer 204 is performed. Therefore, the receiving apparatus 200 can also be configured with a terminal having a low computing capability.

In the above, the code image D ₃ has been described as being switched between one frame and the next frame. However, in a liquid crystal display device that is a hold-type display device, when a display image is switched by a dot sequential scanning method or a line sequential scanning method, it takes one frame period to switch the displayed image. In this case, the “timing immediately before and after the switching” is set to “the frame in which the switching is performed and the immediately preceding frame and the immediately following frame”, and the timing mark D ₂ is set for a period of 3 frames per shot. In some cases, it is not added and is not subject to decoding.

Hereinafter, in this regard, FIGS. 12A, 12B, 13A, 13B, 13C, 13D, 14A, and 14 B), FIG. 14C, FIG. 15A, FIG. 15B, and FIG. FIG. 12A and FIG. 12B are diagrams illustrating changes in the display image within the display frame period. FIG. 13A to FIG. 13D are diagrams showing changes in the display image within the display frame period.
FIGS. 14A to 14C are diagrams showing changes in the display image within the display frame period. FIGS. 15A to 15C are diagrams showing changes in the display image within the display frame period.

FIG. 12A shows a change in the display image displayed on the display unit 130. Upper symbols Fd ₁ to Fd ₇ represent display frame numbers. FIG. 12B shows a change in an image (captured image) obtained by imaging by the imaging unit 202. Upper symbols Fp ₁ to Fp ₇ represent imaging frame numbers. FIGS. 13A to 15C show changes in the display image within the display frame period.
In the example shown in FIGS. 12A and 12B, the display frame rate (display frame period T _t ) and the imaging frame rate (imaging frame period T _r ) are the same. However, it is assumed that the imaging frame is shifted from the display frame by a half frame period. In addition, the imaging screen is not inclined with respect to the display screen. That is, it is assumed that the horizontal direction of the imaging screen matches the horizontal direction of the display screen.

In FIG. 12A, the update of the image within each frame is indicated by an oblique line SC that is sequentially performed from the top to the bottom of the screen over one frame period (display frame period T _t ).
For simplicity, the code image D ₃ is assumed to be located in the center of the screen. Also shows the code image _{D 3} replaces sequentially cut code _{D 3A,} by the symbol _{D 3B} and code _{D 3C.} Further shows the code image _D _3A, D _3B, the upper half of the _{D 3C} respectively _D _3AU, D _3BU, at _{D 3 CU.} The lower half of the code images D _3A , D _3B , D _3C is indicated by D _3AL , D _3BL , D _3CL . Also shows the timing marks to be added to the code image D ₃ in D _2. The upper half of the timing mark _{D 2} shown in _{D 2U.} The lower half of the timing mark _{D 2} indicated by _{D 2L.}

The image update will be described with reference to the display frame Fd ₁ . The time point at which the screen is updated is indicated by a time point t _ns (n is a natural number representing a frame number). A time point t _1b indicates a time point when the update of the screen of the display frame Fd ₁ proceeds to the upper end of the code image. A time point when the screen update of the display frame Fd ₁ has advanced to the center of the screen is indicated by a time point t _1m . A time point t _1e indicates a time point when the update of the screen of the display frame Fd ₁ proceeds to the lower end of the code image. At t _1b , the code image D _3A is displayed in the display frame Fd ₁ . At the time point t _1m , D _3BU is displayed on the upper side of the code image D ₃ of the display frame Fd ₁ . At time t _1m , D _3AL is displayed below the code image D ₃ of the display frame Fd ₁ . At the time point t _1e , the code image D _3B is displayed in the display frame Fd ₁ .

Figure 12 (A) and the timing mark D ₂ in the example shown in and FIG. 12 (B) is located only on the left side of and the code image D ₃ intended strip.
It is assumed that the imaging unit 202 is open for the entire imaging period of each imaging frame.

In the example of FIG. 12 (A), the code image _{D 3} is switched in 6 frame periods. Among them, the switching of the code image _{D 3} in the first frame Fd ₁ is performed. Switching of the image due to the addition of the timing mark D ₂ is Okonawaru in the third frame Fd _3. Switching of the image by erasing timing mark D ₂ is carried out in the fifth frame Fd _5.

The update of the screen of the display frame Fd ₁ will be described with reference to FIG. In the display frame Fd ₁ in which the switching of the code image D ₃ is performed, at the time t _1m when the update of the image proceeds to the center of the screen, as shown in FIG. 13A, the code image D ₃ being displayed is displayed. In the lower half, the lower half D _3AL of the code image D _3A before _switching is displayed. Further, the upper half of the code image D ₃ being displayed, the upper half D _3BU code image D _3B after switched is displayed. In the display frame Fd ₁ , at the time t _1e when the update of the image proceeds to the lower end of the code image, the lower half of the displayed code image D ₃ is also switched as shown in FIG. 13B. It becomes the lower half D _{3BL of} _3B . From time t _1m to time t _1e, the lower half of the displayed code image D ₃ gradually changes from the image D _3A before switching to the image D _3B after switching.

In the next display frame Fd _2, the entire code image D _3B is displayed over the entire display period.

The update of the screen of the display frame Fd ₃ will be described with reference to FIG. In the next display frame Fd _3, switching of the image for the addition of timing marks D ₂ is performed. At time t _3b which advanced to the position of the upper end of the update timing mark D ₂ of the image _(just before the upper end of the update begins), as shown in FIG. 14 (A), timing marks D ₂ are not displayed at all.
Image update, as shown in the advanced time _{t 3m} to the center of the screen FIG. 14 (B), the displayed upper half _{D 2U} timing mark _{D 2} is, the lower half _{D 2L} timing mark _{D 2} are displayed Not.
In advanced time t _3e update image to the lower end of the timing mark D _2, as shown in FIG. 14 (C), the overall timing mark D ₂ are displayed.
In other words, gradually appeared to lower the timing mark D ₂ over a period of from time t _3b to time t _3e is from the upper side of the screen.

In the display frame Fd _4, over all of the display periods, the code image D _3B and the timing mark D ₂ are displayed.

In the next display frame Fd _5, switching of the image for erasing timing mark D ₂ is performed. At time t _5b advanced to the position of the upper end of the update timing mark D ₂ of the image, as shown in FIG. 15 (A), timing marks D ₂ is displayed in its entirety.
Image update, as shown in the advanced time _{t 5 m} to the center of the screen FIG. 15 (B), the display lower half _{D 2L} timing mark _{D 2} is the upper half _{D 2U} timing mark _{D 2} is erased ing.
In the lower end until advanced time t _5e update image timing mark D _2, as shown in FIG. 15 (C), the overall timing mark D ₂ is erased.
In other words, it will be erased gradually to the lower timing mark D ₂ over a period of from time t _5b to the time point t _5e from the upper side.

In the next display frame Fd _6, all of the code image D _3B over the display period are displayed, not displayed timing mark D _2.

Next, in the display frame Fd ₇ in which the switching of the code image D ₃ is performed, at the time t _7b (immediately before the update of the upper end starts) when the image update proceeds to the upper end of the code image D ₃ , FIG. as shown, the code image D ₃ to be displayed are all switched-previous image D _3B.
In advanced time t _7m update image to the center of the code image D _3, as shown in FIG. 13 (D), the lower half of the code image D ₃ to be displayed under the switched before the code image D _3B Half D _3BL . Further, the upper half of the code image _{D 3} is the upper half of the code image _{D 3C} after switched is _{D 3 CU.} In between the time t _7b to time t _7m, it will change the image D _3C after switched the upper half of the code image D ₃ to be displayed from the gradually switched previous image D _3B.

In an image obtained by imaging in the imaging frame Fp ₁ straddling the display frame Fd ₁ and the display frame Fd ₂ , the upper half of the code image D ₃ is the same as the upper half D _3BU of the image D _3B after switching. . However, the lower half of the code image _{D 3} is an image _{D 3ABL} where the lower half _{D 3BL} mixed image _{D 3B} after switches the lower half _{D 3AL} of switched previous image _{D 3A.} The mixing ratio varies depending on the position in the vertical direction in the imaging screen, and is equal to the ratio of the time during which the image D _3A is displayed and the time during which the image D _3B is displayed at each position on the corresponding display screen.

In the image obtained by imaging in the next imaging frame Fp _2, the entire code image D ₃ is the same as the code image D _3B. On the other hand, in the display frame Fd ₂ , the timing mark D ₂ is not added, whereas in the first half of the display frame Fd ₃ (a period until the update reaches the center of the screen), the timing mark D ₂ gradually increases from the top. appear. Therefore, in the image obtained by the imaging in the imaging frame Fp _2, the lower half of the timing mark _{D 2} _{D 2L} it does not appear. Also, half of _{D 2U} on the timing mark _{D 2} is an image _{D 2U0} of the image that do not appear half _{D 2U} appears images on the timing mark _{D 2} are mixed. The ratio of the mixture in each location of the corresponding display screen, the time and the timing mark D ₂ appearing, equal to the ratio of the time when the timing mark D ₂ does not appear. Thus, the timing marks D ₂ in the captured image, as the direction of the upper darker (the gradation value is large), becomes thinner toward the lower side (gradation values is small).

In the image obtained by imaging in the next imaging frame Fp _3, the entire code image D ₃ is the same as the code image D _3B. On the other hand, the second half in (update period after the time of reaching the center of the screen), the timing mark D ₂ of the display frame Fd ₃ is gradually extended progressively further downward from the state appeared to the center. That is, the lower half D _2L of the timing mark D ₂ also gradually appears. On the other hand, the display frame Fd _4, timing marks _{D 2} in its entirety has appeared. Therefore, in the image obtained by the imaging in the imaging frame Fp _3, timing marks D ₂ entirety appears that. However, the lower half of the image obtained by the imaging in the imaging frame Fp _3, the image D _2L0 of the image the bottom half of the timing mark D ₂ does not appear as images D _2L appear are mixed. The ratio of the mixture in each location, time and the timing mark D ₂ appearing, equal to the ratio of the time when the timing mark D ₂ does not appear. Therefore, the lower half D _2L of the timing mark D _{2 in} the captured image is thinner toward the lower side and darker toward the upper side.

Even in an image obtained by imaging in the next imaging frame Fp _4, the entire code image D ₃ is the same as the code image D _3B. On the other hand, in the second half of the display frame Fd ₄ (a period after the time when the update reaches the center of the screen), the entire timing mark D ₂ is displayed. However, in the first half (the period of until the update reaches the center of the screen) of the display frame Fd _5, timing mark D ₂ is going to disappear gradually from the upper side in order. Therefore, in the image obtained by the imaging in the imaging frame Fp _4, timing marks D ₂ entirety appears that. However, the upper half image and image upper half D _2U image and timing marks D ₂ of the upper half D _2U timing mark D ₂ appears does not appear is mixed D of the image obtained by the imaging in the imaging frame Fp ₄ _2U0 It becomes. The ratio of the mixture in each location, time and the timing mark D ₂ appearing, equal to the ratio of the time when the timing mark D ₂ does not appear. Accordingly, the upper half D _2U of the timing mark D _{2 in} the captured image is thinner toward the upper side and darker toward the lower side.

In the image obtained by imaging in the next imaging frame Fp _5, the entire code image D ₃ is the same as the code image D _3B. On the other hand, in the second half (the period after the time when the update has reached the center of the screen) of the display frame Fd _5, timing marks D ₂ fades gradually from the upper side in this order. In the display frame Fd ₆ , since the timing mark D ₂ is not added, the upper half D _2U of the timing mark D ₂ does not appear in the image obtained by imaging in the imaging frame Fp ₅ . The lower half of the timing mark _{D 2} is an image _{D 2L0} to the image lower half _{D 2L} image and timing marks _{D 2} of lower half _{D 2L} timing mark _{D 2} appears does not appear have been mixed. The ratio of the mixture in each location of the corresponding display screen, the time and the timing mark D ₂ appearing, equal to the ratio of the time when the timing mark D ₂ does not appear. Thus, the timing marks D ₂ in the captured image is darker towards the bottom, it becomes thinner toward the upper side.

In an image obtained by imaging in the imaging frame Fp ₆ straddling the display frame Fd ₆ and the display frame Fd ₇ , the lower half of the imaging frame Fp ₆ is the same as the lower half D _3BL of the image D _3B before switching. . However, upper half of the image pickup frame Fp _6, and the upper half _{D 3 CU} image _{D 3C} after switched the upper half _{D 3BU} of switched previous image _{D 3B,} the longitudinal position of the screen, the image _{D 3B} And the image D _3BCU mixed according to the display time ratio of the image D _3C .

As can be seen from the discussion above, in the imaging frame Fp _4, the overall timing mark D ₂ is captured dark. Therefore, the code image D ₃ obtained by the image pickup frame Fp ₄ it can be seen is suitable for decoding.
In the imaging frame Fp ₃ and Fp _5, density of the timing marks _{D 2} varies depending longitudinal position. In this case, it is assumed that it is determined whether or not the timing mark D ₂ appearing in each line of the display screen. For example, if the threshold than the predetermined density, may be determined code image D ₃ of the line and suitable for decoding.

For example, if the threshold value is a medium value, the lower half D _3L of the code image D ₃ of the imaging frame Fp ₃ and a part of the upper half D _3U of the code image D ₃ are suitable for decoding. It is determined that Here, "density is medium value" is the density in the case where the timing mark D ₂ over all imaging period of the image frame is displayed, not displayed timing mark D ₂ over all imaging period of the image frame It is an intermediate value of the darkness of the case. All of the upper half D _3U of the code image D ₃ of the imaging frame Fp ₅ and a part of the lower half D _3L of the code image D ₃ are determined to be suitable for decoding.
When determining in this way, for example, one encoding a combination of a lower half _{D 3L} code image _{D 3} of the imaging frame Fp _3, the upper half _{D 3U} code image _{D 3} of the imaging frame Fp ₅ overall image D _3, it is also possible to perform decoding.

Incidentally, the line density of the timing marks D ₂ is equal to or greater than the threshold is, the code image D ₃ of the frame is determined to be more than a predetermined number, even be determined to be in its entirety suitable for decoding good.

In the imaging frame Fp _2, the lower half of the timing mark _{D 2} _{D 2L} does not appear. Then, the upper half D _2U of the timing mark D ₂ appears. Therefore, in the imaging frame Fp _2, density of the timing marks D ₂ depends longitudinal position.
In such a case, the code image _D3B as a whole may be determined to be unsuitable for decoding. _{Alternatively} , by lowering the threshold, a part of the upper half D _3BU of the code image D _3B may be suitable for decoding. Furthermore, in a predetermined number or more of lines, when the density of the timing marks D ₂ is determined to be equal to or greater than the threshold value, it may be determined to be suitable for decoding the entire code image D _3B .

Similarly, the upper half D _2U of the timing mark D ₂ does not appear in the imaging frame Fp ₅ . In addition, the lower half D _2L of the timing mark D ₂ appears. Therefore, density of the timing marks D ₂ depends longitudinal position.
In such a case, the code image _D3B as a whole may be determined to be unsuitable for decoding. _{Alternatively} , by lowering the threshold value, a part of the lower half D _3BL of the code image D _3B may be suitable for decoding. Furthermore, in a predetermined number or more of lines, when the density of the timing marks D ₂ is determined to be equal to or greater than the threshold value, it may be determined to be suitable for decoding the entire code image D _3B .

In the imaging frame Fp _1, the timing mark _{D 2} does not appear at all. The upper half D _3BU of the code image D _3B is equal to the image D _3BU after switching. Also, the lower half _{D 3BL} code image _{D 3B} is an image _{D 3ABL} that the image _{D 3BL} after switches the switched previous image _{D 3AL} have been mixed. Therefore, the code image D _3B of the imaging frame Fp ₁ is not suitable for decoding.
Similarly, in the imaging frame Fp _6, timing marks _{D 2} does not appear. The lower half of the code image D _3B is equal to the image D _3BL before switching. Further, the upper half of the code image _{D 3B} is an image _{D 3BCU} in which the image _{D 3 CU} of switches and switched preceding image _{D 3BU} mixed. Therefore, the code image D _3B of the imaging frame Fp ₆ is not suitable for decoding.
The code image D ₃ is not suitable for decoding, can be determined based on the timing mark D ₂ does not appear at all.

The case where the shutter opening time (exposure time) of the imaging unit 202 extends over the entire frame period Tr of each imaging has been described above. However, when the shutter opening time indicates only a part of the frame period _Tr of each imaging, the above mixing ratio is switched to the period during which the image before switching is displayed within the shutter opening time. It becomes a ratio with the period in which a subsequent image is displayed.

Note that the display screen of the display unit 130 may be tilted in an image obtained by imaging by the imaging unit 202. Therefore, when it is determined whether or not the code image D ₃ for each line of the display screen suitable for decoding, in the image obtained by the imaging (image data D _7), the display unit 130 of the line Need to specify direction.

Further, instead of the determination for each line of the display screen, may each range of longitudinal positions comprising a plurality of successive lines from each other, that it is determined whether or not those codes image D ₃ is suitable for decoding. In other words, for each band-shaped area composed of a plurality of lines, can be performed is determined whether or not those codes image D ₃ is suitable for decoding.

In the first embodiment, the code image and the timing mark are composed of data of the three primary colors of red (R), blue (B), and green (G). However, the present invention is not limited to this, It may be configured by data of a color different from the above. However, it is easier to configure with the data of the three primary colors when multiplexing by combining by the combining unit 122 as shown in FIG. 2, for example, and when separating by the separating unit 208 as shown in FIG. And efficient. The same applies to the following embodiments.

Embodiment 2. FIG.
In the first embodiment described above, the combining unit 122, the timing marks D ₂ and code image D _3, are synthesized assigned to each color channel. Then, the composition unit 122, and generates a composite image _{D 61} of the collar. However, the present invention is not limited to such an example. For example, the images D ₂ , D ₃₁ , and D ₃₂ output from the mark generation unit 106, the first code generation unit 116A, and the second code generation unit 116B can be separately connected to generate separate videos. . Then, the synthesis unit 122 may synthesize these videos by assigning them to the respective color channels.

FIG. 16 shows the configuration in that case. FIG. 16 is a block diagram schematically showing a configuration of transmitting apparatus 100a in the second embodiment. A transmitting apparatus 100a illustrated in FIG. 16 includes a first video buffer 126A, a second video buffer 126B, a third video buffer 126C, a first video buffer 126A, a video playback unit 128, and a synthesis unit 122 instead of the video buffer 126, the video playback unit 128, and the synthesis unit 122 illustrated in FIG. It has a 1 video playback unit 128A, a second video playback unit 128B, a third video playback unit 128C, and a synthesis unit 122b. Constituent elements similar to those of the transmitting apparatus 100 described in Embodiment 1 are assigned the same reference numerals, and descriptions thereof are omitted. Constituent elements similar to those of the transmission device 100 are a transmission data storage unit 102, a data processing unit 110, a reference clock generation unit 104, a mark generation unit 106, and a display unit 130. In addition, the reading unit 112, the data division unit 114, the first code generation unit 116A, and the second code generation unit 116B of the data processing unit 110 are the same components as those in the first embodiment. Further, the image storage unit 107 of the mark generation unit 106 is a component similar to that of the first embodiment.

First video buffer 126A stores the first code image _{D 31} generated by the first code generating unit 116A in this order. First video buffer 126A stores the first code image _{D 31} in sequence. The first video buffer 126A is, these are connected to generate a first code image _{D 64.} That is, the first image buffer 126A connects the first code image _{D 64} accumulated. The first video buffer 126A generates a first code image _{D 64.}
Second video buffer 126B stores the second code image _{D 32} generated by the second code generating unit 116B in this order. Second video buffer 126B stores the second code image _{D 32} in sequence. The second video buffer 126B is these ligated to generate a second code image _{D 65.} In other words, the second image buffer 126B connects the second code image _{D 65} accumulated. The second video buffer 126B generates a second code image _{D 65.}
Third image buffer 126C stores the timing mark _{D 2} generated by the mark generation unit 106 in order. Third image buffer 126C stores the timing mark _{D 2} in order. The third image buffer 126C is these ligated to generate a timing mark image _{D 21.} That is, the third image buffer 126C connects the accumulated timing mark _{D 2.} The third image buffer 126C generates the timing mark image _{D 21.}

The first video playback unit 128A, among the data file _{D 0,} when the connection code image _{D 31} of the first video buffer 126A is processing is completed, the data stored in the video buffer 126A (first code image _{D 64} ) Are sequentially read for each frame and reproduced as a moving image. That is, the first moving image reproduction unit 128A reproduces a moving image sequentially reads the first code image _{D 64.} The first video playback unit 128A reads the first code image _{D 64} for each frame. The timing at which the first video playback unit 128A reads out the first code image _{D 64} is the time the connection is complete code image _{D 31} in the first image buffer 126A.
The second moving image reproduction unit 128B, of the data file _{D 0,} when the connection code image _{D 32} of the second image buffer 126B to process is completed, the data stored in the video buffer 126B (second code image _{D 65} ) Are sequentially read for each frame and reproduced as a moving image. That is, the second moving image reproduction unit 128B is reproduced as sequentially reads video a second code image _{D 65.} The second moving image reproduction unit 128B reads the second code image _{D 65} for each frame. Timing when the second moving image reproduction unit 128B reads the second code image _{D 65} is the time the connection is complete code image _{D 32} in the second image buffer 126B.
The third moving image playback unit 128C receives the data (timing mark video D ₂₁ ) stored in the video buffer 126C when the conversion of all data of the data file D ₀ to the code video D ₆₄ and D ₆₅ is completed. It reads out every frame sequentially and reproduces it as a moving picture. That is, the third moving image reproduction unit 128C is reproduced as video sequentially reads the timing marks image _{D 21.} The third video playback unit 128C reads the timing marks video _{D 21} for each frame. Timing the third moving image reproduction unit 128C reads the timing marks image _{D 21} is the time when the video buffer 126A, the connection code image _D _{31, D 32} at 126B has been completed.

Combining unit 122b, a first code image _{D 64} given from the first moving image reproduction unit 128A, and the second code image _{D 65} supplied from the second moving image reproduction unit 128B, the timing marks provided from the third moving image reproduction unit 128C The video D ₂₁ is synthesized by being assigned to independent color channels. Then, the synthesis unit 122b generates a synthesized video D ₆₆ (color video) for one data file D ₀ .
Data (synthesized video D ₆₆ ) transmitted from the transmission device 100a in FIG. 16 can be received by the reception device 200 in FIG. 5 described in the first embodiment.

Embodiment 3 FIG.
In the first embodiment, the code image D ₃ suitable for decoding, and the fact indicated by the timing mark D _2. However, instead of the timing mark D ₂ , a code image D ₃ suitable for decoding may be indicated by light from a light emitting unit separate from the display unit 130. Such Embodiment 3 will be described below.
FIG. 17 shows a transmitting apparatus 100c according to the third embodiment of the present invention. FIG. 18 illustrates a receiving device 200c according to the third embodiment. The transmission device 100 c includes a light emitting unit 156 instead of the mark generation unit 106. Further, the transmission device 100 c includes a combining unit 122 c instead of the combining unit 122. The receiving device 200 c includes a light emission detection unit 260 instead of the mark recognition unit 210. Constituent elements similar to those of the transmitting apparatus 100 described in Embodiment 1 are assigned the same reference numerals, and descriptions thereof are omitted. The same components as the transmission apparatus 100 are a transmission data storage unit 102, a data processing unit 110, a reference clock generation unit 104, a video buffer 126, a moving image reproduction unit 128, and a display unit 130. In addition, the reading unit 112, the data division unit 114, the first code generation unit 116A, and the second code generation unit 116B of the data processing unit 110 are the same components as those in the first embodiment.
Constituent elements similar to those of receiving apparatus 200 described in Embodiment 1 are assigned the same reference numerals, and descriptions thereof are omitted. Constituent elements similar to those of the receiving apparatus 200 are an imaging unit 202, a video buffer 204, a reading unit 206, a separating unit 208, an image clipping unit 220, a code decoding unit 230, a combining unit 242, and a received data storage unit 244. Further, the first cutout part 222A and the second cutout part 222B of the image cutout part 220 are the same constituent elements as those in the first embodiment. Further, the first decoding unit 232A and the second decoding unit 232B of the code decoding unit 230 are the same constituent elements as those in the first embodiment.

As described above, the transmission device 100c in FIG. 17 is generally the same as the transmission device 100 in FIG. 2, but includes a light emitting unit 156 instead of the mark generation unit 106. Further, a synthesis unit 122 c is provided instead of the synthesis unit 122.

Combining section 122c is combined with the first code image _{D 31} from the first code generating unit 116A of the second code image _{D 32} from the second code generating unit 116B. Unlike the synthesis unit 122 of FIG. 2, the timing mark D ₂ I am not synthesized.

Result of synthesis by the synthesis unit 122c, the transmission data (transfer data), the combined image D ₆₇ assigned to the red channel and blue channel is obtained.

Video buffer 126 sequentially connecting the composite image _{D 67.} Also in the synthesized video D ₆₈ obtained by linking, the content of the code image D ₃ is switched every one shot period T _d (four frame periods) as in the first embodiment.

The light emitting unit 156 is disposed adjacent to the display screen 131 of the display unit 130. That is, the light emitting unit 156 is arranged so that the imaging unit 202 of the receiving device 200c can shoot with the same field of view as the display screen 131 of the display unit 130.

The light emitting unit 156 is configured by an LED or the like. Emitting unit 156, in synchronization with the generated clock signal CL at the reference clock generator 104, to emit light only just before and the period other than the period immediately after the timing of switching the code image D ₃ in the synthesis image D _68. For example, the period during which the light emitting unit 156 to emit light is switched immediately before and the frame period immediately after the code image D ₃ (guard period) other periods (body period). The light emitting unit 156 emits green light, for example. A monitor lamp having another role in the transmission device 100c may be used as the light emitting unit 156. For example, a monitor lamp used to indicate power on / off may be used as the light emitting unit 156.

In order to synchronize the light emission in the light emitting unit 156 with the output of the code images D ₃₁ and D _{32 in} the

code generating units

116A and 116B, the reading unit 112 uses one shot data (the code image D ₃ displayed in one shot period T _d). ) Is read each time a signal (synchronization signal SY) is output. The light emitting unit 156 receives the signal (synchronization signal SY) from the reading unit 112 and determines the timing of light emission.

As described above, the receiving apparatus 200c shown in FIG. 18 is generally the same as the receiving apparatus 200 of FIG. 5, but differs in that a light emission detecting unit 260 is provided instead of the mark recognizing unit 210.
The imaging unit 202 images the display screen 131 of the display unit 130 and the light emitting unit 156 of the transmission device 100c on which the video (the synthesized video D ₆₈ ) is displayed with the same field of view. Then, the imaging unit 202 acquires an image including the image (synthesized image D ₆₈ ) displayed on the display screen 131 and the light from the light emitting unit 156 by imaging.

As in the first embodiment, data (video data D ₇ ) obtained by imaging by the imaging unit 202 is accumulated in the video buffer 204. After all the data of one data file D ₀ is accumulated in the video buffer 204, the reading unit 206 reads the data one frame (frame data D ₈ ) at a time. The frame data D ₈ is separated by the separation unit 208. The green component value data is supplied to the light emission detection unit 260. The red component value data is supplied to the first cutout unit 222A. The blue component value data is supplied to the second cutout unit 222B.

Based on the green component value data provided from the separation unit 208, the light emission detection unit 260 determines whether or not the light emission unit 156 is reflected in each frame. Luminescent detection unit 260 (when the detection of luminescence) when it is determined that the reflected light-emitting state, it is determined that the code image D ₃ in the frame are those that can be correctly decoded. The light emission detection unit 260 transmits this determination result to the image cutout unit 220. Luminescent detection unit 260 instructs the extraction of the code image _{D 3} to the image clipping unit 220.
Image clipping unit 220 performs extraction of the code image _{D 3} in accordance with the instruction (image clipping signal _{D 10).} Thus, light emission code image D ₃ from the detected frame is cut.
Except for the above, the third embodiment is the same as the first embodiment.

Embodiment 4 FIG.
In the third embodiment, in which the light emitting unit 156 to emit different colors of light from the code image D _3. In the receiving device 200c, the light emission from the light emitting unit 156 is separated by the color of the image portion in the captured image. However, the light from the light emitting unit 156 may be separated based on the position of the light emitting unit 156 or the blinking timing. Such a fourth embodiment will be described below.

FIG. 19 shows a transmitting apparatus 100d according to the fourth embodiment of the present invention. FIG. 20 illustrates a receiving device 200d according to the fourth embodiment.

The transmission device 100d in FIG. 19 is generally the same as the transmission device 100 in FIG. However, the transmission device 100d includes a light emitting unit 158 instead of the mark generation unit 106, similarly to the transmission device 100c of FIG. The transmission device 100d includes a data processing unit 110d instead of the data processing unit 110. The transmission device 100d includes a combining unit 122d instead of the combining unit 122.

The data processing unit 110d is generally the same as the data processing unit 110 of FIG. 2, but includes a data dividing unit 114d instead of the data dividing unit 114. The data processing unit 110d includes a third code generation unit 116C in addition to the first code generation unit 116A and the second code generation unit 116B.

Constituent elements similar to those of the transmitting apparatus 100 described in Embodiment 1 are assigned the same reference numerals, and descriptions thereof are omitted. Constituent elements similar to those of the transmission device 100 are a transmission data storage unit 102, a reference clock generation unit 104, a video buffer 126, a moving image reproduction unit 128, and a display unit 130. Further, the reading unit 112, the first code generation unit 116A, and the second code generation unit 116B of the data processing unit 110 are the same components as those in the first embodiment.
Constituent elements similar to those of receiving apparatus 200 described in Embodiment 1 are assigned the same reference numerals, and descriptions thereof are omitted. The same components as the receiving apparatus 200 are an imaging unit 202, a video buffer 204, a reading unit 206, a separation unit 208, and a reception data storage unit 244. Further, the first cutout part 222A and the second cutout part 222B of the image cutout part 220d are the same constituent elements as those in the first embodiment. Further, the first decoding unit 232A and the second decoding unit 232B of the code decoding unit 230d are the same constituent elements as those in the first embodiment.

The data dividing unit 114d divides the data of each shot read by the reading unit 112 into three data parts. Then, the data dividing unit 114d gives the first data portion (first data unit D ₅₁ ) to the first code generating unit 116A. The data dividing unit 114d gives the second data portion (second data unit D ₅₂ ) to the second code generating unit 116B. The data dividing unit 114d gives the third data portion (third data unit D ₅₃ ) to the third code generating unit 116C.

As described above, the first code generation unit 116A and the second code generation unit 116B are the same as the first code generation unit 116A and the second code generation unit 116B in the first embodiment.
The third code generation unit 116C converts the third data unit D ₅₃ given from the data division unit 114 into a corresponding code image (third code image D ₃₃ ). That is, the third code generation unit 116C converts the third data unit D ₅₃ into a graphic code. The third code generation unit 116C further converts the graphic code into a code image (third code image D ₃₃ ) corresponding to the graphic code. The third code generating unit 116C outputs the third code image _{D 33.}
Thus, the third code generating unit 116C divides one shot data _{D 4} into three channels. Therefore, the third code generating unit 116C is the amount of data D ₄ in 1 shot, it can be made larger than that of the first embodiment.

Combining unit 122d includes a first code image _{D 31} outputted from the first code generating unit 116A, and the second code image _{D 32} outputted from the second code generating unit 116B, output from the third code generating unit 116C and a third code image D ₃₃ that is synthesized assigned to each independent color channel. Then, the synthesis unit 122d generates a synthesized image D ₆₁ (color image).
For example, the composition unit 122d performs composition by assigning the first code image to the red (R) channel. Then, the combining unit 122d performs the combining by assigning the second code image to the blue (B) channel. Then, the combining unit 122d performs the combining by assigning the third code image to the green (G) channel. As a result of such processing, a composite image D ₆₁ assigned to the red, blue and green channels is obtained.

Also in the composite video D ₆₃ obtained by sequentially connecting the composite images D ₆₁ , the contents of the code image D ₃ are switched every one shot period T _d (4 frame periods) as in the first embodiment.

The light emitting unit 158 is disposed adjacent to the display screen 131 of the display unit 130, similarly to the light emitting unit 156 of the third embodiment. And the light emission part 158 is arrange | positioned so that it can image | photograph with the same visual field as the display screen 131 of the display part 130 by the imaging part 202 of the receiver 200d.

The light emitting unit 158 is configured by an LED or the like, similar to the light emitting unit 156 of the third embodiment. Emitting unit 158 in synchronization with the reference clock generator 104 generates clock signals CL, emits light only immediately before and the period other than the period immediately after the timing of switching the code image D ₃ in the synthesis image. For example, the light emitting unit 158, immediately before and frame period (guard period) other periods (body period) immediately after the switching of the code image D ₃ only emits light. As described in the third embodiment, a monitor lamp having another role in the transmission device 100d may be used as the light emitting unit 158.
For example, the transmission device 100d may use a monitor lamp used to indicate power on / off as the light emitting unit 158.

Emitting portion 156 of the third embodiment has been to emit a different color from the color assigned to the first code image D ₃₁ and the second code image D _32. That is, the light emitting unit 156 emits green light in the above example. However, the light emitting unit 158 of Embodiment 4 does not have the above restriction on the color of the emitted light.

In order to synchronize the light emission in the light emitting unit 158 with the output of the code image D ₃₁ , the image D ₃₂ , and the image D _{33 in} the code generation unit 116A, the code generation unit 116B, and the code generation unit 116C, the reading unit 112 performs one shot of data every time reading the D _4, and outputs a signal (synchronization signal SY) indicating that. The light emitting unit 158 receives the signal (synchronization signal SY) from the reading unit 112 and determines the timing of light emission.

A receiving device 200d shown in FIG. 20 is generally the same as the receiving device 200 of FIG. 5, but includes a light emission detection unit 262 instead of the mark recognition unit 210. The receiving device 200d includes an image cutout unit 220d instead of the image cutout unit 220. The receiving apparatus 200d includes a code decoding unit 230d instead of the code decoding unit 230.
The image cutout unit 220d is different from the image cutout unit 220 in that it includes a third cutout unit 222C in addition to the first cutout unit 222A and the second cutout unit 222B.
The code decoding unit 230d is different from the code decoding unit 230 in that it includes a third decoding unit 232C in addition to the first decoding unit 232A and the second decoding unit 232B.

The imaging unit 202 images the display screen 131 of the display unit 130 and the light emitting unit 158 of the transmission device 100d on which an image is displayed with the same field of view. The imaging unit 202 acquires a video including light from the composite video D ₆₃ and the light emitting unit 158 displayed on the display screen 131 by imaging.

As in the first embodiment, data obtained by imaging by the imaging unit 202 is accumulated in the video buffer 204. After all the data of one data file D ₀ is accumulated in the video buffer 204, the reading unit 206 reads the data one frame at a time. The frame data D ₈ read by the reading unit 206 is supplied to the separation unit 208. The frame data D ₈ read by the reading unit 206 is also supplied to the light emission detection unit 262.
The separation unit 208, the data _{D 8} frames from the read unit 206, the red component data _{D 9R,} is separated into data _{D 9G} data _{D 9B} and green component and blue component. Then, they are supplied to the first cutout part 222A, the second cutout part 222B, and the third cutout part 222C, respectively. That is, the red component data D _9R is supplied to the first cutout unit 222A. Data _{D 9B} of the blue component is supplied to the second cutout portion 222B. Data _{D 9G} green component is supplied to a third cutout portion 222C.

Luminescent detection unit 262 sequentially receives the data D ₈ of each frame supplied from the reading unit 206. Luminescent detection unit 262 analyzes the image composed of data D ₈ frames sequentially received. Then, the light emitting detection unit 262, a code image D ₃ included in the image of each frame, to identify the position of the light from the light emitting portion 158 in the image. Here, the “light position” is an image portion corresponding to the light from the light emitting unit 158. The light emission detection unit 262 identifies the position of the light from the light emitting unit 158 in the video from the positional relationship of the light from the light emitting unit 158 and the blinking timing of the light emitting unit 158 in the video. Then, the light emission detection unit 262 determines whether or not the light emission unit 158 is reflected in the data D ₈ of each frame. When it is determined that the image is captured in a light-emitting state (when light emission is detected), the light emission detection unit 262 determines that the code image in that frame can be correctly decoded. The light emission detection unit 262 transmits the determination result to the image cutout unit 220d. Then, the light emission detecting unit 262 instructs the extraction of the code image _{D 3} to the image clipping unit 220d (image clipping signal _D 10).

Image clipping unit 220d performs extraction of the code image _{D 3} in accordance with the instruction (image clipping signal _{D 10).} Thus, light emission code image D ₃ from the detected frame is cut. That is, the first cutout unit 222A cuts out the first code image _D11R from the red component image of the frame in which the light emission is detected.
The second cutout unit 222B cuts out the second code image _D11B from the blue component image of the frame in which the light emission is detected.
The third cutout unit 222C cuts out the third code image _D11G from the green component image of the frame in which the light emission is detected.

Code decoder 230d decodes the code image _{D 11} contained in the cut out images in the image clipping unit 220d. The code decoder 230d obtains the decoded if already decoded data D ₁₂ color units corresponding to the code image D ₁₁ which is the decoded.
The first decoding unit 232A and the second decoding unit 232B are the same as the first decoding unit 232A and the second decoding unit 232B shown in FIG.
Third decoding unit 232C decodes the third code image _{D 11G} contained in the cut out images in the third cutout portion 239C. Then, the third decoding unit 232C generates decoded data D _12C that has been decoded in units of colors.

The synthesizing unit 242d combines the decoded data D _12R , D _12B , and D _12C in units of colors output from the first decoding unit 232A, the second decoding unit 232B, and the third decoding unit 232C. Then, the synthesizing unit 242d synthesizes the decoded data D _12R , D _12B , and D _12C of these color units to restore the transfer data D ₁₃ for one shot.
Except for the above, the fourth embodiment is the same as the first embodiment.

Embodiment 5 FIG.
In the first embodiment, the data D _{4 of} each shot is divided into a plurality of data portions (data units D ₅ ), each of which is assigned to a different color channel, and a code image D ₃ of a different color is generated and displayed. Yes. However, color channel used to generate the code image D ₃ may be one. Such a fifth embodiment will be described below.
FIG. 21 shows a transmitting apparatus 100e according to the fifth embodiment of the present invention. FIG. 22 shows a receiving apparatus 200e of the fifth embodiment.

21 is generally the same as the transmission device 100 of FIG. 2, but includes a data processing unit 110e instead of the data processing unit 110. The transmission device 100e includes a combining unit 122e instead of the combining unit 122.

22 is generally the same as the receiving apparatus 200 of FIG. 5, but includes a separating unit 208e instead of the separating unit 208. The receiving device 200e includes an image cutout unit 220e instead of the image cutout unit 220. The receiving device 200e includes a code decoding unit 230e instead of the code decoding unit 230. The receiving device 200e is not provided with the combining unit 242.

Constituent elements similar to those of the transmitting apparatus 100 described in the first embodiment are denoted by the same reference numerals and description thereof is omitted. Constituent elements similar to those of the transmission device 100 are a transmission data storage unit 102, a reference clock generation unit 104, a mark generation unit 106, a video buffer 126, a moving image reproduction unit 128, and a display unit 130. Further, the image storage unit 107 of the mark generation unit 106 is a component similar to that of the first embodiment.

Constituent elements similar to those of the receiving apparatus 200 are an imaging unit 202, a video buffer 204, a reading unit 206, a mark recognition unit 210, a combining unit 242, and a received data storage unit 244.

Unlike the data processing unit 110 in FIG. 2, the data processing unit 110 e includes a single code generation unit 116. Further, the data processing unit 110e is not provided with the data dividing unit 114. Without data D ₄ of each shot from the reading unit 112e is divided into a plurality of channels, supplied to the code generator 116.

The code generator 116 is the same as the first code generator 116A or the second code generator 116B in FIG. That is, the code generation unit 116 has the same function as the first code generation unit 116A or the second code generation unit 116B in FIG.
Combining section 122e includes a timing mark _{D 2} from the mark generator 106, and a code image _{D 3} from the code generation unit 116, respectively synthesized allocated to green (G) channel, and red (R) channel.
The output of the combining unit 122e (the combined image D ₆₁ ) is supplied to the video buffer 126.
Except for the above, the operation of the transmission device 100e in FIG. 21 is the same as that of the transmission device 100 in FIG.

Separation unit 208e includes data D ₈ of frames read by the reading unit 206, separates the red channel, the green channel.
The image cutout unit 220e has a single image cutout unit 222. Image clipping unit 222 cuts out a code image _{D 11R} from the data D9 _R of the red component values of the red channel from the separation unit 208e.
The code decoding unit 230e has a decoding unit 232. Code decoder 232 decodes the code image _{D 11R} cut out by the image cutting out unit 222.
The output of the code decoding unit 232 (decoded decoded data D _{12R in} color units) is supplied to the received data storage unit 244.
Except for the above, the operation of the receiving device 200e in FIG. 22 is the same as that of the receiving device 200 in FIG.

Embodiment 6 FIG.
In the first embodiment, and to send a timing mark D ₂ with different color channels and the code image D _3. However, there are cases where such a method is not suitable. For example, such a method cannot be performed when either the display unit 130 or the imaging unit 202 is monochrome. In addition, such a method cannot be performed in the case of a system in which colors are separated using a rotary color filter and read sequentially for each color. Further, even when the color channel can more available, there is a case you want to assign all the color channels in the code image D _3.

In these cases, it is also possible to use a figure enclosing the code image D ₃ as a timing mark D _2. Examples of this case are shown in FIGS. 23, 24A, 24B, and 24C. Figure 23 is a schematic diagram showing a layout of a timing mark D ₂ and code image D _3. 24A, 24B, and 24C are schematic diagrams illustrating temporal changes in video displayed on the display unit 130 of the transmission device 100. FIG. FIG. 24 (A) represents a timing mark _{D 2.} FIG. 24 (B) represents a code image _{D 3.} FIG. 14 (C) represents the layout of the timing mark _{D 2} and the code image _{D 3} in each period. 24A and 24B, the vertical axis indicates whether or not each signal is output. A signal is output at “H” and no signal is output at “L”. In FIGS. 24A and 24B, the horizontal axis represents time.

In the example shown in FIG. 23, and a frame-shaped figure timing mark D ₂ surrounds the code image D _3.
24A, 24B, and 24C are schematic diagrams illustrating temporal changes in video displayed on the display unit 130 of the transmission device 100. FIG. Figure 24 (A) and as shown in FIG. 24 (B), in a period _{_{_{T 1, T 3, T 4}}} , T 6, the timing mark _{D 2} is not displayed. Therefore, in these periods _{_{_{T 1, T 3, T 4}}} , T 6, the decoded code image _{D 3} is not performed. On the other hand, in a period _{T 2} and time _{T 5,} the timing mark _{D 2} are displayed. Therefore, during these periods _T 2, _{T 5,} the decoded code image _{D 3} is performed.

The frame code image D ₃ taken at the moment of switching, the code image D ₃ is sometimes incomplete. However, if the entire timing mark D ₂ is taken it can be regarded as being read even code image located inside the timing mark D ₂ correctly.
By using such timing marks D _2, it can be read automatically and efficiently code image D ₃ which is continuously and sequentially displayed.

Sixth and seventh embodiments to be described below are those with reference to the timing mark _{D 2} shown in FIG. 23 and FIG. 24 (A) ~ FIG 24 (C), performs clipping of the code image _{D 3.}

FIG. 25 shows a transmission device 100f according to the sixth embodiment. FIG. 26 illustrates a receiving device 200f according to the sixth embodiment.

A transmission apparatus 100f shown in FIG. 25 is generally the same as the transmission apparatus 100e of FIG. 21, but includes a mark generation unit 106f instead of the mark generation unit 106.
The transmission device 100f includes a combining unit 122f instead of the combining unit 122e.

The receiving device 200f shown in FIG. 26 is generally the same as the receiving device 200e in FIG. 22, but includes a separating unit 208f instead of the separating unit 208e. The receiving device 200f includes a mark recognition unit 210f instead of the mark recognition unit 210.

Constituent elements similar to those of the transmitting apparatus 100 described in the first embodiment are denoted by the same reference numerals and description thereof is omitted. Constituent elements similar to those of the transmission device 100 are a transmission data storage unit 102, a reference clock generation unit 104, a video buffer 126, a moving image reproduction unit 128, and a display unit 130. The image storage unit 107 of the mark generation unit 106f is the same component as that in the first embodiment.

The same components as those of the receiving apparatus 200 described in Embodiment 1 are denoted by the same reference numerals, and the description thereof is omitted. The same components as the receiving apparatus 200 are an imaging unit 202, a video buffer 204, a reading unit 206, and a received data storage unit 244.

Mark generation unit 106f generates a shape surrounding the code image D ₃ generated by the code generation unit 116 as a timing mark D _2. For example, as shown in FIG. 23, to generate a frame-like timing mark D ₂ which is formed to surround the code image D _3.

Combining unit 122f synthesizes the timing mark _{D 2} and code image _{D 3} of the frame-shaped.
In the configuration of FIG. 21, it is assigned a code image D ₃ and the timing mark D ₂ in different color channels. However, in the configuration of FIG. 25, only one color channel is used. The code image D ₃ and the timing mark D ₂ represented by the pixel values of the same color. In FIG. 25, both are shown as representing the luminance value Y.
In other respects, the operation of the transmission device 100f in FIG. 25 is the same as that of the transmission device 100e in FIG.

The separation unit 208f analyzes the composite image (frame data D ₈ ), and separates the code image D ₃ located in the center and the timing mark D ₂ located around the code image D ₃ . Separation unit 208f on the basis of the shape and relative position, separated from the code image _{D 3,} and a timing mark _{D 2.} Separation unit 208f supplies the timing mark _{D 2} generated by the separation in the mark recognition unit 210f (data _{D 91Y).} Separation unit 208f supplies the code image _{D 3} in cutout section 222 (data _{D 92Y).}
The image clipping unit 222, the code image _{D 92Y} from the separation unit 208f, cuts out a code image _{D 93Y.} The code decoding unit 232 decodes the code image _D93Y cut out by the image cutout unit 222. The output (decoded data D _94Y ) of the code decoding unit 232 is supplied to the received data storage unit 244.

The mark recognition unit 210f receives data D _91Y (timing mark D ₂ ) from the separation unit 208f. Mark recognition unit 210f confirms that there is no lack of timing marks _{D 2} supplied from the separation unit 208f. Mark recognition unit 210f, the code image D ₃ in the same frame that was contained timing mark D ₂ is determined to be suitable for decoding. Mark recognition unit 210f instructs the extraction of the code image _{D 3} to clipping unit 222 (image clipping signal _D 10).
In other respects, the operation of the receiving device 200f in FIG. 26 is the same as that of the receiving device 200e in FIG.

Embodiment 7 FIG.
FIG. 27 shows a configuration of transmitting apparatus 100g according to the seventh embodiment. FIG. 28 shows a configuration of receiving apparatus 200g according to the seventh embodiment.

27 is generally the same as the transmission device 100 of FIG. 2, but includes a mark generation unit 106A and a mark generation unit 106B instead of the mark generation unit 106. In addition, the transmission device 100g includes a combining unit 122g instead of the combining unit 122.

Constituent elements similar to those of the transmitting apparatus 100 described in the first embodiment are denoted by the same reference numerals and description thereof is omitted. Constituent elements similar to those of the transmission device 100 include a transmission data storage unit 102, a reading unit 112, a data division unit 114, a first code generation unit 116A, a second code generation unit 116B, a reference clock generation unit 104, a video buffer 126, a moving image A playback unit 128 and a display unit 130. Further, the image storage units 107A and 107B of the

mark generation units

106A and 106B are the same components as the image storage unit 107 of the first embodiment.

In the first embodiment, the timing mark _{D 2} is assigned a code image _D _{31, D 32} to a different color channels. In the seventh embodiment, the timing mark D ₂₄ is assigned to the same color channel as the code image D ₃₁ . The timing mark _{D 25} is assigned the same color channel as the code image _{D 32.} In this respect, the seventh embodiment is the same as the sixth embodiment.

Mark generating unit 106A generates a shape surrounding the code image _{D 31} generated by the first code generating unit 116A as a timing mark _{D 24.} For example, when the code image D ₃₁ generated by the first code generation unit 116A is indicated by the code D ₃ (code image D ₃ ) in FIG. 23, the mark generation unit 106A displays the code image D ₃ generating a frame-like timing mark D ₂₄ which is formed to surround.
Mark generation unit 106B generates a shape surrounding the code image _{D 32} generated by the second code generating unit 116B as a timing mark _{D 25.} For example, when the code image D ₃₂ generated by the second code generation unit 116B is indicated by the code D ₃ (code image D ₃ ) in FIG. 23, the mark generation unit 106B converts the code image D ₃ into the code image D ₃ . generating a frame-like timing mark D ₂₅ which is formed to surround.

Note that the timing mark D ₂₄ generated by the mark generation unit 106A and the timing mark D ₂₅ generated by the mark generation unit 106B may have the same shape and size. Further, the timing mark D ₂₄ generated by the mark generation unit 106A and the timing mark D ₂₅ generated by the mark generation unit 106B may have different shapes or sizes.

Combining section 122g generates a first combined image _{D 69} by combining a frame-like timing mark _{D 24} from the mark generating unit 106A and the code image _{D 31} from the first code generating unit 116A. This first composite image _D69 is assigned to the first color channel. For example, the first composite image _D69 is assigned to the red channel.
Combining section 122g generates a second composite image _{D 70} by combining a frame-like timing mark _{D 25} from the mark generating unit 106B and the code image _{D 32} from the second code generating unit 116B. Assigning the second combined image D ₇₀ to a second color channel. For example, assigning a second combined image D ₇₀ in the blue channel.
Then, the composition unit 122g synthesizes a composite image _{D 69} and the composite image _{D 70.} Combining unit 122g outputs the combined image _{D 71} of the composite image _{D 69} and the composite image _{D 70} synthesized color image.

The video buffer 126 stores the composite image D ₇₁ generated by the combining unit 122. Then, the video buffer 126 connects the accumulated composite image D ₇₁ to generate a composite video D ₇₃ (moving image).

Other than the above, the operation of the transmission device 100g in FIG. 27 is the same as that of the transmission device 100 in FIG.

The receiving device 200g shown in FIG. 28 is generally the same as the receiving device 200 in FIG. However, the receiving device 200g includes a separation unit 208g instead of the separation unit 208. The receiving device 200g includes

mark recognition units

210A and 210B instead of the mark recognition unit 210.

The same components as those of the receiving apparatus 200 described in Embodiment 1 are denoted by the same reference numerals, and the description thereof is omitted. Constituent elements similar to those of the receiving apparatus 200 are an imaging unit 202, a video buffer 204, a reading unit 206, an image cutout unit 220, a code decoding unit 230, a combining unit 242, and a received data storage unit 244. The first cutout part 222A and the second cutout part 222B of the image cutout part 220 are the same constituent elements as those in the first embodiment. The first decoding unit 232A and the second decoding unit 232B of the code decoding unit 230 are the same constituent elements as those in the first embodiment.

Separation unit 208g has data _{D 8} of each frame is read by the reading unit 206, and the red component value of the data _{D 9R} (data of red channel), a blue component value data _{D 9B} (blue channel data ) And separated.

The separation unit 208g further analyzes the red channel data (red component value data D _9R ). Then, the separation unit 208g includes a code image D ₃ located in the center, separating the timing mark D ₂ positioned therearound. Separation of the code image D ₃ and the timing mark D ₂ is carried out based on the shape and relative position. Separation unit 208g supplies the timing mark _{D 2} generated by the separation mark recognition unit 210A (data _{D 91R} color component values). Separation unit 208g supplies the code image _{D 3} produced by the separation in the first cutout portion 222A (data _{D 92R} color component values).

The separation unit 208g also analyzes the blue channel data (blue component value data D _9B ). Then, the separation unit 208g includes a code image D ₃ located in the center, separating the timing mark D ₂ positioned therearound. Separation of the code image D ₃ and the timing mark D ₂ is carried out based on the shape and relative position. Separation unit 208g supplies the timing mark _{D 2} generated by the separation mark recognition unit 210B (data _{D 91B} color component values). Separation unit 208g supplies the code image _{D 3} produced by the separation in the second cutout portion 222B (data _{D 92B} color component values).

Upon receipt of the color component value data D _91R (timing mark D ₂ ) from the separation unit 208g, the mark recognition unit 210A confirms that the timing mark D ₂ supplied from the separation unit 208g is not missing. Then, when there is no lack of timing mark _{D 2} are mark recognition unit 210A instructs the extraction of the image with respect to cutout portion 222A (image clipping signal _{D 10R).}
Upon receipt of the color component value data D _91B (timing mark D ₂ ) from the separation unit 208g, the mark recognition unit 210B confirms that the timing mark D ₂ supplied from the separation unit 208g is not missing. Then, when there is no lack of timing mark _{D 2} are mark recognition unit 210B instructs the extraction of the image with respect to cutout portion 222B (image clipping signal _{D 10B).}

The first cutout portion 222A receives the data _{D 10R} of the red component from the mark recognition unit 210A. The first cutout unit 222A cuts out the first code image from the red component data (video) D _92R given from the separation unit 208g at the timing instructed by the mark recognition unit 210A.
The second cutout portion 222B receives the data _{D 10B} of the blue component from the mark recognition unit 210B. The second cutout unit 222B cuts out the second code image from the blue component data (video) D _92B given from the separation unit 208g at the timing instructed by the mark recognition unit 210B.

Other than the above, the operation of the receiving device 200g in FIG. 28 is the same as that of the receiving device 200 in FIG.

In the above example, a separate mark generator (106A, 106B) is provided for each color channel. However, provided a single mark generating unit for a plurality of color channels, it is also possible to add a timing mark D ₂ generated by the single mark generation unit to the plurality of code image D _3.

Further, in the above example, it is added to generate a separate timing mark D ₂ with respect to a plurality of color channels. However, in the transmission apparatus 100 g, transmits given the timing mark D ₂ in the code image D ₃ of the one of the plurality of color channels. Then, the receiving apparatus 200 g, by detecting the timing mark D ₂ attached to the code image D ₃ of one color channel may be also be used for extraction of the code image D ₃ of the other color channels.

On the other hand, as shown in FIGS. 27 and 28, when generating a timing mark D ₂ in each color channel, the communication using the plurality of color channels, is not necessarily performed in synchronization. Timing of switching of the code image D ₃ to be transmitted may be used as the different between the color channels each other.

Embodiment 8 FIG.
When considering application to signage etc., instead of starting transfer by user's operation, it should be displayed repeatedly so that the user can shoot at any timing. Convenient. FIG. 29 shows an example for enabling such use. Figure 29 is a diagram showing an example of a sequence of QR codes marked with different kinds of timing marks D _2. In the example of FIG. 29, are three types used as timing marks D _2. The first mark is a timing mark D _2h with start mark to be used at the start of the transfer of the series of data (data file D _0). That is, it is a first identification image representing the head portion of a series of data (data file D ₀ ) to be transferred. The second mark is a timing mark D _2m used in the middle part. That is, it is an identification image representing a portion other than the beginning portion and the ending portion of a series of transferred data (data file D ₀ ). The third mark is a timing mark _D2t with an end mark used at the end of a series of data transfer. That is, it is an identification image representing the end portion of a series of transferred data (data file D ₀ ). Although not shown in FIG. 29, it may have a separate timing mark D ₂ for small data as transferred in one of the display is completed. As a result, when the transfer of the data file D ₀ is completed in one shot, it takes time to transfer the data file D ₀ if the start mark and the end mark are sent. Thus, by creating a dedicated mark when transferring data file D ₀ in one shot, it is possible to shorten the transfer time of transferring the small data.

With such a configuration, it is possible to automatically recognize the start and end points of the receiving apparatus 200 in the data file D _0. The transmission device 100 divides the data file D ₀ stored in the transmission data storage unit 102 and repeatedly displays it. Combining unit 242 of the receiving apparatus 200 combines the divided data to reconstruct the original data file D _0. At the time of reconstruction of the data file D _0, the starting point and end point of the data file D ₀ is required. From among the divided data is repeatedly displayed, by combining the data from the start point to the end point, it is possible to reproduce the original data file D _0. Here, “divided data” is described as a QR code in the first embodiment. And a user's operation can be simplified.
For example, when the time (transfer time) required to transfer a series of data (data file D ₀ ) is 2 seconds, a time that is twice or more of 2 seconds, which is the transfer time, is determined in advance. For example, a time of 5 seconds is determined in advance. Here, the "transfer time" is that time that the display unit 130 all of the code image D ₃ representing the data file D ₀ is displayed. The transmission device 100h is made to repeat the transfer of a series of data (data file D ₀ ). Then, a message “Please shoot for 5 seconds” is displayed to the user. When the receiving apparatus 200 starts photographing by a user's operation, the start mark to the end mark are stored as a series of data for the repeatedly displayed video. Decoding is started after a series of data is stored in receiving apparatus 200.

Also, storage can be started immediately in response to user operation. The receiving apparatus 200 ends the storage when the start mark is detected and then the end mark is detected. Then, the receiving apparatus 200 may decode the stored data from the start mark to the end mark.

Further, it is convenient not only when the same data is repeatedly displayed, but also when transferring a plurality of data sequentially. In this case, indicates increase the type of timing mark D _2, the timing marks D ₂ and end indicate the start of a whole in addition to the timing mark D ₂ showing the timing mark D ₂ and end indicating the beginning of individual data it may be used timing mark D _2.

In the example shown in FIG. 29 using the timing marks D ₂ expressed outside the coding region in the linear or rectangular shape. In such a case, when the display unit 130 is of the progressive scanning point, timing marks D _2, in the sub-scanning direction, begins before the start of the coding region, from the end point of the coding region It is desirable to use what ends later. Further, when the display unit 130 is of the line-sequential scanning scheme, the timing mark D _2, in the scanning direction (direction perpendicular to the line), begins before the start of the coding region, the end of the coding region It is desirable to use one that ends after the point. It may be adapted to draw a timing mark D ₂ in accordance with the scan mode of the display unit 130. This is because a video during the rewriting of the display may be shot due to a difference between the display frame and the shooting frame. By drawing this way the timing mark D _2, if it is the entire timing marks D ₂ is captured correctly, it can be expected that coding region is properly captured.
In the example of FIG. 29, it is assumed that the sub-scanning direction of the dot sequential scanning method is from the upper side to the lower side. Alternatively, it is assumed that the scanning direction of the line sequential scanning method is from the upper side to the lower side. Timing mark D ₂ as shown in FIG. 29 is drawn as a figure ending the above coding region below the start coding region. In other words, the timing marks D ₂ is drawn from above the coding region, a figure drawn to below the coding region. Here, “drawing” means drawing a picture.

FIG. 30 is a block diagram schematically showing the configuration of the transmission device 100h. FIG. 31 is a block diagram schematically showing the configuration of the receiving device 200h.
Hereinafter, a head portion of a set of data, and the end portion, in the other portions, and generates a different timing mark D _2, shown in Figure 30 an example of the transmission device 100h for combining a code image D _3.
Further, the head portion of the series of data, and the end portion, in the other portions, and receives the image generated by concatenating the code image D ₃ that is added is different from timing mark D _2, the recording An example of a receiving device 200h that performs control is shown in FIG.

30 is generally the same as the transmission device 100 of FIG. 2, but includes a data processing unit 110h instead of the data processing unit 110. The data processing unit 110 h is generally the same as the data processing unit 110 in FIG. 2, but includes a reading unit 112 h instead of the reading unit 112. The data processing unit 110h includes a mark generation unit 106h instead of the mark generation unit 106.

Constituent elements similar to those of the transmitting apparatus 100 described in the first embodiment are denoted by the same reference numerals and description thereof is omitted. Constituent elements similar to those of the transmission device 100 are a transmission data storage unit 102, a reference clock generation unit 104, a synthesis unit 122, a video buffer 126, a moving image reproduction unit 128, and a display unit 130. The data dividing unit 114, the first code generation unit 116A, and the second code generation unit 116B of the data processing unit 110h are the same constituent elements as those in the first embodiment. Further, the image storage unit 107 of the mark generation unit 106 is a component similar to that of the first embodiment.

The reading unit 112h supplies a signal (synchronization signal SY _h ) indicating that when reading the head of a series of data (data file D ₀ ) from the transmission data storage unit 102 to the mark generation unit 106h. Further, the reading unit 112h supplies a signal (synchronization signal SY _t ) indicating that when reading the end portion of a series of data (data file D ₀ ) from the transmission data storage unit 102 to the mark generation unit 106h.
Based on the above-mentioned signals (synchronization signals SY _h , SY _t ) from the reading unit 112 _h , the mark generation unit 106 _{h is configured to} include a head portion, a tail portion, and other portions of a series of data (data file D ₀ ). generating different timing mark D ₂ in.

That is, the mark generating unit 106h generates a first timing mark D _2h corresponding to the beginning of the data portion of the series of data (data file D _0). First timing mark _{D 2h} is combined with the first code image _{D 31} and the second code image _{D 32} of the first set of data (data file _{D 0).} An example of the first timing mark _{D 2h} is indicated with _{D 2h} in Figure 29.

Mark generating unit 106h generates a second timing mark D _2t corresponding to the end of the data part of the series of data (data file D _0). The second timing mark D _2t is combined with the first code image D ₃₁ and the second code image D ₃₂ at the end of the series of data (data file D ₀ ). The second timing mark D _2t is a mark different from the first timing mark D _2h . An example of the second timing marks _{D 2h} is indicated with _{D 2t} Figure 29.

Mark generating unit 106h generates a third timing mark D _2m corresponding to the head and a data portion other than the end of the series of data (data file D _0). The third timing mark D _2m is combined with the first code image D ₃₁ other than the beginning and end of a series of data (data file D ₀ ). The third timing mark _{D 2m} is combined with the second code image _{D 32} top and non-end of the series of data (data file _{D 0).} The third timing mark D _2m is a mark different from the first timing mark D _2h . The third timing mark _D2m is a mark different from the second timing mark _D2t . An example of the third timing mark D _2m is indicated by a symbol D _2m in FIG.

31 is generally the same as the receiving apparatus 200 of FIG. 5, but includes a mark recognizing unit 210h instead of the mark recognizing unit 210.
Constituent elements similar to those of receiving apparatus 200 described in Embodiment 1 are assigned the same reference numerals, and descriptions thereof are omitted.
The same components as the receiving apparatus 200 are an imaging unit 202, a reading unit 206, a separation unit 208, an image clipping unit 220, a code decoding unit 230, a combining unit 242, and a received data storage unit 244. Further, the first cutout part 222A and the second cutout part 222B of the image cutout part 220 are the same constituent elements as those in the first embodiment.
Further, the first decoding unit 232A and the second decoding unit 232B of the code decoding unit 230 are the same constituent elements as those in the first embodiment.
Mark recognition unit 210h receives the video data _{D 7} which are sequentially output from the imaging unit 202. Mark recognition unit 210h, upon detecting a first timing mark _{D 2h} indicating the beginning to start stored in the video buffer 204 (memory start signal _{D 14h).} Mark recognition unit 210h ends the stored in the video buffer 204 when it detects the second timing mark _{D 2t} indicating the end (storage completion signal _{D 14t).} As a result, the video buffer 204 stores a series of data (data file D ₀ ) from the beginning to the end. The storage start signal _{D 14h} and storing end signal _{D 14t} collectively called a memory timing signal _{D 14.}

Although the second embodiment has been described as a modification of the first embodiment, similar modifications can be made to the third to eighth embodiments. In addition, although it has been described that various modifications are made to the transmission device 100 and the reception device 200 of the first embodiment, the same modifications can be applied to the second to eighth embodiments.
Also relates to the eighth embodiment, the timing mark D _2, when the display unit 130 is of the progressive scanning point, in the sub-scanning direction, begins before the start of the coding region, the end point of the coding region He said it would be desirable to use something that ends later. The timing mark D _2, when the display unit 130 is of the line-sequential scanning scheme, in the scanning direction (direction perpendicular to the line), begins before the start of the coding region, the end of the coding region He stated that it would be desirable to use something that ends after the point. However, this is the same for the first, second, fifth, and seventh embodiments.

As described above, the display unit 130 of the

transmission devices

100, 100a, 100c, 100d, 100e, 100f, 100g, and 110h is a liquid crystal display device that performs display by a dot sequential scanning method or a line sequential scanning method and a progressive display method. explained. However, the present invention can also be applied to the case where the display unit 130 is of the frame sequential scanning method, or can be applied to the case of the interlaced display method.

Each of the

transmission devices

100, 100a, 100c, 100d, 100e, 100f, 100g, and 110h and the

reception devices

200, 200c, 200d, 200e, 200f, 200g, and 200h can be partially realized by software. . That is, each of the

transmission devices

100, 100a, 100c, 100d, 100e, 100f, 100g, and 110h and the

reception devices

200, 200c, 200d, 200e, 200f, 200g, and 200h can be realized by a programmed computer. In that case, the processing performed by a part of the

transmission devices

100, 100a, 100c, 100d, 100e, 100f, 100g, 110h and the

reception devices

200, 200c, 200d, 200e, 200f, 200g, 200h is performed by a computer. A program for causing the program to be executed is also part of the present invention. A computer-readable recording medium that records such a program also forms part of the present invention.

In addition, although embodiment of this invention was described as mentioned above, this invention is not limited to these embodiment.

1 Information processing system, 100, 100c, 100d, 100e, 100f, 100g, 110h Transmission device, 102 Transmission data storage unit, 104 Reference clock generation unit, 106, 106f, 106h Mark generation unit, 107 Image storage unit, 110, 110d , 110e data processing unit, 112, 112h reading unit, 114, 114d data dividing unit, 116A first code generation unit, 116B second code generation unit, 116C third code generation unit, 122, 122b, 122c, 122d, 122e, 122f, 122g composition unit, 126, 126A, 126B, 126C, video buffer, 128, 128A, 128B, 128C video playback unit, 130 display unit, 131 display screen, 156, 158 Light emitting unit, 200, 200c, 200d, 200e, 200f, 200g, 200h receiving device, 202 imaging unit, 204 video buffer, 206 reading unit, 208 separation unit, 210 release mark recognition unit, 220, 220d, 220e image cutout unit, 222A first cutout unit, 222B second cutout unit, 222C third cutout unit, 230, 230d, 230e code decoding unit, 232A first decoding unit, 232B second decoding unit, 232C third decoding unit, 242, 242d combining unit 244 reception

data storage unit

260, 262 light emission detection unit.

Claims

A transmission data storage unit for storing a series of data;
The series of data stored in the transmission data storage unit is divided into a plurality of data parts, each of the data parts obtained by the division is converted into a code image, and each data part generated by the conversion A code generator that sequentially outputs code images corresponding to
An identification image generator for generating and outputting an identification image;
A combining unit that combines the code image generated by the code generation unit with the identification image output from the identification image generation unit to generate a combined image;
An image connecting unit that connects the composite images generated by the combining unit to create a moving image corresponding to the series of data;
A video playback unit that plays back and outputs the video created by the image connection unit;
A display unit that displays the video played by the video playback unit on a screen;
In the moving image output from the moving image reproducing unit, the code images are sequentially switched, and the identification image appears at a timing other than the switching timing.
The transmission apparatus according to claim 1, wherein in the moving image, the code image and the identification image are images of different colors.
A transmission data storage unit for storing a series of data;
The series of data stored in the transmission data storage unit is divided into a plurality of data parts, each of the data parts obtained by the division is converted into a code image, and each data part generated by the conversion A code generator that sequentially outputs code images corresponding to
A first image connecting unit that connects the code images generated by the code generating unit to create a moving image;
A first moving image reproduction unit that reproduces and outputs the moving image created by the first image connection unit;
An identification image generator for generating and outputting an identification image;
A second image connecting unit that connects the identification images generated by the identification image generating unit to create a moving image;
A second moving image reproduction unit that reproduces and outputs the moving image created by the second image connecting unit;
A synthesis unit that synthesizes the video created by the first video playback unit and the video created by the second video playback unit to generate and output a composite video;
A display unit that displays the synthesized video generated by the synthesis unit on a screen;
In the synthesized video output from the synthesizing unit, the code images are sequentially switched, and the identification image appears at a timing other than the timing of the switching.
The transmission apparatus according to claim 3, wherein, in the synthesized video, the code image and the identification image are images of different colors.
The identification image generation unit
As the identification image used for the synthesis with the code image obtained by converting the data portion at the beginning or the end of the series of data, a first identification image is generated,
A second identification image different from the first identification image is generated as the identification image used for synthesis with the code image obtained by converting the data portion other than the head or tail of the series of data. The transmission device according to any one of claims 1 to 4, wherein:
The display on the screen is performed by a dot sequential scanning method or a line sequential scanning method,
The identification image is a figure that starts before and ends after the code image combined with the identification image in the sub-scanning direction in the dot sequential scanning method or the scanning direction in the line sequential scanning method. The transmission device according to any one of claims 1 to 5.
A transmission data storage unit for storing a series of data;
The series of data stored in the transmission data storage unit is divided into a plurality of data parts, each of the data parts obtained by the division is converted into a code image, and each data part generated by the conversion A code generator that sequentially outputs code images corresponding to
An image connecting unit that connects the code images generated by the code generating unit to create a moving image;
A video playback unit that plays back and outputs the video created by the image connection unit;
A display unit for displaying the video played by the video playback unit on a screen;
A transmission device comprising: a light emitting unit that emits light at a timing other than a timing at which the code image is switched in a moving image displayed on the display unit.
The transmission device according to claim 7, wherein the light emitting unit emits light having a color different from a color of the code image included in the moving image displayed on the display unit.
An imaging unit that sequentially switches the code image, captures a screen displaying a video in which the identification image appears at a timing other than the switching timing, and sequentially acquires the image of the screen;
A video buffer for storing images sequentially acquired by the imaging unit;
A reading unit for sequentially reading out images stored in the video buffer;
An identification image detection unit for sequentially detecting the identification images in images sequentially read by the reading unit;
A code decoding unit that decodes the code image that appears at the same timing as the identification image detected by the identification image detection unit, and sequentially acquires corresponding data portions;
A reception apparatus comprising: a reception data storage unit configured to sequentially write data portions sequentially acquired by the code decoding unit and to connect the sequentially written data portions to form a series of data.
The video displayed on the screen is generated by dividing the series of data into a plurality of data parts and converting each data part into a code image,
As the identification image,
A first identification image is displayed together with a code image obtained by converting the data portion at the beginning or end of the series of data,
A second identification image different from the first identification image is displayed together with a code image obtained by converting a data portion other than the beginning or end of the series of data,
The receiving apparatus according to claim 9, wherein the identification image detection unit controls start or end of decoding of the series of data when the first identification image is detected as the identification image. .
The display unit performs display by a dot sequential scanning method or a line sequential scanning method,
The identification image detection unit starts and ends after the code image combined with the identification image in the sub-scanning direction in the dot sequential scanning method or the scanning direction in the line sequential scanning method. The receiving apparatus according to claim 9 or 10, wherein, when it is determined to be a figure, it is determined that the code image combined with the identification image is suitable for decoding.
A screen that displays code images that are switched in sequence,
The light from the light emitting unit that is arranged adjacent to the screen and emits light at a timing other than the switching timing is captured, and the image displayed on the screen and the light image from the light emitting unit are sequentially acquired. An imaging unit;
A video buffer for accumulating images sequentially acquired by the imaging unit;
A reading unit for sequentially reading out images stored in the video buffer;
A detection unit for detecting an image portion by light from the light emitting unit in images sequentially read by the reading unit;
A code decoding unit that decodes the code image that appears at the same timing as the image portion by the light sequentially detected by the detection unit, and sequentially obtains the corresponding data portion;
A reception apparatus comprising: a reception data storage unit configured to sequentially write data portions sequentially acquired by the code decoding unit and to connect the sequentially written data portions to form a series of data.
An information processing system comprising: the transmission device according to claim 1 or 3; and the reception device according to claim 9.
An information processing system comprising the transmission device according to claim 7 and the reception device according to claim 12.