JP6147161B2 - Imaging apparatus, information display apparatus, and information processing system - Google Patents

Description

  The present invention relates to an imaging device, an information display device, and an information processing system.

Conventional network surveillance cameras are often connected to a personal computer for setting via a network in order to set the network, and the setting work is often performed. In this method, a camera for which network setting has not been completed must communicate with the network. Therefore, for the initial setting, use the physical address on the camera side to set the PC side as a special setting, or set the camera to a specific address when manufacturing the camera, and use this address. A method of setting has been performed.
These methods have drawbacks that the operation of the personal computer is complicated, the physical address needs to be managed, and the setting operation cannot be performed when a plurality of cameras are connected to the actual network. This complicates the network camera setting work.

  In the above method, since the setting operation cannot be performed in a state before the physical address of the network interface is written, the above method cannot be applied to the writing of the physical address itself.

Therefore, as a method for capturing data into a camera without using a network interface, Patent Document 1 describes a method using blinking of light, and Patent Document 2 describes a method of color-modulating light. Has been.
In the method described in Patent Document 1 or 2, the screen of the setting terminal is arranged in front of the image sensor of the camera, and the information displayed on the screen of the setting terminal is captured by the camera. Therefore, the camera can be set without using a network and without providing a new interface for the camera.

JP 2004-289324 A (paragraphs 0011 to 0019, FIG. 1) JP 2013-9072 (paragraphs 0019 to 0020, FIG. 2)

The refresh rate of the screen of the setting terminal and the scanning rate of the camera actually take various frequencies and do not have the same phase. In addition, there are various angles of view, screen angles, and the like, and a signal acquired from an image captured by an image sensor of the camera is also subjected to various distortions.
For example, when the refresh rate of the screen of the setting terminal is 30 fps and the scanning rate of the image sensor of the camera is 24 fps, in order to transmit and receive information with minimum distortion, FIGS. As shown in (2), only 6 bits of information can be sent per second. For this reason, it is desired to improve the transfer rate in order to improve work efficiency.

  In addition, when setting a monitoring camera, it is necessary to take measures to prevent an unauthorized setting change when the setting is changed through an image sensor of the monitoring camera.

  Therefore, an object of the present invention is to efficiently transfer information from an information display device to an imaging device and to be able to set the imaging device safely.

The imaging device according to the first aspect of the present invention includes a clock image in which the density of the first color component changes according to the waveform of the clock signal, and a data image in which the intensity of the second color component changes according to the waveform of the data signal. An image capturing unit that captures a screen on which a composite image composed of images is displayed and acquires the image on the screen, and the image of the first color component and the second color from the image acquired by the image capturing unit. A separation unit that separates the component image, a clock waveform identification unit that identifies a waveform of the clock signal from a change in shading in the first color component image separated by the separation unit, and a separation unit a data waveform specifying unit for specifying a waveform of the data signal from a change in shading in the image of the second color component, from the waveform of the clock signal identified by the clock waveform identifying unit is identified by the data waveform specifying unit Based on the remaining waveform obtained by subtracting the waveform of the data signal, and a data acquisition unit for acquiring data represented by the waveform of the data waveform specified data signals in a particular unit, it stores the data acquired by the data acquisition unit And a storage unit.

An information processing system according to a first aspect of the present invention is an information processing system including an information display device and an imaging device, wherein the information display device includes a reference clock generation unit that generates a clock signal, and the reference clock generation A clock imaging unit for generating a clock image whose density changes according to the waveform of the clock signal generated by the unit, a storage unit for storing data, and the data stored in the storage unit are generated by the reference clock generation unit A data processing unit that generates a data signal that is modulated in synchronization with the generated clock signal, and that generates a data image whose density changes according to the waveform of the generated data signal, and is generated by the clock imaging unit. The clock image is synthesized using the first color component and the data image generated by the data processing unit is synthesized using the second color component. A composite unit that generates a composite video; and a display unit that displays the composite video generated by the composite unit on a screen; and the imaging device captures the screen on which the composite video is displayed, and the screen An image capturing unit that acquires the image of the first color component, a separation unit that separates the image of the first color component and the image of the second color component from the image acquired by the image capturing unit, and separation by the separation unit A clock waveform specifying unit for specifying a waveform of the clock signal from a change in shading in the image of the first color component, and the data signal from a change in shading in the image of the second color component separated by the separating unit. Based on the remaining waveform obtained by subtracting the waveform of the data signal specified by the data waveform specifying unit from the waveform of the clock signal specified by the clock waveform specifying unit and the data waveform specifying unit for specifying the waveform of Data waveform Characterized in that it comprises a data acquisition unit for acquiring data represented by the waveform of the specified data signal with a constant section, and a storage unit for storing the data acquired by the data acquisition unit.

In the imaging device according to the second aspect of the present invention, the light emitting unit that emits light for the first period with the first color component and the code drawn with the second color component include the second period. An image of a screen displaying a code image that appears in the period of time, an imaging unit that acquires the image of the light emitting unit and the screen, an image of the first color component from the image acquired by the imaging unit, and the A separation unit that separates the image of the second color component; a detection unit that detects light emission of the light emitting unit from the image of the first color component separated by the separation unit; and the detection unit of the light emitting unit At the timing when light emission is detected, an image cutout unit that cuts out an image from the video of the second color component separated by the separation unit, and a code included in the image cut out by the image cutout unit are decoded. A code decoding unit for acquiring data corresponding to the decoded code, Characterized in that it comprises a storage unit for storing the data acquired by the code decoding unit.

An information display device according to a first aspect of the present invention includes a reference clock generation unit that generates a clock signal, and a first color component that is synchronized with the clock signal generated by the reference clock generation unit. A light emitting unit that emits light for a period, a storage unit that stores data, and converts the data stored in the storage unit into a code and performs the conversion in synchronization with the clock signal generated by the reference clock generation unit A code generation unit that generates a code image in which the generated code appears in a second period including the first period, and a synthesized image obtained by combining the code image generated by the code generation unit using a second color component The image processing apparatus includes: a combining unit that generates the display unit; and a display unit that displays the combined video generated by the combining unit on a screen.

An information processing system according to a second aspect of the present invention is an information processing system including an information display device and an imaging device, wherein the information display device includes a reference clock generation unit that generates a clock signal, and the reference clock generation A light emitting unit that emits light for a first period with a first color component, a storage unit that stores data, and converts the data stored in the storage unit into a code in synchronization with a clock signal generated by the unit And a code generation unit that generates a code image in which the converted code appears in a second period including the first period in synchronization with the clock signal generated by the reference clock generation unit, and the code generation A combining unit that generates a combined image obtained by combining the code image generated by the unit using the second color component, and a display unit that displays the combined image generated by the combining unit on a screen. The imaging device captures the screen on which the light emitting unit and the combined video are displayed, acquires an image of the light emitting unit and the screen, and an image acquired by the imaging unit, A separation unit that separates an image of the first color component and an image of the second color component, and a detection unit that detects light emission of the light emitting unit from the image of the first color component separated by the separation unit And at the timing when the detection unit detects the light emission of the light emitting unit, an image cutout unit that cuts out an image from the video of the second color component separated by the separation unit, and the image cutout unit A code decoding unit that decodes a code included in an image and acquires data corresponding to the decoded code, and a storage unit that stores data acquired by the code decoding unit. And

An imaging device according to a third aspect of the present invention images a light emitting unit that emits light for a first period, and a screen that displays a code image in which a drawn code appears in a second period including the first period. And an image capturing unit that acquires the image of the light emitting unit and the screen, a detection unit that detects light emission of the light emitting unit from the image acquired by the image capturing unit, and the detection unit detects light emission of the light emitting unit. At timing, an image cutout unit that cuts out an image from the video acquired by the image pickup unit, and a code included in the image cut out by the image cutout unit are decoded and corresponds to the decoded code A code decoding unit that acquires data, and a storage unit that stores the data acquired by the code decoding unit.

An information display device according to a second aspect of the present invention includes a reference clock generating unit that generates a clock signal, and a light emitting unit that emits light for a first period in synchronization with the clock signal generated by the reference clock generating unit. And a storage unit for storing data, and the data stored in the storage unit are converted into codes, and the converted code is synchronized with the clock signal generated by the reference clock generation unit. A code generation unit that generates a code image that appears in a second period including one period; and a display unit that displays the code image generated by the code generation unit on a screen.

An information processing system according to a third aspect of the present invention is an information processing system including an information display device and an imaging device, and the information display device includes a reference clock generation unit that generates a clock signal, and the reference clock generation A light emitting unit that emits light for a first period in synchronization with a clock signal generated by the unit, a storage unit that stores data, and converts the data stored in the storage unit into a code, and the reference clock A code generation unit that generates a code image in which the converted code appears in a second period including the first period in synchronization with a clock signal generated by the generation unit, and a code generated by the code generation unit A display unit that displays an image on a screen, and the imaging device images the light emitting unit and the screen, and acquires an image of the light emitting unit and the screen; A detection unit that detects light emission of the light emitting unit from the video acquired by the imaging unit, and an image cutout that extracts an image from the video acquired by the imaging unit at a timing when the detection unit detects light emission of the light emitting unit. An output unit, a code decoding unit that decodes a code included in the image cut out by the image cut-out unit, and obtains data corresponding to the decoded code; and acquired by the code decoding unit And a storage unit for storing data.

An imaging apparatus according to a fourth aspect of the present invention includes a verification information generation unit that generates first verification information, and an encryption that encrypts the first verification information generated by the verification information generation unit. A processing unit, an interface unit that transmits the first verification information encrypted by the cryptographic processing unit to the network, and a screen displaying a video obtained by converting the second verification information are captured, and the video of the screen is displayed. An image acquisition unit that acquires the second verification information from the video acquired by the imaging unit, the second verification information acquired by the decoding unit, and the verification information generation unit The first verification information generated in step (1), a verification unit that determines whether the verification is successful, and the interface unit or the verification unit when the verification unit determines that the verification is successful. A system for acquiring data via the imaging unit And parts,
A storage unit that stores data acquired by the control unit, and the encryption processing unit is generated by the verification information generation unit using the verification information used by the data transmission source as an encryption key. The first verification information is encrypted .

An information display device according to a third aspect of the present invention includes a verification information generation unit that generates first verification information, a storage unit that stores data, a part of the data, and the verification information generation unit A display video generation unit for generating a first video obtained by converting the first verification information generated in step (b), a display unit for displaying the first video generated by the display video generation unit on a screen, and a network. An interface unit that receives the second verification information encrypted via the interface, and decrypts the encrypted second verification information received by the interface unit, thereby obtaining the second verification information. Whether the encryption processing unit to be extracted, the second verification information extracted by the encryption processing unit, and the first verification information generated by the verification information generation unit are collated, and whether or not the verification is successful A collation unit for determining whether the collation unit is If it is determined that the data has been successfully transmitted, the interface unit transmits the remaining data, or the display image generating unit generates a second image obtained by converting the remaining data. A control unit that displays the second video generated by the display video generation unit, and the second verification information is encrypted using the first verification information as an encryption key, The encryption processing unit decrypts the encrypted second verification information received by the interface unit using the first verification information as a decryption key .

An information processing system according to a fourth aspect of the present invention is an information processing system including an imaging device and an information display device, and the imaging device includes a verification information generation unit that generates first verification information; An encryption processing unit that encrypts the first verification information generated by the verification information generation unit, and an interface unit that transmits the first verification information encrypted by the encryption processing unit to the information display device And imaging the screen of the information display device on which the video obtained by converting the second verification information is displayed, acquiring the video of the screen, and the second acquired from the video acquired by the imaging unit. The decoding unit that acquires the verification information, the second verification information acquired by the decoding unit, and the first verification information generated by the verification information generation unit were collated, and the verification was successful A collation unit for determining whether or not A control unit that acquires data from the information display device via the interface unit or the imaging unit and a memory that stores the data acquired by the control unit when the combination unit determines that the verification is successful And the encryption processing unit encrypts the first verification information generated by the verification information generation unit using the verification information used by the data transmission source as an encryption key. It is characterized by.

The information processing system which concerns on the 5th aspect of this invention is an information processing system provided with an imaging device and an information display apparatus, Comprising: The said information display apparatus is a collation information generation part which produces | generates the 1st information for collation. A storage unit for storing data, a display video generation unit for generating a first video obtained by converting a part of the data and the first verification information generated by the verification information generation unit, and the display video A display unit for displaying the first video generated by the generation unit on the screen; an interface unit for receiving the encrypted second verification information via the network; and the encryption received by the interface unit. Decrypting the second verification information, the cryptographic processing unit for extracting the second verification information, the second verification information extracted by the cryptographic processing unit, and the verification information generating unit Generated by the first A collation unit that collates information for collation and determines whether or not the collation is successful, and when the collation unit determines that the collation is successful, the remaining image data is stored in the interface unit in the imaging device. Or a control unit that causes the display video generation unit to generate a second video obtained by converting the rest of the data, and causes the display unit to display the second video generated by the display video generation unit And the second verification information is encrypted using the first verification information as an encryption key, and the encryption processing unit uses the first verification information as a decryption key. And decrypting the encrypted second verification information received by the interface unit .

  According to one embodiment of the present invention, information can be efficiently transferred from an information display device to an imaging device, and the imaging device can be set safely.

(A) And (B) is a wave form diagram which shows an example of the signal communicated when a frame rate differs on the transmission side and the reception side. 1 is an overview diagram schematically showing a configuration of an information processing system according to Embodiments 1 to 3. FIG. FIGS. 4A to 4D are schematic diagrams illustrating waveforms of a clock signal and a data signal transmitted from a setting terminal according to the first embodiment, and waveforms of a clock signal and a data signal received by a camera on the receiving side. It is. FIG. 3 is a block diagram schematically showing a configuration of a setting terminal in the first embodiment. (A)-(C) show the waveforms of the clock signal assigned to the green channel, the first data signal assigned to the red channel, and the second data signal assigned to the blue channel in the first embodiment. FIG. 2 is a block diagram schematically showing a configuration of a camera in Embodiment 1. FIG. 4 is a flowchart showing an operation for displaying an image based on setting data inside the setting terminal according to the first embodiment. 4 is a flowchart illustrating an operation of acquiring and storing setting data from a captured image in the camera according to the first embodiment. 6 is a block diagram schematically showing a configuration of a setting terminal according to Embodiment 2. FIG. (A)-(C) show the display timing of the release mark assigned to the green channel, the first code assigned to the red channel, and the second code assigned to the blue channel in the second embodiment. FIG. FIG. 5 is a block diagram schematically showing a configuration of a camera in a second embodiment. (A) And (B) is the schematic for demonstrating the timing shown with a release mark and the timing which cuts out a code | cord | chord in Embodiment 2. FIG. FIG. 10 is a schematic diagram for explaining synthesis of a code and a release mark in the second embodiment. 10 is a flowchart illustrating an operation of displaying a video based on setting data inside the setting terminal according to the second embodiment. 10 is a flowchart illustrating an operation of acquiring and storing setting data from a captured image in the camera according to the second embodiment. (A)-(M) is the schematic which shows the modification of the release mark in Embodiment 2. FIG. FIG. 11 is a block diagram schematically showing a configuration of a setting terminal in a modification of the second embodiment. FIG. 10 is a block diagram schematically showing a configuration of a camera in a modified example of the second embodiment. In the modification of Embodiment 2, it is the schematic which shows the layout of a release mark and a code | cord | chord. In the modification of Embodiment 2, it is the schematic which shows the time change of the image | video displayed on the display part of the terminal for a setting. FIG. 10 is an overview diagram schematically showing a configuration of an information processing system according to a third embodiment. FIG. 10 is a block diagram schematically showing a configuration of a setting terminal in a third embodiment. FIG. 10 is a block diagram schematically showing a configuration of a camera in a third embodiment. 10 is a flowchart illustrating an operation in the information processing system according to the third embodiment.

Embodiment 1 FIG.
FIG. 2 is an overview diagram schematically showing the configuration of the information processing system 100 according to the first embodiment.
The information processing system 100 includes a setting terminal 110 as an information display device and a camera 130 as an imaging device. The setting terminal 110 visualizes information to be transmitted to the camera 130 such as setting information and displays it on the screen 110a. The camera 130 captures and decodes the video obtained by capturing the screen 110a of the setting terminal 110, and uses it as predetermined data. Here, the image means a still image, and the video means a moving image. Reference numerals in parentheses in FIG. 2 indicate configurations of the second and third embodiments.

In the first embodiment, it is assumed that the refresh rate and scanning method of the screen display of the setting terminal 110 and the scanning rate and scanning method of the image sensor of the camera 130 are different. Here, both the refresh rate of image display and the scanning rate of the image sensor are referred to as frame rates.
Since TV screens and video cameras usually operate at video frame rates, there are few problems. However, information display devices such as computer displays, mobile phones, and smart phones are driven at various frame rates. In addition, imaging devices such as cameras, surveillance cameras, computer cameras, and mobile phone cameras operate at different frame rates. For this reason, when the display screen of the setting terminal 110 is photographed by the camera 130, the display frame rate and the photographing frame rate often do not match.

3A to 3D show the waveforms of the clock signal and data signal transmitted by the setting terminal 110 on the transmission side, and the waveforms of the clock signal and data signal received by the camera 130 on the reception side. FIG.
FIG. 3B shows a waveform of a data signal when the setting terminal 110 that is driven at, for example, 30 fps modulates data by pulse width and transmits the data by blinking the screen. FIG. 3D shows a waveform of a data signal received when the camera 130 driven at 24 fps captures a blinking image displayed on the setting terminal 110. It can be seen that the waveform is greatly disturbed because the frame rate is different between the display side and the imaging side. Actually, it is considered that the distortion becomes larger due to the shutter opening time of the camera 130, the timing difference thereof, the angle of view of the image, and the like. Therefore, if the waveform is disturbed in this way, it becomes difficult to decode the blinking and extract data, and the communication speed cannot be increased.

  FIG. 3A shows a waveform of a clock signal transmitted by the setting terminal 110 driven at 30 fps by repeating blinking every two frames at equal time. FIG. 3C shows a waveform of a clock signal received when the camera 130 driven at 24 fps images the blinking pattern displayed on the setting terminal 110. The distortion of the clock signal due to the difference in the frame rate has periodicity, and the periodicity of three times in 2 seconds can be confirmed in the waveform of the clock signal shown in FIG. From the frame rate of the image sensor of the camera 130 and this periodicity, the clock frequency can be calculated backward. If the clock frequency is obtained, the clock signal can be reproduced if the blinking at the same time is assumed. Further, it can be estimated that the frame rate on the transmission side is an integral multiple of the clock frequency.

  As shown in FIG. 3, if both the clock signal and the data signal can be transmitted through a transmission system subjected to the same distortion, even if the waveform is distorted on the receiving side, the reproducible waveform of the clock signal is obtained. It is possible to correct the waveform of the received data signal and extract the original data to some extent.

  FIG. 4 is a block diagram schematically showing the configuration of the setting terminal 110. In Embodiment 1, the entire screen of the setting terminal 110 or one of the screens can be transmitted so that signals can be transmitted even if the imaging system of the camera 130 is not in focus or the lens of the camera 130 is not provided. Blink part and send data. In the first embodiment, the setting terminal 110 changes the color according to the data to be transmitted so that the camera 130 can extract data from the blinking change of the entire screen or a part of the screen. Change flickering in time series. Note that a part of the screen is an area having a certain area so that signals can be transmitted even if the imaging system of the camera 130 is not focused or the lens of the camera 130 is not provided. It is desirable.

  The setting terminal 110 includes a setting data list storage unit 111, a data processing unit 112, a reference clock generation unit 115, a clock imaging unit 116, a color channel synthesis unit 117, a display video generation unit 118, and a display unit. 119.

  The setting data list storage unit 111 is a storage unit that stores setting data including setting values for setting in the camera 130.

The data processing unit 112 modulates the setting data stored in the setting data list storage unit 111 in synchronization with the clock signal generated by the reference clock generation unit 115 to generate a data signal. A data image whose density changes according to the waveform of the signal is generated.
The data processing unit 112 includes a data dividing unit 113, a first data imaging unit 114A, and a second data imaging unit 114B.
The data dividing unit 113 reads the setting data from the setting data list storage unit 111 and divides the read setting data for a plurality of channels. In the first embodiment, the data dividing unit 113 divides the setting data for two channels.
The first data visualization unit 114A generates a data signal (first data signal) by performing pulse modulation, for example, pulse width modulation, on one data (first data) divided by the data dividing unit 113. Then, the first data imaging unit 114A generates an image (first data image) whose density changes according to the waveform of the generated first data signal. For example, in the first embodiment, the first data video may be a black and white video. Specifically, when the first data is subjected to pulse modulation, when the amplitude is the minimum value, the minimum value (for example, “0”) and the maximum value (for example, “255”) of the gradation value. When either one of the gradation values is assigned and the amplitude is the maximum value, any one of the minimum value and the maximum value of the gradation values may be assigned. As a result, the gradation value changes according to the waveform of the first data signal, and the shading also changes.
The second data visualization unit 114B generates a data signal (second data signal) by performing pulse modulation, for example, pulse width modulation, on the other data (second data) divided by the data dividing unit 113. Then, the second data imaging unit 114B generates an image (second data image) whose density changes according to the waveform of the generated second data signal. For example, in the first embodiment, the second data video may be a black and white video. Specifically, when the second data is pulse-modulated, if the amplitude is the minimum value, one of the gradation values of the gradation value is assigned, and the amplitude is In the case of the maximum value, it is only necessary to assign the other gradation value of the minimum value and the maximum value of the gradation values. As a result, the gradation value changes according to the waveform of the second data signal, and the shading also changes.

The reference clock generation unit 115 generates a clock signal (also referred to as a reference clock signal) synchronized with the frame rate of the display unit 119.
The clock imaging unit 116 generates an image (clock image) whose density changes according to the waveform of the clock signal generated by the reference clock generation unit 115. For example, in the first embodiment, the clock video may be a black and white video. Specifically, when the clock signal is a pulse signal, if the amplitude is the minimum value, one of the gradation values “0” and “255” is assigned, and the amplitude is the maximum value. In this case, it is only necessary to assign one of the gradation values “0” and “255”. As a result, the gradation value changes according to the waveform of the clock signal, and the shading also changes.

  The color channel synthesizing unit 117 includes the clock video generated by the clock video converting unit 116, the first data video generated by the first data video converting unit 114A, and the second data generated by the second data video converting unit 114B. This is a combining unit that generates a combined image by combining the data image and regarding it as belonging to independent color channels. Here, in the first embodiment, the color channel combining unit 117 employs the three primary colors of red (R), blue (B), and green (G) as color channels. This is because the display unit 119 normally emits light in the three primary colors of light to form an image, so that the three color channels are highly independent and easy to process. Then, as shown in FIGS. 5A to 5C, the color channel combining unit 117 displays the clock image as the green (G) channel, the first data image as the red (R) channel, and the second data image. Is regarded as a blue (B) channel, color channel composition is performed, and transmission data is converted into a color composite image. For example, the color channel combining unit 117 may combine the gradation values assigned to the clock image, the first data image, and the second data image as the gradation values of the respective color channels.

The display video generation unit 118 converts the composite video synthesized by the color channel synthesis unit 117 into video (display video) that can be displayed by the display unit 119.
The display unit 119 displays the display video converted by the display video generation unit 118 on the screen 110a (see FIG. 2).

FIG. 6 is a block diagram schematically showing the configuration of the camera 130.
The camera 130 includes an imaging unit 131, a color channel separation unit 132, a data waveform specification unit 133, a clock waveform specification unit 134, a data acquisition unit 135, a data synthesis unit 136, and a setting storage unit 137.

The imaging unit 131 images the screen 110a of the setting terminal 110 on which the video is displayed, and acquires the video on the screen 110a. For example, the imaging unit 131 receives an input of light and converts the input light into an electrical signal to acquire an image.
The color channel separation unit 132 is a separation unit that separates the video provided from the imaging unit 131 into videos of a plurality of predetermined color components. For example, the color channel separation unit 132 converts the video provided from the imaging unit 131 into a green component image that is a green (G) channel image, a red component image that is a red (R) channel image, and a blue component image. (B) The image is separated into a blue component image which is a channel image.

The data waveform specifying unit 133 specifies the waveform of the data signal from the video of the predetermined color component separated by the color channel separation unit 132. The data waveform specifying unit 133 includes a first data waveform specifying unit 133A and a second data waveform specifying unit 133B.
The first data waveform specifying unit 133A specifies the change in the shade of the red component video separated by the color channel separation unit 132, and specifies the waveform of the first data signal.
The second data waveform specifying unit 133B specifies the change in the shade of the blue component image separated by the color channel separation unit 132, and specifies the waveform of the second data signal.

  The clock waveform specifying unit 134 specifies the change in light and shade of the green component image separated by the color channel separation unit 132 and specifies the waveform of the clock signal.

The data acquisition unit 135 acquires the data indicated by the data signal using the waveform of the data signal specified by the data waveform specifying unit 133 and the waveform of the clock signal specified by the clock waveform specifying unit 134. For example, the data acquisition unit 135 acquires data based on the remaining waveform obtained by subtracting the waveform of the data signal specified by the data waveform specification unit 133 from the waveform of the clock signal specified by the clock waveform specification unit 134. . The data acquisition unit 135 includes a first data acquisition unit 135A and a second data acquisition unit 135B.
The first data acquisition unit 135A performs decoding by combining and combining the waveform of the first data signal specified by the first data waveform specifying unit 133A and the waveform of the clock signal specified by the clock waveform acquisition unit 134. First data is acquired.
The second data acquisition unit 135B performs decoding by calculating a combination of the waveform of the second data signal specified by the second data waveform specifying unit 133B and the waveform of the clock signal specified by the clock waveform acquisition unit 134. Second data is acquired.

The data synthesis unit 136 combines and sends the first data specified by the first data acquisition unit 135A and the second data specified by the second data acquisition unit 135B. Restore the setting data.
The setting storage unit 137 stores the setting data restored by the data synthesis unit 136 according to the content.

Next, the operation of the setting terminal 110 will be described with reference to FIG.
FIG. 7 is a flowchart showing an operation of displaying an image based on the setting data in the setting terminal 110. The flowchart of FIG. 7 is started when a predetermined program, for example, a setting transmission program for setting the camera 130 is activated in the setting terminal 110.

  In step S <b> 10, the data dividing unit 113 selects setting data to be transmitted from among a plurality of setting data stored in the setting data list storage unit 111. Usually, the setting data list storage unit 111 stores settings for a plurality of cameras. For example, the operator of the setting terminal 110 selects setting data for the camera 130 to be set from now on while viewing a list displayed on the display unit 119. When an input unit (not shown) receives an operator's selection input, the data dividing unit 113 reads the selected setting data from the setting data list storage unit 111.

  In step S11, the data dividing unit 113 divides the setting data read in step S10 into the number of channels to be used. Then, the data dividing unit 113 gives the divided first data to the first data imaging unit 114A, and gives the divided second data to the second data imaging unit 114B.

On the other hand, in step S12, the reference clock generator 115 generates a reference clock signal synchronized with the frame rate of the display unit 119.
In step S13, the clock imaging unit 116 generates a clock image whose density changes according to the waveform of the reference clock signal generated in step S13. When the reference clock signal is visualized, for example, as shown in FIG. 5A, a simple blinking image of the entire screen is basically generated. The blinking of the screen is basically synchronized with the frame rate, and the blinking width is an integral multiple of the frame rate. FIG. 5A shows a case where the frame rate is 30 fps and the bright and dark images each have a width of 2 frames.

In step S14, the first data imaging unit 114A encodes and visualizes the first data divided in step S11, and generates a first data image whose density changes in synchronization with the frame rate. For example, FIG. 5B shows an encoded signal (first data signal) that is encoded by arranging first data indicated by a bit string of 1011011000 in time series and performing pulse width modulation. . When this encoded signal is synchronized with a clock signal and converted into a blinking image, the first data can be visualized.
In step S15, the second data imaging unit 114B encodes and visualizes the second data divided in step S11, and generates a second data image whose density changes in synchronization with the frame rate. For example, FIG. 5C shows an encoded signal (second data signal) that is encoded by arranging second data indicated by a bit string of 00100100100 in time series and performing pulse width modulation. . When the encoded signal is synchronized with the clock signal and converted into a blinking image, the second data can be visualized.

  In step S16, the color channel synthesizing unit 117 separates the clock image converted in step S14, the first data image converted in step S15, and the second data image converted in step S16 from independent color channels. It is considered to belong to and is synthesized. In the first embodiment, the color channel combining unit 117 regards the first data video as a red (R) channel, the second data video as a blue (B) channel, and a clock video as a green (G) channel. Are combined. Then, the display video generation unit 118 converts the synthesized color video into a video (display video) that can be displayed on the display unit 119, and generates a converted display video. Usually, the display image of a color image display device is often expressed as a combination of the three primary colors of red, blue and green light, so when three colors of red, blue and green are selected as color channels, Synthesis can be performed by overlaying various data.

  In step S17, the display unit 119 displays the display video generated in step S16. Here, an image in which light changes according to time series is displayed.

In this example, it is shown that an image displayed on the screen is used, but the same thing can be performed using an LED or a flashlight.
Many status display LEDs mounted on terminals as information display devices can display two or more colors. For LED flashlights such as smartphones, the color of the flashlight can be adjusted by combining different color flashlights. There is something that can be done. These light emitting elements (light emitting portions) can be controlled and used for data transfer in the same manner.

Next, the operation of the camera 130 will be described with reference to FIG.
FIG. 8 is a flowchart showing an operation of acquiring and storing setting data from the captured video inside the camera 130. The flowchart of FIG. 8 is started when a predetermined program, for example, a setting acquisition program for setting the camera 130 is activated in the camera 130.

  In step S20, the camera 130 starts acquiring video. As shown in FIG. 1, the imaging unit 131 of the camera 130 is opposed to the screen 110 a of the display unit 119 of the setting terminal 110. ) And color change can be captured and read as video.

  In step S21, the color channel separation unit 132 separates the video read in step S20 into videos for each color channel. In the first embodiment, three primary colors of red (R), blue (B), and green (G) are adopted as color channels. Since the color image sensor included in the imaging unit 131 is often composed of elements with red, blue and green color filters, it can be easily separated into three channels of red, blue and green. Can do. The color channel separation unit 132 gives a red component image, which is a red (R) channel image, to the first data waveform specifying unit 133A, and a blue component image, which is a blue (B) channel image, specifies a second data waveform. The image of the green component that is the image of the green (G) channel is supplied to the clock waveform specifying unit 134.

In step S22, the clock waveform specifying unit 134 specifies the change in light and shade of the received green component video, and specifies the waveform of the clock signal. Here, as shown in FIGS. 3A and 3C, if the frame rate is different between the display unit 119 of the setting terminal 110 and the imaging unit 131 of the camera 130, a simple blinking image is displayed. Even when the clock signal is sent in the waveform, the waveform specified from the green component image is distorted. However, in the case of a clock signal, the original waveform is determined when the system is designed, and in the case of the first embodiment, a bright image and a dark image are repeated at equal intervals. When the clock signal is distorted due to the difference in frame rate, periodicity is also observed in the received signal. Therefore, the clock waveform specifying unit 134 can calculate the cycle of the clock signal at the time of transmission from the periodicity of the received signal and the frame rate of the imaging unit 131. For example, in the example shown in FIG. 3C, the clock waveform specifying unit 134 can specify that the period is 16 frames by taking the autocorrelation of the clock waveform. Then, the clock waveform specifying unit 134 can specify 1.5 cycles per second using the following equation (1) using the frame rate of 24 fps of the imaging unit 131.
16/24 = 1 / 1.5 (1)
Further, the clock waveform specifying unit 134 specifies the number of combinations of peaks and valleys of waveforms included in one cycle (16 frames) of the received clock signal. In the example shown in FIG. 3C, five combinations of peaks and valleys are included in one cycle. This specified number is considered to correspond to the waveform of the clock signal. For this reason, the clock waveform specifying unit 134 can calculate the frequency (7.5 Hz) of the clock signal at the time of transmission by the following equation (2).
1.5 × 5 = 7.5 (2)
Thereby, the clock waveform specifying unit 134 can specify the cycle on the transmission side.
The clock waveform identification unit 134 notifies the first correction unit 135A and the second data acquisition unit 135B of the identified transmission-side cycle.

  In step S23, the first data waveform specifying unit 133A specifies the change in light and shade of the red component image separated in step S21, and specifies the waveform of the first data signal. The acquired waveform of the first data signal is distorted due to the difference between the frame rate of the display unit 119 of the setting terminal 110 and the frame rate of the imaging unit 131 of the camera 130. In Embodiment 1, for example, the waveform is as shown in FIG. It is difficult to simply reproduce data from this waveform.

  Therefore, in step S24, the first data acquisition unit 135A corrects the waveform of the first data signal specified in step S23 with the waveform of the clock signal specified in step S22, and decodes the first data signal. To reproduce the first data. Since it can be considered that the same color channel is subject to the same distortion, the waveform of the first data signal can be corrected using a clock waveform that is easy to estimate the original waveform. For example, when the first data acquisition unit 135A subtracts the first data waveform from the clock waveform, only the portion corresponding to “1” where the pulse width is changed by the pulse width modulation remains, and the pulse width is not changed. No waveform remains in the portion corresponding to 0 ". Then, the first data acquisition unit 135A assigns the portion corresponding to “1” and the portion corresponding to “0” specified in this way to the cycle of the clock signal on the transmission side calculated in step S22. The first data can be reproduced.

  In the same manner as described above, in step S25, the second data waveform specifying unit 133B specifies the change in the shade of the blue component image separated in step S21, and specifies the waveform of the second data signal. In step S26, the second data acquisition unit 135B corrects the waveform of the second data signal specified in step S25 with the waveform of the clock signal specified in step S22, and reproduces the second data.

  In step S27, the data synthesizing unit 136 combines the first data reproduced in step S24 and the second data reproduced in step S26, and synthesizes these data. To restore.

  In step S28, the data composition unit 136 causes the setting storage unit 137 to store the setting value of the camera 130 based on the restored setting data. This completes the reading mode.

  As described above, the setting data is transmitted to the camera 130 by causing the imaging unit 131 of the camera 130 to read the video obtained by combining the video of the clock signal and the video of the data signal into separate color channels. The camera 130 itself can be set. In particular, in the case of this method, it is not necessary to form an image on the imaging unit 131, so that the camera 130 can easily read and set data even without a lens or before adjustment of the lens. Can do.

  Even when the refresh rate of the display screen of the setting terminal 110 that sends information and the scanning rate of the imaging unit 131 of the camera 130 that receives the information are different, data is transmitted using a plurality of color channels, so the transfer speed can be increased. Can be improved.

  In addition, when the refresh rate of the display screen of the setting terminal 110 that transmits information is different from the scan rate of the imaging unit 131 of the camera 130 that receives the information, the waveform is distorted. However, according to the first embodiment, by assigning one of a plurality of color channels to the transmission of the clock signal, both the clock signal and the data signal can be transmitted through the transmission system that receives the same distortion. Therefore, even if the waveform is distorted on the receiving side, the waveform of the received data signal can be corrected using the waveform of the reproducible clock signal, and the original data can be extracted.

  In the first embodiment described above, the first data imaging unit 114A and the second data imaging unit 114B of the setting terminal 110 use pulse width modulation, but pulse amplitude modulation or pulse phase modulation. May be used.

In the first embodiment, the data acquisition unit 135 of the camera 130 reproduces the data indicated by the data signal by subtracting the waveform of the data signal from the waveform of the clock signal. It is not limited to.
For example, the clock waveform specifying unit 134 specifies the period by taking the autocorrelation of the waveform of the received clock signal (for example, 16 frames in the above example).
The camera 130 is provided with a storage unit (not shown), and correspondence information in which a waveform of a possible data signal and a value of data in the waveform are associated with each other is stored for each period. Then, the data acquisition unit 135 can identify the value of the associated data by comparing the waveform of the received data signal with the waveform of the data signal stored in the storage unit.

  In the first embodiment described above, the color channel combining unit 117 of the setting terminal 110 performs combining using the three color channels, but is not limited to such an example. . For example, the color channel synthesis unit 117 may perform synthesis using two color channels. In such a case, the data processing unit 112 of the setting terminal 110 may include only one of the first data imaging unit 114A and the second data imaging unit 114B without the data dividing unit 113. Good. Then, the data processing unit 112 may visualize the setting data read from the setting data list storage unit 111 without dividing it. In such a case, the color channel separation unit 132 of the camera 130 separates the video obtained from the imaging unit 131 into two-color video, gives one to the clock waveform specifying unit 134, and sends the other to the data What is necessary is just to give to the waveform specific | specification part 133. FIG. The data waveform specifying unit 133 includes either the first data waveform specifying unit 133A or the second data waveform specifying unit 133B, and the data acquisition unit 135 is also configured by the first data acquisition unit 135A and the second data acquisition unit 135B. Any one of them may be provided.

  In the first embodiment described above, the color channel synthesizing unit 117 generates a color synthesized video by allocating and synthesizing the clock video and the data video to the respective color channels. It is not limited to a specific example. For example, the clock imaging unit 116, the first data imaging unit 114A, and the second data imaging unit 114B create color images in which only the assigned color components change, and the color channel synthesis unit 117 These may be synthesized.

Embodiment 2. FIG.
As shown in FIG. 2, the information processing system 200 according to Embodiment 2 includes a setting terminal 210 and a camera 230. In the second embodiment, the imaging system of the camera 230 forms an image, that is, the case where the focal position is known in advance, and the function of automatically adjusting the focal position by operating the autofocus. Assuming that In the second embodiment, the setting terminal 210 transfers the setting data to the camera 230 by sequentially displaying the codes. Here, the code includes a two-dimensional code including the entire QR code (registered trademark) and other two-dimensional barcodes, and a one-dimensional barcode.

FIG. 9 is a block diagram schematically showing a configuration of setting terminal 210 in the second embodiment.
The setting terminal 210 includes a setting data list storage unit 111, a data processing unit 212, a reference clock generation unit 115, a release mark generation unit 221, a color channel synthesis unit 217, a display video generation unit 118, and a display unit. 119. The setting terminal 210 according to the second embodiment includes a point of processing in the data processing unit 212, a point that a release mark generation unit 221 is provided instead of the clock imaging unit 116, and a color channel synthesis unit 217. This processing is different from the setting terminal 110 in the first embodiment.
However, the reference clock generation unit 115 provides the generated clock signal to the release mark generation unit 221 and the data processing unit 212.

  The release mark generation unit 221 generates a release mark video (identification video) in which a release mark as an identification image appears in a predetermined first period in synchronization with the clock signal generated by the reference clock generation unit 115. In the second embodiment, the release mark image may be a monochrome image. Specifically, either the minimum value or the maximum value of the gradation value is assigned to the area for drawing the release mark, and the gradation value of the other one of the minimum value and the maximum value of the gradation value is assigned to the other area. May be assigned.

The data processing unit 212 converts the setting data stored in the setting data list storage unit 111 into a code, and displays a first period during which the image in which the converted code is drawn displays a release mark. A code image that appears in the second period is generated. In the second embodiment, the code image may be a monochrome image. Specifically, either the minimum or maximum gradation value of the gradation value is assigned to the area where the code is drawn, and the other of the minimum and maximum gradation values is assigned to the other area. It is only necessary to assign a gradation value.
The data processing unit 212 includes a data dividing unit 113, a first code generation unit 220A, and a second code generation unit 220B. The data processing unit 212 according to the second embodiment is implemented in that it includes a first code generation unit 220A and a second code generation unit 220B instead of the first data visualization unit 114A and the second data visualization unit 114B. This is different from the data processing unit 112 in the first embodiment.
However, the data dividing unit 113 divides the setting data read from the setting data list storage unit 111, gives one divided data (first data) to the first code generation unit 220A, and the other divided data (Second data) is provided to the second code generation unit 220B.
The first code generation unit 220A converts the first data divided by the data division unit 113 into a code, and generates a first code video in which the converted code (first code) appears in the second period.
The second code generation unit 220B converts the second data divided by the data division unit 113 into a code, and generates a second code video in which the converted code (second code) appears in the second period.

  The color channel composition unit 217 receives the release mark video provided from the release mark generation unit 221, the first code video provided from the first code generation unit 220A, and the second code video provided from the second code generation unit 220B. A synthesized video is generated by regarding each as belonging to an independent color channel. Here, also in the second embodiment, the color channel combining unit 217 employs three primary colors of red (R), blue (B), and green (G) as color channels. Then, as shown in FIGS. 10A to 10C, the color channel composition unit 217 displays the release mark image as the green (G) channel, the first code image as the red (R) channel, and the second code. The video is regarded as a blue (B) channel, color channel composition is performed, and transmission data is converted into a color video. For example, the color channel combining unit 217 may combine the gradation values assigned to the release mark image, the first code image, and the second code image as the gradation values of the respective color channels.

FIG. 11 is a block diagram schematically showing the configuration of the camera 230 in the second embodiment.
The camera 230 includes an imaging unit 131, a color channel separation unit 132, a release mark recognition unit 238, an image cutout unit 239, a code decoding unit 240, a data synthesis unit 136, and a setting storage unit 137. The camera 230 according to the second embodiment is provided with a release mark recognizing unit 238 instead of the clock waveform specifying unit 134, an image cutting unit 239 instead of the data waveform specifying unit 133, and The difference from the camera 130 in the first embodiment is that a code decoding unit 240 is provided instead of the data acquisition unit 135.
However, the color channel separation unit 132 converts the video supplied from the imaging unit 131 into a green component video that is a green (G) channel video, a red component video that is a red (R) channel video, and a blue color. (B) The image is separated into the blue component image, which is the channel image, and the green component image is supplied to the release mark recognition unit 238, and the red component image and the blue component image are supplied to the image cutout unit 239. Further, the data synthesis unit 136 combines the first data and the second data given from the code decoding unit 240 to synthesize these data and restores the transmitted setting data.

  The release mark recognition unit 238 is a detection unit that detects the release mark from the green component image provided from the color channel separation unit 132. Then, the release mark recognition unit 238 recognizes the timing at which the release mark is detected as the timing at which the code can be read correctly. The release mark recognition unit 238 instructs the image cutout unit 239 to cut out an image at the timing when the release mark is detected.

The image cutout unit 239 cuts out an image from a video having a predetermined color component at the timing when the release mark recognition unit 238 detects the release mark. The image cutout unit 239 includes a first image cutout unit 239A and a second image cutout unit 239B.
The first image cutout unit 239A cuts out the image of the first code from the red component video given from the color channel separation unit 132 at the timing instructed by the release mark recognition unit 238.
The second image cutout unit 239B cuts out the image of the second code from the blue component video given from the color channel separation unit 132 at the timing instructed by the release mark recognition unit 238.

The code decoding unit 240 decodes the code included in the image cut out by the image cutout unit 239, and acquires data corresponding to the decoded code. The code decoding unit 240 includes a first code decoding unit 240A and a second code decoding unit 240B.
The first code decoding unit 240A decodes the first code included in the image cut out by the first image cutout unit 239A, and extracts the first data.
The second code decoding unit 240B decodes the second code included in the image cut out by the second image cutout unit 239B, and extracts the second data.

12A and 12B are schematic diagrams for explaining the timing indicated by the release mark and the timing for cutting out the code.
As shown in FIG. 12B, codes are sequentially displayed in time series on the display portion 119 of the setting terminal 210.
The display device functioning as the display unit 119 is different in display methods such as dot sequential scanning, line sequential scanning, surface sequential scanning, interlaced, and progressive. Similarly, the camera 230 also has a difference in scanning method, a difference in frame rate, an influence of a shutter opening time, and the like. For this reason, when images sequentially displayed are taken as one frame of the camera, a correct video is not always obtained. Therefore, it is important to obtain a timing at which the camera 230 can read a correct image.

  While the line shown in FIG. 12B is on the upper side, it is the timing at which the code can be read correctly. While the line is on the lower side and on the upper side or on the lower side (while the image is switched), the image is incomplete and the code cannot be read correctly.

  On the other hand, when a plurality of codes are sequentially displayed, it is also important to detect that the camera 230 has switched to a new code. In the case where it cannot be detected, it is necessary to check whether the code has been changed after constantly performing the decoding process in the camera 230, and a wasteful decoding process occurs.

Therefore, in the second embodiment, the setting terminal 210 informs the camera 230 of the timing when the code can be correctly read. In the second embodiment, an image that informs the timing when the code can be read correctly is called a release mark.
As shown in FIG. 12A, the setting terminal 210 displays a release mark with a delay of, for example, one frame after drawing of the code is completed. Also, when changing the code, the setting terminal 210 deletes the release mark one frame ahead of the start of drawing change. At this time, in order to reduce the influence of the shutter opening time on the camera 230 side and the difference in frame rate between the setting terminal 210 and the camera 230, it is desirable that the code and the release mark are displayed for a plurality of frames. .
In this way, on the camera 230 side that reads the screen, the image is cut out after confirming that the release mark is displayed, and if it is decoded as a code, image disturbance due to the cut-out timing is suppressed, Decoding can be performed efficiently.
In addition, each time the code changes, the display and non-display of the release mark is switched, so that the same code is read a plurality of times, and when the same content code is continuously displayed, it is prevented from being overlooked. It is possible to read codes that are sequentially displayed with high accuracy.

  Next, a code and release mark transmission method will be described. The code and the release mark can be taken out independently, and it is necessary to perform transmission without shifting the timing. Therefore, in the second embodiment, the setting terminal 210 transmits the code and the release mark by separating into color channels. FIG. 10 is a schematic diagram showing an example. As shown in FIG. 10A, the setting terminal 210 has a green (G) channel for the release mark, and a red (R) channel for the first code as shown in FIG. 10 (B). As shown in FIG. 10C, the blue (B) channel is used for the second code. In this way, when the three primary colors of red (R), blue (B), and green (G) are transmitted as independent color channels, two channels can be used for data transfer in addition to the release mark. Since the display timing can be aligned at the frame rate, the code reading timing of a plurality of channels can be instructed by one release mark.

FIG. 13 is a schematic diagram for explaining synthesis of a code and a release mark.
As shown in FIG. 13, a release mark is drawn on the green (G) channel. For example, a release mark image indicates whether code reading is valid or invalid depending on whether or not a specific area is filled.
In addition, codes are drawn on the red (R) channel and the blue (B) channel.
The red (R) channel and blue (B) channel codes are displayed as an image together with the green (G) channel release mark.

Next, the operation of the setting terminal 210 will be described with reference to FIG.
FIG. 14 is a flowchart showing an operation for displaying an image based on the setting data in the setting terminal 210. The flowchart in FIG. 14 is started when a predetermined program, for example, a setting transmission program for setting the camera 230 is activated in the setting terminal 210.

  In step S30, the data dividing unit 113 selects and reads out the setting data to be transmitted from the plurality of setting data stored in the setting data list storage unit 111. The data dividing unit 113 divides the read setting data into sizes that can be displayed with one code, and distributes the data for each channel to be used. In the second embodiment, a red (R) channel and a blue (B) channel are used for data transmission. Then, the data dividing unit 113 gives the divided first data to the first code generating unit 220A, and gives the divided second data to the second code generating unit 220B.

In step S31, the first code generation unit 220A draws the first code indicating the first data that is one of the data divided in step S30.
In step S32, the second code generation unit 220B draws the second code indicating the second data that is the other data divided in step S30.

In step S <b> 33, the color channel composition unit 217 determines that the image including the first code drawn in step S <b> 31 and the image including the second code drawn in step S <b> 32 belong to independent color channels. Seen and synthesize.
In step S34, the display video generation unit 118 converts the combined color image into an image (display image) that can be displayed on the display unit 119, and the display unit 119 displays the converted display image. At this time, the release mark is not displayed yet, and only two codes drawn and displayed on different color channels are displayed. That is, it is still in an invalid display state. Then, the display unit 119 stands by in a state where only two codes are displayed (S35).

The release mark generation unit 221 generates an image including a release mark in advance. For example, the release mark generation unit 221 stores an image including the release mark in the memory 221a.
In step S36, the release mark generation unit 221 performs time adjustment based on the clock signal supplied from the reference clock generation unit 115, and the release mark is displayed, for example, one frame later than the code drawing in step S34. As described above, the image including the release mark is given to the color channel combining unit 217.

In step S37, the color channel combining unit 217 includes the image including the first code drawn in step S31, the image including the second code drawn in step S32, and the release mark sent in step S36. The images are combined as if they belonged to independent color channels.
In step S38, the display video generation unit 118 converts the synthesized color image into an image (display image) that can be displayed on the display unit 119, and the display unit 119 displays the converted display image. At this point, a release mark is displayed and an effective display state is entered. The display unit 119 continues the effective display for a time sufficient for the camera 230 to recognize the code in a state where the two codes and the release mark are displayed (S39).

The release mark generation unit 221 generates an image not including the release mark in advance. For example, the release mark generation unit 221 stores an image not including the release mark in the memory 221a.
In step S40, the release mark generation unit 221 adjusts the time based on the clock signal supplied from the reference clock generation unit 115, and the release mark is erased with a delay of, for example, two frames from the display of the release mark in step S38. As described above, an image not including the release mark is given to the color channel combining unit 217.

In step S41, the color channel composition unit 217 includes the image including the first code drawn in step S31, the image including the second code drawn in step S32, and the image from which the release mark has been deleted in step S40. Are considered to belong to independent color channels.
In step S42, the display video generation unit 118 converts the synthesized color image into a display image that can be displayed on the display unit 119, and the display unit 119 displays the converted display image. At this point, since the release mark has been erased, only two codes drawn and displayed on different color channels are displayed, and an invalid display state is set. Then, the display unit 119 waits for one frame with only two codes displayed (S43).

  In step S44, the data dividing unit 113 determines whether there is a remaining data to be transmitted. If there is remaining data (Yes in S44), the process returns to steps S30 and S36. If there is no remaining data (No in S44), the flow ends.

Next, the operation of the camera 230 will be described with reference to FIG.
FIG. 15 is a flowchart illustrating an operation of acquiring and storing setting data from a captured image within the camera 230. The flowchart of FIG. 15 is started when a predetermined program, for example, a setting acquisition program for setting the camera 230 is activated in the camera 230.

  In step S50, the camera 230 starts to acquire a video. As shown in FIG. 2, the imaging unit 131 of the camera 230 is configured to face the screen 210 a of the display unit 119 of the setting terminal 210, and thus captures the screen 210 a of the setting terminal 210, Can be read as video.

  In step S51, the color channel separation unit 132 separates the video read in step S50 into videos for each color channel. In the second embodiment, three primary colors of red (R), blue (B), and green (G) are adopted as color channels. The color channel separation unit 132 gives a red component image that is a red (R) channel image to the first image cutout unit 239A, and a blue component image that is a blue (B) channel image is cut out to the second image. The image of the green component that is the image of the green (G) channel is provided to the release mark recognition unit 238.

  In step S52, the release mark recognition unit 238 determines whether a release mark has been detected. For example, when the release mark is included in the green component image separated in step S51 (Yes in S52), the release mark recognition unit 238 proceeds to steps S53 and S55. If the release component is not included in the green component image separated in step S51 (No in S52), the process returns to step S50.

In step S53, the first image cutout unit 239A cuts out an image including the first code from the video of the red component separated in step S51. In step S54, the first code decoding unit 240A extracts the first data by decoding the first code included in the image cut out in step S53.
In step S55, the second image cutout unit 239B cuts out an image including the second code from the blue component video separated in step S51. In step S56, the second code decoding unit 240B extracts the second data by decoding the second code included in the image cut out in step S55.

In step S57, the data combining unit 136 combines the first data extracted in step S54 and the second data extracted in step S54 to restore the setting data.
In step S <b> 58, the data composition unit 136 stores the setting value of the camera 230 in the setting storage unit 137 based on the restored setting data.

In step S59, the camera 230 acquires a video.
In step S60, the color channel separation unit 132 separates the video read in step S50 into videos for each color channel.

  In step S61, the release mark recognition unit 238 determines whether or not the release mark has been erased. For example, if the release mark has been deleted from the green component image separated in step S60 (Yes in S61), the release mark recognition unit 238 proceeds to step S62. If the release mark remains in the green component image separated in step S51 (No in S61), the process returns to step S59.

  In step S62, the camera 230 determines the end. For example, when the release mark recognizing unit 238 determines that the value indicating the end is stored in the setting data synthesized by the data synthesizing unit 136 when the release mark is not detected for a certain period of time or longer, These are determined as a series of transfers completed. Two or more different release marks may be used, and one of them may be used for the purpose of indicating the end of data. If it is determined that the process is to be terminated (Yes in S62), the flow is terminated. If it is not determined that the process is terminated (No in S62), the process returns to Step S50.

  Since the setting terminal 210 and the camera 230 in the second embodiment are configured as described above, the display unit 119 of the setting terminal 210 and the imaging unit of the camera 230 are configured as shown in FIG. By facing 131, the setting data can be automatically taken into the camera 230 automatically and efficiently using sequentially displayed codes.

  Further, in the second embodiment, by realizing a means for automatically reading, the information density of each code can be reduced, and each point constituting the code can be displayed large. For this reason, the tolerance with respect to a focus shift can be improved. In other words, the tolerance for the focus is improved, and when the setting terminal 210 is held toward the camera 230, the workability of the user can be improved.

  When the release mark is sent independently of the code using a plurality of color channels as in the second embodiment, for example, the release mark is a uniform image including at least the entire effective range of the code. Can be represented. In the example shown in FIG. 13, the effective range of the code and the image range of the release mark are the same. Due to the scanning method between the display unit 119 of the setting terminal 210 and the imaging unit 131 of the camera 230, the captured images are partially mixed with each other at the moment when the screen is switched. Incomplete video may be shot. For this reason, by expressing the release mark as a uniform image including the entire effective range of the code, if the entire release mark is read without being missing, it can be considered that the entire code is also read correctly. .

The release mark is not limited to the above. Also, if there is a margin in timing, there is no need to check for missing release marks, and the degree of freedom of the release marks is further increased.
For example, a release mark represented by the presence or absence of drawing on the entire screen, a release mark of a circle or a multiple circle, a square release mark, a release mark with a border that further surrounds the square may be used.
When a plurality of release marks can be identified, the release marks can be given meaning. In other words, different release marks can be used depending on the data. For example, a plurality of release marks can indicate a series of data breaks, data start and end, and the like.
FIGS. 16A to 16M are schematic diagrams illustrating modifications of the release mark according to the second embodiment.
For example, as a release mark, a circular release mark RMA shown in FIG. 16 (A) indicates the start of a series of data, and one circle is drawn outside the circle shown in FIG. 16 (B). The release mark RMB indicates the middle of a series of data, and the release mark RMC in which two circles are drawn outside the circle shown in FIG. 16C indicates the end of the series of data. it can. Here, it is desirable that the circle depicted in FIGS. 16A to 16C is displayed at a position corresponding to the code.
Further, as the release mark, the start of a series of data is indicated by the rectangular release mark RMD shown in FIG. 16D, and one border is drawn outside the rectangle shown in FIG. The release mark RME indicates the middle of a series of data, and the release mark RMF in which two borders are drawn outside the rectangle shown in FIG. 16F indicates the end of the series of data. it can. Here, the rectangle drawn in FIGS. 16D to 16F is preferably drawn in a size corresponding to the code and displayed at a position corresponding to the code.
Further, when the data can be divided into a plurality of groups including a series of data, a rectangular release mark RMG shown in FIG. 16G indicates the start of the series of data, and FIG. A release mark RMH in which one rectangle smaller than the rectangle is arranged on the right side of the rectangle shown in H) indicates that it is in the middle of a series of data, and the rectangle shown in FIG. A release mark RMI in which two rectangles smaller than the rectangle are arranged on the right side of FIG. 16 indicates the end of a series of data, and a rectangle smaller than the rectangle is shown on the right side of the rectangle shown in FIG. The start or end of the group can be indicated by three release marks RMJ arranged. Here, it is desirable that the rectangle drawn in FIGS. 16G to 16J is drawn in a size corresponding to the code and displayed at a position corresponding to the code.
As the release mark, the rectangular release mark RMK shown in FIG. 16 (K) indicates the start of a series of data, and one rectangle is drawn on each side of the rectangle shown in FIG. 16 (L). The release mark RME indicates the middle of a series of data, and the release mark RMM in which two rectangles are drawn on both sides of the rectangle shown in FIG. be able to. Here, the rectangle drawn in FIGS. 16K to 16M is preferably drawn in a size corresponding to the code and displayed at a position corresponding to the code.
In such a case, for example, information indicating the position of a series of data (for example, start, midway, end) and information indicating the start or end of the group to which the data belongs from the data division unit 113 to the release mark generation unit 221. give. Based on such information, the release mark generation unit 221 may generate a corresponding release mark.

In addition, if the image can be taken with the same field of view as the screen, a light emitting unit (not shown) such as an LED or a monitor lamp outside the screen can be blinked instead of the release mark.
In such a case, instead of the release mark generator 221, the setting terminal 210 synchronizes with the clock signal generated by the reference clock generator 115 and emits light for a first predetermined period (see FIG. (Not shown). For example, the light emitting unit may emit green light.
Furthermore, the imaging unit 131 of the camera 230 captures the screen 210a and the light emitting unit of the setting terminal 210 on which the video is displayed, and acquires the images of the screen 210a and the light emitting unit.
The camera 230 may include a light emission detection unit (not shown, hereinafter, also simply referred to as a detection unit) instead of the release mark recognition unit 238. The light emission detection unit detects light emission of the light emission unit from the image of the green component given from the color channel separation unit 132. The light emission detection unit recognizes the timing at which the light emission is detected as the timing at which the code can be read correctly. Then, the light emission detection unit instructs the image cutout unit 239 to cut out an image at the timing when the light emission is detected.

In addition, if the image can be taken with the same field of view as the screen, a light emitting unit such as an LED or a monitor lamp outside the screen can be blinked instead of the release mark.
FIG. 17 is a block diagram schematically showing a configuration of setting terminal 1210 in a modification of the second embodiment.
As shown in FIG. 17, the setting terminal 1210 is determined in advance in synchronization with the clock signal generated by the reference clock generator 115 instead of the release mark generator 221 shown in FIG. The light emitting unit 1221 that emits light for the first period is provided.
By configuring in this way, the data processing unit 1212 is configured to include the data dividing unit 1113, the first code generating unit 1220A, the second code generating unit 1220B, and the third code generating unit 1220C. Can do.
The data dividing unit 1113 divides the setting data for three channels.
The first code generation unit 1220A and the second code generation unit 1220B are the same as the first code generation unit 220A and the second code generation unit 220B in the second embodiment.
The third code generation unit 220C converts the third data divided by the data division unit 113 into a code, and generates a third code video in which the converted code (third code) appears in the second period.
The color channel combining unit 1217 includes a first code image provided from the first code generation unit 220A, a second code image provided from the second code generation unit 220B, and a third code image provided from the third code generation unit 220C. Are synthesized as if they belonged to independent color channels. Here, the color channel composition unit 1217 regards the third code image as a green (G) channel, the first code image as a red (R) channel, and the second code image as a blue (B) channel, and performs color channel composition. Just do it.
FIG. 18 is a block diagram schematically showing the configuration of the camera 1230 in a modification of the second embodiment.
The imaging unit 131 of the camera 1230 captures the screen 210a and the light emitting unit 1221 of the setting terminal 1210 on which the video is displayed, and acquires the images of the screen 210a and the light emitting unit 1221.
The camera 1230 includes a light emission detection unit 1238 instead of the release mark recognition unit 238 shown in FIG.
The light emission detection unit 1238 detects the light emission of the light emitting unit in the video from the positional relationship between the code and the light emitting unit 1221 and the blinking timing. The light emission detection unit 1238 recognizes the timing at which the light emission is detected as the timing at which the code can be read correctly. Then, the light emission detection unit 1238 instructs the image cutout unit 1239 to cut out an image at the timing when the light emission is detected.
The image cutout unit 1239 cuts out an image from a video of a predetermined color component at the timing when the light emission detection unit 1238 detects light emission. The image cutout unit 1239 includes a first image cutout unit 1239A, a second image cutout unit 1239B, and a third image cutout unit 1239C.
The first image cutout unit 1239A cuts out the image of the first code from the red component video given from the color channel separation unit 1132 at the timing instructed by the light emission detection unit 1238.
The second image cutout unit 1239B cuts out the image of the second code from the blue component video given from the color channel separation unit 1132 at the timing instructed by the light emission detection unit 1238.
The third image cutout unit 1239C cuts out an image of the third code from the green component video given from the color channel separation unit 1132 at the timing instructed by the light emission detection unit 1238.
The code decoding unit 1240 decodes a code included in the image cut out by the image cutout unit 1239, and acquires data corresponding to the decoded code. The code decoding unit 1240 includes a first code decoding unit 1240A, a second code decoding unit 1240B, and a third code decoding unit 1240C. The first code decoding unit 1240A and the second code decoding unit 1240B are the same as the first code decoding unit 240A and the second code decoding unit 240B shown in FIG.
The third code decoding unit 1240C decodes the third code included in the image cut out by the third image cutout unit 239C, and extracts third data.
The data synthesizing unit 1136 combines the first data, the second data, and the third data given from the code decoding unit 1240 to synthesize these data, and restores the transmitted setting data.

  In the second embodiment, the release mark is sent using a color channel different from that of the code. However, such a method may not be suitable. For example, when either the display unit or the imaging unit is monochrome, or when the color is separated using a rotary color filter and sequentially read for each color, such a method cannot be performed. Even when a plurality of color channels can be used, there are cases where it is desired to assign all channels to codes.

In these cases, a pattern image surrounding the code can be used as the release mark. 19 and 20 show an example of this case.
FIG. 19 is a schematic diagram showing a layout of release marks and codes.
As shown in FIG. 19, the release mark 260 is an image surrounding the code 261.
FIG. 20 is a schematic diagram showing temporal changes in the video displayed on the display unit 119 of the setting terminal 210. As shown in FIG. 20, since no release mark is displayed during the periods T1, T3, T4, and T6, code reading is not performed during these periods. On the other hand, since the release mark is displayed in the periods T2 and T5, the code is read during these periods.

In the frame shot at the moment of switching screen, the image may be incomplete, but if the entire release mark is complete, it can be considered that the code located inside the release mark is also read correctly .
By using such a release mark, it is possible to automatically and efficiently read codes that are displayed sequentially and sequentially.

  In the second embodiment described above, the color channel composition unit 217 of the setting terminal 210 performs composition using three color channels, but the present invention is not limited to such an example. . For example, the color channel synthesis unit 217 may perform synthesis using two color channels. In such a case, the data processing unit 212 of the setting terminal 210 may include only one of the first code generation unit 220A and the second code generation unit 220B. Then, the data processing unit 212 divides the setting data read from the setting data list storage unit 111 into data sizes that can be included in one code by the data dividing unit 113, and the first code generation unit 220A or What is necessary is just to give to any one of the 2nd code production | generation part 220B. In such a case, the color channel separation unit 132 of the camera 230 separates the video obtained from the imaging unit 131 into two-color video, gives one to the release mark recognition unit 238, and sends the other to the image. What is necessary is just to give to the cutout part 239. The image cutout unit 239 includes either the first image cutout unit 239A or the second image cutout unit 239B, and the code decoding unit 240 also includes the first code decoding unit 240A and the second code decoding unit 240B. Any one of them may be provided.

  In the second embodiment described above, the color channel synthesizing unit 217 generates a color synthesized video by allocating and synthesizing the clock video and the data video to the respective color channels. It is not limited to a specific example. For example, the release mark generation unit 221, the first code generation unit 220 </ b> A, and the second code generation unit 220 </ b> B create color images in which only the assigned color components are changed, and the color channel synthesis unit 217 generates them. You may synthesize.

Embodiment 3 FIG.
FIG. 21 is an overview diagram schematically showing the configuration of the information processing system 300 according to the third embodiment.
The information processing system 300 includes a setting terminal 310 as an information display device and a camera 330 as an imaging device. In the information processing system 300, the setting terminal 310 and the camera 330 are connected to the network 350.
In the third embodiment, information transmission using a combination of video display and imaging and information transmission using the network 350 are combined to realize a highly reliable system and method that can simplify the operation procedure.

The advantage of information transmission by imaging is that an operator needs to stand in front of the camera, and the possibility of erroneous setting can be reduced. There is also a point that unauthorized connection by outsiders can be psychologically suppressed.
Information transmission using a network has advantages such as remote control and transmission / reception of a large amount of data. On the other hand, there is a drawback that the possibility of erroneous setting is high, the erroneous setting is difficult to notice, and correction is difficult. Another disadvantage is that it is difficult to operate, for example, unauthorized connection attempts are normally made via the network.
Therefore, it is recommended that the surveillance network camera be connected to a dedicated network that is disconnected from the outside. On the other hand, in order to realize low cost, there is a demand for realizing a surveillance camera network using the existing Internet or an in-house network connected to the outside.

  Also in the third embodiment shown in FIG. 2, the setting operation is performed by imaging the screen 310 a of the setting terminal 310 with the camera 330 with the setting terminal 310 and the camera 330 facing each other. However, as shown in FIG. 21, in the third embodiment, the setting terminal 310 and the camera 330 are separately connected to the network 350.

FIG. 22 is a block diagram schematically showing the configuration of the setting terminal 310.
The setting terminal 310 includes a setting data list storage unit 111, a tally generation unit 322, a display video generation unit 318, a display unit 119, a network interface unit (hereinafter referred to as an NI / F unit) 323, and an encryption processing unit. 324, a tally collation unit 325, and a control unit 326.

The setting data list storage unit 111 stores setting data including setting values for setting in the camera 330.
The tally generation unit 322 is a verification information generation unit that generates tally as verification information.

The display video generation unit 318 combines a part of the setting data extracted from the setting data list storage unit 111 with the tally generated by the tally generation unit 322 to display video (display video) that can be displayed on the display unit 119. ) To generate a display image.
The display unit 119 displays the video generated by the display video generation unit 318 on the screen.

The NI / F unit 323 communicates with the network 350.
The encryption processing unit 324 decrypts the data received by the NI / F unit 323.
The tally collation unit 325 is a collation unit that collates the tally included in the data decrypted by the encryption processing unit 324 with the tally generated by the tally generation unit 322 and determines whether or not the collation is successful. is there. For example, the tally verification unit 325 determines that the verification is successful when the tally included in the data decrypted by the encryption processing unit 324 matches the tally generated by the tally generation unit 322.
The control unit 326 controls the entire processing at the setting terminal 310.

FIG. 23 is a block diagram schematically showing the configuration of the camera 330.
The camera 330 includes an imaging unit 131, a decoding unit 341, a tally generation unit 342, a tally collation unit 343, an auxiliary setting storage unit 344, a setting storage unit 137, an encryption processing unit 345, a network interface unit (hereinafter referred to as a network interface unit). , NI / F unit) 346 and a control unit 347. Since the imaging unit 131 and the setting storage unit 137 are the same as those in the first embodiment, description thereof is omitted.

The decoding unit 341 decodes the video acquired by the imaging unit 131 and extracts data. For example, the decoding unit 341 acquires a tally from the video acquired by the imaging unit 131.
The tally generation unit 342 is a verification information generation unit that generates tally as verification information.
The tally collation unit 343 is a collation unit that collates the tally included in the data extracted by the decoding unit with the tally generated by the tally generation unit 342 and determines whether or not the collation is successful. . For example, when these tallys match, it is determined that the collation is successful.
The auxiliary setting storage unit 344 is a temporary storage area for performing operations with temporary settings until the settings are confirmed.
The encryption processing unit 345 performs encryption processing and decryption processing.
The NI / F unit 346 communicates with the network 350.
The control unit 347 controls the entire processing in the camera 330.

Next, the operation will be described with reference to FIG.
FIG. 24 is a flowchart showing operations in the information processing system 300 according to the third embodiment.
In the setting terminal 310, when a predetermined program, for example, a setting transmission program for setting the camera 330 is activated, the setting terminal 310 starts operation. The control unit 326 reads out the setting data for the camera 330 to be used for the current setting from the plurality of setting data stored in the setting data list storage unit 111 by the operator's operation, and generates a display image. It gives to the part 318 (S70).
The tally generator 322 of the setting terminal 310 generates a tally T used for the current setting work (S71).
Based on the setting data read out in step S70, the display video generation unit 318 includes the tally generated in step S71 along with information necessary for the camera 330 to connect to the setting terminal 310 via the network 350. T is visualized and the image is displayed on the display unit 119 (S72). Information necessary for network connection is, for example, the network address and port number of the setting terminal 310, the communication method, the temporary network address and mask length for the camera 330, the gateway, route information, and the like.

In the camera 330, when a predetermined program, for example, a setting acquisition program for setting the camera 330 is activated, the camera 330 starts its operation. The control unit 347 confirms the setting state (S90). For example, the control unit 347 performs this confirmation depending on whether or not the setting of the camera 330 is stored in the setting storage unit 137. If it has been set (Yes in Step S90), the control unit 347 proceeds to the monitoring mode in Step S105, and if not set (No in Step S90), the control unit 347 proceeds to Step S91 and starts the setting operation.
The imaging unit 131 images the screen 310a of the setting terminal 310 on which the video is displayed (S92), and the decoding unit 341 decodes the captured video and extracts data (S93). Here, it is assumed that the extracted data includes connection data for connecting to the setting terminal 310 and a tally T. The control unit 347 stores the data extracted by the decoding unit 341 in the auxiliary setting storage unit 344.
The tally generator 342 generates a tally C for continuing the connection with the setting terminal 310 (S94).
The control unit 347 temporarily sets the NI / F unit 346 of the camera 330 itself from the data stored in the auxiliary setting storage unit 344 (S95).
When the temporary setting of the NI / F unit 346 is completed, the encryption processing unit 345 encrypts the tally T transmitted from the setting terminal 310 and the tally C generated by the tally generation unit 342, and the NI / F unit 346 The encrypted information is sent to the setting terminal 310 via the network 350 (S96).

In the setting terminal 310, the NI / F unit 323 receives the information (S73). The encryption processing unit 324 decrypts the received information and extracts the tally T and the tally C.
The tally verification unit 325 confirms whether or not the extracted tally T is issued by the setting terminal 310 (S74). If the tally T is issued by the setting terminal 310 itself, it can be considered that the camera 330 communicating on the network 350 exists within a range where the screen of the setting terminal 330 can be photographed. . If the tally T is issued by the setting terminal 310 itself (Yes in Step S74), the process proceeds to Step S75, and if the tally T is not issued by the setting terminal 310 itself (Step S74). In S74, No), the process returns to step S73.
In step S75, the display video generation unit 318 converts the tally C extracted in step S73 into a display video that can be displayed on the display unit 119, and the display unit 119 displays the converted video of the tally C. To do. At this time, the tally T itself may be used as the encryption key.

In the camera 330, the imaging unit 131 images the screen 310a of the setting terminal 310 on which the video is displayed (S97), and the decoding unit 341 decodes the captured video and extracts data (S98). It is assumed that tally C is included in the data extracted here.
The tally verification unit 343 confirms whether or not the extracted tally C is issued by the camera 330 (S99). When the tally C is issued by the camera 330, it can be considered that the setting terminal 310 communicating via the network 350 is actually present in the shootable range of the camera 330. If the tally C is issued by the camera 330 (Yes in step S99), the process proceeds to step S100. If the tally C is not issued by the camera 330 (No in step S99), the process proceeds to step S100. The process returns to step S97.
In step S <b> 100, the control unit 347 generates a partner-confirmed notification and gives the notification to the encryption processing unit 345. The encryption processing unit 345 encrypts the notification and gives it to the NI / F unit 346, and the NI / F unit 346 transmits the encrypted notification to the setting terminal 310.

In the setting terminal 310, the NI / F unit 323 receives the notification (S76). The encryption processing unit 324 decrypts the received notification and gives it to the control unit 326. As a result, both parties have been able to confirm the communicating party on the network 350.
In this way, by confirming both the tally T and the tally C with the respective issuers, the setting terminal 310 and the camera 330 have a one-to-one relationship with the counterparts connected by the network 350, It was confirmed that they actually exist within the viewing angle of each other spatially. In this way, it is possible to prevent misconfiguration, duplication setting, and the like, and to prevent unauthorized setting attempts via the network 350, such as impersonation (spoofing) of connection destinations and man-in-the-middle attacks.

  Since both parties have been confirmed at this time, the control unit 326 of the setting terminal 310 uses the setting data for the official camera 330 (the remaining portion displayed in step S72) as the encryption processing unit 324 and the NI. The data is transmitted via the network 350 using the / F unit 323 (S77). The data transmitted here is, for example, a camera network address, a mask length, a default gateway address, route information, communication method information, a camera identifier, and an electronic certificate. Here, the data is transmitted via the network 350, but the control unit 310 may use the display video generation unit 318 to visualize the data to be transmitted and display the data on the display unit 119.

In the camera 330, when the NI / F unit 346 receives the setting data (S101), after confirming the consistency, the setting is stored in the setting storage unit 137 based on the received setting data (S102).
After completing the setting storage, the control unit 347 transmits a setting completion notification to the setting terminal 310 using the encryption processing unit 345 and the NI / F unit 346 (S103). Thereafter, the control unit 347 performs formal network setting based on the setting stored in the setting storage unit 137, and switches from the temporary network setting set in step S95 (S104). And the control part 347 transfers to the monitoring operation | movement of step S105.

  In the setting terminal 310, the NI / F unit 323 receives the setting completion notification (S78), and the control unit 321 records the set information in the setting data read in step S70 in the setting data list storage unit 311. (S79). Then, the control unit 321 ends the setting operation.

  Since the information processing system 300 according to the third embodiment is configured as described above, the camera 330 and the setting terminal 310 are one-to-one even in the initial state of the camera 330 that does not yet have an electronic certificate. While confirming that it is compatible, in other words, data can be transmitted to the camera 330 for setting while ensuring that the setting operation is actually performed in front of the camera 330 at the time of setting.

  Once the electronic certificate is stored in the camera 330, the control unit 347 can check the other party using the electronic certificate as long as the other party has an electronic certificate belonging to the same issuer. The setting can be changed using only the network 350.

  In this example, the electronic certificate is sent after confirming the other party using a tally, but in an environment where there is no worry of voyeurism, the electronic certificate may be sent simply by using transfer by video reading. .

  In the third embodiment, when the setting terminal 310 visualizes information and when the camera 330 extracts information from the captured video, the setting terminal 110 described in the first or second embodiment. , 210 and the cameras 130 and 230 may be performed. In such a case, the setting terminal 310 and the camera 330 are assumed to have the configuration described in the first or second embodiment.

  100, 200, 300 Information processing system, 110, 210, 1210, 310 Setting terminal, 111 Setting data list storage unit, 112, 212, 1212 Data processing unit, 113, 1113 Data dividing unit, 114A First data visualization unit , 114B second data visualization unit, 115 reference clock generation unit, 116 clock visualization unit, 117, 217, 1217 color channel synthesis unit, 118, 318 display video generation unit, 119 display unit, 220A, 1220A first code generation , 220B, 1220B second code generation unit, 1220C third code generation unit, 221 release mark generation unit, 1221 light emission unit, 322 tally generation unit, 323 NI / F unit, 324 encryption processing unit, 325 tally verification unit, 326 Control unit 130, 230, 1230, 330 Camera, 131 Imaging unit, 132, 1132 Color channel separation unit, 133 Data waveform specifying unit, 133A First data waveform specifying unit, 133B Second data waveform specifying unit, 134 Clock waveform specifying unit, 135 Data acquisition unit, 135A first data acquisition unit, 135B second data acquisition unit, 136, 1136 data synthesis unit, 137 setting storage unit, 238 release mark recognition unit, 1238 emission detection unit, 239, 1239 image cutout unit, 239A , 1239A First image extraction unit, 239B, 1239B Second image extraction unit, 1239C Third image extraction unit, 240, 1240 Code decoding unit, 240A, 1240A First code decoding unit, 240B, 1240B Second code decoding Part 1240 The third code decoder, 341 decoding unit, 342 tally generation unit, 343 a tally matching unit, 345 encryption processing part, 346 NI / F unit, 347 control unit, 350 network.

Claims (12)

Capture a screen that displays a composite video in which a clock video in which the density of the first color component changes according to the waveform of the clock signal and a data video in which the density of the second color component changes in accordance with the waveform of the data signal An imaging unit for acquiring video of the screen;
A separation unit that separates the first color component image and the second color component image from the image acquired by the imaging unit;
A clock waveform specifying unit that specifies a waveform of the clock signal from a change in shading in the video of the first color component separated by the separation unit;
A data waveform specifying unit for specifying a waveform of the data signal from a change in shading in the video of the second color component separated by the separation unit;
Based on the remaining waveform obtained by subtracting the waveform of the data signal specified by the data waveform specifying unit from the waveform of the clock signal specified by the clock waveform specifying unit, the data waveform specified by the data waveform specifying unit A data acquisition unit for acquiring data indicated by a waveform;
A storage unit that stores data acquired by the data acquisition unit. An information processing system comprising an information display device and an imaging device,
The information display device includes:
A reference clock generator for generating a clock signal;
A clock imaging unit for generating a clock image whose density changes according to the waveform of the clock signal generated by the reference clock generation unit;
A storage unit for storing data;
Data in which the data stored in the storage unit is modulated in synchronization with the clock signal generated by the reference clock generation unit to generate a data signal, and the data whose density changes according to the waveform of the generated data signal A data processing unit for generating video;
A synthesizing unit that generates a synthesized image obtained by synthesizing the clock image generated by the clock imaging unit using the first color component and the data image generated by the data processing unit using the second color component; ,
A display unit for displaying the synthesized video generated by the synthesis unit on a screen,
The imaging device
An imaging unit that captures the screen on which the composite video is displayed and acquires the video of the screen;
A separation unit that separates the first color component image and the second color component image from the image acquired by the imaging unit;
A clock waveform specifying unit that specifies a waveform of the clock signal from a change in shading in the video of the first color component separated by the separation unit;
A data waveform specifying unit for specifying a waveform of the data signal from a change in shading in the video of the second color component separated by the separation unit;
Based on the remaining waveform obtained by subtracting the waveform of the data signal specified by the data waveform specifying unit from the waveform of the clock signal specified by the clock waveform specifying unit, the data waveform specified by the data waveform specifying unit A data acquisition unit for acquiring data indicated by a waveform;
A storage unit that stores the data acquired by the data acquisition unit. A light emitting unit that emits light for the first period with the first color component, and a screen on which a code image in which the code drawn with the second color component appears in the second period including the first period is displayed. An imaging unit that acquires the image of the light emitting unit and the screen;
A separation unit that separates the first color component image and the second color component image from the image acquired by the imaging unit;
A detection unit for detecting light emission of the light emitting unit from an image of the first color component separated by the separation unit;
An image cutout unit that cuts out an image from the video of the second color component separated by the separation unit at a timing when the detection unit detects light emission of the light emitting unit;
A code decoding unit that decodes a code included in the image cut out by the image cutout unit and obtains data corresponding to the decoded code;
An imaging apparatus comprising: a storage unit that stores data acquired by the code decoding unit. A reference clock generator for generating a clock signal;
A light emitting unit that emits light for a first period with a first color component in synchronization with a clock signal generated by the reference clock generating unit;
A storage unit for storing data;
The data stored in the storage unit is converted into a code, and the converted code appears in a second period including the first period in synchronization with the clock signal generated by the reference clock generation unit. A code generator for generating a code image;
A combining unit that generates a combined image obtained by combining the code image generated by the code generating unit using a second color component;
An information display device comprising: a display unit configured to display a synthesized video generated by the synthesis unit on a screen. An information processing system comprising an information display device and an imaging device,
The information display device includes:
A reference clock generator for generating a clock signal;
A light emitting unit that emits light for a first period with a first color component in synchronization with a clock signal generated by the reference clock generating unit;
A storage unit for storing data;
The data stored in the storage unit is converted into a code, and the converted code appears in a second period including the first period in synchronization with the clock signal generated by the reference clock generation unit. A code generator for generating a code image;
A combining unit that generates a combined image obtained by combining the code image generated by the code generating unit using a second color component;
A display unit for displaying the synthesized video generated by the synthesis unit on a screen,
The imaging device
An imaging unit that captures the screen on which the light emitting unit and the composite video are displayed, and acquires the video of the light emitting unit and the screen;
A separation unit that separates the first color component image and the second color component image from the image acquired by the imaging unit;
A detection unit for detecting light emission of the light emitting unit from an image of the first color component separated by the separation unit;
An image cutout unit that cuts out an image from the video of the second color component separated by the separation unit at a timing when the detection unit detects light emission of the light emitting unit;
A code decoding unit that decodes a code included in the image cut out by the image cutout unit and obtains data corresponding to the decoded code;
A storage unit that stores data acquired by the code decoding unit. The light emitting unit that emits light for the first period and the screen on which the drawn code displays the code image that appears in the second period including the first period are captured, and the light emitting unit and the image of the screen are acquired. An imaging unit;
A detection unit for detecting light emission of the light emitting unit from the video acquired by the imaging unit;
An image cutout unit that cuts out an image from the video acquired by the imaging unit at a timing when the detection unit detects light emission of the light emitting unit;
A code decoding unit that decodes a code included in the image cut out by the image cutout unit and obtains data corresponding to the decoded code;
An imaging apparatus comprising: a storage unit that stores data acquired by the code decoding unit. A reference clock generator for generating a clock signal;
A light emitting unit that emits light for a first period in synchronization with the clock signal generated by the reference clock generating unit;
A storage unit for storing data;
The data stored in the storage unit is converted into a code, and the converted code appears in a second period including the first period in synchronization with the clock signal generated by the reference clock generation unit. A code generator for generating a code image;
An information display device comprising: a display unit configured to display a code image generated by the code generation unit on a screen. An information processing system comprising an information display device and an imaging device,
The information display device includes:
A reference clock generator for generating a clock signal;
A light emitting unit that emits light for a first period in synchronization with the clock signal generated by the reference clock generating unit;
A storage unit for storing data;
The data stored in the storage unit is converted into a code, and the converted code appears in a second period including the first period in synchronization with the clock signal generated by the reference clock generation unit. A code generator for generating a code image;
A display unit for displaying the code video generated by the code generation unit on a screen,
The imaging device
An imaging unit that images the light emitting unit and the screen, and acquires an image of the light emitting unit and the screen;
A detection unit for detecting light emission of the light emitting unit from the video acquired by the imaging unit;
An image cutout unit that cuts out an image from the video acquired by the imaging unit at a timing when the detection unit detects light emission of the light emitting unit;
A code decoding unit that decodes a code included in the image cut out by the image cutout unit and obtains data corresponding to the decoded code;
A storage unit that stores data acquired by the code decoding unit. A collation information generating unit that generates first collation information;
An encryption processing unit that encrypts the first verification information generated by the verification information generation unit;
An interface unit for transmitting the first verification information encrypted by the cryptographic processing unit to a network;
An imaging unit that captures a screen on which a video obtained by converting the second verification information is displayed, and acquires the video of the screen;
A decoding unit that acquires the second verification information from the video acquired by the imaging unit;
A collation unit that collates the second collation information acquired by the decoding unit with the first collation information generated by the collation information generation unit and determines whether the collation is successful;
When the collation unit determines that the collation is successful, a control unit that acquires data via the interface unit or the imaging unit;
A storage unit for storing data acquired by the control unit ,
The encryption processing unit encrypts the first verification information generated by the verification information generation unit using the verification information used by the data transmission source as an encryption key. . A collation information generating unit that generates first collation information;
A storage unit for storing data;
A display video generation unit that generates a first video obtained by converting a part of the data and the first verification information generated by the verification information generation unit;
A display unit for displaying the first video generated by the display video generation unit on a screen;
An interface unit that receives the second verification information encrypted via the network;
An encryption processing unit for retrieving the second verification information by decrypting the encrypted second verification information received by the interface unit;
A collation unit that collates the second verification information extracted by the cryptographic processing unit with the first verification information generated by the verification information generation unit and determines whether the verification is successful. When,
When the collation unit determines that the collation is successful, the interface unit transmits the rest of the data, or the display video generation unit generates a second video obtained by converting the remaining data, A control unit that causes the display unit to display the second video generated by the display video generation unit ,
The second verification information is encrypted using the first verification information as an encryption key,
The encryption processing unit uses the first verification information as a decryption key to decrypt the encrypted second verification information received by the interface unit . An information processing system comprising an imaging device and an information display device,
The imaging device
A collation information generating unit that generates first collation information;
An encryption processing unit that encrypts the first verification information generated by the verification information generation unit;
An interface unit that transmits the first verification information encrypted by the cryptographic processing unit to the information display device;
An imaging unit that captures an image of the information display device on which an image obtained by converting the second verification information is displayed;
A decoding unit that acquires the second verification information from the video acquired by the imaging unit;
A collation unit that collates the second collation information acquired by the decoding unit with the first collation information generated by the collation information generation unit and determines whether the collation is successful;
A control unit that acquires data from the information display device via the interface unit or the imaging unit when the verification unit determines that the verification is successful;
A storage unit for storing data acquired by the control unit ,
The encryption processing unit encrypts the first verification information generated by the verification information generation unit using the verification information used by the data transmission source as an encryption key. system. An information processing system comprising an imaging device and an information display device,
The information display device includes:
A collation information generating unit that generates first collation information;
A storage unit for storing data;
A display video generation unit that generates a first video obtained by converting a part of the data and the first verification information generated by the verification information generation unit;
A display unit for displaying the first video generated by the display video generation unit on a screen;
An interface unit for receiving the encrypted second verification information via the network;
An encryption processing unit for retrieving the second verification information by decrypting the encrypted second verification information received by the interface unit;
A collation unit that collates the second verification information extracted by the cryptographic processing unit with the first verification information generated by the verification information generation unit and determines whether the verification is successful. When,
When the collation unit determines that the collation is successful, the interface unit transmits the remainder of the data to the imaging device, or generates a second video obtained by converting the remainder of the data. And a control unit that causes the display unit to display the second video generated by the display video generation unit ,
The second verification information is encrypted using the first verification information as an encryption key,
The information processing system, wherein the encryption processing unit decrypts the encrypted second verification information received by the interface unit, using the first verification information as a decryption key .

Images