JP2012244601A - Information acquisition device, information output device, information communication system, information acquisition method, information output method, control program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To securely communicate information hard to be sensed by a viewer in real time from a display device having a displayed moving image or a still image.SOLUTION: An information acquisition device includes: an image acquisition unit 21 for successively acquiring images of a display screen, having a displayed moving image or a still image, imaged by a camera 13; a representative brightness value identification unit 22 for identifying a representative brightness value from the images acquired by the image acquisition unit 21; a bit stream detection unit 24 for detecting a brightness change coincident with a brightness change, identified by a plurality of representative brightness values identified by the representative brightness value identification unit 22 from the successively acquired images, from conversion information including one or a plurality of sets of communication information associated with the brightness changes in a predetermined period; and a position identification unit 25 for identifying the communication information associated with the brightness change detected by the bit stream detection unit 24 from the conversion information.

Description

  The present invention relates to an information acquisition device, an information output device, an information communication system, an information acquisition method, an information output method, a control program, and a recording medium that communicate information using a signal that is difficult for humans to detect.

  A technique has been developed in which a moving image embedded with a digital watermark displayed on a display device such as a television is captured by a camera, and information that is not perceived by a viewer is acquired from the display device.

  For example, Patent Document 1 describes a method of capturing a moving image of a television or the like with a camera and performing digital watermark detection in real time from the captured moving image. Japanese Patent Application Laid-Open No. 2004-228867 specifies a part in a captured frame image from which a digital watermark should be detected, and reliably performs digital watermark detection under various shooting angles and background image conditions. Is described.

International Publication No. 2007-015452 (Released on February 8, 2007)

  However, the conventional technology as described above has a problem in that there is a restriction on conditions under which real-time information that is difficult to be perceived by a viewer can be reliably acquired from a display device displaying a moving image.

  Specifically, first, the entire display screen of the television needs to be included in the captured frame image. In other words, in the conventional technique, when a part of the display screen of the television is photographed, it is impossible to detect all the embedded digital watermarks and accurate information cannot be acquired.

  Even when the entire television display screen is included in the frame image, if the television display screen is distorted, it is necessary to correct the distortion. Therefore, a processing time for executing correction is required. Therefore, when a television is photographed from an oblique direction, a digital watermark cannot be detected in real time, and a delay may occur.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to obtain information capable of reliably communicating information that is difficult to be perceived by a viewer from a display device displaying a moving image or a still image in real time. An apparatus, an information output device, an information communication system, an information acquisition method, an information output method, a control program, and a recording medium are provided.

  In order to solve the above problems, an information acquisition apparatus according to the present invention acquires an image acquisition unit that sequentially acquires images obtained by capturing a display screen displaying a moving image or a still image with a camera, and the image acquisition unit acquires From the image, the representative luminance value specifying means for specifying the representative luminance value, and the conversion information are associated with one or a plurality of sets of communication information and luminance changes in a predetermined period. A luminance change detecting means for detecting a luminance change corresponding to the luminance change specified by the plurality of representative luminance values specified by the representative luminance value specifying means, and the luminance change detection from the conversion information. Communication information specifying means for specifying communication information associated with the luminance change detected by the means.

  In order to solve the above problems, the information acquisition method according to the present invention is acquired in an image acquisition step of sequentially acquiring images obtained by capturing a display screen displaying a moving image or a still image with a camera, and in the image acquisition step. From the image, the representative luminance value specifying step for specifying the representative luminance value, and the conversion information are associated with one or a plurality of sets of communication information and a change in luminance in a predetermined period. From the conversion information, a luminance change detection step for detecting a luminance change that matches a luminance change specified by a plurality of representative luminance values specified in the representative luminance value specifying step from the image, and from the conversion information, A communication information specifying step for specifying communication information associated with the luminance change detected in the luminance change detecting step. It is characterized by a door.

  According to the above configuration, the representative luminance value specifying unit specifies a representative luminance value from the luminance value of each pixel included in an image obtained by capturing a display screen displaying a moving image or a still image, and the luminance The change detecting means detects, from the conversion information, a luminance change that matches a luminance change specified by the plurality of representative luminance values specified by the representative luminance value specifying means from a plurality of images sequentially acquired from the camera, The communication information specifying unit specifies communication information associated with the luminance change detected by the luminance change detecting unit from the conversion information.

  Therefore, there is an effect that communication information can be acquired based on a luminance change of an image obtained by capturing a display screen displaying a moving image or a still image.

  Here, the display device including the display screen transmits the communication information while changing the luminance value of the entire display screen. Therefore, the information acquisition device can acquire communication information by imaging a part of the display screen of the display device. In addition, even if the entire screen or part of the display screen is shot by moving the shooting range instead of shooting the display screen at a fixed point, by capturing the luminance change of the entire image acquired by the image acquisition means Communication information can be detected.

  Also, even when the shooting range moves, the luminance of the portion other than the display screen (for example, the background) on the captured image usually does not change abruptly. Further, the luminance of the original image of the moving image or still image video content output from the display device is unlikely to change suddenly between consecutive frames.

  Therefore, for example, when the communication device outputs communication information by changing the luminance at a time interval equal to or less than the frame interval (1/60 seconds), the information acquisition device detects a predetermined luminance change, It can be determined that the luminance change is a luminance change corresponding to the communication information transmitted from the display device. Therefore, even if the shooting range moves, the information acquisition device can detect communication information with high accuracy if the captured image includes at least a part of the display screen.

  The information acquisition apparatus according to the present invention associates the communication information specified by the communication information specifying unit with the region information indicating the region where the luminance change detecting unit detects the luminance change corresponding to the communication information. It is preferable to further include position information generating means for acquiring.

  According to the above configuration, the position information generating means includes the communication information specified by the communication information specifying means and area information indicating an area where the luminance change detecting means has detected a luminance change corresponding to the communication information. Acquired in association. Therefore, the position information generating unit can specify the position on the image where the communication information is detected.

  The information acquisition apparatus according to the present invention further includes a transmission unit that transmits communication information associated with the display device to the display device including the display screen, and the position information generation unit includes the transmission information. Preferably, the device position information is generated by associating the device information indicating the display device with the area information corresponding to the communication information specified by the communication information specifying device.

  According to said structure, the said brightness change detection means detects the brightness change corresponding to the communication information which the said transmission means transmitted in the said conversion information. In other words, the luminance change detecting means detects a luminance change corresponding to communication information associated with the display device in the conversion information. Then, the communication information specifying means specifies communication information associated with the display device, and the position information generating means displays the display information in the area information associated with the communication information specified by the communication information specifying means. Device position information is generated in association with device information indicating the device.

  Here, when the communication information specifying unit specifies communication information associated with the display device, the luminance change detected by the luminance change detection unit is considered to be a luminance change of light output from the display device. It is done. That is, it is considered that the display device is imaged in an area where the luminance change detecting unit detects a luminance change associated with the communication information. Therefore, it is possible to estimate the position on the image of the transmission destination display device to which the transmission means has transmitted the communication information.

  The information acquisition device according to the present invention further includes a receiving unit that receives communication information associated with the display device from a display device including the display screen, and the position information generating unit includes: It is preferable to generate device position information by associating the device information indicating the display device with the region information corresponding to the communication information received and specified by the communication information specifying means.

  According to said structure, the said brightness change detection means detects the brightness change corresponding to the communication information which the said receiving means received in the said conversion information. In other words, the luminance change detecting means detects a luminance change corresponding to communication information associated with the display device in the conversion information. Then, the communication information specifying means specifies communication information associated with the display device, and the position information generating means displays the display information in the area information associated with the communication information specified by the communication information specifying means. Device position information is generated in association with device information indicating the device.

  Here, when the communication information specifying unit specifies communication information associated with the display device, the luminance change detected by the luminance change detection unit is considered to be a luminance change of light output from the display device. It is done. That is, it is considered that the display device is imaged in an area where the luminance change detecting unit detects a luminance change associated with the communication information. Therefore, the position on the image of the display device that is the transmission source of the communication information received by the receiving unit can be estimated.

  Further, in the information acquisition device according to the present invention, the representative luminance value specifying unit divides the image acquired by the image acquisition unit into a plurality of regions, and each region included in the region is obtained for each region obtained by the division. The representative luminance value of the area is specified from the luminance value of the pixel, and the luminance change detecting means is configured to output a plurality of representative luminance values specified by the representative luminance value specifying means from the same area of the sequentially acquired images. It is preferable to detect a luminance change that matches the luminance change specified by the conversion information.

  According to the above configuration, the representative luminance value specifying unit divides the image acquired by the image acquiring unit into a plurality of regions, and for each region obtained by the division, the luminance value of each pixel included in the region From the same area of the plurality of sequentially acquired images, the brightness change detecting means is specified by a plurality of representative brightness values specified by the representative brightness value specifying means, respectively. A luminance change that matches the luminance change is detected from the conversion information. That is, the brightness change detecting means detects a brightness change in units of regions obtained by division. Therefore, the position information generating means can specify the position where the communication information is detected in units of areas obtained by division.

  In the information acquisition apparatus according to the present invention, the representative luminance value specifying unit detects a motion vector of a subject from a plurality of images sequentially acquired by the image acquisition unit, and a plurality of the corrected images based on the motion vector are detected. It is preferable to divide the area.

  According to the above configuration, the representative luminance value specifying unit detects a motion vector of the subject from the plurality of images sequentially acquired by the image acquisition unit, and divides the image after correction by the motion vector into a plurality of regions. To do. Therefore, communication information can be detected with high accuracy even if the display screen moves beyond the region in a plurality of captured images.

  The information acquisition device according to the present invention may be configured such that the communication information specifying unit detects the same luminance change when the luminance change detecting unit detects the same luminance change from a plurality of regions included in a predetermined range on the image. It is preferable to specify communication information associated with the luminance change.

  According to the above configuration, the communication information specifying unit responds to the luminance change when the luminance change detecting unit detects the same luminance change from a plurality of areas included in a predetermined range on the image. Identify attached communication information. Therefore, even if the luminance change due to noise coincides with the luminance change included in the conversion information in one region of the plurality of images acquired by the image acquisition means, the luminance change is erroneously detected as being due to communication information. Can be prevented.

  Further, the information acquisition device according to the present invention indicates the device information corresponding to the region information at the display position of the screen of the display unit corresponding to the region indicated by the region information included in the device position information. It is preferable to further include display control means for displaying display device related information regarding the display device.

  According to said structure, the said display control means is a display apparatus which the apparatus information corresponding to the said area | region information shows in the display position of the screen of the said display part corresponding to the area | region which the area | region information contained in the said apparatus positional information shows. Display device-related information about is displayed. Therefore, the display control means can display the display device related information related to the display device in correspondence with the estimated position of the display device on the image. Since the position of the display device on the image captured by the camera may correspond to the actual position of the display device as seen by the user, which is the display device related information displayed on the display unit by the user It is possible to intuitively understand whether the display device is related.

  In the information acquisition device according to the present invention, it is preferable that the display control unit displays the display device related information so as to be superimposed on the image acquired by the image acquisition unit.

  According to said structure, the said display control means displays the said display apparatus related information on the image acquired by the said image acquisition means, superimposed. Therefore, the display control means can display the display device related information regarding the display device at the estimated position on the image of the display device. Therefore, the user can intuitively understand to which display device the display device related information related to the display device shown in the image captured by the camera relates.

  An information output apparatus according to the present invention is an information output apparatus that includes a display unit and displays a moving image or a still image on the display unit in order to solve the above-described problem. Information converting means for converting the luminance value to be output into luminance change transition information indicating the luminance change in a predetermined period, and according to the luminance change indicated by the luminance change transition information converted by the information converting means, for the predetermined period, And a luminance value adjusting means for adjusting the luminance value and displaying it on the display unit.

  An information output method according to the present invention is an information output method of an information output device that includes a display unit and displays a moving image or a still image on the display unit in order to solve the above-described problem. An information conversion step for converting the luminance value output from the display unit into luminance change transition information indicating a luminance change in a predetermined period, and a predetermined period according to the luminance change indicated by the luminance change transition information converted in the information conversion step, And a luminance value adjusting step for adjusting the luminance value of a moving image or a still image and displaying it on the display unit.

  According to said structure, the said information conversion means converts communication information into the brightness change transition information which shows the brightness change in the predetermined period of the brightness value which the said display part outputs, The said brightness value adjustment means is the said information In accordance with the luminance change indicated by the luminance change transition information converted by the conversion means, the luminance value of the moving image or still image is adjusted and displayed on the display unit for a predetermined period.

  Therefore, there is an effect that communication information can be output as a luminance change.

  Here, one frame of a moving image is generally 1/60 second. For example, when the luminance change is performed on a frame basis, it is considered that a human is relatively difficult to detect the luminance change of the display unit. Therefore, the information output device can output communication information without being detected by a human. Furthermore, since it is difficult for humans to detect, it is possible to output communication information without impairing the quality of the moving image or still image displayed on the display unit.

  An information communication system according to the present invention includes the information acquisition device and the information output device.

  According to said structure, an information processing system has an effect similar to the said information acquisition apparatus and the said information output device.

  The information acquisition device and the information output device may be realized by a computer. In this case, the information acquisition device is operated by causing the computer to operate as each unit of the information acquisition device and the information output device. A control program for realizing the information output device by a computer and a computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  As described above, the information acquisition apparatus according to the present invention includes an image acquisition unit that sequentially acquires images obtained by capturing a display screen displaying a moving image or a still image with a camera, and an image acquired by the image acquisition unit. In the representative luminance value specifying means for specifying the representative luminance value, the conversion information is associated with one or a plurality of sets of communication information and a luminance change in a predetermined period. A luminance change detecting unit that detects a luminance change that matches a luminance change specified by a plurality of representative luminance values specified by the representative luminance value specifying unit, and a luminance change detecting unit that detects the luminance change from the conversion information. Communication information specifying means for specifying communication information associated with the luminance change.

  Further, the information acquisition method according to the present invention includes an image acquisition step of sequentially acquiring images obtained by capturing a display screen displaying a moving image or a still image with a camera, and a representative luminance from the image acquired in the image acquisition step. In the representative luminance value specifying step for specifying the value and the conversion information, one or a plurality of sets of communication information and luminance changes in a predetermined period are associated, and each of the representative images is obtained from the sequentially acquired images. The luminance change detecting step for detecting the luminance change matching the luminance change specified by the plurality of representative luminance values specified in the luminance value specifying step, and detecting the luminance change in the luminance change detecting step from the conversion information. A communication information specifying step for specifying communication information associated with the brightness change.

  Therefore, there is an effect that communication information can be acquired based on a luminance change of a predetermined area of an image obtained by capturing a display screen displaying a moving image or a still image.

  Further, the information output device according to the present invention is configured to convert the communication information into luminance change transition information indicating luminance change in a predetermined period of the luminance value output from the display unit, and the information converting unit converts the communication information. According to the luminance change indicated by the luminance change transition information, there is provided a luminance value adjusting unit that adjusts the luminance value of the moving image or the still image for a predetermined period and displays it on the display unit.

  Further, the information output method according to the present invention converts the communication information into luminance change transition information indicating luminance change in a predetermined period of the luminance value output from the display unit, and is converted in the information conversion step. And a luminance value adjusting step of adjusting the luminance value of the moving image or still image for a predetermined period according to the luminance change indicated by the luminance change transition information and displaying it on the display unit.

  Therefore, the communication information can be output as a luminance change.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1, showing an embodiment of the present invention, is a block diagram illustrating an example of a main part configuration of a mobile phone. It is a figure which shows the outline | summary of the optical communication utilized by this invention. It is a figure which shows the outline | summary of the information communication system containing the said mobile phone. It is a figure which shows an example of an image division | segmentation, (a) is a figure which shows the example which divided | segmented the image into 20 area | regions of 4 * 5 in a grid | lattice form, (b) is a figure which shows a vertical 13 in a grid form. It is a figure which shows the example divided | segmented into 221 area | region of * horizontal 17. A luminance value that is added to or subtracted from an original luminance value of video content displayed on a television included in the information communication system, an image captured by a camera mounted on the mobile phone, a luminance representative value specified from the image, and the luminance representative value It is a figure which shows the relationship of the bit value specified from. It is a figure which shows an example of the said bit value. It is a figure which shows an example of the bit value conversion table which shows the correspondence of the said bit value and the luminance change of the output luminance of the said television. It is a figure which shows an example of the bit string conversion table which shows the correspondence of the communication information transmitted from the said television to the said mobile telephone, and the bit string which arranged the said bit value. It is a block diagram which shows an example of the principal part structure of the said television. It is a figure which shows an example of the luminance change transition information which shows the luminance change of the output luminance which the said television converted the said communication information. It is a figure which shows the outline | summary of the said information communication system in one Example. It is a figure which shows an example of the process in which the said mobile telephone specifies the position of the said television using the optical communication by a brightness change. It is a figure which shows an example of the process which the said television outputs communication information using the optical communication by a brightness change. It is a figure which shows the outline | summary of the said information communication system in another Example. It is a figure which shows another example of the process in which the said mobile telephone specifies the position of the said television using the optical communication by a brightness change. It is a figure which shows another example of the process which the said television outputs communication information using the optical communication by a brightness change. It is a figure which shows the outline | summary of the said information communication system in another Example. It is a figure which shows another example of the process in which the said mobile telephone specifies the position of the said television using the optical communication by a brightness change. It is a figure which shows another example of the process which the said television outputs communication information using the optical communication by a brightness change. It is a figure which shows an example of the screen displayed on the display part of the said mobile telephone. It is a figure which shows another example of the screen displayed on the display part of the said mobile telephone.

[Outline of optical communication used in the present invention]
The present invention uses optical communication in which communication information transmitted between an information output device and an information acquisition device is converted into a luminance change for communication. This optical communication will be described with reference to FIG. FIG. 2 is a diagram showing an outline of optical communication used in the present invention.

  As shown in FIG. 2, in the television 2 (display device, information output device) that displays video content, the luminance of the original video content is changed for each frame. The mobile phone 1 that is an information acquisition device captures the display device with the camera 13 and analyzes the captured image to detect a change in luminance.

  As shown in FIG. 2, in order to change the luminance value of the entire screen of the television 2, the mobile phone 1 can acquire communication information by imaging a part of the display screen of the television 2. In addition, even when the entire range or a part of the television 2 is photographed by moving the photographing range instead of photographing the television 2 at a fixed point, by capturing the luminance change of the whole image captured by the camera 13, Communication information can be detected.

  Even when the shooting range moves, the brightness of the portion other than the screen of the television 2 (for example, the background, etc.) on the captured image usually does not change abruptly. Furthermore, the brightness of the original image of the video content output from the television 2 hardly changes between consecutive frames.

  Therefore, for example, when the television 2 outputs the communication information by changing the luminance at a time interval equal to or less than the frame interval (1/60 second), the mobile phone 1 detects a predetermined luminance change, It can be determined that the luminance change is a luminance change corresponding to the communication information transmitted from the television 2. Therefore, even if the shooting range moves, the mobile phone 1 can detect communication information with high accuracy if the captured image includes at least a part of the television 2.

  Here, even if a sudden brightness change occurs in a portion other than the screen of the television 2 on the captured image or in the original image of the video content output by the television 2, the television 2 repeatedly outputs communication information. Thus, the mobile phone 1 can detect communication information with high accuracy.

  Here, the original luminance value of the video content is used as a reference signal, a signal whose brightness is higher than that of the reference signal is a bright signal, and a signal whose brightness is darker than the reference signal is a dark signal.

  Communication information is exchanged between the display device and the device on which the camera is mounted by associating the luminance change combining the reference signal, the bright signal and the dark signal with the bit information of “0” and “1”. can do.

  Since video content is generally 60 frames / second, information of a maximum of 30 bits can be transmitted per second by using the optical communication shown in FIG. In the embodiment described below, as shown in FIG. 7, it is assumed that one bit is represented by a luminance change of three frames. However, the present invention is not limited to this, and the number of frames corresponding to 1 bit may be arbitrary.

  For example, when 1 bit is defined as a change in luminance of 3 frames and 6-bit communication information is transmitted, the communication information can be transmitted in 6 bits × 3 frames × 1/60 seconds = 0.3 seconds. Therefore, optical communication based on a change in luminance can be suitably used for a system that transmits communication information in real time.

  In the embodiment described below, it is assumed that the bright signal is 10% brighter than the reference signal, and the dark signal is 10% brighter than the reference signal. However, the present invention is not limited to this, and the ratio of the bright signal and the dark signal to the reference signal may be arbitrary. In addition, the bright signal and the dark signal may be indicated not by increase / decrease in the ratio of the luminance with respect to the reference signal but by increase / decrease in absolute value.

  Thus, by changing the output luminance value of the display device at a short time interval of 1/60 seconds, communication information can be transmitted without affecting the viewing of video content. Further, as shown in FIG. 7, when the luminance change corresponding to 1 bit is configured with the same ratio of the bright signal and the dark signal, the human perceives the light averaged by the afterimage effect. It may be impossible to perceive. That is, communication information can be transmitted without degrading the quality of the original video content.

  Note that the period during which the output luminance value is changed (bright signal or dark signal period) is not limited to 1/60 seconds, but it is desirable that the period be short in order to prevent humans from perceiving it.

  Hereinafter, an embodiment of the present invention will be described with reference to FIGS.

[Outline of information communication system]
Based on FIG. 3, the outline | summary of the information communication system 3 of this invention is demonstrated. FIG. 3 is a diagram showing an outline of the information communication system 3. As shown in FIG. 3, the information communication system 3 of the present invention includes a mobile phone 1 as an information acquisition device and a television 2 as an information output device.

  The television 2 displays video content such as a television program. In addition, the television 2 converts the communication information transmitted to the mobile phone 1 into the luminance change transition information indicating the luminance change in the predetermined period of the luminance value output from the display unit while displaying the video content. Based on the change transition information, the luminance value output from the display unit is changed.

  The mobile phone 1 is equipped with a camera that captures images, sequentially acquires images obtained by capturing the whole or a part of the display unit of the television 2, and performs a predetermined luminance change from the luminance values of the acquired plurality of images. It detects and acquires communication information.

  In the example illustrated in FIG. 3, the mobile phone 1 equipped with a camera is illustrated as the information acquisition device and the television 2 is illustrated as the information output device, but the present invention is not limited thereto.

  The information acquisition device only needs to acquire an image captured by the camera. That is, the information acquisition apparatus may be equipped with a camera, or may be connected to the camera by wireless communication means or wired communication means so that an image captured by the camera can be acquired. The information acquisition device may be, for example, a PDA (Personal Digital Assistant) equipped with a camera, a portable game machine equipped with a camera, a PC connected to the camera, or the like.

  Moreover, the information output device should just be provided with the display part. The information output device may be, for example, a PC, an audio device, a video recorder, or other home appliances including a display unit such as a refrigerator or a microwave oven.

  In the example shown in FIG. 3, the information communication system 3 includes one mobile phone 1 and one television 2, but is not limited thereto. The information communication system 3 may include a plurality of mobile phones 1 or a plurality of televisions 2.

[Configuration of mobile phone]
Next, the configuration of the mobile phone 1 will be described with reference to FIG. FIG. 1 is a block diagram illustrating an example of a main configuration of the mobile phone 1. As shown in FIG. 1, the mobile phone 1 includes a control unit 11, a storage unit 12, a camera 13, a communication unit 14, a display unit 15, and an operation unit 16. The mobile phone 1 may include members such as a voice input unit, a voice output unit, and a telephone network communication unit, but these members are not shown because they are not related to the features of the invention.

  The camera 13 records an image (or moving image) captured by the image sensor as digital data. Digital data recorded by the camera 13 is referred to as image data (moving image data).

  The communication unit 14 communicates with other devices such as the television 2 by wireless communication means such as a wireless LAN, and exchanges data in accordance with instructions from the control unit 11. The communication unit 14 performs communication using a communication method such as Wi-Fi, WiMAX, Bluetooth (registered trademark), or IrDA.

  The display unit 15 displays an image according to an instruction from the control unit 11. The display unit 15 only needs to display an image in accordance with an instruction from the control unit 11. For example, an LCD (liquid crystal display), an organic EL display, a plasma display, or the like can be applied.

  The operation unit 16 is for a user to input an instruction signal to the mobile phone 1 and operate the mobile phone 1. The operation unit 16 may include an on-hook key, an off-hook key, a function key, a cross key, a numeric keypad, and the like. Further, a touch panel in which the operation unit 16 and the display unit 15 are integrated may be used.

  The control unit 11 performs various calculations by executing a program read from the storage unit 12 to a temporary storage unit (not shown), and comprehensively controls each unit included in the mobile phone 1. .

  In the present embodiment, the control unit 11 includes, as functional blocks, an image acquisition unit (image acquisition unit) 21, a representative luminance value specification unit (representative luminance value specification unit) 22, a bit value generation unit 23, and a bit string detection unit (luminance change). A detection unit) 24, a position specifying unit (communication information specifying unit, position information generating unit) 25, a display control unit (display control unit) 26, an optical communication control unit (transmitting unit, receiving unit) 27, and a device information acquiring unit 28. It is the composition provided. Each functional block (21 to 28) of the control unit 11 includes a CPU (central processing unit) that stores a program stored in a storage device realized by a ROM (read only memory) or the like (RAM (random access memory)). This can be realized by reading out and executing the temporary storage unit realized by the above.

  The image acquisition unit 21 acquires an image captured by the camera. In order to perform information processing or display processing in real time, the image acquisition unit 21 sequentially acquires images captured by the camera and outputs the acquired images to the representative luminance value specifying unit 22 and / or the display control unit 26 each time. .

  The representative luminance value specifying unit 22 divides the image acquired by the image acquisition unit 21 into a plurality of regions, and for each region obtained by the division, the representative luminance value specifying unit 22 calculates the representative luminance of the region from the luminance value of each pixel included in the region. The value is specified. The representative luminance value specifying unit 22 outputs the specified representative luminance value to the bit value generating unit 23.

  Specifically, the representative luminance value specifying unit 22 divides the image into a plurality of regions so that each region obtained by the division includes at least one pixel. The shape of each region (the shape of the pixel group included in the region) may be arbitrary. For example, as shown in FIG. 4, the representative luminance value specifying unit 22 may divide the image into a plurality of regions in a grid pattern. FIG. 4 is a diagram illustrating an example of image division. 4A is a diagram illustrating an example in which an image is divided into 20 regions of 4 × 5 horizontal in a grid pattern, and FIG. 4B illustrates 13 × vertical image in a grid pattern. It is a figure which shows the example divided | segmented into 17 221 area | regions. Further, instead of making the shapes of the respective regions the same as shown in FIG. 4, the image may be divided so that the shapes of the respective regions are different. In the present embodiment, the representative luminance value specifying unit 22 divides the image into 12 regions of 3 × 4 in a grid pattern.

  In this embodiment, the representative luminance value specifying unit 22 sets a value obtained by averaging the luminance values of the pixels included in the region as the representative luminance value of the region. However, the method for specifying the representative luminance value is not limited to this. For example, the representative luminance value specifying unit 22 may use the luminance value of any pixel included in the region as the representative luminance value of the region. Further, the representative luminance value specifying unit 22 may use the median value of the luminance values of the pixels included in the region as the representative luminance value of the region. Further, the representative luminance value specifying unit 22 may use the maximum value or the minimum value of the luminance values of the pixels in the region as the representative luminance value of the region.

  That is, in the present embodiment, the representative luminance value specifying unit 22 divides the image acquired by the image acquisition unit 21 into 12 regions of 3 × 4 in a grid pattern, and the luminance value of each pixel included in the region Are averaged, a representative luminance value is calculated for each region, and twelve representative luminance values are specified from one image. A matrix of representative luminance values obtained from the image of the nth frame (frame n) is a representative luminance value matrix Cn. That is, the representative luminance value matrix Cn is a 3 × 4 matrix, and the representative luminance value of each region is indicated by Cn (i, j). Here, “i” in (i, j) indicates the i-th region from the top of the image, and “j” indicates the j-th region from the left of the image.

  As described above, the representative luminance value specifying unit 22 calculates the representative luminance value by averaging the luminance values of the respective pixels, so that noise of luminance change (color change) can be removed. Further, noise can be further removed by reducing the number of regions to be divided (increasing the area of the regions to be divided).

  In the representative luminance value matrices Cn, Cn-1, and Cn-2 acquired from the representative luminance value specifying unit 22, the bit value generating unit 23 determines whether the representative luminance value of the region satisfies a predetermined condition for each region. Determination is made to generate a bit value matrix Bn. The bit value generation unit 23 outputs the generated bit value matrix Bn to the bit string detection unit 24.

Specifically, the bit value generation unit 23 determines that the representative luminance value matrices Cn (i, j), Cn-1 (i, j), and Cn-2 (i, j) are the following first condition and second condition. If the condition is satisfied, the value of the bit value matrix Bn (i, j) is set to 1.
First condition: first threshold value P <Cn-1 (i, j) / Cn-2 (i, j) <second threshold value Q
Second condition: third threshold R <Cn−2 (i, j) / Cn (i, j) <fourth threshold S
In addition, the bit value generation unit 23 indicates that the representative luminance value matrix Cn (i, j), Cn-1 (i, j), Cn-2 (i, j) is either the first condition or the second condition. Is not satisfied, the value of the bit value matrix Bn (i, j) is set to 0.

  In this way, bit values are generated for all regions (12 regions in this embodiment). That is, the bit value matrix Bn is also a 3 × 4 matrix like the luminance representative value matrix Cn.

  Here, in the present embodiment, the value of the first threshold P is 1.1−α, the value of the second threshold Q is 1.1 + α, the value of the third threshold R is 1.1−β, The value of the threshold S is 1.1 + β. Note that α and β> 0, and in this embodiment, α = β = 0.05.

  As described above, in this embodiment, the bit value “1” is defined by three frames of the reference signal, the bright signal, and the dark signal. The brightness value of the bright signal is 10% larger than that of the reference signal, and the dark signal has the brightness. The value is 10% smaller than the reference signal. The first condition and the second condition are for detecting the bit value “1” replaced by the change in the luminance value. Therefore, the first condition and the second condition may be appropriately set according to the definition of the bit value “1” (or the bit value “0”).

  Here, the processing executed by the image acquisition unit 21, the representative luminance value specification unit 22, and the bit value generation unit 23 will be specifically described with reference to FIG. FIG. 5 is a diagram showing the relationship between the luminance value that is added to or subtracted from the original luminance value of the video content displayed on the television 2, the image captured by the camera, the luminance representative value, and the bit value for each frame.

  In the example illustrated in FIG. 5, it is assumed that the timing at which the television 2 switches images and the timing at which the camera sequentially captures images are synchronized. In addition, the television 2 outputs a bit value “1” in the frames 1 to 3.

  Specifically, the television 2 displays the video content with the original luminance value in the frames 0 to 1. In frame 2, the television 2 increases the original luminance value of the video content by 10% and outputs it. In frame 3, the television 2 outputs the video content with the original luminance value reduced by 10%. Then, in the frame 4, the television 2 displays the video content with the original luminance value.

  First, when the image acquisition unit 21 acquires the image P0 of the frame 0 from the camera, the representative luminance value specifying unit 22 divides the image P0 into 3 × 4 regions, calculates an average luminance value for each region, A representative luminance value matrix C0 is specified. The bit value generation unit 23 generates a bit value matrix B0 from the representative luminance value matrix C0. Note that here, the bit value generation unit 23 acquires only the representative luminance value matrix C0 in the frame 0, and there is no representative luminance value in the previous frame and the previous two frames of the frame 0. A bit value matrix B0 is generated with all zeros.

  Next, when the image acquiring unit 21 acquires the image P1 of frame 1 from the camera, the representative luminance value specifying unit 22 similarly specifies the representative luminance value matrix C1. The bit value generation unit 23 generates a bit value matrix B1 from the representative luminance value matrices C1 and C0. Similarly to the previous step, the bit value generation unit 23 generates a bit value matrix B1 with all bit values set to 0 because there is no representative luminance value of the frame two frames before the frame 1.

  Next, when the image acquisition unit 21 acquires the image P2 of the frame 2 from the camera, the representative luminance value specifying unit 22 similarly specifies the representative luminance value matrix C2. The bit value generation unit 23 generates a bit value matrix B2 from the representative luminance value matrices C2, C1, and C0. In the example shown in the figure, the bit value generation unit 23 sets the bit value to 0 in all the regions because none of the representative luminance value matrices C2, C1, and C0 satisfies the first condition or the second condition. To generate a bit value matrix B2.

  Next, when the image acquisition unit 21 acquires the image P3 of the frame 3 from the camera, the representative luminance value specifying unit 22 similarly specifies the representative luminance value matrix C3. The bit value generation unit 23 generates a bit value matrix B3 from the representative luminance value matrices C3, C2, and C1. In the illustrated example, the representative luminance value matrices C3 (2, 4), C2 (2, 4), and C1 (2, 4) in the region (2, 4) satisfy the first condition and the second condition. Specifically, C2 (2,4) / C1 (2,4) = 110/100 = 1.1, which is larger than the first threshold value P (= 1.05) and smaller than the second threshold value Q (= 1.15). Meet the conditions. Further, C1 (2,4) / C3 (2,4) = 100/90 = about 1.11, which is larger than the third threshold R (= 1.05) and smaller than the fourth threshold S (= 1.15), so that the second condition is satisfied. Fulfill. Therefore, the bit value generation unit 23 sets only the bit value “1” for B3 (2, 4) and sets the bit value in the other region to 0, and generates the bit value matrix B3.

  As described above, the representative luminance value specifying unit 22 specifies the representative luminance value matrix Cn from the image Pn acquired by the image acquiring unit 21, and the bit value generating unit 23 sets the representative luminance value matrices Cn, Cn-1, Cn−. A bit value matrix Bn is generated from 2.

  The bit string detection unit 24 detects a predetermined communication bit string for each region from the bit value matrices B0 to Bn generated by the bit value generation unit 23. When the bit string detection unit 24 detects the communication bit string, the bit string detection unit 24 outputs the detected communication bit string and region information indicating the region where the communication bit sequence is detected to the position specifying unit 25.

  Here, the area information may be information indicating the positional relationship of the area, such as (2, 4), or may be information indicating the coordinates indicating the position of the area on the image. The coordinates indicating the position of the area on the image may be, for example, the coordinates of the end points of the area or the coordinates of the center of the area.

  Specifically, the process executed by the bit string detection unit 24 will be described with reference to FIG. FIG. 6 is a diagram showing the bit value matrices B0 to B18 generated by the bit value generating unit 23. As shown in FIG. Here, it is assumed that the bit string detection unit 24 detects the communication bit string “101101”.

  In the present embodiment, as described above, since the bit value “1” is defined by three frames, the bit string detection unit 24 first sets three bit value matrices B0 to B18 as shown in FIG. Divide into groups (frame 3m, frame 3m + 1, frame 3m + 2). Then, the bit string detection unit 24 determines whether or not there is a communication bit string “101101” for each area of each group.

  In the illustrated example, since the communication bit string “101101” is in the area (2, 4) of the group of the frame 3m, the bit string detection unit 24 includes the communication bit string “101101”, area information indicating the area (2, 4), and Is output to the position specifying unit 25.

  The bit string detection unit 24 detects the communication bit string from the bit value matrices B0 to Bn, but is not limited thereto. For example, in the example shown in FIG. 6, when the maximum number of bits of the communication bit string is 6, it is sufficient for the bit string detection unit 24 to detect the communication bit string from the bit value matrices B3 to B18.

  The optical communication control unit 27 controls or manages optical communication processing executed by the television 2. Specifically, the optical communication control unit 27 transmits a communication information output instruction that instructs the television 2 to output predetermined communication information through optical communication. In addition, the optical communication control unit 27 transmits an output information transmission instruction for requesting the television 2 to transmit the contents of communication information output from the television 2 by optical communication. And the communication content identification information which shows the content of the communication information which the television 2 is outputting by optical communication as a response with respect to the output information transmission instruction | indication is received.

  When the communication information output instruction is transmitted, the optical communication control unit 27 indicates device identification information for uniquely identifying the instruction-destination television 2 and the content of the communication information instructed to output by optical communication. The device output information associated with the communication content identification information is output to the position specifying unit 25. In addition, when the optical communication control unit 27 transmits the output information transmission instruction and receives the communication content identification information, the apparatus identification information for uniquely identifying the television 2 that is the transmission destination of the instruction, and the television 2 Device output information in which communication content identification information indicating the content of communication information output by optical communication is associated is output to the position specifying unit 25.

  When the position specifying unit 25 acquires the communication bit string detected by the bit string detecting unit 24 from the bit string detecting unit 24, the position specifying unit 25 reads a bit string conversion table in which the communication bit string is associated with the communication information from the storage unit 12. Then, the position specifying unit 25 specifies communication information corresponding to the communication bit string detected by the bit string detection unit 24 with reference to the read bit string conversion table.

  When specifying the communication information, the position specifying unit 25 associates the area information acquired from the bit string detection unit 24 with the communication content identification information indicating the content of the specified communication information, and generates communication content acquisition position information.

  Further, when acquiring the device output information from the optical communication control unit 27, the position specifying unit 25, if the communication content identification information included in the acquired device output information matches the specified communication content identification information, the bit string detection unit The device position information is generated by associating the area information acquired from 24 with the device identification information included in the device output information acquired from the optical communication control unit 27. The position specifying unit 25 outputs the generated device position information to the display control unit 26.

  As described above, the position specifying unit 25 specifies the position where the communication information is detected or the position on the image of the television 2 based on the divided areas, and thus, by dividing the image into small areas, Specific accuracy is improved.

  The device information acquisition unit 28 transmits, via the communication unit 14, a display device related information transmission instruction that requests to transmit display device related information, which is information related to the television 2, to the television 2. Then, as a response to the display device related information transmission instruction, the display device related information of the television 2 is received from the television 2 via the communication unit 14. The device information acquisition unit 28 outputs the received display device related information to the display control unit 26.

  Here, the display device related information is, for example, information indicating the function, performance, characteristics, etc. of the television 2, information indicating the content displayed on the television 2, information indicating the content held by the television 2, television, 2 is information indicating a connected device, display information such as an icon for the user to identify the television 2, and the like.

  The device information acquisition unit 28, when the display device related information of the television 2 is stored in the storage unit 12, an external storage device or another device on the network, is stored on the storage unit 12, the external storage device or the network. Display device related information may be acquired from other devices.

  The display control unit 26 generates an image according to the process executed by the mobile phone 1 and displays the generated image on the display unit 15. The display control unit 26 displays the image acquired by the image acquisition unit 21 and captured by the camera 13 on the display unit 15 in real time.

  In addition, when the display control unit 26 acquires the device position information from the position specifying unit 25, when generating an image including device-related information regarding the device indicated by the device identification information included in the device position information, the display control unit 26 An image is generated by arranging device-related information related to the television 2 indicated by the device information corresponding to the region information at the display position of the screen of the display unit 15 corresponding to the region indicated by the region information included in the information.

  When the display control unit 26 acquires the apparatus position information from the position specifying unit 25, the display control unit 26 displays the image captured by the camera 13 when displaying the image acquired by the image acquisition unit 21. The device-related information related to the television 2 indicated by the device information corresponding to the region information is displayed at the display position of the screen of the display unit 15 corresponding to the region indicated by the region information included in the device position information.

  The storage unit 12 stores programs, data, and the like referred to by the control unit 11, and stores the bit value conversion table, the bit string conversion table, and the like, for example. The storage unit 12 may store condition information indicating the first condition and the second condition.

  The bit value conversion table and the bit string conversion table are generated in advance by the optical communication control unit 27. The optical communication control unit 27 transmits the generated bit value conversion table and bit string conversion table to the television 2 in advance. Note that the television 2 may generate a bit value conversion table and a bit string conversion table. In this case, the optical communication control unit 27 acquires a bit value conversion table and a bit string conversion table from the television 2 in advance, and stores the acquired bit value conversion table and the bit string conversion table in the storage unit 12.

  The bit value conversion table and the bit string conversion table may be data as shown in FIGS. 7 and 8, for example. FIG. 7 is a diagram illustrating an example of the bit value conversion table, and FIG. 8 is a diagram illustrating an example of the bit string conversion table.

  In the example shown in FIG. 7, the bit values “1” and “0” are defined by the luminance change of three frames, but the present invention is not limited to this. Further, although the bit value “1” is defined by a combination of the reference signal, the bright signal, and the dark signal in this order, the present invention is not limited to this. The definition of the bit values “1” and “0” may be arbitrary.

  In the example shown in FIG. 8, all communication bit strings include a start code and a communication code indicating the content of communication information. In the example shown in FIG. 8, the start code is “1011”. Noise can be removed by putting a start code in the communication bit string. In this embodiment, the communication bit string includes the start code, but the start code may not be present. Further, the bit string (the number of bits and the bit value) of the start code may be arbitrary.

  That is, the communication code included in the communication bit string indicates the content of the communication information. In the example illustrated in FIG. 8, the communication code “01” indicates the communication information “A”, and the communication code “10” indicates the communication information “B”. FIG. 8 is an example, and the correspondence between the communication code and the communication information may be arbitrarily defined.

  Information including the bit value conversion table and the bit string conversion table is referred to as conversion information.

[Configuration of TV]
FIG. 9 is a block diagram illustrating an example of a main configuration of the television 2. As shown in FIG. 9, the television 2 includes a control unit 31, a storage unit 32, an antenna 33, a communication unit 34, and a display unit 35. The television 2 may include members such as an audio output unit and an operation unit, but these members are not shown because they are not related to the features of the invention.

  The antenna 33 receives a broadcast wave of a television broadcast and outputs the received signal to the control unit 31.

  The communication unit 34 communicates with other devices such as the mobile phone 1 by wireless communication means such as a wireless LAN, and exchanges data according to instructions from the control unit 31. The communication unit 34 communicates with a communication method such as Wi-Fi, WiMAX, Bluetooth (registered trademark), or IrDA.

  The display unit 35 displays an image according to an instruction from the control unit 31. The display unit 35 only needs to display an image in accordance with an instruction from the control unit 31. For example, an LCD (liquid crystal display), an organic EL display, a plasma display, or the like can be applied.

  The control unit 31 executes various programs by executing a program read from the storage unit 32 to a temporary storage unit (not shown), and comprehensively controls each unit included in the television 2.

  In the present embodiment, the control unit 31 includes a content acquisition unit 41, an optical communication processing unit (information conversion unit) 42, a device information transmission unit 43, and a display control unit 44 as functional blocks. Each functional block (41 to 44) of the control unit 11 is executed by the CPU reading a program stored in a storage device realized by a ROM or the like to a temporary storage unit realized by a RAM or the like and executing the program. realizable.

  The content acquisition unit 41 acquires a broadcast wave signal received by the antenna 33, converts the acquired signal into a video / audio signal, outputs the video signal to the display control unit 44, and outputs the audio signal to the audio output unit ( (Not shown). In the present embodiment, the content acquisition unit 41 acquires video content via the antenna 33, but is not limited thereto. For example, when the video content is stored in the storage unit 32, the content acquisition unit 41 may read and acquire the video content from the storage unit 32. In addition, the content acquisition unit 41 may acquire video content from an external storage device or another device on a network via a wireless communication unit or a wired communication unit.

  When receiving the communication information output instruction from the mobile phone 1, the optical communication processing unit 42 reads the bit string conversion table and the bit value conversion table from the storage unit 32, and refers to the read bit string conversion table and bit value conversion table to perform communication. Predetermined communication information included in the information output instruction is converted into luminance change transition information indicating a luminance change in a predetermined period. Then, the optical communication processing unit 42 instructs the display control unit 44 to change the luminance value of the image based on the converted luminance change transition information.

  When the television 2 is set to output predetermined communication information by optical communication, the optical communication processing unit 42 converts the predetermined communication information into luminance change transition information, The display control unit 44 may be instructed to change the luminance value of the image based on the luminance change transition information. In this case, when receiving the output information transmission instruction from the mobile phone 1, the optical communication processing unit 42 sends communication content identification information indicating the content of the predetermined communication information to the mobile phone 1 as a response to the output information transmission instruction. Send.

  Here, the luminance change transition information may be information as shown in FIG. 10, for example. In the example shown in FIG. 10, the luminance change transition information is information indicating a luminance change with respect to the reference signal for each frame in a predetermined period. Note that the luminance change transition information illustrated in FIG. 10 corresponds to the communication information “A”.

  When receiving the device-related information transmission instruction from the mobile phone 1 via the communication unit 34, the device information transmission unit 43 reads the device-related information of the own device from the storage unit 32, and as a response to the received device-related information transmission instruction. Then, the read device-related information is transmitted to the mobile phone 1 via the communication unit 34.

  The display control unit 44 generates an image (moving image or still image) according to the process executed by the television 2, and displays the generated image on the display unit 35. The display control unit 44 includes an image generation unit 45 and a luminance control unit (luminance value adjusting means) 46.

  The image generation unit 45 acquires a video signal from the content acquisition unit 41, generates an image based on the acquired video signal, and sequentially outputs the generated image to the luminance control unit 46.

  Based on an instruction from the optical communication processing unit 42, the luminance control unit 46 changes the luminance value of the image generated by the image generation unit 45 and displays it on the display unit 35 according to the luminance change indicated by the luminance change transition information. The luminance control unit 46 may change the luminance values of all the pixels of the image generated by the image generation unit 45, or may change the luminance values of some pixels of the image generated by the image generation unit 45. Also good.

  The storage unit 32 stores programs, data, and the like referred to by the control unit 31, and stores, for example, the bit string conversion table, the bit value conversion table, the device related information, and the like.

  As described above, the optical communication processing unit 42 receives the bit value conversion table and the bit string conversion table from the mobile phone 1 and stores them in the storage unit 32. The optical communication processing unit 42 may generate a bit value conversion table and a bit string conversion table. In this case, the optical communication processing unit 42 transmits the generated bit value conversion table and bit string conversion table to the mobile phone 1 in advance.

[Example 1]
Next, an example (Example 1) of processing in the cellular phone 1 and the television 2 when communication information is transmitted from the television 2 to the cellular phone 1 by optical communication will be described with reference to FIGS.

  FIG. 11 is a diagram illustrating an overview of the information communication system 3a according to the first embodiment. As shown in FIG. 11, the information communication system 3 a includes a mobile phone 1, a television 2 a, a television 2 b, and a Wi-Fi access point 4. Hereinafter, the television 2 a and the television 2 b are collectively referred to as the television 2.

  The Wi-Fi access point 4 is for connecting the mobile phone 1 and the television 2 to each other or to connect to another network.

  As a premise, it is assumed that the mobile phone 1 and the television 2 have known IP addresses, but have not been able to recognize where they are located. Note that the IP address of the mobile phone 1 is “192.168.0.10”, the IP address of the television 2a is “192.168.0.21”, and the IP address of the television 2b is “192.168.0.22”.

  The television 2 is assumed to display video content on the display unit 35.

  In the first embodiment, an example will be described in which the mobile phone 1 specifies the positions of the television 2a and the television 2b using optical communication based on luminance change. In the present embodiment, the mobile phone 1 transmits a communication information output instruction to the television 2a and the television 2b. At this time, the cellular phone 1 instructs the television 2a and the television 2b to output different communication information. Specifically, the mobile phone 1 instructs the television 2a to output the communication information “A”, and instructs the television 2b to output the communication information “B”. .

  In FIG. 11, “A” and “B” are depicted as being displayed on the television 2 a and the television 2 b, but this is for convenience of explanation. Actually, the television 2a and the television 2b do not display “A” and “B” as images, but output the communication information by changing the luminance information indicating the communication information “A” and the communication information “B”, respectively. To do.

  Below, based on FIG. 12 and FIG. 13, the specific process example of the mobile telephone 1 and the television 2 is demonstrated. FIG. 12 is a diagram illustrating an example of a process in which the mobile phone 1 identifies the positions of the television 2a and the television 2b using optical communication based on a luminance change. FIG. 13 is a diagram illustrating an example of processing in which the television 2 outputs communication information using optical communication based on a luminance change.

  As shown in FIG. 12, the mobile phone 1 activates the camera 13 (S1). Further, the optical communication control unit 27 of the mobile phone 1 transmits a communication information output instruction for instructing the television 2a to output the communication information “A”, and outputs the communication information “B” to the television 2b. A communication information output instruction for instructing to transmit is transmitted (S2).

  Here, the optical communication control unit 27 includes device output information DA in which device identification information indicating the television 2a is associated with communication content identification information indicating the communication information “A”, device identification information indicating the television 2b, The apparatus output information DB in which the communication content identification information indicating the communication information “B” is associated is output to the position specifying unit 25.

  Next, the image acquisition unit 21 sequentially acquires images captured by the camera 13. Here, n = 0 is set (S3), and the image acquisition unit 21 acquires the image Pn of the nth frame, that is, the image P0 of frame 0 (S4).

  The representative luminance value specifying unit 22 divides the entire region of the image P0 acquired by the image acquisition unit 21 into a plurality of regions (i, j) (S5). Then, the representative luminance value specifying unit 22 calculates a representative luminance value for each region (i, j) of the image P0 and specifies a representative luminance value matrix C0 (i, j) (S6).

  Based on the representative luminance value matrix C0 (i, j), the bit value generation unit 23 sets the value of the region satisfying the bit value detection condition (first condition and second condition) to 1, and sets the values of the other regions as 1 The bit value matrix B0 (i, j) of frame 0 is generated (S7).

  The bit string detection unit 24 determines whether or not a predetermined communication bit string is included in each region (i, j) of the bit value matrix B0 (S8). Here, it is assumed that the communication bit string could not be detected because there is only the bit value matrix B0.

  If the bit string detection unit 24 determines that the predetermined communication bit string is not included (NO in S8), the optical communication control unit 27 determines whether or not to end the process of specifying the positions of the television 2a and the television 2b by optical communication. A determination is made based on whether the end condition is satisfied (S12).

  Here, the end condition is that the mobile phone 1 specifies the positions of all the televisions 2 included in the information communication system 3a. Therefore, the termination condition is not satisfied (NO in S12), n = n + 1 is set (S13), and the processes of S4 to S8 are repeated.

  The processing of S4 to S8 is repeated, and when the bit string detection unit 24 determines in S8 that a predetermined communication bit string is included in a certain area (i, j) of the bit value matrix B0 to Bn (YES in S8), the detection is performed. The communication bit string and area information indicating the area where the communication bit string is detected are output to the position specifying unit 25. Here, it is assumed that communication bit string “101101” is detected in area (1, 2), and further communication bit string “101110” is detected in area (1, 4).

  When acquiring the communication bit strings “101101” and “101110” from the bit string detection unit 24, the position specifying unit 25 reads the bit string conversion table from the storage unit 12, refers to the read bit string conversion table, and sets the communication bit string “101101”. The communication information “A” is specified as the corresponding communication information, and further the communication information “B” corresponding to the communication bit string “101110” is specified (S9).

  Then, the position specifying unit 25 includes, in the acquired device output information, device output information DA including communication content identification information indicating communication information “A” and device output information including communication content identification information indicating communication information “B”. Since there is an information DB, device identification information indicating the television 2a included in the device output information DA and device identification information indicating the television 2b included in the device output information DB are specified (S10). The position specifying unit 10 generates device position information by associating the specified device identification information with the region information (S11).

  Here, it is assumed that the position specifying unit 25 generates the apparatus position information by using the IP address of the television 2 as the apparatus identification information and using the coordinates indicating the position of the area (i, j) on the image as the area information. That is, the position specifying unit 25 indicates device identification information indicating the IP address “192.168.0.21” of the television 2a and coordinates “x = 20, y = 20” indicating the position of the area (1, 2) on the image. The apparatus position information EA is generated by associating the area information with each other, and the apparatus identification information indicating the IP address “192.168.0.22” of the television 2b and the coordinates “x” indicating the position of the area (1, 4) on the image. The device position information EB is generated in association with the area information indicating “= 80, y = 20”. Note that the coordinate axes of the coordinates indicating the position on the image captured by the camera 13 are the origin at the upper left corner of the image, the horizontal right direction is the x axis, and the vertical downward direction is the y axis.

  When the position specifying unit 25 generates the apparatus position information, the optical communication control unit 27 determines whether or not the end condition is satisfied. At this time, since the position specifying unit 25 generates the apparatus position information EA and EB and the mobile phone 1 specifies the positions of all the televisions 2 included in the information communication system 3a (YES in S12), the optical communication control unit 27 is The process of specifying the positions of the television 2a and the television 2b by optical communication is terminated.

  Note that the termination condition in S12 is not limited to the above condition. For example, the end condition may be that the mobile phone 1 specifies the position of any of the televisions 2 included in the information communication system 3a. Further, the end condition may be that the mobile phone 1 specifies the position of the predetermined television 2 included in the information communication system 3a. Further, the elapse of a predetermined time may be set as the end condition, or the end instruction from the user may be set as the end condition.

  As shown in FIG. 13, the optical communication processing unit 42 of the television 2 first waits for a communication information output instruction to be transmitted from the mobile phone 1 (S21). Here, when the optical communication processing unit 42 does not receive the communication information output instruction (NO in S21), the display control unit 44 continues to change the luminance based on the video signal acquired by the content acquisition unit 41. The video content is displayed on the display unit 35 as it is.

  On the other hand, when the optical communication processing unit 42 receives the communication information output instruction (YES in S21), the bit string conversion table and the bit value conversion table are read from the storage unit 32, and the read bit string conversion table and bit value conversion table are referred to. The predetermined communication information included in the communication information output instruction is converted into luminance change transition information indicating a luminance change (S22). Then, the optical communication processing unit 42 instructs the display control unit 44 to change the luminance value of the image based on the converted luminance change transition information.

  Based on the instruction from the optical communication processing unit 42, the luminance control unit 46 changes the luminance value of the image generated by the image generation unit 45 in accordance with the luminance change indicated by the luminance change transition information and displays it on the display unit 35 (S23). ).

[Example 2]
Next, another example (Example 2) of processing in the cellular phone 1 and the television 2 when communication information is transmitted from the television 2 to the cellular phone 1 by optical communication will be described with reference to FIGS. .

  FIG. 14 is a diagram illustrating an overview of the information communication system 3b according to the second embodiment. As shown in FIG. 14, the information communication system 3 b includes a mobile phone 1, a television 2 c, a television 2 d, and a Wi-Fi access point 4. Hereinafter, the television 2c and the television 2d are collectively referred to as the television 2.

  As shown in FIG. 14, it is assumed that the television 2c is installed on the first floor of the building and the television 2d is arranged on the second floor of the building. It is assumed that the user of the mobile phone 1 is on the first floor of the building.

  As a premise, it is assumed that the mobile phone 1 and the television 2 have known IP addresses, but have not been able to recognize where they are located. Note that the IP address of the mobile phone 1 is “192.168.0.10”, the IP address of the television 2c is “192.168.0.21”, and the IP address of the television 2d is “192.168.0.22”.

  The television 2 is assumed to display video content on the display unit 35.

  In the second embodiment, an example will be described in which the mobile phone 1 specifies the positions of the television 2c and the television 2d using optical communication based on luminance change. In the present embodiment, the mobile phone 1 sequentially transmits communication information output instructions to the television 2c and the television 2d. At this time, the cellular phone 1 instructs the television 2c and the television 2d to output the same communication information. Specifically, the cellular phone 1 instructs the television 2c and the television 2d to output the communication information “A”.

  That is, in the second embodiment, each television 2 is instructed in turn, its response is detected, and the position of a certain television 2 is specified. In the second embodiment, unlike the first embodiment, the positions of a plurality of televisions 2 cannot be specified at the same time. However, in order to exchange one piece of communication information, the processing load on the mobile phone 1 and the television 2 can be reduced or the processing can be reduced. It can be simplified. When there is communication information that is optimal for reducing the processing load of the mobile phone 1 and the TV 2 and communication information that is optimal for safe or accurate communication, the mobile phone 1 and the TV can be obtained by applying the method of the second embodiment. 2 can be reduced, and communication between the mobile phone 1 and the television 2 can be executed safely or accurately.

  In FIG. 14, “A” is displayed on the television 2c and the television 2d, but this is for convenience of explanation. Actually, the television 2c and the television 2d do not display “A” as an image, but output communication information instead of a luminance change indicating the communication information “A”.

  Below, based on FIG. 15 and FIG. 16, the specific process example of the mobile telephone 1 and the television 2 is demonstrated. FIG. 15 is a diagram illustrating an example of processing in which the mobile phone 1 specifies the positions of the television 2c and the television 2d using optical communication based on luminance change. FIG. 16 is a diagram illustrating an example of processing in which the television 2 outputs communication information using optical communication based on a luminance change.

  As shown in FIG. 15, the mobile phone 1 activates the camera 13 (S31). Further, the optical communication control unit 27 of the mobile phone 1 selects one instruction target television 2 to which a communication information output instruction is transmitted (S32). Here, it is assumed that the optical communication control unit 27 selects the television 2d as the instruction target television.

  When selecting the television 2d, the optical communication control unit 27 transmits a communication information output instruction for instructing the selected television 2d to output the communication information “A” (S33). In addition, the optical communication control unit 27 outputs the device output information DD in which the device identification information indicating the television 2 d and the communication content identification information indicating the communication information “A” are associated with each other to the position specifying unit 25.

  Next, the image acquisition unit 21 sequentially acquires images captured by the camera 13. The following S35 to S38 are the same processes as S4 to S7 in FIG. That is, the image acquisition unit 21 acquires the image Pn, and the representative luminance value specifying unit 22 divides the image Pn into a plurality of regions and specifies the representative luminance value matrix Cn (i, j). Then, the bit value generation unit 23 generates a bit value matrix Bn (i, j) from the representative luminance value matrix Cn (i, j).

  The bit string detection unit 24 determines whether or not a predetermined communication bit string is included in each region (i, j) of the bit value matrix B0 (S39). Since the television 2d is on the second floor of the building and cannot capture the television 2d, the bit string detection unit 24 does not detect a predetermined communication bit string. If the bit string detection unit 24 does not detect a predetermined communication bit string (NO in S39), the optical communication control unit 27 determines whether or not a predetermined period has elapsed since the communication information output instruction was transmitted to the television 2d ( S40).

  If the predetermined period has not elapsed (NO in S40), n = n + 1 is set (S41), and the processes of S35 to S39 are repeated again.

  On the other hand, if the bit string detection unit 24 does not detect the predetermined communication bit string in S39 and the predetermined period has elapsed (YES in S40), the optical communication control unit 27 changes the instruction target television from the television 2d to the television 2c. (S42). The optical communication control unit 27 transmits a communication information output instruction for instructing to output the communication information “A” to the changed television 2c (S33). Further, the optical communication control unit 27 outputs device output information DC in which device identification information indicating the television 2 c and communication content identification information indicating communication information “A” are associated with each other to the position specifying unit 25.

  Similarly to the previous step, the processes of S34 to S39 are performed, and the bit string detection unit 24 does not detect a predetermined communication bit string (NO in S39), and if the predetermined period has not elapsed (NO in S40), n = n + 1 ( S41), the processes of S35 to S39 are repeated again.

  Here, when the bit string detection unit 24 detects a predetermined communication bit string in S39 (YES in S39), the detected communication bit string and area information indicating the area where the communication bit string is detected are output to the position specifying unit 25. . Here, it is assumed that the communication bit string “101101” is detected in the area (1, 2).

  When acquiring the communication bit string “101101” from the bit string detection unit 24, the position specifying unit 25 associates the device identification information indicating the television 2c included in the device output information DC with the region information and generates device position information ( S43).

  Here, it is assumed that the position specifying unit 25 generates the apparatus position information by using the IP address of the television 2 as the apparatus identification information and using the coordinates indicating the position of the area (i, j) on the image as the area information. That is, the position specifying unit 25 indicates the device identification information indicating the IP address “192.168.0.21” of the television 2c and the coordinates “x = 20, y = 20” indicating the position of the area (1, 2) on the image. The apparatus position information EC is generated in association with the area information.

  In S32 or S42, the optical communication control unit 27 may select the television 2 from the televisions 2 included in the information communication system 3b according to a predetermined order, or may randomly select them sequentially. Good. Further, the optical communication control unit 27 may sequentially select the televisions 2 in accordance with a user instruction.

  As shown in FIG. 16, the optical communication processing unit 42 of the television 2 first waits for a communication information output instruction to be transmitted from the mobile phone 1 (S51). Here, when the optical communication processing unit 42 does not receive the communication information output instruction (NO in S51), the display control unit 44 continues to change the luminance based on the video signal acquired by the content acquisition unit 41. The video content is displayed on the display unit 35 as it is.

  On the other hand, when the optical communication processing unit 42 receives the communication information output instruction (YES in S51), the bit string conversion table and the bit value conversion table are read from the storage unit 32, and the read bit string conversion table and bit value conversion table are referred to. The predetermined communication information included in the communication information output instruction is converted into luminance change transition information indicating a luminance change (S52). Then, the optical communication processing unit 42 instructs the display control unit 44 to change the luminance value of the image based on the converted luminance change transition information.

  Based on the instruction from the optical communication processing unit 42, the luminance control unit 46 changes the luminance value of the image generated by the image generation unit 45 according to the luminance change indicated by the luminance change transition information and displays it on the display unit 35 (S53). ).

Example 3
Next, another example (Example 3) of processing in the cellular phone 1 and the television 2 when communication information is transmitted from the television 2 to the cellular phone 1 by optical communication will be described with reference to FIGS. .

  FIG. 17 is a diagram illustrating an overview of the information communication system 3c according to the third embodiment. As shown in FIG. 17, the information communication system 3 c includes a mobile phone 1, a television 2 e, a television 2 f, and a Wi-Fi access point 4. Hereinafter, the television 2e and the television 2f are collectively referred to as the television 2.

  As a premise, it is assumed that the mobile phone 1 and the television 2 have known IP addresses, but have not been able to recognize where they are located. The IP address of the mobile phone 1 is “192.168.0.10”, the IP address of the television 2e is “192.168.0.21”, and the IP address of the television 2f is “192.168.0.22”.

  In the third embodiment, the television 2e and the television 2f regularly output communication information uniquely determined for each television 2 by optical communication, and the mobile phone 1 uses the optical communication by luminance change to perform the television 2a and An example of specifying the position of the television 2b will be described. In this embodiment, the mobile phone 1 transmits an output information transmission instruction to the television 2e and the television 2f.

  In this embodiment, it is assumed that the television 2e and the television 2f are displaying video content on the display unit 35. Further, it is assumed that the television 2e is set to output the communication information “A” by optical communication, and the television 2f is set to output the communication information “B” by optical communication.

  In FIG. 17, “A” and “B” are illustrated as being displayed on the television 2 e and the television 2 f, but this is for convenience of explanation. Actually, the television 2e and the television 2f do not display “A” and “B” as images, but output the communication information instead of the luminance changes indicating the communication information “A” and the communication information “B”, respectively. To do.

  Below, based on FIG. 18 and FIG. 19, the specific process example of the mobile telephone 1 and the television 2 is demonstrated. FIG. 18 is a diagram illustrating an example of processing in which the mobile phone 1 specifies the positions of the television 2e and the television 2f using optical communication based on a luminance change. FIG. 19 is a diagram illustrating an example of a process in which the television 2 outputs communication information using optical communication using a luminance change.

  As shown in FIG. 18, in the mobile phone 1, the optical communication control unit 27 of the mobile phone 1 transmits an output information transmission instruction to the television 2e and the television 2f (S61). And the optical communication control part 27 receives communication content identification information from the television 2e and the television 2f as a response with respect to an output information transmission instruction | indication (S62).

  Here, the optical communication control unit 27 associates the device identification information indicating the television 2e with the communication content identification information received from the television 2e, the device identification information indicating the television 2f, and the television 2f. The apparatus output information DF associated with the communication content identification information received from is output to the position specifying unit 25.

  Next, the mobile phone 1 activates the camera 13 (S63), and the image acquisition unit 21 sequentially acquires images captured by the camera 13. The following S64 to S74 are the same processes as S3 to 13 in FIG.

  That is, the image acquisition unit 21 acquires the image Pn, and the representative luminance value specifying unit 22 divides the image Pn into a plurality of regions and specifies the representative luminance value matrix Cn (i, j). Then, the bit value generation unit 23 generates a bit value matrix Bn (i, j) from the representative luminance value matrix Cn (i, j).

  Next, when the bit string detection unit 24 detects a predetermined communication bit string from the bit value matrices B0 to Bn, the position specifying unit 25 specifies communication information corresponding to the communication bit string detected by the bit string detection unit 24. Then, when there is device output information including communication content identification information indicating communication information in the acquired device output information, the position specifying unit 25 includes device identification information indicating the television 2 included in the device output information, and an area. The apparatus position information is generated in association with the information.

  Here, it is assumed that the position specifying unit 25 generates the apparatus position information by using the IP address of the television 2 as the apparatus identification information and using the coordinates indicating the position of the area (i, j) on the image as the area information. Further, it is assumed that the bit string detection unit 24 detects the communication bit string “101101” in the area (1, 2) and further detects the communication bit string “101110” in the area (1, 4).

  That is, the position specifying unit 25 indicates device identification information indicating the IP address “192.168.0.21” of the television 2e and coordinates “x = 20, y = 20” indicating the position of the area (1, 2) on the image. The apparatus position information EE is generated in association with the area information, and further, the apparatus identification information indicating the IP address “192.168.0.22” of the television 2f and the coordinates “x” indicating the position of the area (1, 4) on the image. The device position information EF is generated in association with the area information indicating “= 80, y = 20”.

  As shown in FIG. 19, the optical communication processing unit 42 of the television 2 performs display control so as to change the luminance value of the image based on the luminance change transition information corresponding to the communication information predetermined for the device itself. The unit 44 is instructed periodically. Based on the instruction from the optical communication processing unit 42, the luminance control unit 46 changes the luminance value of the image generated by the image generation unit 45 in accordance with the luminance change indicated by the luminance change transition information and displays it on the display unit 35 (S81). ).

  In this state, the optical communication processing unit 42 waits for an output information transmission instruction to be transmitted from the mobile phone 1 (S82). When the output information transmission instruction is transmitted from the mobile phone 1 (YES in S82), the optical communication processing unit 42 identifies the communication content indicating the content of the communication information predetermined for the own device as a response to the output information transmission instruction. Information is transmitted to the mobile phone 1 (S83).

[Screen display example 1]
Next, an example of a screen displayed on the display unit 15 after specifying the position of the television 2 on the image taken by the mobile phone 1 with the camera 13 will be described with reference to FIG. FIG. 20 is a diagram illustrating an example of a screen displayed on the display unit 15 of the mobile phone 1.

  The screen example shown in FIG. 20 is a screen for allowing the mobile phone 1 to select a transmission destination device for transmitting content held by the mobile phone 1. More specifically, as shown in FIG. 20, this is a screen for allowing the user to select which of the two televisions the content held by the mobile phone 1 is displayed on which television 2.

  Here, it is assumed that there are a television 2a and a television 2b as the television 2 for displaying content. Further, as exemplified in the first embodiment, it is assumed that the mobile phone 1 generates the device position information EA and the device position information EB.

  At this time, the display control unit 26 of the mobile phone 1 is a device related to the device indicated by the device information corresponding to the region information at the display position of the screen of the display unit 15 corresponding to the region indicated by the region information included in the device position information. Display related information. Specifically, the display control unit 26 displays the icon of the television 2a at the display position of the screen of the display unit 15 corresponding to the area (1, 2) indicated by the area information included in the apparatus position information EA. Further, the display control unit 26 displays the icon of the television 2b at the display position of the screen of the display unit 15 corresponding to the area (1, 4) indicated by the area information included in the apparatus position information EB.

  That is, the display control unit 26 arranges the television 2a icon on the left side and the television 2b icon on the right side, and displays an image for allowing the user to select the television 2.

  In this way, by displaying the device-related information related to the television 2 in correspondence with the position of the television 2 captured in the image captured by the camera 13, the user can operate intuitively and easily.

[Screen display example 2]
Next, another example of the screen displayed on the display unit 15 after the mobile phone 1 specifies the position of the television 2 on the image captured by the camera 13 will be described with reference to FIG. FIG. 21 is a diagram illustrating an example of a screen displayed on the display unit 15 of the mobile phone 1.

  The screen example shown in FIG. 21 is a screen in which the mobile phone 1 displays an image obtained by capturing the television 2 with the camera 13 in real time, and superimposes the captured image on the captured image to display device-related information related to the television 2. is there. The screen example shown in FIG. 21 is a screen displaying so-called augmented reality.

  Here, similarly to the above example, it is assumed that the television 2a and the television 2b are captured by the camera 13. Further, as exemplified in the first embodiment, it is assumed that the mobile phone 1 generates the device position information EA and the device position information EB.

  At this time, the display control unit 26 of the mobile phone 1 superimposes the image acquired by the image acquisition unit 21 from the camera 13 on the display position of the screen of the display unit 15 corresponding to the region indicated by the region information included in the device position information. Thus, the device related information related to the device indicated by the device information corresponding to the area information is displayed.

  Specifically, the display control unit 26 acquires the image acquisition unit 21 from the camera 13 at the display position of the screen of the display unit 15 corresponding to the area (1, 2) indicated by the area information included in the apparatus position information EA. The text information indicating the function of the television 2a is displayed so as to be superimposed on the image. Further, the display control unit 26 applies the image acquired by the image acquisition unit 21 from the camera 13 to the display position of the screen of the display unit 15 corresponding to the region (1, 4) indicated by the region information included in the device position information EB. The text information indicating the content held by the television 2b is displayed in a superimposed manner.

  The device-related information of the television 2 may be displayed as shown in FIG. 21 instead of being superimposed on the position on the screen corresponding to the region indicated by the region information included in the device position information. That is, the position on the screen corresponding to the area indicated by the area information included in the apparatus position information, that is, the position where the leader line is drawn from the position where the television 2 is displayed and away from the position where the television 2 is displayed The device related information of the television 2 may be displayed.

  Further, the display method of the device related information is not limited to this, and the position on the screen corresponding to the region indicated by the region information included in the device position information, the device related information related to the device indicated by the device information corresponding to the region information, As long as the corresponding relationship is indicated, the device-related information may be displayed in any manner.

  In summary, the display control unit 26 of the mobile phone 1 responds to the region information by superimposing it on a position on the image corresponding to the region indicated by the region information included in the device position information or in association with the position. Device related information related to the device indicated by the device information is displayed.

  In this way, when the camera 13 displays an image obtained by capturing the television 2, it is possible to display device related information related to the television 2 at the position where the television 2 is drawn. That is, augmented reality can be displayed. Therefore, it is possible to display a screen that the user can intuitively understand.

[Modification 1]
In this embodiment, the position specifying unit 25 specifies communication information corresponding to a communication bit string detected from a predetermined area, but the present invention is not limited to this.

  For example, when the image is divided into 64 × 64 areas and the same communication bit string is detected from a plurality of areas included in a predetermined range on the image, the position specifying unit 25 performs communication corresponding to the communication bit string. Information may be specified.

  If the number of areas into which an image is divided is increased, the position where the communication information is detected on the image can be specified with high accuracy. However, since the number of pixels included in each area is reduced, it is easily affected by noise. However, as described above, when substantially the same luminance change is detected in a plurality of areas, it is determined that the luminance change is a luminance change corresponding to the communication information transmitted from the television 2, so that the image is displayed. Even if the number of areas to be divided is increased, the influence of noise can be reduced, and communication information can be detected with high accuracy.

[Modification 2]
Here, a method using a motion vector will be described in order to enable detection of communication information even when there is a motion that greatly exceeds a previously assumed divided region. A motion vector represents a moving distance and moving direction of an object from a reference frame in a video, and is also used in a moving image compression technique such as MPEG4. An example of using this motion vector in the present invention is shown below.

  For example, in frame n-2, frame n-1, and frame n, the coordinate values of the position of the television 2 (subject) (for example, the position of the upper left corner) in the acquired image are (300, 500), (250, 500) and (280, 430), the motion vector of the television 2 based on the frame n-2 estimated by the image processing of the representative luminance knowledge specifying unit 22 is (− 50, 0) and (−20, −70) in frame n.

  Here, with reference to the frame n−2, the television 2 of the frame n−1 is corrected to move by the coordinate value (50, 0), and the frame n is moved by the coordinate value (20, 70). Correction is performed. In the corrected image, the representative luminance values of the divided areas are compared. In this comparison, it is not always necessary to divide and compare the areas, and the luminance may be compared for the area occupied by the television 2 corrected in each frame by the motion vector.

  As described above, by correcting the acquired image using the motion vector, it is possible to accurately detect communication information even if the shooting range or the shooting target moves violently beyond the size of each divided region. . In addition, since the luminance change of the image due to the movement of the shooting range or the shooting target is corrected, the conditions for detecting the bright signal and the dark signal can be strict. That is, the values of α and β in the first condition and the second condition can be reduced. Thereby, erroneous detection of the bit value can be reduced.

  In the present embodiment, the image is divided into a plurality of areas, and the area where the predetermined luminance change is detected is specified as the area where the communication information is detected. However, the present invention is not limited to this. For example, the position where the communication information is detected may be specified based on information indicating the coordinates on the image included in the motion vector information.

  For example, according to Japanese Patent Laid-Open No. 2008-236096, when detecting a motion vector from a plurality of reference images, the final motion vector can be detected while changing the block size and determining the optimum block size. I can do it. The motion vector obtained from this process and the optimum block size specify information (position and size) of an area having a correspondence relationship among a plurality of images. Obviously, the block size is not limited to 8x8, 16x16, and can be expanded to any size. Such identification may be performed for each vector and held, the representative value luminance may be calculated for each vector, and the communication information may be detected to associate the communication information with the region information.

[Supplement]
The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  Finally, each block of the mobile phone 1 and the television 2, in particular, the control unit 11 and the control unit 31 may be configured by hardware logic, or may be realized by software using a CPU as follows.

  In other words, the mobile phone 1 and the television 2 are provided with a central processing unit (CPU) that executes instructions of a control program for realizing each function, a read only memory (ROM) that stores the program, and a random access memory (RAM) that expands the program. memory), a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is a record in which program codes (execution format program, intermediate code program, source program) of control programs for the mobile phone 1 and the television 2 which are software for realizing the functions described above are recorded so as to be readable by a computer. This can also be achieved by supplying the medium to the mobile phone 1 and the television 2 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  The mobile phone 1 and the television 2 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  INDUSTRIAL APPLICABILITY The present invention can be used for a device equipped with a camera such as a mobile phone and a display device for displaying an image such as a television, and can be suitably used for a system or device that realizes augmented reality.

1 Mobile phone (information acquisition device)
2 Television (information output device, display device)
3 Information Communication System 13 Camera 15 Display Unit 21 Image Acquisition Unit (Image Acquisition Unit)
22 representative luminance value specifying unit (representative luminance value specifying means)
23 bit value generation unit 24 bit string detection unit (luminance change detection means)
25 Position specifying unit (communication information specifying means, position information generating means)
26 Display control unit (display control means)
27 Optical communication control unit (transmission means, reception means)
35 Display unit 42 Optical communication processing unit (information conversion means)
46 Luminance control unit (luminance value adjusting means)

Claims (15)

Image acquisition means for sequentially acquiring images obtained by capturing a display screen displaying a moving image or a still image with a camera;
Representative luminance value specifying means for specifying a representative luminance value from the image acquired by the image acquisition means;
In the conversion information, one or more sets of communication information and luminance changes in a predetermined period are associated,
A luminance change detecting means for detecting, from the conversion information, a luminance change corresponding to a luminance change specified by a plurality of representative luminance values specified by the representative luminance value specifying means from the plurality of sequentially acquired images;
An information acquisition device comprising: communication information specifying means for specifying communication information associated with the luminance change detected by the luminance change detecting means from the conversion information.   And further comprising position information generating means for acquiring the communication information specified by the communication information specifying means and the area information indicating the area where the luminance change detecting means has detected the luminance change corresponding to the communication information. The information acquisition apparatus according to claim 1, wherein For a display device comprising the display screen, further comprising a transmission means for transmitting communication information associated with the display device,
The position information generating means generates device position information by associating device information indicating the display device with region information corresponding to the communication information specified by the communication information specifying device. The information acquisition apparatus according to claim 2. From a display device comprising the display screen, further comprises receiving means for receiving communication information associated with the display device,
The position information generation means generates device position information by associating device information indicating the display device with area information corresponding to the communication information received by the receiving device and specified by the communication information specifying device. The information acquisition apparatus according to claim 2. The representative luminance value specifying means divides the image acquired by the image acquisition means into a plurality of areas, and for each area obtained by the division, the representative luminance value is determined from the luminance value of each pixel included in the area. Identify the value,
The luminance change detecting means converts the luminance change corresponding to the luminance change specified by the plurality of representative luminance values specified by the representative luminance value specifying means from the same region of the sequentially acquired images, to the conversion information. The information acquisition apparatus according to claim 1, wherein the information acquisition apparatus detects the information from the information acquisition apparatus.   The representative luminance value specifying means detects a motion vector of a subject from a plurality of images sequentially acquired by the image acquisition means, and divides the image corrected by the motion vector into a plurality of regions. Item 6. The information acquisition device according to Item 5.   The communication information specifying unit, when the luminance change detecting unit detects the same luminance change from a plurality of areas included in a predetermined range on the image, displays communication information associated with the luminance change. The information acquisition apparatus according to claim 5, wherein the information acquisition apparatus is specified. A display unit;
Display control means for displaying display device related information related to the display device indicated by the device information corresponding to the region information at a display position of the screen of the display unit corresponding to the region indicated by the region information included in the device position information; The information acquisition apparatus according to claim 3, further comprising:   9. The information acquisition apparatus according to claim 8, wherein the display control means displays the display apparatus related information superimposed on the image acquired by the image acquisition means. An information output device that includes a display unit and displays a moving image or a still image on the display unit,
Information conversion means for converting the communication information into luminance change transition information indicating a luminance change in a predetermined period of the luminance value output by the display unit;
Brightness value adjusting means for adjusting the brightness value of the moving image or still image for a predetermined period in accordance with the brightness change indicated by the brightness change transition information converted by the information converting means, and displaying the brightness value on the display unit. Information output device. An image acquisition step for sequentially acquiring images obtained by capturing a display screen displaying a moving image or a still image with a camera;
A representative luminance value specifying step for specifying a representative luminance value from the image acquired in the image acquiring step;
In the conversion information, one or more sets of communication information and luminance changes in a predetermined period are associated,
A luminance change detecting step of detecting, from the conversion information, a luminance change that matches a luminance change specified by the plurality of representative luminance values specified in the representative luminance value specifying step, from the plurality of sequentially acquired images; ,
A communication information specifying step for specifying communication information associated with the luminance change detected in the luminance change detecting step from the conversion information. An information output method of an information output device comprising a display unit and displaying a moving image or a still image on the display unit,
An information conversion step of converting the communication information into luminance change transition information indicating a luminance change in a predetermined period of the luminance value output by the display unit;
A luminance value adjusting step of adjusting the luminance value of the moving image or still image for a predetermined period according to the luminance change indicated by the luminance change transition information converted in the information converting step, and displaying the luminance value on the display unit. Information output method. The information acquisition device according to any one of claims 1 to 9,
An information communication system comprising the information output device according to claim 10.   A control program for operating the information acquisition device according to any one of claims 1 to 9, wherein the control program causes a computer to function as each of the means.   A control program for operating the information output apparatus according to claim 10 for causing a computer to function as each means.

Images