JP2017118160A - Transmission method, transmission device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a transmission method capable of communicating between various devices including a device out of a lighting device.SOLUTION: A transmission method includes a step SC11 of determining luminance variation pattern by modulating a visible light signal; a step SC12 of switching a common switch for lightning a plurality of light sources for indicating a pixel in a moving image included in a light source group comprised in a display on the basis of the luminance variation pattern; and a step SC13 of transmitting the visible light signal by turning on a first pixel switch for lightning the first light source of the plurality of light source included in the light source group, and by lightning the first light source in a period that the common switch is On and the first pixel switch is On.SELECTED DRAWING: Figure 470A

Description

  The present invention relates to a visible light signal transmission method, a transmission apparatus, and a program.

  In recent home networks, in addition to cooperation of AV home appliances by IP (Internet Protocol) connection in Ethernet (registered trademark) and wireless LAN (Local Area Network), management of power consumption corresponding to environmental problems, and from outside the home Introduction of a home appliance linkage function in which various home appliances are connected to a network by a home energy management system (HEMS) having a function of turning on / off the power of the home appliance. However, there are home appliances that do not have sufficient computing power to have a communication function, and home appliances that are difficult to install a communication function in terms of cost.

  In order to solve such a problem, in Patent Literature 1, in an optical space transmission device that transmits information to free space using light, limited transmission is performed by performing communication using a plurality of monochromatic light sources of illumination light. A technology for efficiently realizing communication between devices is described in the apparatus.

JP 2002-290335 A

  However, the conventional method is limited to a case where a device to be applied has a three-color light source such as illumination.

  The present invention solves such problems and provides a transmission method that enables communication between various devices including devices other than lighting.

  A transmission method according to an aspect of the present invention is a transmission method for transmitting a visible light signal according to a luminance change, and includes a determination step for determining a luminance change pattern by modulating the visible light signal, and a display. A common switch control step of switching a common switch included in the light source group for commonly lighting a plurality of light sources for representing pixels in the video according to the luminance change pattern; and a plurality of light sources included in the light source group By turning on the first pixel switch for turning on the first light source among the light sources, the common switch is on and only during the period when the first pixel switch is on. And a first pixel switch control step of transmitting the visible light signal by turning on the first light source.

  Note that these comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, and computer program. And any combination of recording media.

  ADVANTAGE OF THE INVENTION According to this invention, the transmission method which enables the communication between apparatuses of the aspect containing apparatuses other than illumination is realizable.

FIG. 1 is a diagram illustrating an example of an observation method of luminance of a light emitting unit in the first embodiment. FIG. 2 is a diagram illustrating an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 3 is a diagram illustrating an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 4 is a diagram illustrating an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5A is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5B is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5C is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5D is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5E is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5F is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5G is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5H is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 6A is a flowchart of the information communication method in Embodiment 1. FIG. 6B is a block diagram of the information communication apparatus according to Embodiment 1. FIG. 7 is a diagram illustrating an example of each mode of the receiver in the second embodiment. FIG. 8 is a diagram illustrating an example of a photographing operation of the receiver in the second embodiment. FIG. 9 is a diagram illustrating another example of the photographing operation of the receiver in the second embodiment. 10A is a diagram illustrating another example of imaging operation of a receiver in Embodiment 2. FIG. FIG. 10B is a diagram illustrating another example of imaging operation of a receiver in Embodiment 2. FIG. 10C is a diagram illustrating another example of imaging operation of a receiver in Embodiment 2. FIG. 11A is a diagram illustrating an example of a camera arrangement of a receiver in Embodiment 2. FIG. 11B is a diagram illustrating another example of camera arrangement of a receiver in Embodiment 2. FIG. 12 is a diagram illustrating an example of display operation of the receiver in Embodiment 2. FIG. 13 is a diagram illustrating an example of display operation of the receiver in Embodiment 2. FIG. 14 is a diagram illustrating an example of operation of a receiver in Embodiment 2. FIG. 15 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 16 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 17 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 18 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 19 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 20 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 21 is a diagram illustrating an example of operations of the receiver, the transmitter, and the server in the second embodiment. FIG. 22 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 23 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 24 is a diagram illustrating an example of initial setting of a receiver in the second embodiment. FIG. 25 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 26 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 27 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 28 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 29 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 30 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 31A is a diagram illustrating a pen used for operation of a receiver in Embodiment 2. FIG. 31B is a diagram illustrating operation of a receiver using a pen in Embodiment 2. FIG. 32 is a diagram illustrating an example of appearance of a receiver in Embodiment 2. FIG. 33 is a diagram illustrating another example of appearance of a receiver in Embodiment 2. FIG. 34 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 35A is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 35B is a diagram illustrating an application example using the receiver in Embodiment 2. FIG. 36A is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 36B is a diagram illustrating an application example using the receiver in Embodiment 2. FIG. 37A is a diagram illustrating an example of operation of a transmitter in Embodiment 2. FIG. 37B is a diagram illustrating another example of operation of a transmitter in Embodiment 2. FIG. 38 is a diagram illustrating another example of operation of a transmitter in Embodiment 2. FIG. 39 is a diagram illustrating another example of operation of a transmitter in Embodiment 2. FIG. 40 is a diagram illustrating an example of a communication form between the plurality of transmitters and the receiver in the second embodiment. FIG. 41 is a diagram illustrating an example of operations of a plurality of transmitters in the second embodiment. FIG. 42 is a diagram illustrating another example of a communication form between a plurality of transmitters and a receiver in the second embodiment. FIG. 43 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 44 is a diagram illustrating an example of application of a receiver in Embodiment 2. FIG. 45 is a diagram illustrating an example of application of a receiver in Embodiment 2. FIG. 46 is a diagram illustrating an example of application of a receiver in Embodiment 2. FIG. 47 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 48 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 49 is a diagram illustrating an example of application of the reception method in Embodiment 2. FIG. 50 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 51 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 52 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 53 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 54 is a flowchart illustrating an example of operation of the receiver in Embodiment 3. FIG. 55 is a flowchart illustrating another example of operation of a receiver in Embodiment 3. FIG. 56A is a block diagram illustrating an example of a transmitter in Embodiment 3. FIG. 56B is a block diagram illustrating another example of the transmitter in Embodiment 3. FIG. 57 is a diagram illustrating a configuration example of a system including a plurality of transmitters in the third embodiment. FIG. 58 is a block diagram illustrating another example of the transmitter in the third embodiment. FIG. 59A is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 59B is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 59C is a diagram illustrating an example of a transmitter in Embodiment 3. 60A is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. FIG. 60B is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 61 is a diagram illustrating an example of processing operations of a receiver, a transmitter, and a server in Embodiment 3. FIG. 62 is a diagram illustrating an example of processing operations of a receiver, a transmitter, and a server in Embodiment 3. FIG. 63 is a diagram illustrating an example of processing operations of the receiver, the transmitter, and the server in Embodiment 3. FIG. 64A is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment. FIG. 64B is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment. FIG. 65 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 66 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 67 is a diagram illustrating an example of operation of a transmitter, a receiver, and a server in Embodiment 3. FIG. 68 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 69 is a diagram illustrating an example of appearance of a receiver in Embodiment 3. FIG. 70 is a diagram illustrating an example of operation of a transmitter, a receiver, and a server in Embodiment 3. 71 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. FIG. 72 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 73 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 74 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 75A is a diagram illustrating an example of a structure of information transmitted by a transmitter in Embodiment 3. FIG. 75B is a diagram illustrating another example of a structure of information transmitted by a transmitter in Embodiment 3. FIG. 76 is a diagram illustrating an example of a 4-level PPM modulation scheme by a transmitter in Embodiment 3. 77 is a diagram illustrating an example of a PPM modulation scheme by a transmitter in Embodiment 3. FIG. FIG. 78 is a diagram illustrating an example of a PPM modulation scheme in a transmitter in Embodiment 3. FIG. 79A is a diagram showing an example of a luminance change pattern corresponding to the header (preamble portion) in the third embodiment. FIG. 79B is a diagram showing an example of a luminance change pattern in Embodiment 3. FIG. 80A is a diagram showing an example of a luminance change pattern in Embodiment 3. FIG. 80B is a diagram showing an example of a luminance change pattern in Embodiment 3. FIG. 81 is a diagram illustrating an example of operation of a receiver in a situation in front of a store in Embodiment 4. FIG. 82 is a diagram illustrating another example of operation of a receiver in a situation in front of a store in Embodiment 4. FIG. 83 is a diagram illustrating an example of next operation of the receiver in the situation in front of the store in the fourth embodiment. FIG. 84 is a diagram illustrating an example of next operation of the receiver in the situation in front of the store in the fourth embodiment. FIG. 85 is a diagram illustrating an example of next operation of the receiver in the situation in front of the store in the fourth embodiment. FIG. 86 is a diagram illustrating an example of operation of the display device in the in-store situation according to Embodiment 4. FIG. 87 is a diagram illustrating an example of next operation of the display device in the in-store situation in the fourth embodiment. FIG. 88 is a diagram illustrating an example of next operation of the display device in the in-store situation in the fourth embodiment. FIG. 89 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 90 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 91 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 92 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 93 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 94 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 95 is a diagram illustrating an example of operation of a receiver in a store search situation in the fourth embodiment. FIG. 96 is a diagram illustrating an example of next operation of the receiver in the store search situation in the fourth embodiment. FIG. 97 is a diagram illustrating an example of next operation of the receiver in the store search situation in the fourth embodiment. FIG. 98 is a diagram illustrating an example of operation of a receiver in a movie advertisement situation in Embodiment 4. FIG. 99 is a diagram illustrating an example of next operation of a receiver in a movie advertisement situation in the fourth embodiment. FIG. 100 is a diagram illustrating an example of next operation of a receiver in a movie advertisement situation in Embodiment 4. FIG. 101 is a diagram illustrating an example of next operation of a receiver in a movie advertisement situation in Embodiment 4. FIG. 102 is a diagram illustrating an example of operation of a receiver in a museum situation in Embodiment 4. FIG. 103 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 104 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 105 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 106 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 107 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 108 is a diagram illustrating an example of operation of a receiver in a bus stop situation in Embodiment 4. FIG. 109 is a diagram illustrating an example of next operation of a receiver in a bus stop situation in the fourth embodiment. FIG. 110 is a diagram for describing imaging in the fourth embodiment. FIG. 111 is a diagram for explaining transmission and imaging in the fourth embodiment. FIG. 112 is a diagram for explaining transmission in the fourth embodiment. FIG. 113 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. 114 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. FIG. 115 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. 116 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 117 is a diagram illustrating an example of operation of a receiver in Embodiment 5. FIG. 118 is a diagram illustrating an example of operation of a receiver in Embodiment 5. FIG. FIG. 119 is a diagram illustrating an example of operation of a system including a transmitter, a receiver, and a server in Embodiment 5. 120 is a block diagram illustrating a configuration of a transmitter in Embodiment 5. In FIG. FIG. 121 is a block diagram illustrating a configuration of a receiver in Embodiment 5. In FIG. 122 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. FIG. 123 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. 124 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 125 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. FIG. 126 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. 127 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 128 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. FIG. 129 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 130 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. 131 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. In FIG. 132 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 133 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. In FIG. 134 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. FIG. 135 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. 136 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. In FIG. FIG. 137 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 138 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 139 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 140 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 141 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 142 is a diagram illustrating an encoding method according to the fifth embodiment. FIG. 143 is a diagram illustrating an encoding method capable of receiving light even when an image is captured from an oblique direction according to the fifth embodiment. FIG. 144 is a diagram illustrating an encoding scheme in which the amount of information differs according to the distance in the fifth embodiment. FIG. 145 is a diagram illustrating an encoding method in which the amount of information differs depending on the distance in the fifth embodiment. FIG. 146 is a diagram illustrating a coding scheme obtained by dividing data according to the fifth embodiment. FIG. 147 is a diagram illustrating the effect of inserting a reverse phase image in the fifth embodiment. FIG. 148 is a diagram showing the effect of inserting a reverse phase image in the fifth embodiment. FIG. 149 is a diagram illustrating super-resolution processing according to the fifth embodiment. FIG. 150 is a diagram illustrating a display indicating that it supports visible light communication in the fifth embodiment. FIG. 151 is a diagram illustrating information acquisition using a visible light communication signal in Embodiment 5. FIG. 152 is a diagram showing a data format in the fifth embodiment. FIG. 153 is a diagram illustrating reception by estimating a three-dimensional shape according to Embodiment 5. FIG. 154 is a diagram illustrating reception by estimating a three-dimensional shape according to Embodiment 5. FIG. 155 is a diagram illustrating stereoscopic projection in the fifth embodiment. FIG. 156 is a diagram illustrating stereoscopic projection in the fifth embodiment. FIG. 157 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 158 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 159 is a diagram illustrating an example of a transmission signal according to the sixth embodiment. FIG. 160 is a diagram illustrating an example of a transmission signal according to the sixth embodiment. 161A is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 161B is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 161C is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. FIG. 162A is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 162B is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 163A is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 163B is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 163C is a diagram illustrating an example of an image (bright line image) captured by the receiver in Embodiment 6. 164 is a diagram illustrating an example of a captured image (bright line image) of a receiver in Embodiment 6. FIG. FIG. 165 is a diagram illustrating an example of a transmission signal in Embodiment 6. FIG. 166 is a diagram illustrating an example of operation of a receiver in Embodiment 6. FIG. 167 is a diagram illustrating an example of instructions to the user displayed on the receiver screen in the sixth embodiment. FIG. 168 is a diagram illustrating an example of instructions to the user displayed on the receiver screen in the sixth embodiment. FIG. 169 is a diagram illustrating an example of a signal transmission method in Embodiment 6. 170 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. FIG. 171 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. 172 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. 173 is a diagram for describing a use case in the sixth embodiment. FIG. 174 is a diagram illustrating an information table transmitted to the server by the smartphone according to Embodiment 6. FIG. 175 is a block diagram of the server in the sixth embodiment. FIG. 176 is a flowchart showing overall processing of the system in the sixth embodiment. FIG. 177 is a diagram illustrating an information table transmitted from the server according to the sixth embodiment to a smartphone. FIG. 178 is a diagram illustrating a screen flow displayed on the wearable device from when the user receives information from the server in front of the store until when the user actually purchases the product. FIG. 179 is a diagram for explaining another use case in the sixth embodiment. FIG. 180 is a diagram illustrating a service providing system using the reception method described in each embodiment. FIG. 181 is a flowchart illustrating a service provision flow. FIG. 182 is a flowchart showing service provision in another example. FIG. 183 is a flowchart illustrating service provision in another example. FIG. 184A is a diagram for describing a modulation scheme easily received in the eighth embodiment. FIG. 184B is a diagram for explaining a modulation scheme easy to receive in the eighth embodiment. FIG. 185 is a diagram for explaining a modulation scheme easy to receive in the eighth embodiment. FIG. 186 is a diagram for describing the combination of bright line communication and image recognition in the eighth embodiment. FIG. 187A is a diagram for describing a method of using an imaging element suitable for visible light signal reception in Embodiment 8. FIG. 187B is a diagram for describing a method of using an imaging element suitable for visible light signal reception in Embodiment 8. FIG. 187C is a diagram for describing a usage method of an imaging element suitable for visible light signal reception in Embodiment 8. FIG. 187D is a diagram for describing a method of using an imaging element suitable for receiving visible light signals according to Embodiment 8. FIG. 187E is a flowchart for describing a method of using an imaging element suitable for visible light signal reception in Embodiment 8. FIG. 188 is a diagram illustrating captured image sizes suitable for visible light signal reception in the eighth embodiment. 189A is a diagram illustrating captured image sizes suitable for visible light signal reception in Embodiment 8. FIG. FIG. 189B is a flowchart illustrating an operation for switching to a captured image size suitable for reception of a visible light signal in the eighth embodiment. FIG. 189C is a flowchart illustrating an operation for switching to a captured image size suitable for visible light signal reception in the eighth embodiment. FIG. 190 is a diagram for describing reception of a visible light signal using zoom according to the eighth embodiment. FIG. 191 is a diagram for describing a compression method of an image data size suitable for visible light signal reception in the eighth embodiment. FIG. 192 is a diagram for explaining a modulation scheme with high reception error detection accuracy in the eighth embodiment. FIG. 193 is a diagram for describing a change in operation of the receiver due to a difference in situation in the eighth embodiment. FIG. 194 is a diagram for describing notification of visible light communication to a human according to the eighth embodiment. FIG. 195 is a diagram for explaining expansion of the reception range by the light diffusing plate in the eighth embodiment. FIG. 196 is a diagram for describing a synchronization method of signal transmission from a plurality of projectors according to the eighth embodiment. FIG. 197 is a diagram for describing a synchronization method of signal transmission from a plurality of displays in the eighth embodiment. FIG. 198 is a diagram for describing reception of a visible light signal by the illuminance sensor and the image sensor in the eighth embodiment. FIG. 199 is a diagram for describing a trigger to start reception in the eighth embodiment. FIG. 200 is a diagram for describing a reception start gesture in the eighth embodiment. FIG. 201 is a diagram for describing an application example to the car navigation system in the eighth embodiment. FIG. 202 is a diagram for describing an example of application to car navigation in the eighth embodiment. FIG. 203 is a diagram for describing an application example to content protection in the eighth embodiment. 204A is a diagram for describing an application example as an electronic lock in Embodiment 8. FIG. FIG. 204B is a flowchart of an information communication method in Embodiment 8. FIG. 204C is a block diagram of an information communication device in Embodiment 8. FIG. 205 is a diagram for explaining an application example as store visit information transmission in the eighth embodiment. FIG. 206 is a diagram for describing an application example of order control according to a place in the eighth embodiment. FIG. 207 is a diagram for describing an example of application to route guidance in the eighth embodiment. FIG. 208 is a diagram for describing an example of application to location communication in the eighth embodiment. FIG. 209 is a diagram for describing an application example to use log accumulation and analysis in the eighth embodiment. FIG. 210 is a diagram for describing an example of application to screen sharing in the eighth embodiment. FIG. 211 is a diagram for describing an example of application to screen sharing in the eighth embodiment. FIG. 212 is a diagram for describing an application example of position estimation using a wireless access point in the eighth embodiment. FIG. 213 is a diagram illustrating a configuration in which position estimation is performed by visible light communication and wireless communication in the eighth embodiment. FIG. 214 is a diagram illustrating an example of application of the information communication method in Embodiment 8. FIG. 215 is a flowchart illustrating an application example of the information communication method in the eighth embodiment. FIG. 216 is a flowchart illustrating an application example of the information communication method in the eighth embodiment. FIG. 217 is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9. FIG. 218 is a diagram illustrating an example of application of the transmitter in Embodiment 9. FIG. 219 is a flowchart of the information communication method in the ninth embodiment. FIG. 220 is a block diagram of an information communication device according to the ninth embodiment. FIG. 221A is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9. FIG. 221B is a flowchart illustrating operation of the receiver in Embodiment 9. FIG. 222 is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9. FIG. 223 is a diagram illustrating an example of application of the transmitter in Embodiment 9. FIG. 224A is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9. FIG. 224B is a flowchart illustrating operation of the receiver in Embodiment 9. 225 is a diagram illustrating operation of a receiver in Embodiment 9. FIG. 226 is a diagram illustrating an example of application of a transmitter in Embodiment 9. FIG. FIG. 227 is a diagram illustrating an example of application of the receiver in Embodiment 9. FIG. 228A is a flowchart illustrating an example of operation of a transmitter in Embodiment 9. FIG. 228B is a flowchart illustrating an example of operation of a transmitter in Embodiment 9. FIG. 229 is a flowchart illustrating an example of operation of a transmitter in Embodiment 9. FIG. 230 is a flowchart illustrating an example of operation of the imaging device according to the ninth embodiment. FIG. 231 is a flowchart illustrating an example of operation of the imaging device according to the ninth embodiment. FIG. 232 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9. FIG. 233 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9. 234 is a diagram illustrating an example of signals transmitted by a transmitter in Embodiment 9. FIG. FIG. 235 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9. FIG. 236 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. FIG. 237 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. 238 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. In FIG. 239 is a diagram illustrating an example of operation of a transmitter in Embodiment 9. FIG. 240 is a diagram illustrating an example of operation of a transmitter in Embodiment 9. FIG. FIG. 241 is a diagram illustrating an example of a structure and operation of a transmitter in Embodiment 9. 242 is a diagram illustrating an example of a structure and operation of a transmitter in Embodiment 9. FIG. FIG. 243 is a diagram illustrating a timepiece including the optical sensor according to the tenth embodiment. FIG. 244 is a diagram illustrating an example of a receiver in Embodiment 10. FIG. 245 is a diagram illustrating an example of a receiver in Embodiment 10. FIG. 246A is a flowchart of an information communication method according to an aspect of the present invention. FIG. 246B is a block diagram of a mobile terminal according to one embodiment of the present invention. FIG. 247 is a diagram illustrating an example of a reception system in Embodiment 10. FIG. 248 is a diagram illustrating an example of a reception system in Embodiment 10. FIG. 249A is a diagram illustrating an example of a modulation scheme in Embodiment 10. FIG. 249B is a diagram illustrating an example of a modulation scheme in Embodiment 10. FIG. 249C is a diagram illustrating an example of a modulation scheme in Embodiment 10. FIG. 249D is a diagram illustrating an example of mixing signal separation in Embodiment 10. FIG. 249E is a diagram illustrating an example of mixing signal separation in Embodiment 10. FIG. 249F is a flowchart illustrating processing of the information processing program in the tenth embodiment. FIG. 249G is a block diagram of an information processing device in Embodiment 10. FIG. 250A is a diagram illustrating an example of a visible light communication system in Embodiment 10. FIG. 250B is a diagram for describing a use case in Embodiment 10. FIG. 250C is a diagram illustrating an example of a signal transmission / reception system in Embodiment 10. FIG. 251 is a flowchart illustrating a reception method in which interference is eliminated in the tenth embodiment. FIG. 252 is a flowchart illustrating a transmitter direction estimation method according to the tenth embodiment. FIG. 253 is a flowchart illustrating a reception start method in Embodiment 10. FIG. 254 is a flowchart illustrating an ID generation method using information of another medium according to the tenth embodiment. FIG. 255 is a flowchart illustrating a reception method selection method based on frequency separation in the tenth embodiment. FIG. 256 is a flowchart showing a signal reception method when the exposure time is long in the tenth embodiment. FIG. 257 is a diagram illustrating an example of a transmitter dimming (adjusting brightness) method in Embodiment 10. 258 is a diagram illustrating an example of a method of configuring a dimming function of a transmitter in Embodiment 10. [FIG. FIG. 259A is a flowchart illustrating an example of operation of a receiver in Embodiment 11. FIG. 259B is a flowchart illustrating an example of operation of a receiver in Embodiment 11. FIG. 259C is a flowchart illustrating an example of operation of a receiver in Embodiment 11. FIG. 259D is a flowchart illustrating an example of operation of a receiver in Embodiment 11. FIG. 260 is a diagram for explaining the EX zoom. FIG. 261A is a flowchart illustrating processing of a reception program in the tenth embodiment. FIG. 261B is a block diagram of a receiving apparatus in Embodiment 10. FIG. 262 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 263 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 264 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 265 is a diagram illustrating an example of a screen display method of a receiver in Embodiment 12. 266 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. FIG. 267 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 268 is a flowchart illustrating an example of a signal reception method in Embodiment 12. FIG. 269 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 270A is a flowchart showing processing of a reception program in the twelfth embodiment. FIG. 270B is a block diagram of a receiving apparatus in Embodiment 12. FIG. 271 is a diagram illustrating an example of display on the receiver when a visible light signal is received. FIG. 272 is a diagram illustrating an example of display on the receiver when a visible light signal is received. FIG. 273 is a diagram illustrating an example of a display of the acquired data image. FIG. 274 is a diagram illustrating an example of an operation for saving or discarding acquired data. FIG. 275 is a diagram illustrating a display example when browsing acquired data. FIG. 276 is a diagram illustrating an example of a transmitter in Embodiment 12. FIG. 277 is a diagram illustrating an example of a reception method in Embodiment 12. FIG. 278 is a diagram illustrating an example of a header pattern in the thirteenth embodiment. FIG. 279 is a diagram for describing an example of the structure of a communication protocol packet in the thirteenth embodiment. FIG. 280 is a flowchart illustrating an example of a reception method in Embodiment 13. FIG. 281 is a flowchart illustrating an example of a reception method in Embodiment 13. FIG. 282 is a flowchart illustrating an example of a reception method in Embodiment 13. FIG. 283 is a diagram for illustrating a reception method in which the receiver in Embodiment 13 uses an exposure time longer than the period of the modulation frequency (modulation period). FIG. 284 is a diagram for illustrating a reception method in which the receiver in Embodiment 13 uses an exposure time longer than the period of the modulation frequency (modulation period). FIG. 285 is a diagram illustrating an efficient division number with respect to the size of transmission data in the thirteenth embodiment. FIG. 286A is a diagram illustrating an example of a setting method in Embodiment 13. FIG. 286B is a diagram illustrating another example of the setting method according to the thirteenth embodiment. FIG. 287A is a flowchart illustrating processing of the information processing program in the thirteenth embodiment. FIG. 287B is a block diagram of an information processing device in Embodiment 13. FIG. 288 is a diagram for describing an example of application of the transmission and reception system in Embodiment 13. FIG. 289 is a flowchart illustrating processing operations of the transmission and reception system in the thirteenth embodiment. FIG. 290 is a diagram for describing an example of application of the transmission and reception system in Embodiment 13. FIG. 291 is a flowchart showing processing operations of the transmission / reception system in the thirteenth embodiment. FIG. 292 is a diagram for describing an example of application of the transmission and reception system in Embodiment 13. FIG. 293 is a flowchart illustrating processing operations of the transmission and reception system in the thirteenth embodiment. FIG. 294 is a diagram for describing an example of application of a transmitter in Embodiment 13. FIG. 295 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. FIG. 296 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. FIG. 297 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. FIG. 298 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. 299 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. FIG. 300 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. 301 is a diagram for describing an application example of the transmission and reception system in Embodiment 14. 302 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. FIG. 303 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. 304 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. 305 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. FIG. 306 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. FIG. 307 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. 308 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. FIG. 309 is a diagram for describing operation of a receiver in Embodiment 15. FIG. 310A is a diagram for describing another operation of the receiver in Embodiment 15. FIG. 310B is a diagram illustrating an example of indicators displayed by the output unit 1215 in the fifteenth embodiment. FIG. 310C is a diagram illustrating a display example of AR in the fifteenth embodiment. FIG. 311A is a diagram for describing an example of a transmitter in Embodiment 15. FIG. 311B is a diagram for describing another example of the transmitter in Embodiment 15. FIG. 312A is a diagram for describing an example of synchronous transmission by a plurality of transmitters in Embodiment 15. FIG. 312B is a diagram for describing another example of synchronous transmission by a plurality of transmitters in Embodiment 15. FIG. 313 is a diagram for describing another example of synchronous transmission by a plurality of transmitters in Embodiment 15. 314 is a diagram for describing signal processing of a transmitter in Embodiment 15. FIG. FIG. 315 is a flowchart illustrating an example of a reception method in Embodiment 15. 316 is an explanatory diagram for describing an example of a reception method in Embodiment 15. FIG. FIG. 317 is a flowchart illustrating another example of the reception method in Embodiment 15. FIG. 318 is a diagram illustrating an example in the case where the exposure time is three times the transmission period and the transmission signal is binary of 0 or 1 in the fifteenth embodiment. FIG. 319 is a diagram illustrating state transition paths according to the fifteenth embodiment. FIG. 320 is a diagram illustrating an image obtained by capturing an object blinking at high speed according to the sixteenth embodiment. FIG. 321 is a diagram illustrating a reception period and a blind period when LSS is used in Embodiment 16. FIG. 322 is a diagram illustrating cutout scanning that enables continuous reception according to the sixteenth embodiment. FIG. 323 is a diagram illustrating an example of symbols by frequency modulation in Embodiment 16. FIG. 324 is a diagram illustrating a frequency response by the LSS in the sixteenth embodiment. FIG. 325 is a diagram illustrating an example of 4PPM symbols and V4PPM symbols in Embodiment 16. In FIG. 326 is a diagram illustrating an example of Manchester code symbols and VPPM symbols in Embodiment 16. FIG. FIG. 327 is a diagram for comparing and explaining the efficiency of V4PPM and VPPM in the sixteenth embodiment. FIG. 328 is a diagram illustrating frequency-domain signals and noise levels according to the sixteenth embodiment. 329A is a diagram illustrating a difference between a transmission frequency and a reception frequency (maximum frequency component of a reception signal) in Embodiment 16. FIG. FIG. 329B is a diagram illustrating an example of an error rate with respect to the frequency margin in Embodiment 16. FIG. 329C is a diagram illustrating another example of the error rate with respect to the frequency margin in Embodiment 16. FIG. 329D is a diagram illustrating another example of the error rate with respect to the frequency margin in Embodiment 16. FIG. 329E is a diagram illustrating another example of the error rate with respect to the frequency margin in the sixteenth embodiment. FIG. 329F is a diagram illustrating another example of the error rate with respect to the frequency margin in the sixteenth embodiment. FIG. 330 is a diagram illustrating a packet reception error rate of the V4PPM symbol in the sixteenth embodiment. FIG. 331 is a block diagram illustrating a configuration of the display system in the seventeenth embodiment. FIG. 332 is a diagram illustrating a transmission / reception mode between the video standard signal transmission unit and the video standard signal reception unit according to the seventeenth embodiment. FIG. 333 is a diagram illustrating an example of a specific transmission / reception form between the video standard signal transmission unit and the video standard signal reception unit according to the seventeenth embodiment. FIG. 334 is a diagram illustrating another example of a specific transmission / reception form between the video standard signal transmission unit and the video standard signal reception unit in Embodiment 17. FIG. 335 is a diagram illustrating another example of a specific transmission / reception form between the video standard signal transmission unit and the video standard signal reception unit in Embodiment 17. FIG. 336A is a diagram illustrating an example of power transmitted in the power transmission transmission line in the seventeenth embodiment. FIG. 336B is a diagram illustrating another example of power transmitted in the power transmission transmission line in the seventeenth embodiment. FIG. 337 is a diagram illustrating another example of the specific transmission / reception form between the video standard signal transmission unit and the video standard signal reception unit in the seventeenth embodiment. FIG. 338 is a diagram illustrating another example of the specific transmission / reception form between the video standard signal transmission unit and the video standard signal reception unit according to the seventeenth embodiment. FIG. 339 is a schematic diagram illustrating an example of a visible light communication system according to the eighteenth embodiment. FIG. 340 is a block diagram illustrating an example of a schematic configuration of the display apparatus according to the eighteenth embodiment. FIG. 341A is a diagram illustrating B. It is a figure which shows an example of the state before superimposing a visible light communication signal on L control signal. FIG. 341B is a diagram illustrating B.3 according to Example 1 of Embodiment 18. FIG. It is a figure which shows an example of the state which superimposed the visible light communication signal on L control signal. FIG. 342 is a timing chart for describing the first method in the second example of the eighteenth embodiment. FIG. 343 is a timing chart for describing the first method in the second example of the eighteenth embodiment. FIG. 344A is a timing chart for describing the second method in the second example of the eighteenth embodiment. FIG. 344B is a timing chart for describing the second method in the second example of the eighteenth embodiment. FIG. 344C is a timing chart for describing the second method in the second example of the eighteenth embodiment. FIG. 344D is a timing chart for describing the second method in the second example of the eighteenth embodiment. FIG. 345A is a timing chart for describing the second method in the second example of the eighteenth embodiment. FIG. 345B is a timing chart for describing the second method in the second example of the eighteenth embodiment. FIG. 345C is a timing chart for describing the second method in the second example of the eighteenth embodiment. FIG. 345D is a timing chart for explaining the second method in the second example of the eighteenth embodiment. FIG. 346 is a diagram illustrating B.3 in Example 3 of the eighteenth embodiment. It is a timing chart for demonstrating the method of superimposing a visible light communication signal on L control signal. FIG. 347 is a flowchart for explaining the operation of the second processing unit in the nineteenth embodiment. FIG. 348A is a diagram illustrating B.3 in the nineteenth embodiment. It is a figure for demonstrating the specific method of superimposing an encoding signal on L control signal. FIG. 348B is a diagram illustrating B.3 in the nineteenth embodiment. It is a figure for demonstrating the specific method of superimposing an encoding signal on L control signal. FIG. 348C is a diagram illustrating B.3 in the nineteenth embodiment. It is a figure for demonstrating the specific method of superimposing an encoding signal on L control signal. FIG. 348D is a diagram illustrating B. in the nineteenth embodiment. It is a figure for demonstrating the specific method of superimposing an encoding signal on L control signal. FIG. 349 is a diagram illustrating the process of B.E. It is a figure for demonstrating another specific method of superimposing an encoding signal on L control signal. FIG. 350 is a flowchart for explaining the operation of the second processing unit in the twentieth embodiment. FIG. 351 is a timing chart illustrating an example of group division of regions in the twentieth embodiment. FIG. 352 is a timing chart illustrating another example of area group division according to the twentieth embodiment. FIG. 353 is a timing chart illustrating another example of group division of regions in the twentieth embodiment. FIG. 354 is a flowchart for describing operation of the second processing unit in the twenty-first embodiment. FIG. 355A is a diagram illustrating B.3 in Embodiment 21. FIG. It is a figure for demonstrating the relationship of the phase of L control signal and a visible light communication signal. FIG. 355B is a diagram illustrating B.3 in Embodiment 21. FIG. It is a figure for demonstrating the relationship of the phase of L control signal and a visible light communication signal. FIG. 356A is a timing chart for explaining the operation of the second processing unit in the twenty-first embodiment. FIG. 356B is a timing chart for explaining the operation of the second processing unit in the twenty-first embodiment. FIG. 356C is a timing chart for explaining the operation of the second processing unit in the twenty-first embodiment. FIG. 357A is a timing chart for explaining the operation of the second processing unit in the twenty-second embodiment. FIG. 357B is a timing chart for explaining the operation of the second processing unit in the twenty-second embodiment. FIG. 358 is a timing chart showing backlight control during local dimming adaptation in the twenty-third embodiment. FIG. 359 is a flowchart for describing an example of operation of the second processing unit in the twenty-third embodiment. FIG. 360 is a timing chart for explaining an example of the operation of the second processing unit in the twenty-third embodiment. FIG. 361 is a flowchart for describing an example of operation of the second processing unit in the twenty-third embodiment. FIG. 362 is a timing chart for explaining an example of the operation of the second processing unit in the twenty-third embodiment. FIG. 363 is a timing chart for explaining an example of operation of the second processing unit in the twenty-third embodiment. FIG. 364 is a schematic diagram of the visible light communication system according to the twenty-fourth embodiment. FIG. 365 is a block diagram of the display device according to the twenty-fourth embodiment. FIG. 366 is a diagram illustrating an example of generating a visible light communication signal according to the twenty-fourth embodiment. FIG. 367 is a block diagram of the receiving apparatus according to the twenty-fourth embodiment. FIG. 368 is a diagram illustrating a captured image of the reception device when the backlight of the display device according to the twenty-fourth embodiment is turned on / off. FIG. 369 is a diagram for explaining a captured image of the reception device with respect to the transmission frame of the display device according to the twenty-fourth embodiment. FIG. 370 is a diagram for explaining the relationship between the transmission clock frequency of the display device according to the twenty-fourth embodiment and the frame rate of the imaging unit of the reception device. FIG. 371 is a diagram for explaining a first generation example of a transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372A is a diagram for explaining a second generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372B is a diagram for explaining a third generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372C is a diagram for explaining a fourth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372D is a diagram for explaining a fifth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372E is a diagram for explaining a sixth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 373 is a flowchart for explaining the operation of the visible light communication signal processing unit in the display apparatus according to the twenty-fourth embodiment. FIG. 374 is a flowchart for explaining the operation of the visible light communication signal processing unit of the display device according to the twenty-fifth embodiment. FIG. 375 is a diagram for explaining an example of a method for determining the number of transmissions of an arbitrary block of a transmission frame for one signal unit according to the twenty-fifth embodiment. FIG. 376 is a diagram for explaining a generation example of a transmission frame for one signal unit according to the twenty-fifth embodiment. FIG. 377 is a flowchart for explaining the operation of the visible light communication signal processing unit in the display apparatus according to the twenty-sixth embodiment. FIG. 378 is a diagram for explaining an example of a method for determining the number of transmissions of an arbitrary block of a transmission frame for one signal unit according to the twenty-sixth embodiment. FIG. 379 is a diagram for explaining a generation example of transmission frames for one signal unit output from the display apparatus according to the twenty-sixth embodiment. FIG. 380 is a diagram illustrating another example of generating a transmission frame for one signal unit output from the display apparatus according to the twenty-sixth embodiment. FIG. 381 is a diagram for explaining a first generation example of a transmission frame for one signal unit according to the twenty-seventh embodiment. FIG. 382A is a diagram for explaining a second generation example of the transmission frame for one signal unit according to the twenty-seventh embodiment. FIG. 382B is a diagram for explaining a third generation example of the transmission frame for one signal unit according to the twenty-seventh embodiment. FIG. 382C is a diagram for explaining a fourth generation example of the transmission frame for one signal unit according to the twenty-seventh embodiment. FIG. 383 is a flowchart for explaining the operation of the visible light communication signal processing unit of the display device according to the twenty-seventh embodiment. FIG. 384 is a diagram for describing visible light communication switching control when the transmission device according to Embodiment 28 is a moving image display device such as a television set. FIG. 385 is a diagram illustrating a procedure of transmitting logical data according to Embodiment 29 through visible light communication. FIG. 386 is a diagram illustrating a procedure of transmitting logical data according to Embodiment 29 through visible light communication. FIG. 387 is a diagram for explaining the dividing process by the logical data dividing unit according to the twenty-ninth embodiment. FIG. 388 is a diagram for explaining the dividing process by the logical data dividing unit according to the twenty-ninth embodiment. 389 is a diagram illustrating an example of a transmission signal in Embodiment 29. FIG. 390 is a diagram illustrating another example of a transmission signal in Embodiment 29. FIG. 391 is a diagram illustrating another example of a transmission signal in Embodiment 29. FIG. 392A is a diagram for illustrating a transmitter in Embodiment 30. FIG. FIG. 392B is a diagram illustrating a change in luminance of each of RGB in Embodiment 30. FIG. 393 is a diagram illustrating the afterglow characteristics of the green fluorescent component and the red fluorescent component in the thirtieth embodiment. FIG. 394 is a diagram for describing a problem newly generated in order to suppress occurrence of a barcode reading error in the thirtieth embodiment. FIG. 395 is a diagram for describing downsampling performed by the receiver in Embodiment 30. FIG. 396 is a flowchart illustrating processing operations of a receiver in Embodiment 30. FIG. 397 is a diagram illustrating processing operation of the reception device (imaging device) in Embodiment 31. 398 is a diagram illustrating processing operation of a reception device (imaging device) in Embodiment 31. FIG. 399 is a diagram illustrating processing operation of a reception device (imaging device) in Embodiment 31. FIG. FIG. 400 is a diagram illustrating processing operation of a reception device (imaging device) in Embodiment 31. FIG. 401 is a diagram illustrating an example of an application in the thirty-second embodiment. FIG. 402 is a diagram illustrating an example of an application in the thirty-second embodiment. 403 is a diagram illustrating an example of a transmission signal and an example of an audio synchronization method in Embodiment 32. FIG. 404 is a diagram illustrating an example of a transmission signal in Embodiment 32. FIG. 405 is a diagram illustrating an example of a process flow of a receiver in Embodiment 32. FIG. 406 is a diagram illustrating an example of a user interface of a receiver in Embodiment 32. FIG. 407 is a diagram illustrating an example of a process flow of a receiver in Embodiment 32. FIG. 408 is a diagram illustrating another example of processing flow of a receiver in Embodiment 32. FIG. FIG. 409A is a diagram for describing a specific method of synchronous playback in Embodiment 32. FIG. 409B is a block diagram illustrating a configuration of a playback device (receiver) that performs synchronized playback according to Embodiment 32. FIG. 409C is a flowchart illustrating a processing operation of a playback device (receiver) that performs synchronized playback according to Embodiment 32. FIG. 410 is a diagram for describing preparations for synchronized playback in the thirty-second embodiment. 411 is a diagram illustrating an example of application of a receiver in Embodiment 32. FIG. FIG. 412A is a front view of a receiver held by a holder in Embodiment 32. FIG. FIG. 412B is a rear view of a receiver held by a holder in Embodiment 32. FIG. 413 is a diagram for describing a use case of a receiver held by a holder in Embodiment 32. FIG. FIG. 414 is a flowchart illustrating processing operation of a receiver held by a holder in Embodiment 32. 415 is a diagram illustrating an example of an image displayed by the receiver in Embodiment 32. FIG. FIG. 416 is a diagram illustrating another example of the holder according to Embodiment 32. In FIG. 417A is a diagram illustrating an example of a visible light signal in Embodiment 33. FIG. 417B is a diagram illustrating an example of a visible light signal in Embodiment 33. FIG. 417C is a diagram illustrating an example of a visible light signal in Embodiment 33. FIG. 417D is a diagram illustrating an example of a visible light signal in Embodiment 33. FIG. 418 is a diagram illustrating a structure of a visible light signal in Embodiment 33. FIG. FIG. 419 is a diagram illustrating an example of bright line images obtained by imaging with a receiver in Embodiment 33. In FIG. FIG. 420 is a diagram illustrating another example of bright line images obtained by imaging by the receiver in Embodiment 33. 421 is a diagram illustrating another example of a bright line image obtained by imaging by a receiver in Embodiment 33. FIG. 422 is a diagram for describing adaptation of the receiver in Embodiment 33 to a camera system that performs HDR combining. FIG. 423 is a diagram for describing a processing operation of the visible light communication system according to Embodiment 33. FIG. 424A is a diagram illustrating an example of vehicle-to-vehicle communication using visible light in Embodiment 33. FIG. FIG. 424B is a diagram illustrating another example of vehicle-to-vehicle communication using visible light according to Embodiment 33. 425 is a diagram illustrating an example of a position determination method of a plurality of LEDs in Embodiment 33. FIG. 426 is a diagram illustrating an example of bright line images obtained by capturing an image of the vehicle in Embodiment 33. FIG. 427 is a diagram illustrating an example of application of a receiver and a transmitter in Embodiment 33. FIG. FIG. 427 is a view of the automobile from the back. FIG. 428 is a flowchart illustrating an example of processing operations of a receiver and a transmitter in Embodiment 33. 429 is a diagram illustrating an example of application of a receiver and a transmitter in Embodiment 33. FIG. FIG. 430 is a flowchart illustrating an example of processing operations of the receiver 7007a and the transmitter 7007b in Embodiment 33. FIG. 431 is a diagram illustrating a configuration of a visible light communication system applied to the inside of a train according to Embodiment 33. FIG. 432 is a diagram illustrating a configuration of a visible light communication system applied to a facility such as an amusement park in Embodiment 33. FIG. 433 is a diagram illustrating an example of a visible light communication system including a playground device and a smartphone according to Embodiment 33. 434 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 435 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 436 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 437 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 438 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 439 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 440 is a diagram illustrating an example of a transmission signal in Embodiment 34. FIG. 441 is a diagram illustrating an example of a reception algorithm in Embodiment 34. FIG. 442 is a diagram illustrating an example of a reception algorithm in Embodiment 34. FIG. 443 is a diagram illustrating an example of a reception algorithm in Embodiment 34. FIG. 444 is a diagram illustrating an example of a reception algorithm in Embodiment 34. FIG. 445 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 446 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 447 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 448 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 449 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. FIG. 450 is a diagram illustrating an example of a transmission signal in Embodiment 35. In FIG. 451 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. FIG. 452 is a diagram illustrating an example of a transmission signal in Embodiment 35. 453 is a diagram illustrating an example of a transmission / reception system in Embodiment 35. FIG. FIG. 454 is a flowchart illustrating an example of processing of the transmission / reception system in the thirty-fifth embodiment. FIG. 455 is a flowchart showing the operation of the server in the thirty-fifth embodiment. FIG. 456 is a flowchart illustrating an example of operation of a receiver in Embodiment 35. FIG. 457 is a flowchart illustrating a method for calculating the progress in the simple mode according to the thirty-fifth embodiment. FIG. 458 is a flowchart showing a method for calculating the progress in the maximum likelihood estimation mode in the thirty-fifth embodiment. FIG. 459 is a flowchart showing a display method in which the progress status does not decrease in the thirty-fifth embodiment. FIG. 460 is a flowchart illustrating a progress display method when there are a plurality of packet lengths according to the thirty-fifth embodiment. 461 is a diagram illustrating an example of an operation state of a receiver in Embodiment 35. FIG. 462 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 463 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 464 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 465 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 466 is a block diagram illustrating an example of a transmitter in Embodiment 35. FIG. FIG. 467 is a timing chart in the case where the LED display in Embodiment 35 is driven with the optical ID modulation signal of the present invention. FIG. 468 is a diagram showing a timing chart in the case where the LED display in Embodiment 35 is driven by the optical ID modulation signal of the present invention. FIG. 469 is a timing chart when the LED display in Embodiment 35 is driven with the optical ID modulation signal of the present invention. FIG. 470A is a flowchart illustrating a transmission method according to one embodiment of the present invention. FIG. 470B is a block diagram illustrating a functional configuration of the transmission device according to one embodiment of the present invention. 471 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 472 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 473 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 474 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 475 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG. 476 is a diagram illustrating an example of a transmission signal in Embodiment 35. FIG.

  A transmission method according to an aspect of the present invention is a transmission method for transmitting a visible light signal by a luminance change, and includes a determination step of determining a luminance change pattern by modulating the visible light signal, and a display. A common switch control step of switching a common switch included in the light source group for commonly lighting a plurality of light sources for representing pixels in the video according to the luminance change pattern; and a plurality of light sources included in the light source group By turning on the first pixel switch for turning on the first light source among the light sources, the common switch is on and only during the period when the first pixel switch is on. And a first pixel switch control step of transmitting the visible light signal by turning on the first light source.

  Accordingly, for example, as shown in FIGS. 463 to 470B, a visible light signal can be appropriately transmitted from a display including a plurality of LEDs as light sources. Therefore, the communication between the apparatuses of the aspect containing apparatuses other than illumination is enabled. When the display is a display for displaying an image by controlling the common switch and the first pixel switch, a visible light signal may be transmitted using the common switch and the first pixel switch. it can. Therefore, a visible light signal can be easily transmitted without making a significant change to the configuration for displaying an image on a display.

  In the determination step, the luminance change pattern may be determined for each symbol period, and in the first pixel switch control step, the first pixel switch may be switched in synchronization with the symbol period.

  Thereby, as shown in FIG. 463, for example, even if the symbol period is 1/2400 seconds, for example, the visible light signal can be appropriately transmitted according to the symbol period.

  Further, in the first pixel switch control step, when displaying an image on the display, the visible period of the lighting period for expressing the pixel value of the pixel in the image corresponding to the first light source is displayed. The first pixel switch may be switched so as to compensate for the lighting period only during a period in which the first light source is turned off to transmit an optical signal. For example, the lighting period may be supplemented by changing pixel values of pixels in the video.

  As a result, as shown in FIGS. 463 and 465, for example, even when the first light source is turned off for the transmission of a visible light signal, the lighting period is supplemented, so that the original image is appropriately displayed without being destroyed. can do.

  Further, the pixel value may be changed at a period that is 1/2 of the symbol period.

  As a result, for example, as shown in FIG. 465, it is possible to appropriately display an image and transmit a visible light signal.

  In the transmission method, the common switch is further turned on by turning on a second pixel switch included in the light source group for turning on a second light source around the first light source. A second pixel switch control step of transmitting the visible light signal by turning on the second light source only during a period in which the second pixel switch is on. In the first and second pixel switch control steps, when the same symbol included in the visible light signal is simultaneously transmitted from each of the first and second light sources, the first and second pixel switches Of the plurality of timings at which each is turned on or off to transmit the same symbol, a timing at which a rise in luminance specific to the same symbol is obtained. The same timing is used for each of the first and second pixel switches, and other timings are made different for each of the first and second pixel switches. You may make the average brightness | luminance of the whole 1st and 2nd light source correspond to a predetermined brightness | luminance.

  As a result, for example, as shown in FIG. 464, in the spatially averaged luminance, the rising edge can be made sharp only at the timing when the rising edge unique to the symbol is obtained, and the occurrence of a reception error can be prevented. Can be suppressed.

  In the first pixel switch control step, when the same symbol included in the visible light signal is transmitted in the first period and the second period following the first period, In each of the second period and the second period, among the plurality of timings at which the first pixel switch is turned on or off to transmit the same symbol, a timing at which a rise in luminance specific to the same symbol is obtained. The same brightness may be set at different timings so that the average brightness of the first light source in the whole of the first and second periods matches the predetermined brightness.

  As a result, for example, as shown in FIG. 464, in the luminance averaged over time, the rising edge can be made sharp only at the timing when the rising edge inherent in the symbol is obtained, and the occurrence of a reception error can be prevented. Can be suppressed.

  In the first pixel switch control step, the first pixel switch is turned on during a signal transmission period in which the common switch is switched according to the luminance change pattern, and the transmission method further includes the signal During the video display period different from the transmission period, turning on the common switch, and turning on the first pixel switch according to the display target video in the video display period, and the common switch is on, and A video display step of displaying pixels in the video by turning on the first light source only during a period in which the first pixel switch is on may be included.

  As a result, since the display of the video and the transmission of the visible light signal are performed in different periods, the display and transmission can be easily performed.

  Note that these comprehensive or specific aspects may be realized by a recording medium such as an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM. You may implement | achieve in arbitrary combinations of a circuit, a computer program, or a recording medium.

  Hereinafter, embodiments will be specifically described with reference to the drawings.

  It should be noted that each of the embodiments described below shows a comprehensive or specific example. The numerical values, shapes, materials, constituent elements, arrangement positions and connecting forms of the constituent elements, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Embodiment 1)
The first embodiment will be described below.

(Observation of luminance of light emitting part)
We propose an imaging method that starts and ends the exposure at different times for each image sensor, rather than exposing all the image sensors at the same timing when capturing one image. FIG. 1 shows an example in which imaging devices arranged in one row are exposed simultaneously, and imaging is performed by shifting the exposure start time in the order of closer rows. Here, the exposure line of the image sensor that is exposed simultaneously is referred to as an exposure line, and the pixel line on the image corresponding to the image sensor is referred to as a bright line.

  When this image capturing method is used to capture an image of a blinking light source on the entire surface of the image sensor, bright lines (light and dark lines of pixel values) along the exposure line appear on the captured image as shown in FIG. . By recognizing the bright line pattern, it is possible to estimate the light source luminance change at a speed exceeding the imaging frame rate. Thereby, by transmitting a signal as a change in light source luminance, communication at a speed higher than the imaging frame rate can be performed. When a signal is expressed by the light source taking two types of luminance values, the lower luminance value is called low (LO), and the higher luminance value is called high (HI). Low may be in a state where the light source is not shining, or may be shining weaker than high.

  By this method, information is transmitted at a speed exceeding the imaging frame rate.

  When there are 20 exposure lines in which the exposure time does not overlap in one captured image and the imaging frame rate is 30 fps, it is possible to recognize a luminance change with a period of 1.67 milliseconds. When there are 1000 exposure lines whose exposure times do not overlap, it is possible to recognize a luminance change with a period of 1 / 30,000 second (about 33 microseconds). The exposure time is set shorter than 10 milliseconds, for example.

  FIG. 2 shows a case where the exposure of the next exposure line is started after the exposure of one exposure line is completed.

  In this case, when the number of frames per second (frame rate) is f and the number of exposure lines constituting one image is 1, if information is transmitted depending on whether or not each exposure line receives a certain amount of light, the maximum Can transmit information at a rate of fl bits per second.

  Note that when exposure is performed with a time difference for each pixel, not for each line, communication at higher speed is possible.

  At this time, when the number of pixels per exposure line is m pixels and information is transmitted depending on whether each pixel receives light above a certain level, the transmission speed is a maximum of flm bits per second.

  As shown in FIG. 3, if the exposure state of each exposure line by the light emission of the light emitting unit can be recognized at a plurality of levels, the light emission time of the light emitting unit is controlled by a unit time shorter than the exposure time of each exposure line. More information can be transmitted.

  If the exposure state can be recognized in the Elv stage, information can be transmitted at a maximum rate of flElv bits per second.

  Moreover, the basic period of transmission can be recognized by causing the light emitting unit to emit light at a timing slightly shifted from the exposure timing of each exposure line.

  FIG. 4 shows a case where the exposure of the next exposure line is started before the exposure of one exposure line is completed. That is, the exposure times of adjacent exposure lines are partially overlapped in time. With such a configuration, (1) it is possible to increase the number of samples within a predetermined time as compared with the case where the exposure of the next exposure line is started after waiting for the end of the exposure time of one exposure line. By increasing the number of samples in a predetermined time, it becomes possible to more appropriately detect the optical signal generated by the optical transmitter that is the subject. That is, it is possible to reduce the error rate when detecting an optical signal. Further, (2) the exposure time of each exposure line can be made longer than when the exposure time of the next exposure line is started after waiting for the end of the exposure time of one exposure line. However, a brighter image can be acquired. That is, the S / N ratio can be improved. In all exposure lines, it is not necessary that the exposure time of adjacent exposure lines has a partial overlap in time, and a configuration in which some exposure lines have no partial overlap. It is also possible. By configuring a part of the exposure lines so as not to partially overlap in time, it is possible to suppress the generation of intermediate colors due to the overlap of exposure times on the imaging screen, and to detect bright lines more appropriately. .

  In this case, the exposure time is calculated from the brightness of each exposure line, and the light emission state of the light emitting unit is recognized.

  When the brightness of each exposure line is determined by a binary value indicating whether the luminance is equal to or higher than a threshold value, in order to recognize the state where no light is emitted, the state where the light emitting unit does not emit light is indicated for each line. It must last longer than the exposure time.

  FIG. 5A shows the influence of the difference in exposure time when the exposure start times of the exposure lines are equal. 7500a is the case where the exposure end time of the previous exposure line is equal to the exposure start time of the next exposure line, and 7500b is the case where the exposure time is longer than that. As in the case of 7500b, the exposure time of adjacent exposure lines is partially overlapped in time, so that the exposure time can be increased. That is, the light incident on the image sensor increases and a bright image can be obtained. In addition, since the imaging sensitivity for capturing images with the same brightness can be suppressed to a low level, an image with less noise can be obtained, so that communication errors are suppressed.

  FIG. 5B shows the influence of the difference in the exposure start time of each exposure line when the exposure times are equal. 7501a is the case where the exposure end time of the previous exposure line is equal to the exposure start time of the next exposure line, and 7501b is the case where the exposure of the next exposure line is started earlier than the end of exposure of the previous exposure line. By adopting a configuration in which the exposure times of adjacent exposure lines partially overlap in time as in 7501b, the number of lines that can be exposed per time can be increased. Thereby, the resolution becomes higher and a large amount of information can be obtained. Since the sample interval (= difference in exposure start time) becomes dense, the change in the light source luminance can be estimated more accurately, the error rate can be reduced, and the change in the light source luminance in a shorter time is recognized. be able to. By making the exposure time overlap, it is possible to recognize blinking of the light source that is shorter than the exposure time by using the difference in exposure amount between adjacent exposure lines.

  As described with reference to FIGS. 5A and 5B, in the configuration in which each exposure line is sequentially exposed so that the exposure times of adjacent exposure lines partially overlap in time, the exposure time is set to be longer than that in the normal shooting mode. By using the bright line pattern generated by setting it short for signal transmission, the communication speed can be dramatically improved. Here, it is possible to generate an appropriate bright line pattern by setting the exposure time during visible light communication to 1/480 seconds or less. Here, when the frame frequency = f, the exposure time needs to be set as exposure time <1/8 × f. Blanking that occurs during shooting is at most half the size of one frame. That is, since the blanking time is less than half of the shooting time, the actual shooting time is 1 / 2f at the shortest time. Furthermore, since it is necessary to receive quaternary information within a time of 1 / 2f, at least the exposure time needs to be shorter than 1 / (2f × 4). Since the normal frame rate is 60 frames / second or less, it is possible to generate an appropriate bright line pattern in the image data and perform high-speed signal transmission by setting the exposure time to 1/480 seconds or less. Become.

FIG. 5C shows an advantage when the exposure times are short when the exposure times of the exposure lines do not overlap. When the exposure time is long, even if the light source has a binary luminance change as in 7502a, the captured image has an intermediate color portion as in 7502e, and it becomes difficult to recognize the luminance change of the light source. There is. However, as the 7502D, after completion exposure of one exposure line, by a configuration in which the free time (predetermined waiting time) t _D2 not predetermined exposure start exposure of the next exposure line, the luminance variation of the light source Can be easily recognized. That is, a more appropriate bright line pattern such as 7502f can be detected. As in 7502D, be provided with a free time without predetermined exposure becomes an exposure time t _E can be realized to be smaller than the time difference t _D of the exposure start time of each exposure line. When the normal shooting mode has a configuration in which the exposure times of adjacent exposure lines partially overlap in time, the exposure time is set shorter than the normal shooting mode until a predetermined idle time occurs. This can be realized. Further, even when the normal photographing mode is the case where the exposure end time of the previous exposure line and the exposure start time of the next exposure line are equal, by setting the exposure time short until a predetermined non-exposure time occurs, Can be realized. Further, as 7502G, also by increasing the distance t _D of the exposure start time of each exposure line, after the exposure of one exposure line, following exposure line exposure start until a predetermined exposure was not free time (predetermined Waiting time) t _D2 can be provided. In this configuration, since the exposure time can be extended, a bright image can be taken, and noise is reduced, so that error tolerance is high. On the other hand, in this configuration, since the number of exposure lines that can be exposed within a certain time is reduced, there is a disadvantage that the number of samples is reduced as in 7502h. For example, when the imaging target is bright, the former configuration is used, and when the imaging target is dark, the latter configuration can be used to reduce the estimation error of the light source luminance change.

  In all exposure lines, it is not necessary that the exposure time of adjacent exposure lines has a partial overlap in time, and a configuration in which some exposure lines have no partial overlap. It is also possible. Further, in all exposure lines, it is not necessary to provide a configuration in which an idle time (predetermined waiting time) in which a predetermined exposure is not performed is provided after the exposure of one exposure line is completed until the exposure of the next exposure line is started. It is also possible to have a configuration in which the lines partially overlap in time. With such a configuration, it is possible to take advantage of the advantages of each configuration. Further, the same readout method is used in the normal shooting mode in which shooting is performed at a normal frame rate (30 fps, 60 fps) and in the visible light communication mode in which shooting is performed with an exposure time of 1/480 second or less in which visible light communication is performed. Alternatively, a signal may be read by a circuit. By reading out signals with the same reading method or circuit, it is not necessary to use different circuits for the normal imaging mode and the visible light communication mode, and the circuit scale can be reduced.

FIG. 5D shows the relationship between the minimum change time t _S of the light source luminance, the exposure time t _E , the time difference t _D of the exposure start time of each exposure line, and the captured image. When t _E + t _D <t _S , since one or more exposure lines are always imaged in a state where the light source does not change from the start to the end of exposure, an image with clear brightness as in 7503d is obtained. It is easy to recognize the luminance change of the light source. When 2t _E > t _S , a bright line having a pattern different from the luminance change of the light source may be obtained, and it becomes difficult to recognize the luminance change of the light source from the captured image.

Figure 5E, the transition and time t _T of the light source luminance, which shows the relationship between the time difference t _D of the exposure start time of each exposure line. as t _D is larger than the t _T, exposure lines to be neutral is reduced, it is easy to estimate the light source luminance. When t _D > t _T , the exposure line of the intermediate color is continuously 2 or less, which is desirable. t _T, the light source is less than 1 microsecond in the case of LED, light source for an approximately 5 microseconds in the case of organic EL, a t _D by 5 or more microseconds, to facilitate estimation of the light source luminance be able to.

Figure 5F shows a high frequency noise _{t HT} of light source luminance, the relationship between the exposure time _{t E.} As t _E is larger than t _HT , the captured image is less affected by high frequency noise, and light source luminance is easily estimated. When t _E is an integral multiple of t _HT , the influence of high frequency noise is eliminated, and the light source luminance is most easily estimated. For estimation of the light source luminance, it is desirable that t _E > t _HT . The main cause of high frequency noise derived from the switching power supply circuit, since many of the t _HT in the switching power supply for the lamp is less than 20 microseconds, by the t _E and 20 micro-seconds or more, the estimation of the light source luminance It can be done easily.

Figure 5G is the case t _HT is 20 microseconds, which is a graph showing the relationship between the size of the exposure time t _E and the high frequency noise. When t _HT is considered that there is variation by the light source, from the graph, t _E is the value becomes equal to the value when the amount of noise takes a maximum, 15 microseconds or more, or, 35 microseconds or more, or, It can be confirmed that the efficiency is good when it is set to 54 microseconds or more, or 74 microseconds or more. From the viewpoint of reducing high-frequency noise, it is desirable that t _E be large. However, as described above, there is a property that light source luminance can be easily estimated in that the smaller the t _E , the more difficult the intermediate color portion is generated. Therefore, the period of variation of the light source luminance is 15 to 35 _{t E} when microseconds 15 microseconds or more, the period of variation of the light source luminance thirty-five to fifty-four _{t E} when microseconds 35 microseconds or more, the light source luminance t _E is 54 microseconds or more when the period of the change is 54 to 74 microseconds, t _E when the period of the change in light source luminance is 74 microseconds or more may be set as 74 microseconds or more.

Figure 5H shows the relationship between the exposure time t _E and the recognition success rate. Since the exposure time t _E has a relative meaning with respect to the time when the luminance of the light source is constant, the value (relative exposure time) obtained by dividing the period t _S where the luminance of the light source changes by the exposure time t _E is taken as the horizontal axis. Yes. From the graph, it can be seen that if the recognition success rate is desired to be almost 100%, the relative exposure time should be 1.2 or less. For example, when the transmission signal is 1 kHz, the exposure time may be about 0.83 milliseconds or less. Similarly, when it is desired to set the recognition success rate to 95% or more, the relative exposure time may be set to 1.25 or less, and when the recognition success rate is set to 80% or more, the relative exposure time may be set to 1.4 or less. Recognize. Also, the recognition success rate drops sharply when the relative exposure time is around 1.5, and becomes almost 0% at 1.6, so it can be seen that the relative exposure time should not be set to exceed 1.5. . It can also be seen that after the recognition rate becomes 0 at 7507c, it rises again at 7507d, 7507e, and 7507f. Therefore, when it is desired to take a bright image by extending the exposure time, use an exposure time in which the relative exposure time is 1.9 to 2.2, 2.4 to 2.6, and 2.8 to 3.0. Just do it. For example, these exposure times may be used as the intermediate mode in FIG.

  FIG. 6A is a flowchart of the information communication method in the present embodiment.

  The information communication method in the present embodiment is an information communication method for acquiring information from a subject, and includes steps SK91 to SK93.

  That is, in this information communication method, a plurality of bright lines corresponding to a plurality of exposure lines included in the image sensor are generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. A first exposure time setting step SK91 for setting a first exposure time of the image sensor; and the image sensor shoots the subject whose luminance changes with the set first exposure time, First image acquisition step SK92 for acquiring a bright line image including a plurality of bright lines, and information acquisition for acquiring information by demodulating data specified by the patterns of the plurality of bright lines included in the acquired bright line image In the first image acquisition step SK92, each of the plurality of exposure lines is included. Starts exposure at successively different times, and the exposure of the adjacent exposure line after a predetermined idle time from the end, the exposure is started adjacent to the exposure line.

  FIG. 6B is a block diagram of the information communication apparatus in the present embodiment.

  The information communication device K90 in the present embodiment is an information communication device that acquires information from a subject, and includes constituent elements K91 to K93.

  That is, the information communication apparatus K90 causes a plurality of bright lines corresponding to a plurality of exposure lines included in the image sensor to be generated in response to a change in luminance of the subject in an image obtained by photographing the subject by an image sensor. And an exposure time setting unit K91 for setting an exposure time of the image sensor, and the image sensor for acquiring a bright line image including the plurality of bright lines by photographing the subject whose luminance changes with the set exposure time. And an information acquisition unit K93 that acquires information by demodulating data specified by the patterns of the plurality of bright lines included in the acquired bright line image, and the plurality of exposure lines. Each of the exposure starts sequentially at different times and is adjacent to the exposure line. Exposure after a predetermined idle time from the end, to initiate the exposure of.

  In the information communication method and the information communication apparatus K90 shown in FIGS. 6A and 6B as described above, for example, as shown in FIG. 5C, each of the plurality of exposure lines is exposed to the adjacent exposure line adjacent to the exposure line. Since exposure is started after a lapse of a predetermined idle time after the end, it is possible to easily recognize a change in luminance of the subject. As a result, information can be appropriately acquired from the subject.

  In the above embodiment, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. For example, the program causes the computer to execute the information communication method shown by the flowchart of FIG. 6A.

(Embodiment 2)
In the present embodiment, each application example using a receiver such as a smartphone that is the information communication device K90 in the first embodiment and a transmitter that transmits information as a blinking pattern of a light source such as an LED or an organic EL. explain.

  FIG. 7 is a diagram illustrating an example of each mode of the receiver in this embodiment.

  In the normal shooting mode, the receiver 8000 acquires a normal shot image by shooting at a shutter speed of 1/100 seconds, for example, and displays the normal shot image on the display. In this case, the normal photographed image clearly shows a subject such as a streetlight or signage configured as a store signboard and its surroundings.

  Further, in the visible light communication mode, the receiver 8000 acquires a visible light communication image by shooting at a shutter speed of 1/10000 seconds, for example. For example, in the case where the streetlight or signage described above transmits a signal by luminance change as the light source shown in the first embodiment, that is, a transmitter, a location where the signal is transmitted in this visible light communication image One or a plurality of bright lines (hereinafter referred to as a bright line pattern) is projected, and nothing is projected except for the portions. That is, in this visible light communication image, only the bright line pattern is displayed, and the portion where the luminance of the subject is not changed and the periphery of the subject are not projected.

  Further, in the intermediate mode, the receiver 8000 acquires an intermediate image by shooting at a shutter speed of 1/3000 seconds, for example. In this intermediate image, a bright line pattern is displayed, and the above-described portion of the subject where the luminance is not changed and the periphery of the subject are also shown. Therefore, the receiver 8000 displays the intermediate image on the display, so that the user can know where or from where the signal is transmitted. Note that the bright line pattern, the subject, and the surrounding area displayed by the intermediate image are not clearer than the bright line pattern of the visible light communication image and the subject and the surrounding area of the normal captured image, respectively, but have a sharpness recognized by the user. .

  In the following description, shooting in the normal shooting mode or the normal shooting mode is referred to as normal shooting, and shooting in the visible light communication mode or the visible light communication mode is referred to as visible light shooting (visible light communication). Further, instead of normal shooting and visible light shooting, shooting in an intermediate mode may be used, and an intermediate image may be used instead of a composite image described later.

  FIG. 8 is a diagram illustrating an example of a photographing operation of the receiver in this embodiment.

  The receiver 8000 switches the shooting mode to normal shooting, visible light communication, normal shooting, and so on. Then, the receiver 8000 generates a composite image in which the bright line pattern, the subject, and the surrounding area are clearly displayed by combining the normal captured image and the visible light communication image, and displays the composite image on the display. . This composite image is an image generated by superimposing the bright line pattern of the visible light communication image on the portion where the signal in the normal captured image is transmitted. Further, the bright line pattern, the subject, and the surroundings displayed by the composite image are clear and have a sharpness sufficiently recognized by the user. By displaying such a composite image, the user can more clearly know from where or from where the signal is transmitted.

  FIG. 9 is a diagram illustrating another example of the photographing operation of the receiver in this embodiment.

  The receiver 8000 includes a camera Ca1 and a camera Ca2. In such a receiver 8000, the camera Ca1 performs normal photographing, and the camera Ca2 performs visible light photographing. Thereby, the camera Ca1 acquires the normal captured image as described above, and the camera Ca2 acquires the visible light communication image as described above. Then, the receiver 8000 generates the above-described combined image by combining the normal captured image and the visible light communication image, and displays the combined image on the display.

  FIG. 10A is a diagram illustrating another example of imaging operation of a receiver in this embodiment.

  In the receiver 8000 having two cameras, the camera Ca1 switches the shooting mode to normal shooting, visible light communication, normal shooting, and so on. On the other hand, the camera Ca2 continuously performs normal shooting. When the normal shooting is performed simultaneously with the cameras Ca1 and Ca2, the receiver 8000 receives from the normal shooting images acquired by these cameras using stereo vision (the principle of triangulation). The distance from the machine 8000 to the subject (hereinafter referred to as subject distance) is estimated. By using the subject distance estimated in this way, the receiver 8000 can superimpose the bright line pattern of the visible light communication image on an appropriate position of the normal captured image. That is, an appropriate composite image can be generated.

  FIG. 10B is a diagram illustrating another example of imaging operation of a receiver in this embodiment.

  The receiver 8000 includes, for example, three cameras (camera Ca1, camera Ca2, and camera Ca3). In such a receiver 8000, two cameras (camera Ca2 and camera Ca3) continue to perform normal imaging, and the remaining one camera (camera Ca1) continues to perform visible light communication. Accordingly, the subject distance can be estimated based on the normal captured images obtained by the two cameras that are normally capturing at any timing.

  FIG. 10C is a diagram illustrating another example of imaging operation of a receiver in this embodiment.

  The receiver 8000 includes, for example, three cameras (camera Ca1, camera Ca2, and camera Ca3). In such a receiver 8000, each camera switches the shooting mode to normal shooting, visible light communication, normal shooting, and so on. Here, in one period, any two of these cameras perform normal shooting, and the remaining one camera performs visible light communication so that the shooting mode of each camera is switched for each period. It is done. In other words, the combination of cameras that are normally shooting changes periodically. Thereby, in any period, the subject distance can be estimated based on the normal captured images obtained by the two cameras that are normally capturing.

  FIG. 11A is a diagram illustrating an example of a camera arrangement of a receiver in this embodiment.

  When the receiver 8000 includes two cameras Ca1 and Ca2, as shown in FIG. 11A, the two cameras Ca1 and Ca2 are arranged at positions separated from each other. As a result, the subject distance can be estimated with high accuracy. That is, the longer the distance between the two cameras, the more accurately the subject distance can be estimated.

  FIG. 11B is a diagram illustrating another example of the camera arrangement of the receiver in this embodiment.

  When receiver 8000 includes three cameras Ca1, camera Ca2, and camera Ca3, as shown in FIG. 11B, two cameras Ca1 and Ca2 for normal photographing are arranged at positions separated from each other. Moreover, the camera Ca3 for visible light communication is arrange | positioned, for example between camera Ca1 and camera Ca2. As a result, the subject distance can be estimated with high accuracy. That is, the subject distance can be accurately estimated by using the two most distant cameras for normal shooting.

  FIG. 12 is a diagram illustrating an example of display operation of the receiver in this embodiment.

As described above, the receiver 8000 switches the photographing mode to visible light communication, normal photographing, visible light communication, and so on. Here, the receiver 8000 activates an application program when performing visible light communication for the first time. Then, the receiver 8000 estimates its own position based on the signal received by visible light communication. Next, when performing normal shooting, the receiver 8000 displays AR (Augmented Reality) information on the normal shot image acquired by the normal shooting. This AR information is acquired based on the position estimated as described above. Further, the receiver 8000 estimates the movement and direction change of the receiver 8000 based on the detection result by the 9-axis sensor, the motion detection of the normal captured image, and the like.
The display position of the AR information is moved in accordance with the estimated movement and change in direction. Thereby, the AR information can be made to follow the subject image of the normal captured image.

  In addition, when the imaging mode is switched from normal imaging to visible light communication, the receiver 8000 superimposes AR information on the latest normal captured image acquired at the time of the normal imaging immediately before the visible light communication. The receiver 8000 displays a normal captured image on which the AR information is superimposed. Similarly to the normal shooting, the receiver 8000 estimates the movement and direction change of the receiver 8000 on the basis of the detection result by the 9-axis sensor, and AR in accordance with the estimated movement and direction change. Move information and normal captured images. Thereby, AR information can be made to follow the subject image of the normal captured image in accordance with the movement of the receiver 8000 or the like in the case of visible light communication as in the case of normal imaging. Further, the normal image can be enlarged and reduced in accordance with the movement of the receiver 8000 or the like.

  FIG. 13 is a diagram illustrating an example of display operation of the receiver in this embodiment.

  For example, the receiver 8000 may display the composite image on which the bright line pattern is projected, as shown in FIG. Further, as shown in FIG. 13B, the receiver 8000 normally captures a signal explicit object that is an image having a predetermined color for notifying that a signal is transmitted instead of the bright line pattern. A composite image may be generated by superimposing on the image, and the composite image may be displayed.

  In addition, as shown in FIG. 13C, the receiver 8000 normally has a location where a signal is transmitted indicated by a dotted frame and an identifier (for example, ID: 101, ID: 102, etc.). The captured image may be displayed as a composite image. Further, as shown in FIG. 13D, the receiver 8000 recognizes a signal that is an image having a predetermined color for notifying that a specific type of signal is transmitted instead of the bright line pattern. A composite image may be generated by superimposing an object on a normal captured image, and the composite image may be displayed. In this case, the color of the signal identification object differs depending on the type of signal output from the transmitter. For example, when the signal output from the transmitter is position information, a red signal identification object is superimposed, and when the signal output from the transmitter is a coupon, the green signal identification object is Superimposed.

  FIG. 14 is a diagram illustrating an example of operation of a receiver in this embodiment.

  For example, when receiving a signal through visible light communication, the receiver 8000 may display a normal captured image and output a sound for notifying the user that the transmitter has been found. In this case, the receiver 8000 varies the type of output sound, the number of outputs, or the output time depending on the number of transmitters found, the type of received signal, or the type of information specified by the signal. It may be allowed.

  FIG. 15 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, when the user touches the bright line pattern displayed in the composite image, the receiver 8000 generates an information notification image based on a signal transmitted from the subject corresponding to the touched bright line pattern, and the information notification Display an image. This information notification image indicates, for example, a store coupon or a place. The bright line pattern may be a signal explicit object, a signal identification object, a dotted line frame, or the like shown in FIG. The same applies to the bright line patterns described below.

  FIG. 16 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, when the user touches the bright line pattern displayed in the composite image, the receiver 8000 generates an information notification image based on a signal transmitted from the subject corresponding to the touched bright line pattern, and the information notification Display an image. For example, the information notification image indicates the current location of the receiver 8000 by a map or the like.

  FIG. 17 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, the receiver 8000 receives signals from two street lamps that are subjects configured as transmitters. Then, similarly to the above, the receiver 8000 estimates its current location based on these signals. The receiver 8000 displays a normal captured image and displays an information notification image (an image indicating latitude and longitude) indicating the estimation result superimposed on the normal captured image. Note that the receiver 8000 may display the auxiliary information notification image superimposed on the normal captured image. This auxiliary information notification image recommends to the user an operation for calibrating, for example, a 9-axis sensor (in particular, a geomagnetic sensor), that is, an operation for canceling drift. By performing such an operation, the current location is estimated with high accuracy.

  Further, when the displayed information notification image is touched by the user, the receiver 8000 may display a map indicating the estimated position instead of the normal captured image.

  FIG. 18 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, when the user swipes the receiver 8000 displaying the composite image, the receiver 8000 performs normal shooting having a dotted frame and an identifier similar to the normal shot image illustrated in FIG. An image is displayed and a list of information is displayed so as to follow the swipe operation. In this list, information specified by a signal transmitted from a location (transmitter) indicated by each identifier is shown. The swipe may be, for example, an operation of moving a finger from outside the right side of the display in the receiver 8000. The swipe may be an operation of moving a finger from the upper side, the lower side, or the left side of the display.

  Further, when information included in the list is tapped by the user, the receiver 8000 may display an information notification image (for example, an image showing a coupon) that shows the information in more detail.

  FIG. 19 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, when the user performs a swipe on the receiver 8000 on which the composite image is displayed, the receiver 8000 displays an information notification image superimposed on the composite image so as to follow the swipe operation. This information notification image shows the subject distance with an arrow in an easy-to-understand manner for the user. The swipe may be, for example, an operation of moving a finger from outside the lower side of the display in the receiver 8000. The swipe may be an operation of moving a finger from the left side of the display, from the upper side, or from the right side.

  FIG. 20 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, the receiver 8000 images a transmitter, which is a signage indicating a plurality of stores, as a subject, and displays a normal captured image acquired by the imaging. Here, when the user taps the signage image of one store included in the subject displayed in the normal captured image, the receiver 8000 generates an information notification image based on a signal transmitted from the signage of the store Then, the information notification image 8001 is displayed. This information notification image 8001 is an image showing, for example, a vacant seat situation in a store.

  FIG. 21 is a diagram illustrating an example of operations of the receiver, the transmitter, and the server in this embodiment.

  First, a transmitter 8012 configured as a television transmits a signal to a receiver 8011 by a luminance change. This signal includes, for example, information for prompting the user to purchase content related to the program being viewed. When the receiver 8011 receives the signal through visible light communication, the receiver 8011 displays an information notification image that prompts the user to purchase content based on the signal. When the user performs an operation for purchasing the content, the receiver 8011 receives information included in a SIM (Subscriber Identity Module) card inserted into the receiver 8011, user ID, terminal ID, credit card information, billing At least one of the information, the password, and the transmitter ID is transmitted to the server 8013. The server 8013 manages a user ID and payment information in association with each user. Then, the server 8013 identifies the user ID based on the information transmitted from the receiver 8011, and confirms the payment information associated with the user ID. By this confirmation, the server 8013 determines whether or not to allow the user to purchase content. If the server 8013 determines to permit, the server 8013 transmits permission information to the receiver 8011. When receiving the permission information, the receiver 8011 transmits the permission information to the transmitter 8012. The transmitter 8012 that has received the permission information acquires and reproduces the content via a network, for example.

  Further, the transmitter 8012 may transmit information including the ID of the transmitter 8012 to the receiver 8011 by changing the luminance. In this case, the receiver 8011 transmits the information to the server 8013. When the server 8013 obtains the information, the server 8013 can determine that, for example, a television program is being viewed by the transmitter 8012, and can perform a viewing rate survey of the television program.

  Further, the receiver 8011 includes the content operated by the user (voting or the like) in the above information and transmits the content to the server 8013, so that the server 8013 can reflect the content in the television program. That is, a viewer participation type program can be realized. Further, when the receiver 8011 accepts writing by the user, the contents of the writing are included in the above-described information and transmitted to the server 8013 so that the server 8013 can write the writing to the TV program or a bulletin board on the network. Etc. can be reflected.

  Further, when the transmitter 8012 transmits the above-described information, the server 8013 can charge for viewing a television program by pay broadcasting or an on-demand program. Further, the server 8013 displays an advertisement on the receiver 8011, displays detailed information of a TV program displayed on the transmitter 8012, and displays a URL of a site indicating the detailed information. Can do. Further, the server 8013 obtains the number of times the advertisement is displayed by the receiver 8011 or the amount of the product purchased by the advertisement, and thereby charges the advertiser according to the number or amount. be able to. Such billing can be made even if the user who saw the advertisement does not purchase the product immediately. Further, when the server 8013 acquires information indicating the manufacturer of the transmitter 8012 from the transmitter 8012 via the receiver 8011, the server 8013 provides a service (for example, a reward for sales of the above-described product to the manufacturer indicated by the information). Payment).

  FIG. 22 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, the user points the camera of the receiver 8021 toward a plurality of transmitters 8020a-8020d configured as illumination. At this time, the receiver 8021 is moved so that each of the transmitters 8020a to 8020d is sequentially photographed as a subject. The receiver 8021 receives signals from the transmitters 8020a to 8020d by performing visible light communication while being moved. These signals include information indicating the position of the transmitter. The receiver 8021 is based on the position indicated by the received signals from the transmitters 8020a to 8020d, the detection result of the 9-axis sensor provided in the receiver 8021, and the movement of the image obtained by photographing. The position of the receiver 8021 is estimated using the principle of triangulation. In this case, since the receiver 8021 is moved, the drift of the 9-axis sensor (particularly, the geomagnetic sensor) is eliminated, so that a more accurate position can be estimated.

  FIG. 23 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 is configured as, for example, a head mounted display provided with a camera. The receiver 8030 starts photographing in the visible light communication mode, that is, visible light communication when the start button is pressed. When a signal is received through visible light communication, the receiver 8030 notifies the user of information corresponding to the received signal. This notification is performed, for example, by outputting sound from a speaker provided in the receiver 8030 or by displaying an image. In addition, when the start button is pressed, the visible light communication is received by the receiver 8030 when an input of a voice instructing the start is received by the receiver 8030 or a signal instructing the start is received by wireless communication. It may be started when it is done. In addition, visible light communication is started when the change width of the value obtained by the 9-axis sensor provided in the receiver 8030 exceeds a predetermined range or when a bright line pattern appears even in a normal photographed image. May be.

  FIG. 24 is a diagram illustrating an example of initial setting of the receiver in this embodiment.

  The receiver 8030 displays an alignment image 8031 at the time of initial setting. This alignment image 8031 is for performing alignment between the position indicated by the user in the image obtained by photographing with the camera of the receiver 8030 and the image displayed by the receiver 8030. When the user aligns the fingertip with the position of the circle indicated by the alignment image 8031, the receiver 8030 performs alignment by associating the position of the fingertip with the position of the circle. That is, the position indicated by the user is calibrated.

  FIG. 25 is a diagram illustrating another example of operation of a receiver in this embodiment.

  When the receiver 8030 identifies a location where a signal is transmitted by visible light communication, the receiver 8030 displays a composite image 8034 on which a bright line pattern is projected. The user performs an operation such as a tap or a double tap on the bright line pattern. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 26 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of moving the fingertip so as to surround the bright line pattern in the composite image 8034. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 27 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of placing the fingertip on the bright line pattern in the composite image 8034 for a predetermined time or more. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 28 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of moving the fingertip toward the bright line pattern of the composite image 8034 by swiping. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 29 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation to keep the line of sight toward the bright line pattern of the composite image 8034 for a predetermined time or more. Alternatively, the user performs an operation of blinking a predetermined number of times with the line of sight directed toward the bright line pattern. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 30 is a diagram illustrating another example of operation of a receiver in this embodiment.

  Similarly to the above, the receiver 8030 displays the composite image 8034 and displays an arrow associated with each bright line pattern in the composite image 8034. These arrows point in different directions for each bright line pattern. Here, the user performs an operation of moving the head along any of the arrows. When receiving this operation based on the detection result by the 9-axis sensor, the receiver 8030 specifies the bright line pattern associated with the arrow corresponding to the operation, that is, the arrow pointing in the direction in which the head is moved. The receiver 8030 displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 31A is a diagram illustrating a pen used for operation of a receiver in this embodiment.

  The pen 8033 includes a transmitter 8033a that transmits a signal according to a luminance change, a button 8033b, and a button 8033c. When the button 8033b is pressed, the transmitter 8033a transmits a predetermined first signal, and when the button 8033c is pressed, the transmitter 8033a has a predetermined first signal different from the first signal. 2 signal is transmitted.

  FIG. 31B is a diagram illustrating operation of a receiver using a pen in this embodiment.

  The pen 8033 is used in place of the above-described user's fingertip, and is used like a stylus pen. Further, by properly using the button 8033b and the button 8033c, the pen 8033 can be used as a normal pen or used like an eraser.

  FIG. 32 is a diagram illustrating an example of appearance of a receiver in this embodiment.

  The receiver 8030 includes a first touch sensor 8030a and a second touch sensor 8030b. These touch sensors are attached to the frame of the receiver 8030. For example, when the user moves the fingertip on the first touch sensor 8030a, the receiver 8030 moves the pointer in accordance with the movement of the fingertip on the image displayed to the user. Further, when the user touches the second touch sensor 8030b, the receiver 8030 performs a process of selecting an object to which the pointer is placed on an image displayed to the user.

  FIG. 33 is a diagram illustrating another example of the appearance of the receiver in this embodiment.

  The receiver 8030 includes a touch sensor 8030c. The touch sensor 8030c is attached to the frame of the receiver 8030. For example, when the user touches and moves the touch sensor 8030c with the fingertip, the receiver 8030 moves the pointer in accordance with the movement of the fingertip on the image displayed to the user. When the user presses the touch sensor 8030c, the receiver 8030 performs a process of selecting an object to which the pointer is placed on an image displayed to the user. That is, the touch sensor 8030c is configured as a so-called clickable touch sensor.

  FIG. 34 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 in the same manner as described above, and displays the pointer 8035 in the composite image 8034. When the receiver 8030 includes the first touch sensor 8030a and the second touch sensor 8030b, the user moves the pointer by moving the first touch sensor 8030a while placing the fingertip on the first touch sensor 8030a. Point the object at. Then, the user touches the second touch sensor 8030b to cause the receiver 8030 to select the bright line pattern. When the receiver 8030 selects a bright line pattern, the receiver 8030 displays an information notification image 8032 based on a signal transmitted from the portion of the bright line pattern.

  In the case where the receiver 8030 includes the touch sensor 8030c, the user moves the pointer by placing the fingertip on the touch sensor 8030c and moves the pointer to the object having the bright line pattern. Then, the user causes the receiver 8030 to select the bright line pattern by pressing the touch sensor 8030c. When the receiver 8030 selects a bright line pattern, the receiver 8030 displays an information notification image 8032 based on a signal transmitted from the portion of the bright line pattern.

  FIG. 35A is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays a gesture confirmation image 8036 based on a signal obtained by performing visible light communication. This gesture confirmation image 8036 prompts the user for a predetermined gesture in order to provide a service to the user, for example.

  FIG. 35B is a diagram illustrating an application example using the receiver in this embodiment.

  A user 8038 wears the receiver 8030 and is at a store, for example. Here, the receiver 8030 displays the above-described gesture confirmation image 8036 on the user 8038. The user 8038 performs a predetermined gesture according to the gesture confirmation image 8036. Here, a clerk 8039 in the store is wearing a receiver 8037. The receiver 8037 is configured as a head mounted display provided with a camera, and may have the same configuration as the receiver 8030. The receiver 8037 also displays a gesture confirmation image 8036 based on a signal obtained by performing visible light communication. The salesclerk 8039 determines whether or not the predetermined gesture indicated by the displayed gesture confirmation image 8036 matches the gesture performed by the user 8038. When the clerk 8039 determines that they match, the clerk 8039 provides the user 8038 with a service associated with the gesture confirmation image 8036.

  FIG. 36A is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays a gesture confirmation image 8040 based on a signal obtained by performing visible light communication. This gesture confirmation image 8040 prompts the user for a predetermined gesture in order to permit wireless communication, for example.

  FIG. 36B is a diagram illustrating an application example using the receiver in this embodiment.

  A user 8038 is wearing a receiver 8030. Here, the receiver 8030 displays the above-described gesture confirmation image 8040 on the user 8038. The user 8038 performs a predetermined gesture according to the gesture confirmation image 8040. Here, a person 8041 around the user 8038 wears the receiver 8037. The receiver 8037 is configured as a head mounted display provided with a camera, and may have the same configuration as the receiver 8030. The receiver 8037 acquires authentication information such as a password included in the gesture by photographing a predetermined gesture performed by the user 8038. When the receiver 8037 determines that the authentication information matches the predetermined information, the receiver 8037 establishes a wireless connection with the receiver 8030. After the establishment, the receiver 8030 and the receiver 8037 can perform wireless communication with each other.

  FIG. 37A is a diagram illustrating an example of operation of a transmitter in this embodiment.

  The transmitter transmits the signal 1 and the signal 2 alternately at a predetermined cycle, for example. Transmission of the signal 1 and transmission of the signal 2 are performed by luminance changes such as blinking of visible light. Further, the luminance change pattern for transmitting the signal 1 and the luminance change pattern for transmitting the signal 2 are different from each other.

  FIG. 37B is a diagram illustrating another example of operation of a transmitter in this embodiment.

  The transmitter may transmit the signal 1 and the signal 2 intermittently with a buffer time without continuously transmitting the signal 1 and the signal 2 as described above. Here, the transmitter does not change the luminance during the buffer time. Alternatively, the transmitter may transmit a signal indicating that it is a buffer time by a luminance change or a luminance change different from the luminance change for transmitting each of the signal 1 and the signal 2 during the buffer time. . As a result, the receiver can appropriately receive the signal 1 and the signal 2 without interference.

  FIG. 38 is a diagram illustrating another example of operation of a transmitter in this embodiment.

  The transmitter repeatedly transmits a signal sequence of a unit composed of a preamble, block 1, block 2, block 3, and a check signal according to a luminance change. Here, the block 1 has a preamble, an address 1, data 1 and a check signal. Block 2 and block 3 are configured in the same manner as block 1. Further, specific information can be obtained by using data included in each of block 1, block 2 and block 3.

  That is, in the signal sequence as described above, one piece of data or information is stored in three blocks. Therefore, even if a receiver that requires a blanking period for imaging cannot receive all the data of block 1, block 2 and block 3 from one signal string, the remaining data from other signal strings Can be received. As a result, even a receiver that requires a blanking period can appropriately acquire specific information from at least one signal sequence.

  In the signal sequence as described above, a preamble and a check signal are arranged for a set of three blocks. Therefore, a receiver that can receive light without requiring a blanking period, for example, a receiver equipped with an illuminance sensor or the like, uses a preamble and a check signal that are arranged for the set, and thereby uses one signal sequence. It can be received at once and specific information can be acquired in a short time.

  FIG. 39 is a diagram illustrating another example of operation of a transmitter in this embodiment.

  As described above, when the transmitter repeatedly transmits the signal sequence of the structural unit including the block 1, the block 2, and the block 3, for each signal sequence, the transmitter changes the arrangement of the blocks included in the signal sequence. Also good. For example, each block is arranged in the order of block 1, block 2, and block 3 in the first signal sequence, and each block is arranged in the order of block 3, block 1, and block 2 in the next signal sequence. Thereby, it can be avoided that only the same block is acquired by a receiver that requires a periodic blanking period.

  FIG. 40 is a diagram illustrating an example of a communication form between a plurality of transmitters and a receiver in this embodiment.

  The receiver 8050 may receive a signal (visible light) transmitted from the transmitters 8051a and 8051b configured as illumination and reflected by the reflecting surface. Thereby, signals from many transmitters can be received together. In this case, the transmitter 8051a and the transmitter 8051b transmit signals of different frequencies or protocols. Thereby, the receiver 8050 can receive the signals from those transmitters without interference.

  FIG. 41 is a diagram illustrating an example of operation of a plurality of transmitters in this embodiment.

  One of the transmitter 8051a and the transmitter 8051b may monitor the transmission status of the signal from the other and transmit a signal so as to prevent interference with the other signal. For example, one transmitter receives a signal transmitted from the other transmitter and transmits a signal of a protocol different from that signal. Alternatively, one transmitter detects a period during which no signal is transmitted from the other transmitter, and transmits a signal during that period.

  FIG. 42 is a diagram illustrating another example of a communication form between a plurality of transmitters and a receiver in this embodiment.

  The transmitter 8051a and the transmitter 8051b may transmit signals having the same frequency or protocol. In this case, the receiver 8050 specifies the intensity of the signal transmitted from those transmitters, that is, the edge intensity of the bright line included in the image obtained by imaging. This intensity becomes weaker as the distance between the receiver 8050 and the transmitter is longer. When the distance between the receiver 8050 and each of the transmitter 8051a and the transmitter 8051b is different, such a difference in distance can be used. That is, the receiver 8050 can appropriately separate and receive the signals transmitted from the transmitter 8051a and the transmitter 8051b according to the specified strength.

  FIG. 43 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8050 receives a signal transmitted from the transmitter 8051a and reflected by the reflecting surface. At this time, the receiver 8050 may estimate the position of the transmitter 8051a based on the luminance intensity distribution (luminance difference at a plurality of positions) in an image obtained by photographing.

  FIG. 44 is a diagram illustrating an example of application of a receiver in this embodiment.

  For example, the receiver 7510a configured as a smartphone captures an image of the light source 7510b with a back camera (out camera) 7510c, receives a signal transmitted from the light source 7510b, and acquires the position and orientation of the light source 7510b from the received signal. The receiver 7510a estimates the position and orientation of the receiver 7510a itself from how the light source 7510b is captured in the captured image and the sensor values of the 9-axis sensor provided in the receiver 7510a. The receiver 7510a captures the user 7510e with a front camera (face camera, in-camera) 7510f, and estimates the position and orientation of the head of the 7510e and the line-of-sight direction (eyeball position and orientation) by image processing. . The receiver 7510a transmits the estimation result to the server. The receiver 7510a changes the behavior (display content and playback sound) according to the viewing direction of the user 7510e. The imaging by the back camera 7510c and the imaging by the front camera 7510f may be performed simultaneously or alternately.

  FIG. 45 is a diagram illustrating an example of application of a receiver in this embodiment.

  For example, similarly to the above, the receivers 7511d and 7511i configured as smartphones receive signals from the light sources 7511b and 7511g, estimate their positions and orientations, and estimate the gaze directions of the users 7511e and 7511j. In addition, the receivers 7511d and 7511i acquire information on the surrounding objects 7511a to 7511c and 7511f to 7511h from the server based on the received data. The receivers 7511d and 7511i change the display on the display as if the object on the other side is seen through the receivers 7511d and 7511i when viewed from the user. The receivers 7511d and 7511i display an AR (augmented reality) object such as 7511k in accordance with the content shown on the display. When the line of sight of the user 7511j exceeds the imaging range of the imaging of the camera, the receiver 7511i displays that it is out of the range, such as 7511l. Alternatively, the AR object and other information are displayed in an area outside the range. Alternatively, the images obtained when the out-of-range region is captured in the past are connected and displayed.

  FIG. 46 is a diagram illustrating an example of application of a receiver in this embodiment.

  For example, similarly to the above, the receiver 7512c configured as a smartphone receives a signal from the light source 7512a, estimates its own position and orientation, and estimates the visual line direction of the user 7512d. The receiver 7512c performs processing related to the object 7512b that is in the line-of-sight direction of the user 7512d. For example, information related to the object 7512b is displayed on the screen. When the line-of-sight direction of the user 7512h moves from the object 7512f to the receiver 7512g, the receiver 7512g determines that the user 7512h is interested in the object 7512f, and continues processing related to the object 7512f. For example, information on the object 7512f is kept displayed on the screen.

  FIG. 47 is a diagram illustrating an example of application of a transmitter in this embodiment.

  For example, the transmitter 7513a configured as an illumination has a high luminance, and when the luminance is high (high) or low (low) as a transmission signal, it exceeds the upper limit brightness when imaged by the receiver. , 7513b, no bright line appears. Therefore, as shown in 7513c, by providing a portion 7513d that diffuses or weakens light such as a diffuser plate or a prism to reduce the luminance, the receiver can image bright lines like 7513e. .

  FIG. 48 is a diagram illustrating an example of application of a transmitter in this embodiment.

  For example, in the transmitter 7514a configured as illumination, since the light source is not uniform, the picked-up image causes unevenness in brightness as in the case of 7514b and induces a reception error. Therefore, as shown by 7514c, a reception error can be suppressed by providing a portion 7514d for diffusing light such as a diffuser plate or a prism so that the luminance is uniform.

  FIG. 49 is a diagram illustrating an application example of the reception method in this embodiment.

  In the transmitters 7515a and 7515b, the luminance of the central portion is high, and bright lines do not appear in the captured image of the receiver, but bright lines appear in the peripheral portions. The receiver cannot receive a signal from the portion 7515d because the bright line is interrupted, but can receive a signal from the portion 7515c. The receiver can receive signals from more bright lines than the part 7515c by reading the bright lines in the path 7515e.

  FIG. 50 is a diagram illustrating an example of application of a transmitter in this embodiment.

  For example, transmitters 7516a, 7516b, 7516c, and 7516d configured as illumination have high luminance like 7513a, and bright lines are less likely to occur when imaged by a receiver. Therefore, the diffuser plate / prism 7516e, the reflector plate 7516f, the reflector plate / half mirror 7516g, the reflector plate 7516h, and the diffuser plate / prism 7516j are provided to diffuse the light and widen the portion where the bright line is generated. Can do. With these transmitters, the picked-up image is picked up in the form of bright lines around 7515a. Since the receiver estimates the distance between the receiver and the transmitter using the size of the transmitter on the captured image, the part where the light is diffused is set as the size of the light source, and the server is associated with the transmission ID. By storing in the receiver, the receiver can accurately estimate the distance to the transmitter.

  FIG. 51 is a diagram illustrating an example of application of a transmitter in this embodiment.

  For example, the transmitter 7517a configured as illumination has high luminance like the 7513a, and a bright line is hardly generated when imaged by the receiver. Therefore, by providing the reflecting plate 7517b, it is possible to diffuse light and widen a portion where a bright line is generated.

  FIG. 52 is a diagram illustrating an example of application of a transmitter in this embodiment.

  The transmitter 7518a reflects light from the light source at 7518c, so that the receiver can capture bright lines in a wide range. The transmitter 7518d directs the light source toward the diffuser plate or the prism 7518e, so that the receiver can image the bright lines in a wide range.

  FIG. 53 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver displays the bright line pattern by the composite image or the intermediate image as described above. At this time, the receiver may not be able to receive a signal from the transmitter corresponding to the bright line pattern. Here, when the bright line pattern is selected by the user performing an operation (for example, tapping) on the bright line pattern, the receiver performs an optical zoom to enlarge the composite line image or the intermediate image in which the bright line pattern is enlarged. Display an image. By performing such optical zoom, the receiver can appropriately receive a signal from the transmitter corresponding to the bright line pattern. That is, even if an image obtained by imaging is too small to acquire a signal, the signal can be appropriately received by performing optical zoom. Even when an image having a size capable of acquiring a signal is displayed, fast reception can be performed by performing optical zoom.

(Summary of this embodiment)
The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, and an bright line corresponding to an exposure line included in the image sensor is included in an image obtained by photographing the subject with an image sensor. A first exposure time setting step for setting an exposure time of the image sensor so as to occur in accordance with a change in luminance of the subject; and the image sensor photographs the subject whose luminance changes with the set exposure time. The bright line image acquisition step of acquiring a bright line image that is an image including the bright line, and the spatial position of the portion where the bright line appears can be identified based on the bright line image, and the subject and the subject An image display step for displaying a display image in which the surroundings of the subject are projected, and the image included in the acquired bright line image Including an information acquisition step of acquiring transmission information by demodulating the data identified by the pattern of lines.

  For example, a composite image or an intermediate image as shown in FIGS. 7 to 9 and FIG. 13 is displayed as a display image. In the display image in which the subject and the surroundings of the subject are projected, the spatial position of the part where the bright line appears is identified by a bright line pattern, a signal explicit object, a signal identification object, a dotted line frame, or the like. Therefore, the user can easily find a subject that is transmitting a signal due to a change in luminance by viewing such a display image.

  The information communication method further includes a second exposure time setting step for setting an exposure time longer than the exposure time, and the image sensor photographs the subject and the surroundings of the subject with the long exposure time. A normal image acquisition step of acquiring a normal photographed image, a part where the bright line appears in the normal photographed image is identified based on the bright line image, and a signal object which is an image indicating the part is designated as the normal image A composite step of generating a composite image by superimposing the captured image, and the composite image may be displayed as the display image in the image display step.

  For example, the signal object is a bright line pattern, a signal explicit object, a signal identification object, a dotted line frame, or the like, and a composite image is displayed as a display image as shown in FIGS. Thus, the user can more easily find the subject that is transmitting the signal due to the luminance change.

  In the first exposure time setting step, an exposure time is set to 1/3000 sec. In the bright line image acquisition step, the bright line image in which the periphery of the subject is projected is acquired, and in the image display step. The bright line image may be displayed as the display image.

  For example, as shown in FIG. 7, the bright line image is acquired and displayed as an intermediate image. Therefore, it is not necessary to perform processing such as acquiring and synthesizing the normal captured image and the visible light communication image, and the processing can be simplified.

  The image sensor includes a first image sensor and a second image sensor. In the normal image acquisition step, the first image sensor captures the normal captured image, and the bright line is acquired. In the image acquisition step, the bright line image may be acquired by capturing the second image sensor simultaneously with the capturing of the first image sensor.

  For example, as shown in FIG. 9, a normal photographed image and a visible light communication image that is a bright line image are acquired by each camera. Therefore, compared with the case where a normal captured image and a visible light communication image are acquired with one camera, those images can be acquired earlier, and the processing can be speeded up.

  Further, in the information communication method, when the part where the bright line appears in the display image is designated by a user operation, the information communication method further includes the transmission information acquired from the pattern of the bright line of the designated part. An information presentation step of presenting presentation information based on the information may be included. For example, the operation by the user is shown in association with a tap, swipe, an operation in which a fingertip is continuously applied to the part for a predetermined time, an operation in which a line of sight is directed to the part for a predetermined time, or the like. An operation of moving a part of the user's body to an arrow, an operation of applying a pen tip that changes in luminance to the part, or a touch sensor is touched, and a pointer displayed on the display image is applied to the part It is an operation.

  For example, as shown in FIGS. 15 to 20 and FIGS. 25 to 34, the presentation information is displayed as an information notification image. Thereby, desired information can be presented to the user.

  Further, the image sensor may be provided in a head mounted display, and in the image display step, a projector mounted on the head mounted display may display the display image.

  Thereby, for example, as shown in FIGS. 23 to 30, information can be easily presented to the user.

  In addition, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. As described above, the first exposure time setting step of setting the exposure time of the image sensor, and the image including the bright line by the image sensor photographing the subject whose luminance changes with the set exposure time. A bright line image acquiring step for acquiring the bright line image, and an information acquiring step for acquiring information by demodulating data specified by the pattern of the bright line included in the acquired bright line image. In the obtaining step, a plurality of the subjects are photographed during the period in which the image sensor is moved. By acquiring the bright line image including a plurality of parts where the bright line appears, in the information acquisition step, for each part, by demodulating data specified by the pattern of the bright line of the part, The position of each of the subjects is acquired, and the information communication method further includes a position for estimating the position of the image sensor based on the acquired positions of the plurality of subjects and the movement state of the image sensor. An estimation step may be included.

  Thereby, for example, as shown in FIG. 22, the position of the receiver including the image sensor can be accurately estimated based on a luminance change caused by a subject such as a plurality of lights.

  In addition, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. As described above, the first exposure time setting step of setting the exposure time of the image sensor, and the image including the bright line by the image sensor photographing the subject whose luminance changes with the set exposure time. A bright line image acquisition step of acquiring a bright line image, an information acquisition step of acquiring information by demodulating data specified by a pattern of the bright line included in the acquired bright line image, and the acquired information An information presenting step for presenting the image sensor, wherein in the information presenting step, the user of the image sensor Against THE, it may present an image that prompts the gesture that has been predetermined as the information.

  As a result, for example, as shown in FIGS. 35A to 36B, depending on whether or not the user performs the gesture as prompted, the user can be authenticated and the convenience can be improved.

  In addition, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. As described above, the exposure time setting step for setting the exposure time of the image sensor, and the image sensor captures the bright line image including the bright line by photographing the subject whose luminance changes at the set exposure time. And an information acquisition step of acquiring information by demodulating data specified by the bright line pattern included in the acquired bright line image. In the image acquisition step, the image is reflected on the reflection surface. The bright line image is acquired by photographing a plurality of the subjects, and the information acquisition step is performed. Then, in accordance with the intensity of the bright line included in the bright line image, the bright line is separated into bright lines corresponding to each of the plurality of subjects, and each subject is specified by a bright line pattern corresponding to the subject. Information may be acquired by demodulating data.

  Thereby, for example, as shown in FIG. 42, even when the subject such as a plurality of illuminations changes in luminance, appropriate information can be acquired from each of the subjects.

  In addition, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. As described above, the exposure time setting step for setting the exposure time of the image sensor, and the image sensor captures the bright line image including the bright line by photographing the subject whose luminance changes at the set exposure time. And an information acquisition step of acquiring information by demodulating data specified by the bright line pattern included in the acquired bright line image. In the image acquisition step, the image is reflected on the reflection surface. The bright line image is acquired by photographing the subject, and the information communication method further includes: , Based on the luminance distribution of the bright line image may include position estimation step for estimating the position of the object.

  Thereby, for example, as shown in FIG. 43, an appropriate subject position can be estimated based on the luminance distribution.

  An information communication method for transmitting a signal according to a luminance change, wherein a first determination step of determining a first pattern of luminance change by modulating a first signal to be transmitted, A second determination step of determining a second pattern of luminance change by modulating the signal of 2; and a luminance change according to the first pattern determined by the light emitter; A transmission step of transmitting the first and second signals by alternately performing a luminance change according to the second pattern.

  Thereby, for example, as shown in FIG. 37A, the first signal and the second signal can be transmitted without delay.

  In the transmission step, when the luminance change is switched between the luminance change according to the first pattern and the luminance change according to the second pattern, the luminance change may be performed after a buffer time.

  Thereby, for example, as shown in FIG. 37B, interference between the first signal and the second signal can be suppressed.

  An information communication method for transmitting a signal according to a luminance change, wherein a determination step of determining a luminance change pattern by modulating a signal to be transmitted, and the light emitter changes in luminance according to the determined pattern And transmitting the signal to be transmitted, the signal comprising a plurality of large blocks, each of the plurality of large blocks including first data and a preamble for the first data. And a check signal for the first data, wherein the first data is composed of a plurality of small blocks, and the small blocks include second data, a preamble for the second data, and the first data. And a check signal for the second data may be included.

  Thus, for example, as shown in FIG. 38, data can be appropriately acquired by a receiver that requires a blanking period or a receiver that does not require a blanking period.

  An information communication method for transmitting a signal by luminance change, wherein a plurality of transmitters each modulate a signal to be transmitted to determine a luminance change pattern, and for each transmitter, And a transmitting step in which a light emitter provided in the transmitter changes the luminance according to the determined pattern and transmits the signal to be transmitted. In the transmitting step, signals having different frequencies or protocols are transmitted. May be.

  Thereby, for example, as shown in FIG. 40, interference of signals from a plurality of transmitters can be suppressed.

  An information communication method for transmitting a signal by luminance change, wherein a plurality of transmitters each modulate a signal to be transmitted to determine a luminance change pattern, and for each transmitter, A transmitter in which a light emitter provided in the transmitter transmits a signal to be transmitted by changing in luminance according to the determined pattern, wherein in the transmitting step, one of the plurality of transmitters One transmitter may receive a signal transmitted from the other transmitter and transmit another signal in a manner that does not interfere with the received signal.

  Thereby, for example, as shown in FIG. 41, interference of signals from a plurality of transmitters can be suppressed.

(Embodiment 3)
In this embodiment, each application example using a receiver such as a smartphone in Embodiment 1 or 2 and a transmitter that transmits information as a blinking pattern such as an LED or an organic EL will be described.

  FIG. 54 is a flowchart illustrating an example of operation of the receiver in Embodiment 3.

  First, the receiver receives a signal with an illuminance sensor (8101). Next, the receiver acquires information such as position information from the server based on the received signal (8102). Next, the receiver activates an image sensor that can capture the light receiving direction of the illuminance sensor (8103). Next, the receiver receives part or all of the signal by the image sensor, and checks whether part or all of the signal is the same as the signal received by the illuminance sensor (8104). Next, the receiver estimates the position of the receiver from the position of the transmitter in the captured image (captured image), the information from the 9-axis sensor provided in the receiver, and the position information of the transmitter. (8105). As described above, the receiver activates the illuminance sensor with low power consumption, and activates the image sensor when a signal is received by the illuminance sensor. Then, the receiver performs position estimation using imaging by the image sensor. As a result, the position of the receiver can be accurately estimated while suppressing power consumption.

  FIG. 55 is a flowchart illustrating another example of operation of a receiver in Embodiment 3.

  The receiver recognizes a periodic change in luminance from the sensor value of the illuminance sensor (8111). Next, the receiver activates an image sensor that can capture the light receiving direction of the illuminance sensor, and receives a signal (8112). That is, similarly to the above, the receiver activates an illuminance sensor with low power consumption, and activates the image sensor when a periodic change in luminance is received by the illuminance sensor. And a receiver receives an exact signal by imaging with the image sensor. Thereby, it is possible to receive an accurate signal while suppressing power consumption.

  FIG. 56A is a block diagram illustrating an example of a transmitter in Embodiment 3.

  The transmitter 8115 includes a power supply unit 8115a, a signal control unit 8115b, a light emitting unit 8115c, and a light emitting unit 8115d. The power supply unit 8115a supplies power to the signal control unit 8115b. The signal control unit 8115b distributes the power supplied from the power supply unit 8115a to the light emitting unit 8115c and the light emitting unit 8115d, and controls the luminance change of the light emitting unit 8115c and the light emitting unit 8115d.

  FIG. 56B is a block diagram illustrating another example of the transmitter in Embodiment 3.

  The transmitter 8116 includes a power supply unit 8116a, a signal control unit 8116b, a light emitting unit 8116c, and a light emitting unit 8116d. The power supply unit 8116a supplies power to the light emitting unit 8116c and the light emitting unit 8116d. Here, the signal control unit 8116b controls changes in luminance of the light emitting unit 8116c and the light emitting unit 8116d by controlling the power supplied from the power supply unit 8116a. In this manner, the power control unit 8116a that supplies power to each of the light emitting unit 8116c and the light emitting unit 8116d is controlled by the signal control unit 8116b, so that the power use efficiency can be increased.

  FIG. 57 is a diagram illustrating a configuration example of a system including a plurality of transmitters in the third embodiment.

  This system includes a central control unit 8118, a transmitter 8117, and a transmitter 8120. The central control unit 8118 controls transmission of signals due to luminance changes of the transmitter 8117 and the transmitter 8120, respectively. For example, the central control unit 8118 causes the transmitter 8117 and the transmitter 8120 to transmit the same signal at the same timing, or causes only one of the transmitters to transmit a unique signal to the transmitter.

  The transmitter 8120 includes two transmission units 8121 and 8122, a signal change unit 8123, a signal storage unit 8124, a synchronization signal input unit 8125, a synchronization control unit 8126, and a light receiving unit 8127.

  Each of the two transmission units 8121 and 8122 has a configuration similar to that of the transmitter 8115 illustrated in FIG. 56A, and transmits a signal by changing the luminance. Specifically, the transmission unit 8121 includes a power supply unit 8121a, a signal control unit 8121b, a light emitting unit 8121c, and a light emitting unit 8121d. The transmission unit 8122 includes a power supply unit 8122a, a signal control unit 8122b, a light emitting unit 8122c, and a light emitting unit 8122d.

  The signal changing unit 8123 modulates the transmission target signal into a signal indicating a luminance change pattern. The signal storage unit 8124 stores a signal indicating the luminance change pattern. The signal control unit 8121b of the transmission unit 8121 reads the signal stored in the signal storage unit 8124, and changes the luminance of the light emitting unit 8121c and the light emitting unit 8121d according to the signal.

  The synchronization signal input unit 8125 acquires a synchronization signal in accordance with control by the central control unit 8118. When the synchronization signal is acquired, the synchronization control unit 8126 synchronizes the luminance change between the transmission unit 8121 and the transmission unit 8122. That is, the synchronization control unit 8126 synchronizes the luminance change between the transmission unit 8121 and the transmission unit 8122 by controlling the signal control unit 8121b and the signal control unit 8122b. Here, the light receiving unit 8127 detects light emission from the transmission unit 8121 and the transmission unit 8122. The synchronization control unit 8126 performs feedback control to the signal control unit 8121b and the signal control unit 8122b in accordance with the light detected by the light receiving unit 8127.

  FIG. 58 is a block diagram illustrating another example of the transmitter in the third embodiment.

  The transmitter 8130 includes a transmission unit 8131 that transmits a signal according to a luminance change, and a non-transmission unit 8132 that emits light without transmitting the signal.

  The transmission unit 8131 has a configuration similar to that of the transmitter 8115 illustrated in FIG. 56A and includes a power supply unit 8131a, a signal control unit 8131b, and light emitting units 8131c to 8131f. The non-transmission unit 8132 includes a power supply unit 8132a and light emitting units 8132c to 8132f, and does not include a signal control unit. In other words, when there are multiple units including a power supply and synchronization control of luminance change is not possible between these units, only one unit is provided with a signal control unit as shown in FIG. Change the brightness of only one unit.

  Here, in such a transmitter 8130, the light emitting units 8131c to 8131f of the transmission unit 8131 are continuously arranged in a line. That is, any of the light emitting units 8132c to 8132f of the non-transmission unit 8132 is not mixed with the set of the light emitting units 8131c to 8131f. As a result, the size of the illuminant that changes in luminance increases, so that the receiver can easily receive a signal transmitted by the luminance change.

  FIG. 59A is a diagram illustrating an example of a transmitter in Embodiment 3.

  The transmitter 8134 is configured as a signage, for example, and includes three light emitting units (light emitting regions) 8134a to 8134c. In addition, the light from these light emission parts 8134a-8134c does not interfere with each other. Here, when only one of the light emitting units 8134a to 8134c can change the luminance and transmit the signal, as shown in FIG. 59A (a), the light emitting unit 8134b at the center is connected. It is desirable to change the brightness. When two of the light emitting units 8134a to 8134c can change in luminance, as shown in FIG. 59A (b), the light emitting unit 8134b at the center and the light emitting unit 8134a at the end or the light emitting unit It is desirable to change the brightness of the portion 8134c. By changing the luminance of the light emitting unit at such a position, the receiver can appropriately receive a signal transmitted by the luminance change.

  FIG. 59B is a diagram illustrating an example of a transmitter in Embodiment 3.

  The transmitter 8135 is configured as a signage, for example, and includes three light emitting units 8135a to 8135c. Note that light from the light emitting units adjacent to each other among these light emitting units 8135a to 8135c interferes with each other. Here, when only one of the light emitting units 8135a to 8135c can change the luminance to transmit a signal, as shown in (a) of FIG. 59B, the light emission arranged at the end. It is desirable to change the luminance of the portion 8135a or the light emitting portion 8135c. Thereby, it can suppress that the luminance change for transmitting a signal interferes with the light from another light emission part. When two of the light emitting units 8135a to 8135c can be changed in luminance, as shown in FIG. 59B (b), the light emitting unit 8135b at the center and the light emitting unit 8135a at the end or the light emitting unit It is desirable to change the brightness of the portion 8135c. By changing the luminance of the light emitting unit at such a position, the area of the luminance change increases, so that the receiver can appropriately receive a signal transmitted by the luminance change.

  FIG. 59C is a diagram illustrating an example of a transmitter in Embodiment 3.

  If the transmitter 8134 can change the luminance of two of the three light emitting units 8134a to 8134c, the transmitter 8134 can change the luminance of the light emitting units 8134a and 8134c at both ends as shown in FIG. 50C. Good. In this case, in the imaging by the receiver, the imaging range in which the part where the luminance changes is reflected can be expanded.

  60A is a diagram illustrating an example of a transmitter in Embodiment 3. FIG.

  The transmitter 8137 is configured as a signage, for example, and transmits a signal when the character portion “A Shop” and the light emitting unit 8137a change in luminance. The light emitting unit 8137a is formed in, for example, a rectangular shape that is long in the horizontal direction, and changes in luminance uniformly. When the light emitting unit 8137a uniformly changes in luminance, the receiver can appropriately receive a signal transmitted by the luminance change.

  FIG. 60B is a diagram illustrating an example of a transmitter in Embodiment 3.

  The transmitter 8138 is configured as a signage, for example, and transmits a signal when the character portion “A Shop” and the light emitting unit 8138a change in luminance. The light emitting unit 8138a is formed in a frame shape along the edge of the signage, for example, and the luminance changes uniformly. That is, the light emitting portion 8138a is formed so that the length of the continuous projected portion is maximized when the light emitting portion is projected onto an arbitrary straight line. When the light emitting unit 8138a changes in luminance uniformly, the receiver can more appropriately receive a signal transmitted by the luminance change.

  FIG. 61 is a diagram illustrating an example of processing operations of a receiver, a transmitter, and a server in Embodiment 3.

  For example, the receiver 8142 configured as a smartphone acquires position information indicating its own position, and transmits the position information to the server 8141. Note that the receiver 8142 acquires position information when, for example, GPS is used or other signals are received. The server 8141 transmits the ID list associated with the position indicated by the position information to the receiver 8142. The ID list includes, for each ID such as “abcd”, the ID and information associated with the ID.

  The receiver 8142 receives a signal from a transmitter 8143 configured as, for example, a lighting device. At this time, the receiver 8142 may receive only a part of the ID (for example, “b”) as the above-described signal. In this case, the receiver 8142 searches the ID list for an ID including a part of the ID. If the unique ID is not found, the receiver 8142 further receives a signal from the transmitter 8143 that includes another portion of that ID. Thereby, the receiver 8142 obtains a larger part (for example, “bc”) of the ID. Then, the receiver 8142 searches the ID list again for an ID including a part of the ID (for example, “bc”). By performing such a search, the receiver 8142 can specify all of the IDs even if only a part of the IDs can be acquired. When receiving a signal from the transmitter 8143, the receiver 8142 receives not only a part of the ID but also a check part such as CRC (Cyclic Redundancy Check).

  FIG. 62 is a diagram illustrating an example of processing operations of a receiver, a transmitter, and a server in Embodiment 3.

  For example, the receiver 8152 configured as a smartphone acquires position information indicating its own position. Note that the receiver 8152 acquires position information when, for example, GPS is used or other signals are received. Furthermore, the receiver 8152 receives a signal from a transmitter 8153 configured as, for example, a lighting device. At this time, the signal includes only a part of the ID (for example, “b”). Here, the receiver 8152 transmits the position information and a part of the ID to the server 8151.

  The server 8151 searches for an ID including a part of the ID from the ID list associated with the position indicated by the position information. If the unique ID is not found, the server 8151 notifies the receiver 8152 that the identification of the ID has failed.

  Next, the receiver 8152 receives a signal including another part of the ID from the transmitter 8153. Thereby, the receiver 8152 obtains a larger part (for example, “be”) of the ID. Then, the receiver 8152 transmits a part of the ID (for example, “be”) and the position information to the server 8151.

  The server 8151 searches for an ID including a part of the ID from the ID list associated with the position indicated by the position information. When the unique ID is found, the server 8151 notifies the receiver 8152 that the ID (for example, “abef”) has been specified, and transmits information associated with the ID to the receiver 8152.

  FIG. 63 is a diagram illustrating an example of processing operations of the receiver, the transmitter, and the server in Embodiment 3.

  The receiver 8152 may transmit not only a part of the ID but also all of the ID to the server 8151 together with the position information. At this time, if the ID of the complete state (for example, “wxyz”) is not included in the ID list, the server 8151 notifies the receiver 8152 of an error.

  FIG. 64A is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment.

  The transmitter 8155a and the transmitter 8155b transmit signals by changing the luminance. Here, the transmitter 8155a transmits a synchronization signal to the transmitter 8155b, whereby the luminance changes in synchronization with the transmitter 8155b. In addition, the transmitter 8155a and the transmitter 8155b each acquire a signal from the transmission source, and change the luminance according to the signal. Here, the time required for signal transmission from the transmission source to the transmitter 8155a (first delay time) may be different from the time required for signal transmission from the transmission source to the transmitter 8155b (second delay time). is there. Therefore, the round trip time of signals between the transmitters 8155a and 8155b and the transmission source is measured, and ½ of the round trip time is specified as the first or second delay time. The transmitter 8155a performs a luminance change synchronized with the transmitter 8155b by transmitting a synchronization signal such that the difference between the first and second delay times is canceled.

  FIG. 64B is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment.

  The light receiving sensor 8156 detects light from the transmitter 8155a and the transmitter 8155b, and outputs the result as a detection signal to the transmitter 8155a and the transmitter 8155b. When the transmitter 8155a and the transmitter 8155b receive the detection signal from the light receiving sensor 8156, the transmitter 8155a and the transmitter 8155b perform luminance changes synchronized with each other based on the detection signal and adjust the intensity of the signal.

  FIG. 65 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  For example, the transmitter 8165 configured as a television acquires an image and an ID (ID 1000) associated with the image from the control unit 8166. The transmitter 8165 displays the image and transmits the ID (ID 1000) to the receiver 8167 by changing the luminance. The receiver 8167 receives the ID (ID 1000) by imaging and displays information associated with the ID (ID 1000). Here, the control unit 8166 changes the image output to the transmitter 8165 to another image. At this time, the control unit 8166 also changes the ID output to the transmitter 8165. That is, the control unit 8166 outputs the other ID (ID 1001) associated with the other image to the transmitter 8165 together with the other image. Thus, the transmitter 8165 displays another image and transmits another ID (ID 1001) to the receiver 8167 by changing the luminance. The receiver 8167 receives the other ID (ID 1001) by imaging and displays information associated with the other ID (ID 1001).

  FIG. 66 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  The transmitter 8170 is configured as a signage, for example, and switches and displays images. Then, when displaying the image, the transmitter 8170 transmits ID time information indicating the ID corresponding to the displayed image and the time when the image is displayed to the receiver 8171 by changing the luminance. For example, the transmitter 8170 displays an image showing a round graphic at time t1, and also displays an ID (ID: 1000) corresponding to the image and a time (TIME: t1) when the image is displayed. ID time information shown is transmitted.

  Here, the transmitter 8170 transmits not only the ID time information corresponding to the currently displayed image but also at least one ID time information corresponding to the image displayed in the past. For example, the transmitter 8170 displays an image showing a square graphic at time t2, and also displays an ID (ID: 1001) corresponding to the image and a time (TIME: t2) at which the image is displayed. ID time information shown is transmitted. Further, at this time, the transmitter 8170 transmits ID time information indicating the ID (ID: 1000) corresponding to the image showing the round graphic and the time (TIME: t1) when the image was displayed. Similarly, at time t3, the transmitter 8170 displays an image showing a triangular figure, ID corresponding to the image (ID: 1002), and time (TIME: t3) when the image is displayed. ID time information indicating is transmitted. Further, at this time, the transmitter 8170 transmits ID time information indicating the ID (ID: 1001) corresponding to the image indicating the square graphic and the time (TIME: t2) when the image was displayed. That is, the transmitter 8170 transmits a plurality of ID time information at the same timing.

  For example, in order to obtain information related to an image showing a square graphic, the user holds the image sensor of the receiver 8171 over the transmitter 8170 at time t2 when the image showing the square graphic is displayed. Imaging with 8171 is started.

  Here, even if the receiver 8171 starts imaging at time t2, the receiver 8171 cannot acquire ID time information corresponding to the image while the image indicating the square graphic is displayed on the transmitter 8170. There is a case. Even in such a case, as described above, since the ID time information corresponding to the image displayed in the past is also transmitted from the transmitter 8170, the receiver 8171 has a triangular shape at time t3. Not only ID time information (ID: 1002, TIME: t3) corresponding to an image showing a graphic but also ID time information (ID: 1001, TIME: t2) corresponding to an image showing a square graphic can be acquired. The receiver 8171 selects ID time information (ID: 1001, TIME: t2) indicating the time (t2) held over the transmitter 8170 from the ID time information, and the ID time information indicates the ID time information. The indicated ID (ID: 1001) is specified. Accordingly, the receiver 8171 can obtain information on an image showing a square graphic from a server or the like based on the specified ID (ID: 1001) at time t3.

  Note that the above-described time is not limited to an absolute time, and is a time between the time when the receiver 8171 is held over the transmitter 8170 and the time when the receiver 8171 acquires the ID time information (so-called relative time). It may be. The transmitter 8170 has transmitted ID time information corresponding to an image displayed in the past, together with ID time information corresponding to the currently displayed image, but corresponds to an image scheduled to be displayed in the future. ID time information may be transmitted. Further, the transmitter 8170 may increase the number of past or future ID time information to be transmitted when reception by the receiver 8171 is difficult.

  Further, when the transmitter 8170 is configured as a television instead of signage, the transmitter 8170 may transmit information indicating a channel corresponding to the displayed image instead of the ID time information. That is, when an image of a broadcast television program is displayed on the transmitter 8170 in real time, the display time of the image displayed on the transmitter 8170 can be uniquely specified for each channel. Therefore, the receiver 8171 specifies the time when the receiver 8171 is held over the transmitter 8170, that is, the time when the receiver 8171 starts imaging, based on the image obtained by imaging and the channel. Can do. The receiver 8171 can obtain information on an image obtained by imaging from, for example, a server based on the channel and the time. The transmitter 8170 may transmit information indicating the display time of the displayed image instead of the ID time information. In this case, the receiver 8171 searches for the TV program including the image obtained by imaging from all the TV programs broadcast at that time, and sets the channel of the TV program and its display time. Based on this, information related to the image can be obtained from a server or the like.

  FIG. 67 is a diagram illustrating an example of operation of a transmitter, a receiver, and a server in Embodiment 3.

  As shown in FIG. 67A, the receiver 8176 captures an image of the transmitter 8175 to acquire an image including a bright line, and specifies (acquires) the ID of the transmitter 8175 from the image. Furthermore, the receiver 8176 transmits the ID to the server 8177, and acquires information associated with the ID from the server 8177.

  On the other hand, as illustrated in FIG. 67B, the receiver 8176 may acquire an image including a bright line by capturing an image of the transmitter 8175 and transmit the image to the server 8177 as captured data. The receiver 8176 may perform preprocessing for an image including a bright line so that the amount of information of the image is reduced, and transmit the preprocessed image to the server 8177 as imaging data. This preprocessing is, for example, image binarization processing. When the server 8177 acquires the imaging data, the server 8177 identifies (acquires) the ID of the transmitter 8175 from the image indicated by the imaging data. Furthermore, the server 8177 transmits information associated with the ID to the receiver 8176.

  FIG. 68 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  When the user is at position A, the receiver 8183 identifies the position of the receiver 8183 by acquiring a signal transmitted from the transmitter 8181 that changes in luminance. As a result, the receiver 8183 displays the point 8183b indicating the specified position together with the error range 8183a of the position.

  Next, when the user moves from the position A and arrives at the position B, the receiver 8183 cannot acquire a signal from the transmitter 8181. At this time, the receiver 8183 estimates its own position by using a 9-axis sensor provided in the receiver 8183. Then, the receiver 8183 displays a point 8183b indicating the estimated position together with an error range 8183a of the position. At this time, since the position is estimated by the 9-axis sensor, the error range 8183a is displayed widely.

  Next, when the user moves from the position B and arrives at the position C, the receiver 8183 identifies the position of the receiver 8183 by acquiring a signal transmitted from another transmitter 8182 whose luminance changes. As a result, the receiver 8183 displays the point 8183b indicating the specified position together with the error range 8183a of the position. Here, the receiver 8183 smoothly displays the point 8183b indicating the position estimated using the 9-axis sensor and the error range 8183a without immediately switching to the position and error range specified as described above and displaying them. Move them to switch. At this time, the error range 8183a becomes small.

  FIG. 69 is a diagram illustrating an example of appearance of a receiver in Embodiment 3.

  The receiver 8183 is configured as, for example, a smartphone (high-function mobile phone), and an image sensor 8183c, an illuminance sensor 8183d, and a display 8183e are arranged on the front surface of the receiver 8183, as shown in FIG. ing. The image sensor 8183c acquires an image including a bright line by imaging the subject whose luminance changes as described above. The illuminance sensor 8183d detects the change in luminance of the subject described above. Therefore, the illuminance sensor 8183d can be used in place of the image sensor 8183c depending on the state or situation of the subject. The display 8183e displays an image or the like. Here, the receiver 8183 may have a function as a subject whose luminance changes. In this case, the receiver 8183 transmits a signal by changing the brightness of the display 8183e.

  As shown in FIG. 69B, an image sensor 8183f, an illuminance sensor 8183g, and a flash light emitting unit 8183h are arranged on the back surface of the receiver 8183. The image sensor 8183f is the same as the image sensor 8183c described above, and acquires an image including a bright line by imaging the subject whose luminance changes as described above. The illuminance sensor 8183g is the same as the illuminance sensor 8183d described above, and detects a change in luminance of the subject. Therefore, the illuminance sensor 8183g can be used in place of the image sensor 8183f depending on the state or situation of the subject. The flash light emitting unit 8183h emits a flash for imaging. Here, the receiver 8183 may have a function as a subject whose luminance changes. In this case, the receiver 8183 transmits a signal by changing the luminance of the flash light emitting unit 8183h.

  FIG. 70 is a diagram illustrating an example of operation of a transmitter, a receiver, and a server in Embodiment 3.

  For example, the transmitter 8185 configured as a smartphone displays information indicating “coupon 100 yen discount”, for example, by changing the luminance of the display 8185a except for the barcode portion 8185b, that is, by visible light communication. Send. The transmitter 8185 displays the barcode on the barcode portion 8185b without changing the luminance of the barcode portion 8185b. This bar code indicates the same information as the information transmitted by the visible light communication described above. Furthermore, the transmitter 8185 displays a character or a picture indicating information transmitted by visible light communication, for example, the character “coupon 100 yen discount” on a portion of the display 8185a excluding the barcode portion 8185b. By displaying such characters or pictures, the user of the transmitter 8185 can easily grasp what information is being transmitted.

  The receiver 8186 acquires the information transmitted by visible light communication and the information indicated by the barcode by imaging, and transmits the information to the server 8187. The server 8187 determines whether or not these pieces of information match or relate to each other. When it determines that these pieces of information match or relate to each other, the server 8187 executes processing according to the pieces of information. Alternatively, the server 8187 transmits the determination result to the receiver 8186, and causes the receiver 8186 to execute processing according to the information.

  The transmitter 8185 may transmit a part of information indicated by the barcode by visible light communication. The barcode may indicate the URL of the server 8187. Further, the transmitter 8185 may acquire information associated with the ID by acquiring the ID as a receiver and transmitting the ID to the server 8187. The information associated with this ID is the same as the information transmitted by the above visible light communication or the information indicated by the barcode. Further, the server 8187 may transmit an ID associated with information (visible light communication information or barcode information) transmitted from the transmitter 8185 via the receiver 8186 to the transmitter 8185.

  71 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG.

  For example, the transmitter 8185 configured as a smartphone transmits a signal by changing the luminance of the display 8185a. The receiver 8188 includes a light-shielding cone-shaped container 8188b and an illuminance sensor 8188a. The illuminance sensor 8188a is stored inside the container 8188b and is disposed near the tip of the container 8188b. When a signal is transmitted from the transmitter 8185 by visible light communication, the opening (bottom) of the container 8188b in the receiver 8188 is directed to the display 8185a. Thereby, since light other than the light from the display 8185a does not enter the container 8188b, the illuminance sensor 8188a of the receiver 8188 appropriately receives the light from the display 8185a without being affected by noise light. It can receive light. As a result, the receiver 8188 can appropriately receive the signal from the transmitter 8185.

  FIG. 72 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  The transmitter 8190 is configured, for example, as a sign tower at a bus stop, and transmits operation information indicating the operation status of the bus to the receiver 8183 by changing the luminance. For example, the destination of the bus, the time at which the destination bus arrives at the bus stop, and operation information indicating the current location of the bus are transmitted to the receiver 8183. When receiving the operation information, the receiver 8183 displays the content indicated by the operation information on the display.

  Here, for example, when buses with different destinations stop at the bus stop, the transmitter 8190 transmits operation information regarding the destination buses. When the receiver 8183 receives the operation information, the receiver 8183 selects the operation information of the destination bus frequently used by the user from the operation information, and displays the contents indicated by the operation information on the display. To do. Specifically, the receiver 8183 specifies the destination of the bus used by the user by using, for example, GPS, and records the destination history. By referring to this history, the receiver 8183 selects operation information of a destination bus frequently used by the user. Alternatively, the receiver 8183 may display the content indicated by the operation information selected by the user's operation from the operation information on the display. Alternatively, the receiver 8183 may preferentially display the operation information of the destination bus that is frequently selected by the user's operation.

  FIG. 73 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  For example, the transmitter 8191 configured as a signage transmits information on a plurality of stores to the receiver 8183 by changing the luminance. This information is a collection of information related to a plurality of stores, and is not unique to each store. Therefore, when the receiver 8183 receives the information by imaging, the receiver 8183 can display information on a plurality of stores as well as one store. Here, the receiver 8183 selects information regarding a store (for example, “B shop”) included in the imaging range from information regarding the plurality of stores, and displays the selected information. In addition, when displaying the information, the receiver 8183 translates a language for expressing the information into a pre-registered language, and displays the information in the translated language. Further, the transmitter 8191 may display a message prompting the user to take an image with the image sensor (camera) of the receiver 8183 in characters or the like. Specifically, when a dedicated application program is activated and imaging is performed with a camera, a message (for example, “GET information with camera”) indicating that information can be received is displayed on the transmitter 8191. The

  FIG. 74 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  For example, the receiver 8183 images a subject including a plurality of persons 8197 and street lamps 8195. The streetlight 8195 includes a transmitter 8195a that transmits information according to a change in luminance. By this imaging, the receiver 8183 acquires an image in which the image of the transmitter 8195a appears as the bright line pattern described above. Furthermore, the receiver 8183 acquires the AR object 8196a associated with the ID indicated by the bright line pattern from, for example, a server. Then, the receiver 8183 superimposes the AR object 8196a on a normal captured image 8196 obtained by normal imaging, and displays a normal captured image 8196 on which the AR object 8196a is superimposed.

  FIG. 75A is a diagram illustrating an example of a structure of information transmitted by a transmitter in Embodiment 3.

  For example, the information transmitted by the transmitter includes a preamble part, a fixed-length data part, and a check part. The receiver checks the data part using the check part, and normally receives information including these parts. Here, when the receiver receives the preamble part and the data part and fails to receive the check part, the check using the check part is omitted. Even in the case where such a check is omitted, the receiver can normally receive the information composed of these parts.

  FIG. 75B is a diagram illustrating another example of a structure of information transmitted by a transmitter in Embodiment 3.

  For example, the information transmitted by the transmitter includes a preamble part, a check part, and a variable length data part. The next information transmitted by the transmitter also includes a preamble part, a check part, and a variable length data part. Here, when the receiver receives the preamble part during reception and further receives the next preamble part, the information from the previous preamble part to immediately before the next preamble part has one meaning. Recognize as information. Further, the receiver may specify the end of the data part received next to the check part by using the check part. In this case, even if the receiver cannot receive the above-described next preamble part (all or a part of the preamble part), it appropriately receives the information that makes one meaning transmitted immediately before. be able to.

  FIG. 76 is a diagram illustrating an example of a 4-level PPM modulation scheme by a transmitter in Embodiment 3.

  The transmitter modulates a signal to be transmitted into a luminance change pattern by a four-value PPM modulation method. At this time, the transmitter can keep the brightness of the light whose luminance changes constant regardless of the signal to be transmitted.

  For example, when the brightness is kept at 75%, the transmitter transmits each of the signals to be transmitted “00”, “01”, “10”, and “11” to any one of four consecutive slots. One shows luminance L (Low), and the remaining three modulate to a luminance change pattern showing luminance H (High). Specifically, the transmitter transmits a signal “00” to be transmitted, a luminance change pattern (L, H, H, etc.) in which the first slot indicates luminance L and the second to fourth slots indicate luminance H. H). That is, in this luminance change, there is a rise in luminance between the first slot and the second slot. Similarly, the transmitter transmits a signal “01” to be transmitted, a luminance change pattern (H, H) in which the second slot indicates luminance L, and the first, third, and fourth slots indicate luminance H. L, H, H). That is, in this luminance change, there is a rise in luminance between the second slot and the third slot.

  When the brightness is maintained at 50%, the transmitter transmits each of the signals to be transmitted “00”, “01”, “10”, and “11” to any two of the four slots. The luminance L (Low) is indicated, and the remaining two are modulated into a luminance change pattern indicating luminance H (High). Specifically, the transmitter transmits a signal “00” to be transmitted, a luminance change pattern (L, L) in which the first and fourth slots indicate luminance L, and the second and third slots indicate luminance H. H, H, L). That is, in this luminance change, there is a rise in luminance between the first slot and the second slot. Similarly, the transmitter transmits a signal “01” to be transmitted, a luminance change pattern (L, L) in which the first and second slots indicate luminance L, and the third and fourth slots indicate luminance H. L, H, H). Alternatively, the transmitter transmits a signal “01” to be transmitted, a luminance change pattern (H, L, H) in which the second and fourth slots indicate luminance L and the first and third slots indicate luminance H. , L). That is, in these luminance changes, there is a rise in luminance between the second slot and the third slot.

  When the brightness is kept at 25%, the transmitter transmits each of the signals to be transmitted “00”, “01”, “10”, and “11” to any three of the four slots. The luminance L (Low) is indicated, and the remaining one is modulated into a luminance change pattern indicating the luminance H (High). Specifically, the transmitter transmits a signal “00” to be transmitted, a luminance change pattern (L, L) in which the first, third and fourth slots indicate luminance L, and the second slot indicates luminance H. H, L, L). That is, in this luminance change, there is a rise in luminance between the first slot and the second slot. Similarly, the transmitter transmits the signal “01” to be transmitted to a luminance change pattern (L, L, in which the first, second, and fourth slots indicate luminance L, and the third slot indicates luminance H). L, H, L). That is, in this luminance change, there is a rise in luminance between the second slot and the third slot.

  The transmitter can suppress flickering and can easily adjust the brightness step by step by the four-value PPM modulation method as described above. In addition, the receiver can appropriately demodulate the luminance change pattern by specifying the position where the luminance rises. Note that the receiver ignores the presence or absence of a rise in luminance at the boundary between the slot group consisting of four slots and the next slot group without using it for demodulating the luminance change pattern.

  77 is a diagram illustrating an example of a PPM modulation scheme by a transmitter in Embodiment 3. FIG.

  The transmitter modulates the signal to be transmitted into a luminance change pattern as in the 4-value PPM modulation method shown in FIG. 76, but performs PPM modulation without switching the luminance between L and H for each slot. Also good. That is, the transmitter performs PPM modulation by switching the rising edge of the luminance in the time width (hereinafter referred to as unit time width) of four consecutive slots shown in FIG. 76 according to the signal to be transmitted. For example, as shown in FIG. 77, the transmitter modulates the transmission target signal “00” into a luminance change pattern in which the luminance rises at a position of 25% of the unit time width. Similarly, as shown in FIG. 77, the transmitter modulates the signal to be transmitted “01” into a luminance change pattern in which the luminance rises at a position of 50% of the unit time width.

  Further, when the transmitter keeps the brightness at 75%, the transmission target signal “00” indicates the luminance L at the position of 0 to 25% in the above unit time width, and the position of 25 to 100%. To modulate to a luminance change pattern indicating luminance H. Here, when the transmitter keeps the brightness at 99%, the transmission target signal “00” indicates the luminance L at the position of 24 to 25% in the above unit time width, and is 0 to 24%. The pattern is modulated into a luminance change pattern indicating the luminance H at the position and the position of 25 to 100%. Similarly, when the transmitter keeps the brightness at 1%, the transmitter sets the signal “00” to be transmitted to the luminance L at the position of 0 to 25% and the position of 26 to 100% in the above unit time width. It is modulated into a luminance change pattern indicating luminance H at a position of 25 to 26%.

  In this way, the brightness can be continuously adjusted by switching the luminance to L and H at any position in the unit time width without switching the luminance to L and H for each slot.

  FIG. 78 is a diagram illustrating an example of a PPM modulation scheme in a transmitter in Embodiment 3.

  The transmitter performs modulation in the same manner as the PPM modulation method shown in FIG. 77. However, regardless of the signal to be transmitted, the transmitter always indicates luminance H at the beginning of the unit time width, And at the end of the unit time width, it is always modulated into a luminance change pattern indicating luminance L. Thereby, since a rise in luminance occurs at the boundary between the unit time width and the next unit time width, the receiver can appropriately specify the boundary. Therefore, the receiver and the transmitter can correct the clock shift.

  FIG. 79A is a diagram showing an example of a luminance change pattern corresponding to the header (preamble portion) in the third embodiment.

  For example, when transmitting the header (preamble portion) shown in FIGS. 75A and 75B, the transmitter changes in luminance according to the pattern shown in FIG. 79A. That is, when the header is composed of 7 slots, the transmitter changes in luminance according to a pattern indicated by L, H, L, H, L, H, and H. When the header is composed of 8 slots, the transmitter changes in luminance according to a pattern indicated by H, L, H, L, H, L, H, and H. Since these patterns can be distinguished from the luminance change patterns shown in FIG. 76, it is possible to clearly notify the receiver that the signal indicated by these patterns is a header.

  FIG. 79B is a diagram showing an example of a luminance change pattern in Embodiment 3.

  As shown in FIG. 76, in the four-value PPM modulation method, when the signal “01” to be transmitted included in the data portion is modulated with the brightness maintained at 50%, the transmitter Is modulated into one of two patterns. That is, it modulates to the 1st pattern shown by L, L, H, and H, or the 2nd pattern shown by H, L, H, and L.

  Here, when the luminance change pattern corresponding to the header is a pattern as shown in FIG. 79A, the transmitter uses the first transmission signal “01” indicated by L, L, H, and H as described above. It is desirable to modulate the pattern. For example, the transmission target signal “11, 01, 11” included in the above-described data portion is “H, H, L, L, L, L, H, H, when the first pattern is used. , H, H, L, L ". On the other hand, when the second pattern is used, the transmission target signals “11, 01, 11” included in the data portion are “H, H, L, L, H, L, H, L , H, H, L, L ". In this case, the pattern “H, H, L, L, H, L, H, L, H, H, L, L” has the same pattern as the header pattern configured by 7 slots shown in FIG. 79A. Is included. Therefore, in order to clarify the distinction between the header and the data portion, it is desirable to modulate the above-described transmission target signal “01” into the first pattern.

  FIG. 80A is a diagram showing an example of a luminance change pattern in Embodiment 3.

  As shown in FIG. 76, in the quaternary PPM modulation method, when the signal “11” to be transmitted is modulated, the transmitter converts the signal to “H, H, H, so that no rise in luminance occurs. , L ”pattern,“ H, H, L, L ”pattern, or“ H, L, L, L ”pattern. However, as shown in FIG. 80A, the transmitter transmits the signal “11” to be transmitted to the pattern “H, H, H, H” or “L, L, L, L,” in order to adjust the brightness. May be modulated into a pattern.

  FIG. 80B is a diagram showing an example of a luminance change pattern in Embodiment 3.

  As shown in FIG. 76, in the 4-level PPM modulation method, when the signal “11,0” to be transmitted is modulated with the brightness maintained at 75%, the transmitter converts the signal to “H, H , H, L, L, H, H, H ”. However, when the luminance L is to be generated continuously, luminances other than the last luminance L among the continuous luminances L may be changed to H so that the luminance L does not continue. In this case, the transmitter modulates the signal “11.00” into a pattern of “H, H, H, H, L, H, H, H”.

  Thereby, since the luminance L is not continuous, the load on the transmitter can be suppressed. Moreover, the capacity | capacitance of the capacitor | condenser with which a transmitter is equipped can be made small, and the volume of a control circuit can be made small. Furthermore, since the load on the light source of the transmitter is small, the light source can be easily made. Furthermore, the power efficiency of the transmitter can be increased. Further, since it is guaranteed that the luminance L is not continuous, the receiver can easily demodulate the luminance change pattern.

(Summary of this embodiment)
The information communication method according to the present embodiment is an information communication method for transmitting a signal according to a luminance change, wherein a determination step for determining a luminance change pattern by modulating a signal to be transmitted and a light emitter are determined. A transmission step of transmitting the signal to be transmitted by changing the luminance according to the pattern, wherein the luminance change pattern includes two different luminances at arbitrary positions in a predetermined time width. In the pattern in which one of the values appears, in the determination step, the luminance change position that is the rising or falling position of the luminance in the time width is different for each of the different signals to be transmitted. In addition, the integrated value of the luminance of the luminous body in the time width is the same according to the preset brightness As a value to determine the pattern of the luminance change.

  For example, as shown in FIG. 77, for each of the signals “00”, “01”, “10”, and “11” that are different from each other, the luminance rising position (luminance change position) is different from each other, and The luminance change so that the integrated value of the luminance of the light emitter in a predetermined time width (unit time width) becomes the same value according to a predetermined brightness (for example, 99% or 1%). Pattern is determined. As a result, the brightness of the illuminant can be kept constant for each signal to be transmitted, flicker can be suppressed, and the receiver that images the illuminant is based on the luminance change position. The brightness change pattern can be demodulated appropriately. In addition, the luminance change pattern is a pattern in which one of two different luminance values (luminance H (High) or luminance L (Low)) appears at any position in the unit time width. The brightness of the body can be changed continuously.

  The information communication method further includes an image display step of sequentially switching and displaying each of the plurality of images. In the determination step, the displayed image is displayed each time the image is displayed in the image display step. The luminance change pattern for the identification information is determined by modulating the identification information corresponding to the signal to be transmitted. In the transmission step, the image is displayed every time the image is displayed in the image display step. The identification information may be transmitted by the luminance change of the light emitter according to the luminance change pattern determined with respect to the identification information corresponding to the existing image.

  As a result, for example, as shown in FIG. 65, each time an image is displayed, identification information corresponding to the displayed image is transmitted. Therefore, the user receives the received information on the receiver based on the displayed image. The identification information to be made can be easily selected.

  In the transmission step, each time an image is displayed in the image display step, the light emitter is further radiated according to a luminance change pattern determined for identification information corresponding to an image displayed in the past. The identification information may be transmitted by changing.

  As a result, for example, as shown in FIG. 66, even if the receiver cannot receive the identification signal transmitted before switching because the displayed image is switched, the identification information corresponding to the currently displayed image is displayed. At the same time, since the identification information corresponding to the images displayed in the past is also transmitted, the identification information transmitted before the switching can be properly received again by the receiver.

  In the determination step, each time an image is displayed in the image display step, the identification information corresponding to the displayed image and the time when the image is displayed are modulated as the signal to be transmitted. Thus, a pattern of luminance change with respect to the identification information and the time is determined, and in the transmission step, each time an image is displayed in the image display step, the identification information and time corresponding to the displayed image are determined. The identification information and the time are transmitted when the luminous body changes in luminance according to the luminance change pattern determined in the above, and the identification information and the time corresponding to an image displayed in the past are further determined. Even if the identification information and the time are transmitted by the luminance change of the light emitter according to the luminance change pattern. There.

  As a result, for example, as shown in FIG. 66, each time an image is displayed, a plurality of ID time information (information including identification information and time) is transmitted. From the information, an identification signal that has been transmitted in the past and cannot be received can be easily selected based on the time included in each of the ID time information.

  In addition, each of the light emitters has a plurality of regions that emit light, light in regions adjacent to each other among the plurality of regions interfere with each other, and only one of the plurality of regions is determined. When the luminance changes according to the luminance change pattern, in the transmission step, only the region arranged at the end of the plurality of regions may change in luminance according to the determined luminance change pattern.

  As a result, for example, as shown in FIG. 59B (a), only the region arranged at the end (light emitting portion) changes in luminance, so that only the region arranged other than the end changes in luminance. The influence of the light from the region on the luminance change can be suppressed. As a result, the receiver can appropriately capture the luminance change pattern by photographing.

  In addition, when only two of the plurality of regions change in luminance according to the determined luminance change pattern, the transmission step includes: a region disposed at an end of the plurality of regions; The area adjacent to the area arranged at the end may change in luminance according to the determined luminance change pattern.

  As a result, for example, as shown in FIG. 59B (b), the luminance changes between the region arranged at the end (light emitting portion) and the region adjacent to the region arranged at the end (light emitting portion). Compared to the case where the luminance changes in a distant area, the area of the range in which the luminance continuously changes can be kept large. As a result, the receiver can appropriately capture the luminance change pattern by photographing.

  The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, a location information transmission step for transmitting location information indicating a location of an image sensor used for photographing the subject, and the location information. A list receiving step for receiving an ID list including a plurality of pieces of identification information associated with the position indicated by, and an image obtained by photographing the subject by the image sensor corresponding to an exposure line included in the image sensor An exposure time setting step for setting an exposure time of the image sensor so that a bright line to be generated is generated according to a change in luminance of the subject, and the subject in which the image sensor changes in luminance is photographed at the set exposure time. An image acquisition step of acquiring a bright line image including the bright line, An information acquisition step of acquiring information by demodulating data specified by the bright line pattern included in the bright line image, and a search step of searching identification information including the acquired information from the ID list. .

  Thereby, as shown in FIG. 61, for example, since the ID list is received in advance, even if the acquired information “bc” is only a part of the identification information, the appropriate identification information “ abcd "can be specified.

  If the identification information including the acquired information is not uniquely specified in the search step, new information is acquired by repeatedly performing the image acquisition step and the information acquisition step, and the information communication The method may further include a re-search step of searching the ID list for identification information including the acquired information and the new information.

  Thereby, as shown in FIG. 61, for example, even when the acquired information “b” is only a part of the identification information and the identification information is not uniquely specified by the information alone, the new information “b” Since “c” is acquired, the appropriate identification information “abcd” can be specified based on the new information and the ID list.

  The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, and an bright line corresponding to an exposure line included in the image sensor is included in an image obtained by photographing the subject with an image sensor. An exposure time setting step for setting an exposure time of the image sensor so as to occur according to a change in luminance of the subject, and the image sensor photographing the subject whose luminance changes with the set exposure time, An image acquisition step of acquiring a bright line image including the bright line, an information acquisition step of acquiring identification information by demodulating data specified by the pattern of the bright line included in the acquired bright line image, and A transmission step of transmitting the identification information and position information indicating the position of the image sensor Error reception for receiving error notification information for notifying an error when there is no acquired identification information in an ID list that includes a plurality of identification information associated with the position indicated by the position information Steps.

  Thus, for example, as shown in FIG. 63, when the acquired identification information is not in the ID list, the error notification information is received, so that the user of the receiver that has received the error notification information has acquired the error notification information. It can be easily understood that information associated with the identification information cannot be obtained.

(Embodiment 4)
In the present embodiment, an application example for each situation using a receiver such as a smartphone in the above first to third embodiments and a transmitter that transmits information as a blinking pattern such as an LED or an organic EL will be described.

<Situation: In front of the store>
First, an application example in a situation where there is a user carrying a receiver in front of a store that has a billboard for advertisement configured as a transmitter will be described with reference to FIGS.

  FIG. 81 is a diagram illustrating an example of the operation of the receiver in the situation in front of the store.

  For example, a user finds a store sign 8301 when walking while carrying a receiver 8300 (terminal device) configured as a smartphone. The sign 8301 is a transmitter (subject) that transmits a signal by luminance change like the transmitter of any of the first to third embodiments. Here, when the user is interested in the store and determines that the sign 8301 is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to perform the visible light communication of the receiver 8300. Application software (hereinafter referred to as communication application).

  FIG. 82 is a diagram illustrating another example of operation of the receiver 8300 in the store front situation.

  The receiver 8300 may automatically start the communication application without accepting an operation by the user. For example, the receiver 8300 detects its current location by using a GPS or a 9-axis sensor, and determines whether or not the current location has entered a specific area predetermined for the signboard 8301. This specific area is an area around the signboard 8301. When the receiver 8300 determines that the current location of the receiver 8300 has entered the specific area, the receiver 8300 activates the communication application. Further, the receiver 8300 may detect an operation of projecting the receiver 8300 or an operation of rotating the receiver 8300 by using a built-in 9-axis sensor or the like, and may activate a communication application. Thereby, a user's operation can be omitted and usability can be improved.

  FIG. 83 is a diagram illustrating an example of next operation of the receiver 8300 in the store front situation.

  The receiver 8300 that has activated the communication application as described above captures an image (visible light imaging) of the signboard 8301 configured as a transmitter that transmits a signal according to a change in luminance. That is, the receiver 8300 performs visible light communication with the sign 8301.

  FIG. 84 is a diagram illustrating an example of next operation of the receiver 8300 in the situation in front of the store.

  The receiver 8300 acquires an image including a bright line by the above-described imaging. Then, the receiver 8300 acquires the device ID of the sign 8301 by demodulating the data specified by the bright line pattern. That is, the receiver 8300 acquires the device ID from the signboard 8301 by visible light imaging or visible light communication in the first to third embodiments. Furthermore, the receiver 8300 transmits the device ID to the server, and acquires advertisement information (service information) associated with the device ID from the server.

  Note that the receiver 8300 may acquire advertisement information associated with the device ID from a plurality of advertisement information held in advance. In this case, when the receiver 8300 determines that the current location of the receiver 8300 has entered the specific area, the receiver 8300 notifies the server of the specific area or the current position, and all the device IDs corresponding to the specific area and The advertisement information associated with each of the device IDs is acquired in advance from the server and held (cached). Thus, when the receiver 8300 acquires the device ID of the signboard 8301 within the specific area, the receiver 8300 associates the advertisement information corresponding to the device ID with each device ID held in advance without requesting the server. The advertisement information associated with the device ID of the sign 8301 can be quickly acquired from the advertisement information.

  When the receiver 8300 acquires the advertisement information associated with the device ID of the signboard 8301, the receiver 8300 displays the advertisement information. For example, the receiver 8300 displays the coupon of the store indicated by the sign 8301, the vacant seat status, and a bar code indicating the same content as them.

  Here, the receiver 8300 may acquire not only the device ID but also the privilege data from the signboard 8301 by visible light communication. The privilege data indicates, for example, a random ID (random number) or a time or a time zone when the privilege data is transmitted. When receiving the privilege data, the receiver 8300 transmits the privilege data together with the device ID to the server. Then, the receiver 8300 acquires advertisement information associated with the device ID and privilege data from the server. Thereby, the receiver 8300 can receive different advertising information according to privilege data. For example, if the time zone when the signboard 8301 is imaged is early morning, the receiver 8300 can acquire and display advertisement information indicating an early morning discount coupon. That is, the advertisement according to the same signboard can be changed according to privilege data (such as time zone). As a result, the user can receive a service suitable for a time zone. In the present embodiment, presentation (display) of information such as service information to a user is referred to as service provision.

  The receiver 8300 may acquire three-dimensional information indicating the spatial arrangement of the sign 8301 with high accuracy (within an error of 1 m) from the sign 8301 together with the device ID by visible light communication. Alternatively, the receiver 8300 may acquire three-dimensional information associated with the device ID from the server. In addition, the receiver 8300 may acquire size information indicating the size of the sign 8301 instead of or together with the three-dimensional information. When the receiver 8300 acquires the size information, the receiver 8300 receives the size information from the receiver 8300 based on the difference between the size of the sign 8301 indicated by the size information and the size of the sign 8301 displayed in the image obtained by imaging. A distance up to 8301 can be calculated.

  Further, when transmitting the device ID acquired by visible light communication to the server, the receiver 8300 may transmit the holding information (attached information) held in advance by itself to the server together with the device ID. For example, the retained information is personal information (such as gender or age) of the user of the receiver 8300 or a user ID. The server that has received such retained information together with the device ID receives the advertisement information associated with the retained information (personal information or user ID) from at least one piece of advertisement information associated with the device ID. Send to. That is, the receiver 8300 can receive store advertisement information that matches personal information, store information corresponding to the user ID, and the like. As a result, the user can be provided with a more useful service.

  Alternatively, the holding information indicates reception conditions set in advance in the receiver 8300. This reception condition is, for example, the number of visitors when the store is a restaurant. The server that has received such retained information together with the device ID transmits, to at least one piece of advertisement information associated with the device ID, advertisement information associated with the reception condition (number of visitors) to the receiver 8300. . That is, the receiver 8300 can receive the advertisement information of the store that matches the number of visitors, specifically, information indicating the vacant seat status for the number of visitors. In addition, the store can display the advertisement information with the discount rate changed according to the number of customers, the day of the week, and the time zone, thereby optimizing customer attraction and profit.

  Alternatively, the holding information indicates the current location detected in advance by the receiver 8300. The server that has received such retained information together with the device ID is not limited to the advertisement information associated with the device ID, but also other device IDs corresponding to the current location (current location and its surroundings) indicated by the retained information, and others. The advertisement information associated with the device ID is transmitted to the receiver 8300. Thereby, the receiver 8300 can cache the other device ID and the advertisement information associated with the other device ID. Therefore, when the receiver 8300 performs visible light communication with another transmitter at the current location (current location and its surroundings), the advertisement associated with the device ID of the other transmitter without accessing the server. Information can be acquired quickly.

  FIG. 85 is a diagram illustrating an example of next operation of the receiver 8300 in the store front situation.

  When the receiver 8300 acquires the advertisement information from the server as described above, for example, the receiver 8300 displays a button described as “vacant seats” as the vacant seat status indicated by the advertisement information. Here, when the user performs an operation of touching the button “available seat” with the finger, the receiver 8300 notifies the server of the operation result. Upon receiving the notification, the server makes a temporary reservation for the store of the signboard 8301 and notifies the receiver 8300 that the temporary reservation has been completed. Upon receiving the notification from the server, the receiver 8300 displays a character string “temporary reservation” indicating that the temporary reservation has been completed, instead of the button “vacant seats”. The receiver 8300 stores, as a pre-acquired image, an image including a store coupon indicated by the signboard 8301, a character string “provisional reservation” certifying that the provisional reservation has been made, and a bar code indicating the same content as the pre-acquired image Store it.

  Here, the server can log information related to visible light communication performed between the sign 8301 and the receiver 8300 by the operation described with reference to FIGS. 84 and 85. That is, the server provides privilege data indicating the device ID of the transmitter (signboard) that performed visible light communication, the location where visible light communication was performed (current location of the receiver 8300), the time zone during which visible light communication was performed, and the like. And personal information of the user of the receiver 8300 that performed visible light communication can be logged. The server can analyze the value of the sign 8301, that is, the degree of contribution of the sign 8301 to the advertisement and promotion of the store as an advertising effect using at least one of these pieces of logged information.

<Situation: In-store>
Next, application examples in situations where the user carrying the receiver 8300 enters a store corresponding to the displayed advertisement information (service information) will be described with reference to FIGS. 86 to 94.

  FIG. 86 is a diagram showing an example of the operation of the display device in the in-store situation.

  For example, the user of the receiver 8300 that has performed visible light communication with the above-described sign 8301 enters a store corresponding to the displayed advertisement information. At this time, the receiver 8300 detects that the user has entered the store corresponding to the advertisement information displayed using visible light communication (entering the store). For example, after performing visible light communication with the sign 8301, the receiver 8300 acquires store information indicating the store location associated with the device ID of the sign 8301 from the server. Then, the receiver 8300 determines whether or not the current location of the receiver 8300 obtained by using a GPS or a 9-axis sensor has entered the store location indicated by the store information. Here, the receiver 8300 detects the above-mentioned entrance by determining that the current location has entered the store location.

  Then, when the receiver 8300 detects the entry, the receiver 8300 notifies the display device 8300b of the entry via, for example, a server. Alternatively, the receiver 8300 notifies the display device 8300b that the store has entered by visible light communication or wireless communication. Upon receiving the notification, the display device 8300b acquires product service information indicating a product or service menu provided at the store, and displays the menu indicated by the product service information. Note that the display device 8300b may be a portable terminal carried by a user of the receiver 8300 or a store clerk, or a device provided in the store.

  FIG. 87 is a diagram illustrating an example of the next operation of the display device 8300b in the in-store situation.

  The user selects a desired product from the menu displayed on the display device 8300b. In other words, the user performs an operation of touching a part of the menu where the name of the desired product is displayed. Thereby, the display device 8300b receives the operation result of the product selection.

  FIG. 88 is a diagram illustrating an example of the next operation of the display device 8300b in the in-store situation.

  The display device 8300b that has received the operation result of the product selection displays an image representing the selected product and the price of the product. As a result, the display device 8300b prompts the user to confirm the selected product. In addition, the image showing the above-mentioned goods, the information which shows the price of goods, etc. are contained in the above-mentioned goods service information, for example.

  FIG. 89 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The user who is prompted for confirmation performs an operation for ordering the product. When the operation is performed, the receiver 8300 notifies the POS (Pint of Sale) system of the store via the display device 8300b or the server of payment information necessary for electronic payment. Furthermore, the receiver 8300 determines whether or not there is the above-described pre-acquired image acquired and stored using visible light communication with the sign 8301 of the store. When the receiver 8300 determines that there is the pre-acquired image, the receiver 8300 displays the pre-acquired image.

  Note that although the display device 8300b is used in this situation, the receiver 8300 may perform processing by the display device 8300b instead of using the display device 8300b. In this case, when the receiver 8300 detects entry, the receiver 8300 acquires product service information indicating a product or service menu provided at the store from the server, and displays the menu indicated by the product service information. Further, when receiving an operation for ordering a product, the receiver 8300 notifies the POS system of the store of the ordered product and payment information necessary for electronic payment via the server.

  FIG. 90 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The store clerk applies the barcode scanner 8302 of the POS system to the barcode of the pre-acquired image displayed on the receiver 8300. The barcode scanner 8302 reads the barcode of the pre-acquired image. As a result, the POS system performs electronic settlement according to the coupon indicated by the barcode. Then, the barcode scanner 8302 of the POS system transmits payment completion information indicating that the electronic payment is completed to the receiver 8300 by the luminance change. That is, the barcode scanner 8302 also has a function as a transmitter for visible light communication. When the payment completion information is acquired by visible light communication, the receiver 8300 displays the payment completion information. This settlement completion information indicates, for example, a message “Thank you for your purchase” and the amount of the settlement. By performing such an electronic payment, the POS system, the server, and the receiver 8300 allow the service indicated by the advertisement information at the store corresponding to the advertisement information (service information) displayed in front of the store. It can be determined that it has been used.

  As described above, merchandise is ordered in the store by the operations of the receiver 8300 and the POS system as shown in FIGS. Therefore, when the user enters the store, the user can place an order for a product from the store menu that is automatically displayed on the display device 8300b or the receiver 8300. That is, the store clerk does not need to show the menu to the user and directly accept an order for the product from the user. As a result, the burden on the store clerk can be greatly reduced. In the above example, the barcode scanner 8302 reads the barcode, but the barcode scanner 8302 may not be used. For example, the receiver 8300 may transmit information indicated by the barcode to the POS system via the server. Then, the receiver 8300 may acquire the payment completion information from the POS system via the server. Thereby, the work by the store clerk can be further reduced, and the user can place an order for the product without going through the store clerk. Alternatively, the display device 8300b and the receiver 8300 may exchange ordering and billing data by visible light communication, or the data may be exchanged by wireless communication using a key exchanged by visible light communication.

  In addition, the sign 8301 may be put out by one of a plurality of stores belonging to the chain store. In such a case, the advertisement information acquired using visible light communication with the sign 8301 can be used at all stores belonging to the chain store. However, even in the same chain store, the service received by the user may be different between a store (advertisement store) where a signboard 8301 is displayed and a store (non-advertisement store) where the signboard 8301 is not displayed. For example, when a user enters a non-advertising store, the user receives a service with a discount rate (for example, 20%) according to the coupon indicated in the pre-acquired image. Receive a service with a discount rate (for example, 30%) higher than the coupon discount rate. That is, when the receiver 8300 detects entering the advertising store, the receiver 8300 acquires additional service information indicating a further discount of 10% from the server, and a discount rate of 30% (20% + 10%). Is displayed instead of the pre-acquired image shown in FIG. The receiver 8300 detects whether the user has entered an advertising store or a non-advertising store based on the above-described store information acquired from the server. The store information indicates the location of each of a plurality of stores belonging to the chain store and whether the stores are advertising stores or non-advertising stores.

  Moreover, when there are a plurality of non-advertisement stores in the same chain store, a difference may be provided in the service received by the user in each non-advertisement store. For example, a service corresponding to the distance from the current location of the receiver 8300 when performing visible light communication with the sign 8301 to the non-advertisement store is provided to the user who entered the non-advertisement store Is done. Alternatively, a service corresponding to the difference (time difference) between the time when the receiver 8300 and the sign 8301 perform visible light communication and the time when the user enters the non-advertisement store is provided to the user who enters the non-advertisement store. Is done. That is, the receiver 8300 acquires additional service information indicating a further discount, which differs depending on the above-described distance (the position of the signboard 8301) and the time difference, from the server, and a discount rate (for example, 30) reflecting the further discount. %) Is displayed instead of the pre-acquired image shown in FIG. Such a service is determined by the server or the POS system or by mutual cooperation. In addition, such a service may be applied to all stores belonging to the chain store, regardless of whether the store is an advertisement store or a non-advertisement store.

  When a user enters a non-advertisement store and places an order using advertisement information, the POS system at the non-notification store returns a portion of the amount obtained by the order to the POS system at the advertisement store. May be.

  Further, each time the advertisement information is displayed, the server can determine whether or not the advertisement information has been used, and analyze the advertisement effect of the sign 8301 by accumulating the determined results. Can do. The server further includes at least one of the position of the sign 8301, the time when the advertisement information is displayed, the position of the store where the advertisement information is used, the time when the advertisement information is used, and the time when the user enters the store. By collecting the two, it is possible to improve the analysis accuracy of the advertising effect of the sign 8301 and to find the position of the sign 8301 having the highest advertising effect.

  In addition, the receiver 8300 acquires additional service information indicating a further discount for the number of uses for which the advertisement information has been used for the product order from the server, and a discount rate that reflects the further discount for the number of uses ( For example, an image showing 30%) may be displayed instead of the pre-acquired image shown in FIG. For example, the server may provide a service for further increasing the discount rate in cooperation with the POS system if the number of uses is large.

  In addition, the server receives the advertisement information associated with each of the device IDs of all the signboards 8301 provided by the store by the receiver 8300 (when acquisition of all advertisement information is completed). The profitable service may be provided to the user who enters the store of the signboard 8301. The profitable service is, for example, a service with a very high discount rate or a service that provides products other than ordered products free of charge. That is, when the receiver 8300 detects the user's entry, the server determines whether the receiver 8300 has performed processing including visible light communication or the like for each of all the signboards associated with the store. judge. If it is determined that the processing has been performed, the receiver 8300 acquires additional service information indicating further discount from the server as the above-mentioned profitable service, and a discount rate that reflects the further discount ( For example, an image indicating 50%) is displayed instead of the pre-acquired image illustrated in FIG.

  Further, the receiver 8300 receives additional service information indicating a further discount rate that differs depending on the difference between the time when the advertisement information is displayed by performing visible light communication with the sign 8301 and the time when the user enters the store. An image obtained from the server and showing a discount rate (for example, 30%) reflecting a further discount may be displayed instead of the pre-acquired image shown in FIG. For example, the receiver 8300 acquires additional service information indicating a higher discount rate as the difference is smaller from the server.

  FIG. 91 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The receiver 8300 that has completed the order and the electronic payment receives a signal transmitted by a luminance change from a transmitter configured as a lighting device in the store, and transmits the signal to the server, whereby the seat position of the user ( For example, a guide map in a store indicating a black circle) is acquired. Furthermore, the receiver 8300 specifies the position of the receiver 8300 using the received signal, as in any of the first to third embodiments. Then, the receiver 8300 displays the specified position of the receiver 8300 (for example, an asterisk) in the guide map. Thereby, the user can easily grasp how the user can go to his / her seat in the store.

  In addition, the receiver 8300 identifies the position of the receiver 8300 as described above by performing visible light communication with a nearby transmitter configured as a lighting device in the store even when the user is moving. Is going from time to time. Therefore, the receiver 8300 sequentially updates the position (for example, star) of the displayed receiver 8300. Thereby, a user can be appropriately guided to a seat.

  FIG. 92 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  When the user arrives at the seat, the receiver 8300 identifies the position of the receiver 8300 by performing visible light communication with a transmitter 8303 configured as a lighting device, and determines that the position is at the seat position of the user. To do. Then, the receiver 8300 notifies the terminal in the store that the user has arrived at the seat together with the user name or nickname via the server. Thereby, the salesclerk can grasp which user is sitting in which seat.

  FIG. 93 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The transmitter 8303 transmits a signal including a customer ID and a message notifying that an order item has been made by changing the luminance. Note that the receiver 8300 acquires the above-described customer ID from the server and holds the product ID, for example, when acquiring the product service information indicating the menu of the product from the server. The receiver 8300 receives the above-described signal by photographing the transmitter 8303 with visible light. Furthermore, the receiver 8300 determines whether or not the customer ID included in the signal matches a customer ID that is held in advance. Here, when the receiver 8300 determines that they match, the receiver 8300 displays a message (for example, “Product is ready”) included in the signal.

  FIG. 94 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The clerk who delivered the ordered product to the user's seat points the handy terminal 8302a to the receiver 8300 in order to prove that the ordered product has been delivered. The handy terminal 8302a is configured as a transmitter, and transmits a signal indicating that the ordered product has been delivered to the receiver 8300 by a change in luminance. The receiver 8300 receives the signal by imaging the handy terminal 8302a, and displays a message (for example, “Please enjoy your meal”) indicated by the signal.

<Situation: Finding a store>
Next, an application example in a situation where a user carrying the receiver 8300 is searching for a store of interest will be described with reference to FIGS. 95 to 97.

  FIG. 95 is a diagram illustrating an example of operation of the receiver 8300 in the store search situation.

  The user finds a signage 8304 that lists restaurants of interest. At this time, if the user determines that the signage 8304 is transmitting a signal due to a change in luminance, the communication application of the receiver 8300 is activated by operating the receiver 8300 as in the example shown in FIG. Let Note that, similarly to the example illustrated in FIG. 82, the receiver 8300 may automatically start the communication application.

  FIG. 96 is a diagram illustrating an example of next operation of the receiver 8300 in the store search situation.

  The receiver 8300 obtains an ID for identifying the signage 8304 or the restaurant by imaging the entire signage 8304 or a portion of the signage 8304 where a restaurant in which the user is interested is posted. Receive.

  FIG. 97 is a diagram illustrating an example of next operation of the receiver 8300 in the store search situation.

  When receiving the above-mentioned ID, the receiver 8300 transmits the ID to the server, acquires advertisement information (service information) associated with the ID from the server, and displays it. At this time, the receiver 8300 may notify the server of the number of persons (attached information) scheduled to enter the restaurant together with the ID. Thereby, the receiver 8300 can acquire the advertisement information according to the number of people. For example, the receiver 8300 can acquire advertisement information indicating whether or not there are vacant seats for the notified number of people in the restaurant.

<Situation: Movie advertisement>
Next, application examples in situations where a user carrying the receiver 8300 is in front of a signage on which an interesting movie advertisement is posted will be described with reference to FIGS. 98 to 101.

  FIG. 98 is a diagram illustrating an example of operation of the receiver 8300 in the movie advertisement situation.

  The user finds a signage 8305 on which a movie advertisement of interest is posted and a signage 8306 configured as, for example, a liquid crystal display and displaying a moving image for movie advertisement. The signage 8305 includes, for example, a transparent film on which an image showing a movie advertisement is drawn, and a plurality of LEDs that are arranged on the back side of the film and illuminate the film. That is, the signage 8305 displays the image drawn on the film brightly as a still image by the light emission of the plurality of LEDs. The signage 8305 is configured as a transmitter that transmits a signal by changing luminance.

  Here, if the user determines that the signage 8305 is transmitting a signal due to a change in luminance, the communication application of the receiver 8300 is activated by operating the receiver 8300 as in the example shown in FIG. Let Note that, similarly to the example illustrated in FIG. 82, the receiver 8300 may automatically start the communication application.

  FIG. 99 is a diagram illustrating an example of next operation of the receiver 8300 in the movie advertisement situation.

  The receiver 8300 acquires the ID of the signage 8305 by imaging the signage 8305. Then, the receiver 8300 transmits the ID to the server, downloads the moving image data for movie advertisement associated with the ID as service information from the server, and reproduces it.

  FIG. 100 is a diagram illustrating an example of next operation of the receiver 8300 in the movie advertisement situation.

  The moving image displayed by reproducing the moving image data downloaded as described above is the same as the moving image displayed by the signage 8306, for example. Therefore, when the user wants to see a movie advertisement moving image, the user can view the moving image at an arbitrary place without stopping in front of the signage 8306.

  FIG. 101 is a diagram illustrating an example of next operation of the receiver 8300 in the movie advertisement situation.

  The receiver 8300 may download not only the moving image data but also the moving image data together with the moving image data showing the screening information indicating the movie showing time as service information. Thus, the receiver 8300 can display the contents of the screening information and notify the user, and can share the screening information with other terminals (such as other smartphones).

<Situation: Museum>
Next, an application example in a situation where the user carrying the receiver 8300 enters the museum and appreciates the exhibits in the hall will be described with reference to FIGS.

  FIG. 102 is a diagram illustrating an example of operation of the receiver 8300 in the museum situation.

  For example, when a user tries to enter a museum, the user finds a guidance bulletin board 8307 hung at the entrance of the museum. At this time, if the user determines that the guidance bulletin board 8307 is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to change the communication application of the receiver 8300 as in the example shown in FIG. Start. Note that, similarly to the example illustrated in FIG. 82, the receiver 8300 may automatically start the communication application.

  FIG. 103 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  The receiver 8300 acquires the ID of the guidance bulletin board 8307 by imaging the guidance bulletin board 8307. Then, the receiver 8300 transmits the ID to the server, downloads the museum guidance application program (hereinafter referred to as the museum application) from the server as service information associated with the ID, and activates it.

  FIG. 104 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  When the museum application is activated, the receiver 8300 displays a guide map in the museum according to the museum application. Furthermore, the receiver 8300 specifies the position of the receiver 8300 in the museum as in any of Embodiments 1 to 3. Then, the receiver 8300 displays the specified position of the receiver 8300 (for example, an asterisk) in the guide map.

  In order to specify the position as described above, the receiver 8300 acquires form information indicating the size and shape of the guidance bulletin board 8307 from the server, for example, when downloading a museum application. The receiver 8300 then performs triangulation based on the size and shape of the guidance bulletin board 8307 indicated by the form information and the size and shape of the guidance bulletin board 8307 displayed on the image obtained by the above imaging. The relative position of the receiver 8300 with respect to the guidance bulletin board 8307 is specified in accordance with the above method.

  FIG. 105 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  As described above, when the user enters the museum, the receiver 8300 that has activated the museum application performs visible light communication with a nearby transmitter configured as a lighting device in the museum, and thus the receiver 8300 The position is specified at any time. For example, the receiver 8300 acquires the ID of the transmitter 8308 from the transmitter 8308 by capturing an image of the transmitter 8308 configured as a lighting device. Then, the receiver 8300 acquires position information indicating the position of the transmitter 8308 and form information indicating the size and shape of the transmitter 8308 associated with the ID from the server. The receiver 8300 then performs triangulation based on the size and shape of the transmitter 8308 indicated by the form information and the size and shape of the transmitter 8308 displayed in the image obtained by the above imaging. The relative position of the receiver 8300 with respect to the transmitter 8308 is estimated according to the above method. In addition, the receiver 8300 determines whether the receiver 8300 is in the museum based on the position of the transmitter 8308 indicated by the position information acquired from the server and the relative position of the receiver 8300 estimated as described above. Identify the location.

  Then, every time the position of the receiver 8300 is specified, the receiver 8300 moves the displayed star to the latest specified position. As a result, a user in the museum can easily grasp where he / she is in the museum by looking at the guide map and the star displayed on the receiver 8300.

  FIG. 106 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  When the user who has entered the museum finds an exhibit 8309 of interest, the user performs an operation of holding the receiver 8300 over the exhibit 8309 so that the receiver 8300 can capture an image of the exhibit 8309. Here, the exhibit 8309 is illuminated by light from the lighting device 8310. The lighting device 8310 is used exclusively for the exhibit 8309, and is configured as a transmitter that transmits a signal according to a change in luminance. Therefore, the exhibit 8309 is illuminated by light that changes in luminance, and indirectly transmits a signal from the lighting device 8310.

  When the receiver 8300 detects an operation of holding the receiver 8300 over the exhibit 8309 based on, for example, an output from a built-in 9-axis sensor, the receiver 8300 captures the signal from the lighting device 8310 by imaging the exhibit 8309. Receive. This signal indicates, for example, the ID of the exhibit 8309. Then, the receiver 8300 acquires the introduction information (service information) of the exhibit 8309 associated with the ID from the server. This introduction information shows a diagram for introducing the exhibit 8309, and the introduction text is shown in the language of each country such as Japanese, English, and French.

  When the introduction information is acquired from the server, the receiver 8300 displays a diagram and a text indicated by the introduction information. Here, when displaying a sentence, the receiver 8300 extracts a sentence in a language preset by the user from sentences in the language of each country, and displays only the sentence in that language. The receiver 8300 may change the language by a selection operation by the user.

  FIG. 107 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  When the display of the introduction information figure and the text is terminated by the user's operation, the receiver 8300 again performs visible light communication with a nearby transmitter configured as a lighting device (for example, the lighting device 8311). Then, the position of the receiver 8300 is specified. Then, when the new position of the receiver 8300 is specified, the receiver 8300 moves the displayed star to the specified new position. Accordingly, the user who has viewed the exhibition 8309 can easily move to the next exhibition to view by viewing the guide map and the star displayed on the receiver 8300.

<Situation: Bus stop>
Next, an application example in a situation where the user carrying the receiver 8300 is at a bus stop will be described with reference to FIGS.

  FIG. 108 is a diagram illustrating an example of operation of the receiver 8300 in a bus stop situation.

  The user goes to a bus stop, for example, to get on a bus. Therefore, when the user determines that the sign tower 8312 at the bus stop is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to perform the operation of the receiver 8300 as in the example shown in FIG. Start the communication application. Note that, similarly to the example illustrated in FIG. 82, the receiver 8300 may automatically start the communication application.

  FIG. 109 is a diagram illustrating an example of next operation of the receiver 8300 in the bus stop situation.

  The receiver 8300 images the sign tower 8312 to obtain the ID of the bus stop where the sign tower 8312 is located. Then, the receiver 8300 transmits the ID to the server, and acquires operation status information associated with the ID from the server. The operation status information is information indicating traffic conditions, and is service information indicating services provided to the user.

  Here, the server manages the operation status of these buses by collecting information from each bus operating in the area including the bus stop. Therefore, when the server acquires the ID of the bus stop from the receiver 8300, the server estimates the time until the bus arrives at the bus stop of that ID based on the managed operation status, and the estimated time Is transmitted to the receiver 8300.

  The receiver 8300 that has acquired the operation status information displays the time indicated by the operation status information, for example, “10 minutes after arrival”. Thereby, the user can grasp | ascertain easily the operation condition of a bus | bath.

(Supplement)
When the scanning direction on the imaging side is the vertical direction (vertical direction) of the mobile terminal, when imaging is performed with a short exposure time, the ON / OFF state of the entire LED illumination device is as shown in FIG. In addition, it is possible to image bright lines that are white and black patterns in the same direction as the scanning direction. In (a) of FIG. 110, since the long side direction of the vertically long LED lighting device is imaged so as to be perpendicular to the scanning direction on the imaging side (left and right direction of the portable terminal), the same direction as the scanning direction In addition, a large number of white / black bright lines can be imaged. That is, the amount of information that can be transmitted and received can be increased. On the other hand, as shown in FIG. 110B, when a vertically long LED illumination device is imaged so as to be parallel to the scanning direction on the imaging side (vertical direction of the portable terminal), the white / black pattern that can be imaged There are fewer bright lines. That is, the amount of information that can be transmitted is reduced.

  As described above, when a large number of white and black pattern bright lines can be imaged depending on the orientation of the LED illumination device with respect to the scanning direction on the imaging side (the long side direction of the vertically long LED illumination device is perpendicular to the scanning direction on the imaging side). And a case where only a few white / black pattern bright lines can be imaged (when the long side direction of the vertically long LED illumination device is made parallel to the scanning direction on the imaging side). Arise.

  In the present embodiment, a description will be given of a control method of an illumination device that can image a large number of bright lines even when only a small number of bright lines of white / black patterns can be captured.

  FIG. 111 shows an example of a lighting device in which a plurality of LEDs are arranged in the vertical direction and a drive signal thereof. FIG. 111A shows an illumination device in which a plurality of LEDs are arranged in the vertical direction. It is assumed that each LED element corresponds to a minimum unit of horizontal stripes that encodes a visible light communication signal and corresponds to an encoded ON / OFF signal. In this way, by generating a white / black pattern, turning on / off each LED element, and illuminating, it is assumed that the scanning direction on the imaging side and the long side direction of the vertically long LED illumination device are parallel The white / black pattern of the LED element unit can be photographed.

  111 (c) and (d) show an example in which a white / black pattern is generated and each LED element is turned on / off for illumination. When the illumination device illuminates as a white / black pattern, the light may be uneven even for a short time. Therefore, an example in which an antiphase pattern is generated and alternately illuminated is shown in (c) and (d) of FIG. 111 (c), the element that was turned on is turned off in FIG. 111 (d), and the element that was turned off in FIG. 111 (c) is turned off in FIG. In d), it is ON. In this way, by illuminating the white / black pattern alternately with the positive phase pattern and the reverse phase pattern one after the other, without causing light unevenness, the scanning direction on the imaging side and the illumination A lot of information can be transmitted and received in visible light communication without being affected by the relationship with the orientation of the apparatus. In addition, two types of patterns of a positive phase pattern and an antiphase pattern are alternately generated and illuminated, and it is also conceivable to generate and illuminate three or more types of patterns. FIG. 112 shows an example of sequentially illuminating four types of patterns.

  Normally, a configuration in which the entire LED illumination is blinked ((b) in FIG. 111), a white / black pattern is generated for a predetermined time, and illumination is performed in units of LED elements is also conceivable. For example, the transmission / reception time of a predetermined data unit may blink in the whole LED illumination, and then illuminate a white / black pattern in LED element units in a short time. Here, the predetermined data unit refers to, for example, a data unit from the first header to the next second header. At this time, when the image is taken in the direction of (a) in FIG. 110, a signal is received from the bright line obtained by imaging the blinking of the entire LED illumination, and when received in the direction of (b) in FIG. A signal is received from the light emission pattern.

  In addition, this Embodiment is not limited to an LED lighting apparatus, What kind of element may be sufficient if it is an element which can control ON / OFF by a small element unit like a LED element. Further, the apparatus is not limited to a lighting device, and may be a device such as a television, a projector, or a signage.

  In this embodiment, an example in which illumination is performed with a white / black pattern has been described, but a color may be used instead of the white / black pattern. For example, among RGB, RG may be always lit, and only B may be used and blinked. Using B alone rather than R and G is less likely to be recognized by humans, and flicker can be suppressed. As another example, ON / OFF is displayed by using colors (red and cyan patterns, green and magenta patterns, yellow and blue patterns, etc.) that are complementary colors in additive color mixing instead of white and black patterns. Also good. By using a complementary color in additive color mixing, flicker can be suppressed.

  Further, in the present embodiment, the description has been given by using the example in which the LED elements are arranged one-dimensionally. However, the LED elements are not arranged in one dimension but arranged in two dimensions and displayed like a two-dimensional barcode. May be performed.

(Summary of this embodiment)
The service providing method according to the present embodiment is a service providing method for providing a service to a user of the terminal device using a terminal device including an image sensor having a plurality of exposure lines, and each exposure line of the image sensor. Exposure times of 1/480 seconds or less so that the exposure times of the respective exposure lines are partially overlapped in time with adjacent exposure lines. An image acquisition step of acquiring image data by photographing the subject with a visible light communication for acquiring identification information of the subject by demodulating a bright line pattern corresponding to each of the exposure lines appearing in the image data And a service presentation step of presenting service information associated with the identification information of the subject to the user. .

  As a result, service information related to the subject is presented to the user of the terminal device by utilizing the fact that the subject and the terminal device communicate with each other as a transmitter and a receiver, respectively. It can be provided to the user in various forms. For example, in the service presentation step, information indicating a store advertisement, availability or reservation status related to the subject, information indicating a discount rate of a product or service price, a moving image for movie advertisement, and a screening time , Information for guiding the inside of the building, information for introducing exhibits, and information indicating traffic conditions are presented as the service information.

  The service providing method may further include an identification information transmitting step in which the terminal device transmits identification information of the subject to a server, and the terminal device transmits the service information associated with the identification information of the subject. In the service presenting step, the terminal device may present the obtained service information to the user.

  Accordingly, since the service information can be managed by the server in association with the identification information of the subject, maintenance such as updating of the service information can be facilitated.

  In the identification information transmission step, the accessory information is transmitted to the server together with the subject identification information, and in the service acquisition step, the service information associated with the subject identification information and the accessory information is acquired. May be.

  Accordingly, it is possible to provide a more appropriate service for the user according to the attached information. For example, as in the operation described with reference to FIGS. 84 and 97, in the identification information transmission step, the personal information of the user, the identification information of the user, the number information indicating the number of groups including the user, or Position information indicating the position of the terminal device is transmitted as the attached information.

  The service providing method may further include a position transmission step in which the terminal apparatus transmits position information indicating the position of the terminal apparatus to a server, and at least one in a predetermined range including the position indicated by the position information. A pre-acquisition step in which the terminal device acquires and holds from the server at least one service information associated with each of the identification information, and the service presentation step includes: The terminal device selects service information associated with the identification information of the subject from the at least one service information held in the pre-article acquisition step, and presents the service information to the user. Also good.

  Thus, for example, when the terminal device acquires the identification information of the subject as in the operation described with reference to FIG. 82, the terminal device does not communicate with the server or the like and then stores at least one service information held in advance. Therefore, the service information associated with the identification information of the subject can be acquired and presented. Therefore, the service provision can be speeded up.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. When the entry determination step determines that the user has entered the store, the terminal device acquires product service information related to the store product or service from the server and presents it to the user. A presentation step.

  Thereby, for example, as in the operation described with reference to FIGS. 86 to 90, when the user enters the store, the store menu or the like can be automatically presented to the user as the merchandise service information. Therefore, the store clerk does not need to present a menu or the like to the user, and the user can easily make an order to the store.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. And when it is determined in the store entry step that the user has entered the store, the location of the subject and the time of the store that differs depending on at least one of the time when the service information was presented An additional service presenting step in which the terminal device presents the additional service information to the user may be included.

  Thus, for example, as the processing described with reference to FIGS. 86 to 90, the closer the subject is to the store where the user entered, or the time when the user entered the store and the time when the service information was presented (or Service information that is more useful to the user can be presented to the user as additional service information as the time of shooting of the subject is closer. Specifically, each of a plurality of stores belonging to the chain store is a store corresponding to the service information presented in the service presentation step, and a signboard that is a subject by one of the stores (advertisement store) May have been issued. In such a case, the advertising store among the plurality of stores belonging to the chain store is typically located closest to the subject (signboard). Therefore, the closer the subject is to the store where the user entered, or the closer the time when the user entered the store and the time when the service information was presented, the more likely the store where the user entered was an advertising store. . Therefore, when there is a high possibility that the user has entered the advertising store, service information more useful to the user can be presented to the user as additional service information.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. When the store entry step determines that the user has entered the store, additional services of the store differ according to the number of times the user has used the service indicated by the service information at the store. An additional service presenting step in which the terminal device presents the information to the user may be included.

  As a result, service information that is more useful to the user can be presented to the user as additional service information as the number of times the service is used, for example, as described with reference to FIGS. For example, if the usage count of service information indicating a 20% discount on the product price exceeds a threshold, additional service information indicating a further discount of 10% can be presented to the user.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. When the entry determination step determines that the user has entered the store, the image acquisition step is performed for each of all other subjects associated with the store other than the subject. A complete determination step for determining whether or not processing including the visible light communication step and the service presentation step has been performed; and when it is determined in the complete determination step that the processing has been performed, An additional service presenting step in which the terminal device presents the service information to the user. Good.

  Thereby, for example, as in the operation described with reference to FIGS. 86 to 90, for example, the store puts out several subjects as signboards, and the image acquisition step, the visible light communication step, and the service presentation for all of the signboards. When the step is performed, service information most useful to the user can be presented to the user as additional service information.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. And when it is determined in the store entry determining step that the user has entered the store, the store differs according to a difference between a time at which the service information is presented and a time at which the user enters the store. The additional service information step may be included in which the terminal device presents the additional service information to the user.

  Accordingly, for example, as in the operation described with reference to FIGS. 86 to 90, the difference between the time when the service information is presented (or the time when the subject is photographed) and the time when the user enters the store is The smaller the service information, the more useful service information for the user can be presented to the user as additional service information. That is, a more useful service can be additionally provided to a user who takes a short time from entering the store after receiving service information by imaging the subject.

  The service providing method further includes a use determination step of determining whether or not the user has used the service indicated by the service information at a store corresponding to the service information presented in the service presentation step; Each time the service information is presented, the results determined in the use determination step may be accumulated, and an analysis step of analyzing the advertising effect of the subject based on the accumulated contents may be included.

  For example, when a service such as a 20% discount on the product price is indicated in the service information as in the operation described with reference to FIGS. 86 to 90, it is determined whether or not the service is used by, for example, electronic payment. Is done. In other words, every time a service is provided to the user at the time of imaging a subject, it is determined whether or not that service has been used. As a result, for example, when it is often determined that the subject has been used, it can be analyzed that the advertising effect of the subject is high. That is, the advertising effect of the subject can be appropriately analyzed based on the use result.

  In the analyzing step, together with the result determined in the use determining step, the position of the subject, the time when the service information is presented, the position of the store, and the time when the user enters the store The advertisement effect of the subject may be analyzed based on the accumulated contents.

  Thereby, the advertising effect of the subject can be analyzed in more detail. For example, when the position of the subject is changed, the advertising effect before and after the position change can be compared, and as a result, the subject can be placed at a position with a high advertising effect.

  The service providing method further includes a use determination step of determining whether or not the user has used the service indicated by the service information at a store corresponding to the service information presented in the service presentation step; When it is determined that the service is used in the use determination step, a store determination step of determining whether a use store that is a store where the service is used is a specific store associated with the subject; When it is determined in the store determination step that the use store is not the specific store, at least a part of the amount settled using the service at the use store is returned to the specific store using electronic commerce. A reduction step.

  Thereby, for example, even when the service is not used at a specific store (for example, an advertising store that has a signboard as a subject) as in the operation described with reference to FIGS. For example, a profit can be obtained as a price for installing a signboard as a subject.

  In the service presentation step, the terminal device presents the service information for introducing the subject to the user when the subject illuminated by light whose luminance changes in the image acquisition step is photographed. When the lighting device that changes in luminance is photographed as the subject in the image acquisition step, the terminal device presents the service information for guiding the inside of the building where the subject is arranged to the user. Also good.

  Accordingly, for example, as in the operation described with reference to FIGS. 105 and 106, it is possible to appropriately provide a user with a guidance service in a museum or the like and an introduction service for an exhibit that is a subject.

  The information communication method according to the present embodiment is an information communication method for acquiring information from a subject having a plurality of light emitting elements, and is included in the image sensor in an image obtained by photographing the subject with an image sensor. The exposure time setting step for setting the exposure time of the image sensor so that the bright line corresponding to the exposure line is generated according to the luminance change of the subject, and the plurality of the luminance lines according to the luminance change pattern for indicating the first information. An image acquisition step of acquiring a bright line image, which is an image including the bright lines, by the image sensor capturing the subject whose luminance changes in the same manner in all of the light emitting elements in the image sensor for the set exposure time; By demodulating the data specified by the bright line pattern included in the bright line image, A first information acquisition step of acquiring first information and each of the plurality of light emitting elements is obtained by photographing the subject that emits light with one of two different luminance values. And a second information acquisition step of acquiring second information by demodulating data indicated by an array of brightness intensities along a direction parallel to the exposure line, which is indicated in the image.

  Alternatively, the information communication method according to the present embodiment is an information communication method for transmitting a signal by a luminance change, and determining a pattern of the luminance change by modulating the first signal to be transmitted; A first transmission step of transmitting the first signal when all of the plurality of light emitting elements included in the light emitter change in luminance in the same manner according to the determined luminance change pattern, and the plurality of light emission Each of the elements emits light with one of the two different brightness values, so that a light brightness / darkness array appears in the space in which the light emitters are arranged, whereby the second transmission target. And a second transmission step of transmitting the signal.

  Thereby, for example, as in the operation described with reference to FIGS. 110 to 112, even if the illumination device that is a subject or a light emitter is an elongated device including a plurality of LEDs arranged in a row, the receiver Regardless of the direction of shooting, information or signals from the lighting device can be appropriately acquired. In other words, when the exposure line (the operation direction on the imaging side) of the image sensor provided in the receiver and the arrangement direction of the plurality of LEDs are not parallel, the receiver detects information from the luminance change of the entire lighting device. Or a signal can be acquired appropriately. Further, even when the exposure line and the above-described arrangement direction are parallel to each other, the receiver can appropriately acquire information or a signal from the bright and dark arrangement of luminance along the direction parallel to the exposure line. it can. In other words, the dependence of the imaging direction on the reception of information can be suppressed.

(Embodiment 5)
In this embodiment, an application example using a receiver such as a smartphone in Embodiments 1 to 4 and a transmitter that transmits information as a blinking pattern such as an LED or an organic EL will be described.

  FIG. 113 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

  Each of transmitter 8321, transmitter 8322, and transmitter 8323 has the same function as that of any of the transmitters in Embodiments 1 to 4, and is configured as a lighting device that transmits a signal (changes in visible light communication) by luminance change. Has been. These transmitters 8321 to 8323 transmit signals by changing in luminance at different frequencies. For example, the transmitter 8321 transmits the ID “1000” of the transmitter 8321 by changing the luminance at the frequency a (eg, 9200 Hz), and the transmitter 8322 changes the luminance at the frequency b (eg, 9600 Hz). The transmitter 8322 transmits the ID “2000”, and the transmitter 8323 transmits the ID “3000” of the transmitter 8322 by changing the luminance at the frequency c (for example, 10000 Hz).

  The receiver images (transmits visible light) the transmitters 8321 to 8323 in the same manner as in the first to fourth embodiments so that these transmitters 8321 to 8323 are all included in the angle of view. In the image obtained by the imaging, a bright line pattern corresponding to each transmitter appears. From the bright line pattern, the frequency of the luminance change of the transmitter corresponding to the bright line pattern can be specified.

  Here, if the frequencies of the transmitters 8321 to 8323 are the same, the frequencies specified from the bright line patterns corresponding to the transmitters are also the same. Furthermore, when the bright line patterns are adjacent to each other, the frequencies specified from the bright line patterns are the same, so it is difficult to distinguish the bright line patterns.

  Therefore, as described above, when the transmitters 8321 to 8323 change in luminance at different frequencies, the receiver can easily distinguish these bright line patterns, and the data specified by each bright line pattern can be identified. By demodulating, the IDs of the transmitters 8321 to 8323 can be appropriately acquired. That is, the receiver can appropriately distinguish the signals from the transmitters 8321 to 8323.

  In addition, each frequency of the transmitters 8321 to 8323 may be set by a remote controller or may be set at random. In addition, each of the transmitters 8321 to 8323 may communicate with an adjacent transmitter and automatically set the frequency of its own transmitter so as to be different from the frequency of the adjacent communication device.

  114 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  In the above example, each of the transmitters changes in luminance at a different frequency. However, when there are five or more transmitters, each of the transmitters may not change in luminance at different frequencies. That is, each of the five or more transmitters only needs to change in luminance at any one of the four types of frequencies.

  For example, as shown in FIG. 114, even in a situation where bright line patterns (rectangular regions in FIG. 114) corresponding to each of five or more transmitters are adjacent to each other, the frequency types are the same as the number of transmitters. If there are four types (frequency a, b, c, d), the frequencies of the bright line patterns adjacent to each other can be reliably varied. This is reasoned by the four-color problem or the four-color theorem.

  That is, in this embodiment, each of the plurality of transmitters changes in luminance at any one of at least four types of frequencies, and two or more light emitters of the plurality of transmitters are the same. The luminance changes at the frequency of. In addition, when a plurality of transmitters are projected on the light receiving surface of the image sensor of the receiver, the luminance changes between all transmitters (bright line patterns that are images of the transmitter) adjacent to each other on the light receiving surface. Each of the plurality of transmitters changes in luminance so that their frequencies are different.

  FIG. 115 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

  The transmitter changes in luminance by outputting high-luminance light (H) or low-luminance light (L) for each predetermined time unit (slot), and transmits a signal according to the luminance change. Here, the transmitter transmits a signal for each block including a header and a body. The header is expressed as (L, H, L, H, L, H, H) using, for example, 7 slots as shown in FIG. 79A. The body is composed of a plurality of symbols (00, 01, 10 or 11), and each symbol is expressed using 4 slots (4-value PPM). A block is expressed using a predetermined number of slots (19 in the example of FIG. 115). The ID is obtained by combining bodies included in each of four blocks, for example. A block may be expressed using 33 slots.

  The bright line pattern obtained by imaging by the receiver includes a pattern (header pattern) corresponding to the header and a pattern (data pattern) corresponding to the body. The data pattern does not include the same pattern as the header pattern. Therefore, the receiver can easily find the header pattern from the bright line pattern, and can measure the number of pixels (the number of exposure lines corresponding to the block) between the header pattern and the next header pattern. In the receiver, the number of slots in one block (19 in the example of FIG. 115) is set to a fixed number regardless of the frequency, and therefore the frequency of the transmitter (the time of one slot is determined according to the measured number of pixels. The reciprocal of the width) can be specified. That is, the receiver specifies a lower frequency as the number of pixels is larger, and identifies a higher frequency as the number of pixels is smaller.

  Thus, the receiver can acquire the ID of the transmitter and identify the frequency of the transmitter by imaging the transmitter. Here, the receiver can determine whether or not the acquired ID is appropriate by using the frequency specified in this way, that is, can perform error detection of the ID. Specifically, the receiver calculates a hash value for the ID, and compares the hash value with the identified frequency. When the hash value and the frequency match, the receiver determines that the acquired ID is appropriate, and when it does not match, the acquired ID is inappropriate (error). Judge that there is. For example, the receiver treats the remainder obtained by dividing the ID by a predetermined divisor as a hash value. In other words, the transmitter transmits the ID of the transmission target by changing the luminance at the same frequency as the hash value for the transmission target ID (the reciprocal of the time width of one slot).

  116 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG.

  The transmitter may use an arbitrary frequency instead of using the same frequency as the hash value as described above, and change the luminance at the arbitrary frequency. In this case, the transmitter transmits a signal indicating a value different from the ID of the transmitter. For example, when the ID of the transmitter is “100” and the transmitter uses 2 kHz as an arbitrary frequency, the transmitter transmits a signal “1002” in which the ID and the frequency are combined. Similarly, when the ID of another transmitter is “110” and the other transmitter uses 1 kHz as an arbitrary frequency, the other transmitter uses a signal “1101” that is a combination of the ID and the frequency. Send.

  In this case, the receiver uses the first digit of the signal transmitted and acquired from the transmitter for error detection, and extracts the remaining digits as the transmitter ID. Then, the receiver compares the frequency specified from the luminance pattern with the numerical value of the lower first digit included in the acquired signal. The receiver determines that the extracted ID is appropriate when the numerical value of the first digit matches the frequency, and determines that the extracted ID is inappropriate when the frequency does not match. (Error) is determined.

  As a result, it is possible to increase the degree of freedom in setting the frequency of the luminance change in the transmitter while enabling error detection in the receiver.

  117 is a diagram illustrating an example of operation of a receiver in Embodiment 5. FIG.

  As shown in FIG. 117, in an image obtained by imaging (visible light imaging) with a receiver, a part of each of the bright line pattern 8327a and the bright line pattern 8327b may overlap. In such a case, the receiver does not demodulate data from the portion 8327c where the bright line pattern 8327a and the bright line pattern 8327b overlap, and demodulates data from portions other than the respective portions 8327c of the bright line pattern 8327a and the bright line pattern 8327b. Do. Thereby, the receiver can acquire an appropriate ID from each of the bright line pattern 8327a and the bright line pattern 8327b.

  118 is a diagram illustrating an example of operation of a receiver in Embodiment 5. FIG.

  As shown in (a) of FIG. 118, the transmitter switches the frequency of the luminance change for transmitting each block, for example, for each block. As a result, the receiver can more easily discriminate between the blocks.

  Also, as shown in FIG. 118 (b), for example, the transmitter makes the frequency of the luminance change for transmitting the header of the block different from the frequency of the luminance change for transmitting the body of the block. Thereby, it can suppress that the same pattern as a header appears in a body. As a result, the receiver can more appropriately determine the distinction between the header and the body.

  FIG. 119 is a diagram illustrating an example of operation of a system including a transmitter, a receiver, and a server in Embodiment 5.

  The system in this embodiment includes a transmitter 8331, a receiver 8332, and a server 8333. The transmitter 8331 has the same function as that of any of the transmitters of Embodiments 1 to 4, and is configured as an illumination device that transmits the ID of the transmitter 8331 by changing the luminance (visible light communication). The receiver 8332 has the function as the receiver in any of Embodiments 1 to 4 above, and acquires the ID of the transmitter 8331 from the transmitter 8331 by imaging the transmitter 8331 (visible light imaging). To do. The server 8333 communicates with the transmitter 8331 and the receiver 8332 via a network such as the Internet.

  In the present embodiment, the ID of transmitter 8331 is fixed without being changed. On the other hand, the frequency used for the luminance change (visible light communication) of the transmitter 8331 can be arbitrarily changed by setting.

  In such a system, the transmitter 8331 first registers the frequency used in the luminance change (visible light communication) with the server 8333. Specifically, transmitter 8331 transmits its ID, registered frequency information indicating the frequency, and related information related to transmitter 8331 to server 8333. When the server 8333 receives the ID of the transmitter 8331, the registered frequency information, and the related information, the server 8333 records them in association with each other. That is, the ID of the transmitter 8331, the frequency used for the luminance change of the transmitter 8331, and the related information are recorded in association with each other. Thereby, the frequency used for the luminance change of the transmitter 8331 is registered.

  Next, the transmitter 8331 transmits the ID of the transmitter 8331 according to the luminance change of the registered frequency. The receiver 8332 acquires the ID by imaging the transmitter 8331 and specifies the frequency of the luminance change of the transmitter 8331 as described above.

  Next, the receiver 8332 transmits the acquired ID and specific frequency information indicating the specified frequency to the server 8333. When the server 8333 receives the ID and specific frequency information transmitted by the receiver 8332, the server 8333 searches for a frequency (frequency indicated by the registered frequency information) recorded in association with the ID, and the recorded frequency and Then, it is determined whether or not the frequency indicated by the specific frequency information matches. If it is determined that they match, the server 8333 transmits related information (data) recorded in association with the ID and frequency to the receiver 8332.

  Accordingly, if the frequency specified by the receiver 8332 does not match the frequency registered in the server 8333, the related information is not transmitted from the server 8333 to the receiver 8332. Therefore, if the receiver 8332 obtains the ID from the transmitter 8331 even once, the frequency registered in the server 8333 indicates that the receiver 8332 can receive related information from the server 8333 at any time. Can be prevented by changing from time to time. That is, the transmitter 8331 can prohibit the acquisition of related information by the receiver 8332 that has acquired the ID before the change by changing the frequency registered in the server 8333 (that is, the frequency used for the luminance change). . In other words, an expiration date can be set for acquisition of related information by changing the frequency. For example, when the user of the receiver 8332 stays at a hotel where the transmitter 8331 is installed, the hotel administrator changes the frequency after staying. Accordingly, the receiver 8332 can acquire related information only on the day the user stays, and can prohibit the receiver 8332 from acquiring related information after staying.

  Note that the server 8333 may register a plurality of frequencies associated with one ID. For example, each time the server 8333 acquires the registered frequency information from the receiver 8332, the server 8333 always registers the frequency indicated by the latest four registered frequency information in association with the ID. Thereby, even if the receiver 8332 has acquired the ID in the past, the related information can be acquired from the server 8333 until the frequency is changed three times. Further, the server 8333 may manage the time or the time zone when the frequency is set in the transmitter 8331 for each registered frequency. In this case, when the server 8333 receives the ID and specific frequency information from the receiver 8332, the server 8333 confirms the time zone managed for the frequency indicated by the specific frequency information, and the receiver 8333 The time zone when 8332 acquired the ID can be specified.

  120 is a block diagram illustrating a configuration of a transmitter in Embodiment 5. In FIG.

  The transmitter 8334 has the same function as that of any of the transmitters of Embodiments 1 to 4, and includes a frequency storage unit 8335, an ID storage unit 8336, a check value storage unit 8337, a check value comparison unit 8338, a check value. A calculation unit 8339, a frequency calculation unit 8340, a frequency comparison unit 8341, a transmission unit 8342, and an error notification unit 8343 are provided.

  The frequency storage unit 8335 stores a frequency used for luminance change (visible light communication). The ID storage unit 8336 stores the ID of the transmitter 8334. The check value storage unit 8337 stores a check value for determining whether the ID stored in the ID storage unit 8336 is appropriate.

  The check value calculation unit 8339 reads the ID stored in the ID storage unit 8336 and applies a predetermined function to the ID to calculate a check value (calculation check value) for the ID. The check value comparison unit 8338 reads the check value stored in the check value storage unit 8337, and compares the check value with the calculated check value calculated by the check value calculation unit 8339. If the check value comparison unit 8338 determines that the calculated check value is different from the check value, the check value comparison unit 8338 notifies the error notification unit 8343 of an error. For example, the check value storage unit 8337 stores a value “0” indicating that the ID stored in the ID storage unit 8336 is an even number as a check value. The check value calculation unit 8339 reads the ID stored in the ID storage unit 8336 and divides it by the value “2”, thereby calculating the remainder as a calculation check value. Then, the check value comparison unit 8338 compares the check value “0” with the calculated check value that is the remainder of the above division.

  The frequency calculation unit 8340 reads the ID stored in the ID storage unit 8336 via the check value calculation unit 8339 and calculates a frequency (calculated frequency) from the ID. For example, the frequency calculation unit 8340 calculates the remainder as a frequency by dividing the ID by a predetermined value. The frequency comparison unit 8341 compares the frequency (storage frequency) stored in the frequency storage unit 8335 with the calculated frequency. If the frequency comparison unit 8341 determines that the calculated frequency is different from the storage frequency, the frequency comparison unit 8341 notifies the error notification unit 8343 of an error.

  The transmission unit 8342 transmits the ID stored in the ID storage unit 8336 by changing the luminance at the calculated frequency calculated by the frequency calculation unit 8340.

  When an error is notified from at least one of the check value comparison unit 8338 and the frequency comparison unit 8341, the error notification unit 8343 notifies the error by a buzzer sound, blinking, or lighting. Specifically, the error notification unit 8343 includes a lamp for error notification, and notifies the error by lighting or blinking the lamp. Alternatively, when the power switch of the transmitter 8334 is turned ON, the error notification unit 8343 sets a light source that changes in luminance to transmit a signal such as an ID for a predetermined period (for example, 10 seconds). An error is notified by blinking at a recognizable cycle.

  Thereby, since it is checked whether the ID stored in the ID storage unit 8336 and the frequency calculated from the ID are appropriate, transmission of an incorrect ID and a change in luminance at the wrong frequency Can be prevented.

  FIG. 121 is a block diagram illustrating a configuration of a receiver in Embodiment 5. In FIG.

  The receiver 8344 has the same function as the receiver of any of Embodiments 1 to 4 above, and also includes a light receiving unit 8345, a frequency detecting unit 8346, an ID detecting unit 8347, a frequency comparing unit 8348, and a frequency calculating unit 8349. It has.

  The light receiving unit 8345 includes, for example, an image sensor, and acquires an image including a bright line pattern by imaging (visible light imaging) a transmitter that changes in luminance. The ID detection unit 8347 detects the transmitter ID from the image. That is, the ID detection unit 8347 acquires the ID of the transmitter by demodulating data specified by the bright line pattern included in the image. The frequency detector 8346 detects the frequency of the luminance change of the transmitter from the image. That is, the frequency detection unit 8346 specifies the frequency of the transmitter from the bright line pattern included in the image as in the example described with reference to FIG.

  The frequency calculation unit 8349 calculates the frequency of the transmitter by dividing the ID from the ID detected by the ID detection unit 8347, for example, as described above. The frequency comparison unit 8348 compares the frequency detected by the frequency detection unit 8346 with the frequency calculated by the frequency calculation unit 8349. Here, if these frequencies are different, the frequency comparison unit 8348 determines that the detected ID is an error, and causes the ID detection unit 8347 to detect the ID again. Thereby, it can prevent acquiring wrong ID.

  122 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter may distinguish and transmit each of the symbols “00, 01, 10, 11” by changing the position where the luminance change occurs in a predetermined time unit.

  For example, when transmitting the symbol “00”, the transmitter transmits the symbol “00” by changing the luminance only in the first interval which is the first interval in the time unit. Further, when transmitting the symbol “01”, the transmitter transmits the symbol “01” by changing the luminance only in the second interval which is the second interval in the time unit. Similarly, when transmitting the symbol “10”, the transmitter transmits the symbol “10” and transmits the symbol “11” by changing the luminance only in the third interval which is the third interval in the time unit. When transmitting, the transmitter transmits the symbol “11” by changing the luminance only in the fourth interval, which is the fourth interval in the time unit.

  As described above, in this embodiment, when any symbol is transmitted, the luminance changes within one section, so that it is lower than the above-described transmitter that reduces the entire luminance of one section (slot). , Flicker can be suppressed.

  FIG. 123 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

  The transmitter may distinguish and transmit each of the symbols “0, 1” by changing the presence / absence of a luminance change in a predetermined time unit. For example, when transmitting the symbol “0”, the transmitter transmits the symbol “0” by not changing the luminance in time units. Further, when transmitting the symbol “1”, the transmitter transmits the symbol “1” by changing the luminance in time units.

  124 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter may distinguish and transmit each of the symbols “00, 01, 10, 11” by changing the intensity (frequency) of the luminance change in a predetermined time unit. For example, when transmitting the symbol “00”, the transmitter transmits the symbol “00” by not changing the luminance in time units. Further, when transmitting the symbol “01”, the transmitter transmits the symbol “01” by changing the luminance in a time unit (changing the luminance at a low frequency). Further, when transmitting the symbol “10”, the transmitter transmits the symbol “10” when the luminance changes drastically in a time unit (luminance changes at a high frequency). Then, when transmitting the symbol “11”, the transmitter transmits the symbol “11” by changing the luminance more intensely in time units (by changing the luminance at a higher frequency).

  125 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter may distinguish and transmit each of the symbols “0, 1” by changing the phase of the luminance change in a predetermined time unit. For example, when transmitting the symbol “0”, the transmitter transmits the symbol “0” by changing the luminance at a predetermined phase in a time unit. Further, when transmitting the symbol “1”, the transmitter transmits the symbol “1” by changing the luminance in a phase opposite to the above phase in a time unit.

  FIG. 126 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

  When transmitting a signal such as an ID, the transmitter may change the luminance for each color such as red, green, and blue. Thus, the transmitter can transmit a signal having a larger amount of information to a receiver that can recognize a luminance change for each color. Further, the luminance change of any color may be used for clock adjustment. For example, a red luminance change may be used for clock adjustment. In this case, the red luminance change serves as a header. As a result, it is not necessary to use a header for luminance changes of colors other than red (green and blue), and redundant data transmission can be suppressed.

  127 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter may express the luminance of the combined color (for example, white) by combining a plurality of colors such as red, green, and blue. That is, the transmitter expresses the luminance change of the composite color by changing the luminance for each color such as red, green, and blue. A signal to be transmitted is transmitted by the luminance change of the composite color, similarly to the above-described visible light communication. Here, the luminance of one or more of red, green, and blue may be used for adjustment for expressing a predetermined luminance of the composite color. Accordingly, a signal can be transmitted by a luminance change of the composite color, and a signal can be transmitted by a luminance change of any two colors of red, green, and blue. In other words, the transmitter can transmit a signal to a receiver that can recognize only the luminance change of the composite color (for example, white) as described above, and can also recognize each color such as red, green, and blue. For such a receiver, more signals can be transmitted as incidental information, for example.

  128 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter has four light sources. Each of these four light sources (for example, LED lamps) emits light of colors represented by different positions 8351a, 8351b, 8352a, and 8352b in the CIExy chromaticity diagram shown in FIG.

  The transmitter transmits each signal by switching between the first lighting transmission and the second lighting transmission. Here, the first lighting transmission is a process of transmitting a signal “0” by turning on a light source that emits light of the color at position 8351a and a light source that emits light of the color at position 8351b among the four light sources. It is. The second lighting transmission is a process of transmitting the signal “1” by turning on the light source that emits the light of the color at the position 8352a and the light source that emits the light of the color at the position 8352b. The image sensor of the receiver can identify the colors represented by each of the positions 8351a, 8351b, 8352a, 8352b, so that the receiver can properly receive the signal “0” and the signal “1”. it can.

  Here, when the first lighting transmission is performed, the color represented by the position in the middle of the positions 8351a and 8351b in the CIExy chromaticity diagram is seen by human eyes. Similarly, when the second lighting transmission is performed, a color represented by a position in the middle of the positions 8352a and 8352b in the CIExy chromaticity diagram is seen by human eyes. Accordingly, by appropriately adjusting the colors and luminances of the four light sources, the position between the positions 8351a and 8351b can be matched with the position between the positions 8352a and 8352b (to the position 8353). As a result, even if the first lighting transmission and the second lighting transmission are switched, it appears to the human eye that the light emission color of the transmitter is fixed, so that it is possible to suppress the human from feeling flickering. .

  FIG. 129 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5.

  The transmitter includes an ID storage unit 8361, a random number generation unit 8362, an addition unit 8363, an encryption unit 8364, and a transmission unit 8365. The ID storage unit 8361 stores the ID of the transmitter. The random number generation unit 8362 generates different random numbers every certain time. Adder 8363 combines the latest random number generated by random number generator 8362 with the ID stored in ID storage unit 8361, and outputs the result as an edit ID. The encryption unit 8364 generates an encrypted edit ID by encrypting the edit ID. The transmission unit 8365 transmits the encrypted edit ID to the receiver by changing the luminance.

  The receiver includes a receiving unit 8366, a decoding unit 8367, and an ID acquisition unit 8368. The receiving unit 8366 receives the encrypted edit ID from the transmitter by imaging the transmitter (visible light imaging). The decryption unit 8367 restores the edit ID by decrypting the received encrypted edit ID. The ID acquisition unit 8368 acquires the ID by extracting the ID from the restored editing ID.

  For example, the ID storage unit 8361 stores the ID “100”, and the random number generation unit 8362 generates the latest random number “817” (Example 1). In this case, the adding unit 8363 generates and outputs the edit ID “100817” by combining the random number “817” with the ID “100”. The encryption unit 8364 generates an encrypted edit ID “abced” by encrypting the edit ID “100817”. The decryption unit 8367 of the receiver restores the edit ID “100817” by decrypting the encrypted edit ID “abced”. Then, the ID acquisition unit 8368 extracts the ID “100” from the restored editing ID “100817”. In other words, the ID acquisition unit 8368 acquires the ID “100” by deleting the last three digits of the edit ID.

  Next, the random number generation unit 8362 generates a new random number “619” (example 2). In this case, the adding unit 8363 generates and outputs the edit ID “100619” by combining the random number “619” with the ID “100”. The encryption unit 8364 generates an encrypted edit ID “diffia” by encrypting the edit ID “100619”. The decryption unit 8367 of the transmitter restores the edit ID “100619” by decrypting the encrypted edit ID “diffia”. Then, the ID acquisition unit 8368 extracts the ID “100” from the restored editing ID “100619”. In other words, the ID acquisition unit 8368 acquires the ID “100” by deleting the last three digits of the edit ID.

  In this way, the transmitter simply encrypts a combination of random numbers that change every fixed time without simply encrypting the ID, so the ID can be easily decrypted from the signal transmitted from the transmitter 8365. Can be prevented. In other words, when a simple encrypted ID is transmitted from the transmitter to the receiver several times, even if the ID is encrypted, if the ID is the same, the transmitter to the receiver. Since the transmitted signal is the same, the ID may be deciphered. However, in the example shown in FIG. 129, a random number that is changed at regular intervals is combined with the ID, and the ID combined with the random number is encrypted. Therefore, even when the same ID is transmitted to the receiver several times, the signals transmitted from the transmitter to the receiver can be made different if the transmission timings of these IDs are different. As a result, it is possible to prevent the ID from being easily decoded.

  FIG. 130 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5.

  The transmitter includes an ID storage unit 8371, a timing unit 8372, an addition unit 8373, an encryption unit 8374, and a transmission unit 8375. The ID storage unit 8371 stores the ID of the transmitter. The timer unit 8372 keeps time and outputs the current date and time (current date and time). The adding unit 8373 combines the ID stored in the ID storage unit 8371 with the current date and time output from the time measuring unit 8372 as the transmission date and time, and outputs the result as the edit ID. The encryption unit 8374 generates an encrypted edit ID by encrypting the edit ID. The transmission unit 8375 transmits the encrypted edit ID to the receiver by changing the luminance.

  The receiver includes a receiving unit 8376, a decoding unit 8377, a validity determining unit 8378, and a time measuring unit 8379. The receiving unit 8376 receives the encrypted editing ID from the transmitter by imaging the transmitter (visible light imaging). The decryption unit 8377 restores the edit ID by decrypting the received encrypted edit ID. The timer unit 8379 measures time and outputs the current date and time (current date and time). The validity determination unit 8378 acquires the ID by extracting the ID from the restored editing ID. Further, the validity determination unit 8378 extracts the transmission date / time from the restored edit ID, and compares the transmission date / time with the current date / time output from the time measuring unit 8379 to determine the validity of the ID. For example, the validity determination unit 8378 determines that the ID is invalid when the difference between the transmission date and time and the current date and time is longer than a predetermined time, or when the transmission date and time is newer than the current date and time. .

  For example, the ID storage unit 8371 stores the ID “100”, and the time measuring unit 8372 outputs the current date and time “201305011200” (May 1, 2013 12:00) as the transmission date and time (Example 1). In this case, the adding unit 8373 generates and outputs the edit ID “100201305011200” by combining the transmission date “2013030501200” with the ID “100”. The encryption unit 8374 generates an encrypted edit ID “ei39ks” by encrypting the edit ID “100201305011200”. The decryption unit 8377 of the receiver restores the edit ID “100201305011200” by decrypting the encrypted edit ID “ei39ks”. Then, the validity determination unit 8378 extracts the ID “100” from the restored edit ID “100201305011200”. In other words, the validity determination unit 8378 acquires the ID “100” by deleting the last 12 digits of the edit ID. Further, the validity determination unit 8378 extracts the transmission date and time “201305011200” from the restored editing ID “100201305011200”. If the transmission date and time “201305011200” is older than the current date and time output from the timing unit 8379 and the difference between the transmission date and time and the current date and time is within 10 minutes, for example, the validity determination unit 8378 has the ID “100 Is determined to be valid.

  On the other hand, the ID storage unit 8371 stores the ID “100”, and the time measuring unit 8372 outputs the current date and time “201401101730” (January 1, 2014, 17:30) as the transmission date (Example 2). In this case, the adding unit 8373 generates and outputs the edit ID “100201401101730” by combining the transmission date and time “201401101730” with the ID “100”. The encryption unit 8374 generates the encrypted edit ID “002jflk” by encrypting the edit ID “100201401411730”. The decryption unit 8377 of the receiver restores the edit ID “100201401101730” by decrypting the encrypted edit ID “002jflk”. Then, the validity determination unit 8378 extracts the ID “100” from the restored editing ID “1002014014101730”. In other words, the validity determination unit 8378 acquires the ID “100” by deleting the last 12 digits of the edit ID. Further, the validity determination unit 8378 extracts the transmission date and time “201401101730” from the restored editing ID “100201401101730”. Then, the validity determination unit 8378 determines that the ID “100” is invalid if the transmission date and time “201401101730” is newer than the current date and time output from the time measuring unit 8379.

  In this way, the transmitter encrypts a combination of the current date and time that is changed at regular intervals without simply encrypting the ID, so that the ID can be easily obtained from the signal transmitted from the transmission unit 8375. It can be prevented from being deciphered. In other words, when a simple encrypted ID is transmitted from the transmitter to the receiver several times, even if the ID is encrypted, if the ID is the same, the transmitter to the receiver. Since the transmitted signal is the same, the ID may be deciphered. However, in the example shown in FIG. 130, the current date and time that are changed at regular intervals are combined with the ID, and the ID that is combined with the current date and time is encrypted. Therefore, even when the same ID is transmitted to the receiver several times, the signals transmitted from the transmitter to the receiver can be made different if the transmission timings of these IDs are different. As a result, it is possible to prevent the ID from being easily decoded.

  Furthermore, since it is determined whether or not the acquired ID is valid by comparing the transmission date and time when the encrypted edit ID is transmitted with the current date and time, the validity of the ID is managed based on the transmission and reception time. be able to.

  Note that the receiver shown in FIG. 129 and FIG. 130 may acquire the encrypted edit ID, transmit the encrypted edit ID to the server, and acquire the ID from the server.

(Guidance at the station)
FIG. 131 shows an example of a usage form of the present invention in a train platform. A user holds the portable terminal over an electronic bulletin board or lighting, and acquires information displayed on the electronic bulletin board, train information of a station where the electronic bulletin board is installed, information on a station in the station, or the like by visible light communication. Here, the information itself displayed on the electronic bulletin board may be transmitted to the portable terminal by visible light communication, or ID information corresponding to the electronic bulletin board is transmitted to the portable terminal, and the ID information acquired by the portable terminal The information displayed on the electronic bulletin board may be acquired by inquiring the server. When the ID information is transmitted from the mobile terminal, the server transmits the content displayed on the electronic bulletin board to the mobile terminal based on the ID information. The train ticket information stored in the memory of the mobile terminal is compared with the information displayed on the electronic bulletin board, and the ticket information corresponding to the user's ticket is displayed on the electronic bulletin board. An arrow indicating the destination to the home where the user's scheduled train arrives is displayed on the display. When getting off, the route to the vehicle near the exit or the transfer route may be displayed. If a seat has been designated, the route to that seat may be displayed. When the arrow is displayed, it can be displayed more easily by displaying the arrow using the same color as the color of the train route in the map or the train guide information. In addition to the arrow display, the user's reservation information (home number, vehicle number, departure time, seat number) can also be displayed. By displaying the user reservation information together, it is possible to prevent erroneous recognition. If the ticket information is stored in the server, query the server from the mobile terminal to obtain and compare the ticket information, or compare the ticket information with the information displayed on the electronic bulletin board on the server side. Thus, information related to the ticket information can be acquired. The target route may be estimated from the history of the user performing a transfer search, and the route may be displayed. Further, not only the contents displayed on the electronic bulletin board but also the train information / premises information of the station where the electronic bulletin board is installed may be acquired and compared. Information related to the user may be highlighted with respect to the display of the electronic bulletin board on the display, or may be rewritten and displayed. When the user's boarding schedule is unknown, an arrow for guiding to the boarding place on each route may be displayed. When station premises information is acquired, an arrow for guiding to a store or restroom may be displayed on the display. The user's behavior characteristics may be managed in advance by a server, and an arrow for guiding the user to a store / restroom may be displayed on the display when the user often stops at a store / restaurant in the station. Only users who have behavioral characteristics of stopping at a store / restroom will display an arrow that directs them to a store / restroom, etc., and will not be displayed to other users, so the amount of processing can be reduced. . The color of the arrow leading to a store / restroom may be different from the arrow guiding the destination to the home. When both arrows are displayed simultaneously, it is possible to prevent erroneous recognition by using different colors. Note that although an example of a train is shown in FIG. 131, it is possible to perform display with a similar configuration even on an airplane or a bus.

(Translation of signage)
FIG. 132 shows an example in the case where information is acquired by visible light communication from an electronic guidance display board installed in an airport, a station premises, a sightseeing spot, a hospital, or the like. Information on the display content is acquired from the electronic guidance display board by visible light communication, the display content information is translated into language information set in the mobile terminal, and then displayed on the display of the mobile terminal. Since it is translated into the user's language and displayed, the user can easily understand the information. Language translation may be performed in the mobile terminal or in the server. When translating in the server, the display content information acquired by visible light communication and the language information of the portable terminal are transmitted to the server, the translation is performed in the server, the transmitted to the portable terminal, and the display on the portable terminal May be displayed. Moreover, when ID information is acquired from an electronic guidance display board, the structure which transmits ID information to a server and acquires the display content information corresponding to ID information from a server may be sufficient. Furthermore, based on nationality information, ticket information, and baggage deposit information stored in the mobile terminal, an arrow that guides the user to the next place may be displayed.

(Coupon pop-up)
FIG. 133 shows an example in which coupon information acquired by visible light communication is displayed or a popup is displayed on the display of the mobile terminal when the user approaches the store. A user acquires coupon information of a store from an electronic bulletin board etc. by visible light communication using a portable terminal. Next, when the user enters the predetermined range from the store, the coupon information of the store or a pop-up is displayed. Whether or not the user has entered the predetermined range from the store is determined using the GPS information of the mobile terminal and the store information included in the coupon information. Not only coupon information but also ticket information may be used. Since it automatically alerts when a store where coupons and tickets can be used approaches, the user can use coupons and tickets appropriately.

  FIG. 134 shows an example in which coupon information / ticket information or a pop-up is displayed on the display of the portable terminal at a cash register or a ticket gate. Display is performed when position information is acquired from visible light communication from lighting installed in a cash register or ticket gate, and the acquired position information matches information included in coupon information / ticket information. The barcode reader may have a light emitting unit, and the position information may be acquired by performing visible light communication with the light emitting unit, or the position information may be acquired from the GPS of the mobile terminal. A transmitter is installed near the cash register, and the user may hold the receiver over the transmitter to display coupons and payment information on the display of the receiver, or the receiver communicates with the server. Payment processing may be performed. When coupon information / ticket information includes Wi-Fi information installed in a store or the like, and the user's portable terminal acquires the same information as the Wi-Fi information included in the coupon information / ticket information In addition, display may be performed.

(Launch operation application)
FIG. 135 illustrates an example in which a user acquires information from a home appliance by visible light communication using a mobile terminal. When ID information or information related to the home appliance is acquired from the home appliance by visible light communication, an application for operating the home appliance is automatically started. FIG. 135 shows an example using a television. With such a configuration, it is possible to start an application for operating a home appliance simply by holding the portable terminal over the home appliance.

(Stop transmission while the barcode reader is operating)
FIG. 136 shows an example in which data communication for visible light communication is stopped near the barcode reader 8405a when the barcode reader 8405a reads a barcode of a product. By stopping the visible light communication when reading the barcode, the barcode reader 8405a can be prevented from erroneously recognizing the barcode. The barcode reader 8405a transmits a transmission stop signal to the visible light signal transmitter 8405b when the barcode reading button is pressed. When the finger is released from the button or when the finger is released from the button, a predetermined signal is transmitted. When the time has elapsed, a transmission resumption signal is transmitted to the visible light signal transmitter 8405b. The transmission stop signal and the transmission restart signal are performed by wired / wireless communication, infrared communication, or sound wave communication. The barcode reader 8405a transmits a transmission stop signal when it is estimated that the barcode reader 8405a is moved from the measurement value of the acceleration sensor provided in the barcode reader 8405a, and the barcode reader 8405a is not moved for a predetermined time. A transmission resumption signal may be transmitted when estimated. The barcode reader 8405a transmits a transmission stop signal when it is estimated that the barcode reader 8405a is grasped from the measured value of the electrostatic sensor or the illuminance sensor provided in the barcode reader 8405a, and when it is estimated that the hand is released. A transmission resumption signal may be transmitted. The barcode reader 8405a detects that the barcode reader 8405a has been lifted because the switch provided on the grounding surface of the barcode reader 8405a is released from the pressed state, and transmits a transmission stop signal. It may be detected that the barcode reader 8405a has been placed because it is pressed, and a transmission resumption signal may be transmitted. The bar code reader 8405a detects that the bar code reader 8405a has been lifted from the measured values of the bar code reader storage switch and the infrared sensor, transmits a transmission stop signal, and detects that the bar code reader 8405a has been returned. Then, a transmission resumption signal may be transmitted. The cash register 8405c may transmit a transmission stop signal when the operation is started, and may transmit a transmission restart signal when the settlement operation is performed.

  For example, the transmitter 8405b configured as illumination stops signal transmission when receiving a transmission stop signal, and operates so that a ripple (luminance change) of 100 Hz to 100 kHz becomes small. Alternatively, signal transmission is continued by reducing the luminance change of the signal pattern. Alternatively, the carrier period is set longer than the barcode reading time of the barcode reader 8405a, or the carrier period is set shorter than the exposure time of the barcode reader 8405a. Thereby, malfunction of the barcode reader 8405a can be prevented.

  As shown in FIG. 137, for example, the transmitter 8406b configured as illumination detects that there is a person near the barcode reader 8406a with a human sensor or camera, and stops signal transmission. Alternatively, the transmitter 8405b performs the same operation as when a transmission stop signal is received. The transmitter 8406b resumes signal transmission when detecting that there is no person in the vicinity of the barcode reader 8406a. The transmitter 8406b may detect an operation sound of the barcode reader 8406a and stop signal transmission for a predetermined time.

(Information transmission from PC)
FIG. 138 shows an example of a usage form of the present invention.

  For example, the transmitter 8407a configured as a personal computer transmits a visible light signal through a display device such as an attached display, a connected display, or a projector. The transmitter 8407a transmits the URL of the website displayed by the browser, information on the clipboard, and information determined by the application having the focus. For example, coupon information acquired on a website is transmitted.

(Database)
FIG. 139 shows an example of the configuration of a database held by a server that manages IDs transmitted by a transmitter.

  The database has an ID-data table that holds data to be passed in response to an inquiry using the ID as a key, and an access log table that stores a record of the inquiry using the ID as a key. The ID-data table clears the ID transmitted by the transmitter, the data passed in response to the inquiry using the ID as a key, the condition for passing the data, the number of times of access using the ID as a key, and the condition. Have the number of times. The conditions for passing data include the date and time, the number of accesses, the number of successful accesses, the information of the terminal of the inquiry source (terminal model, the application that made the inquiry, the current location of the terminal, etc.) Age, gender, occupation, nationality, language, religion, etc.). By setting the number of successful accesses as a condition, a service method such as “1 yen per access, but 100 yen as the upper limit and no data returned thereafter” becomes possible. When there is access using ID as a key, the log table clears the ID, requested user ID, time, other incidental information, whether or not the data is passed, and the passed data Record the contents of.

(Reception start gesture)
FIG. 140 shows an example of a gesture operation for starting reception by this communication method.

  For example, the user protrudes a receiver configured as a smartphone, and starts reception by an operation of rotating the wrist to the left and right. The receiver detects this operation from the measurement value of the 9-axis sensor and starts reception. The receiver may start reception when detecting at least one of these operations. The operation of projecting the receiver has an effect that the transmitter is closer to the transmitter, so that the transmitter is imaged larger, and the reception speed and accuracy are improved. The operation of rotating the wrist to the left and right has the effect of changing the angle of the receiver to eliminate the angle dependency of this method and enable stable reception.

  This operation may be performed only when the home screen is in the foreground. Thereby, it is possible to prevent this communication from being performed against the user's intention while using another application.

  Note that the image sensor may be activated when the operation of the receiver protruding is detected, and reception may be stopped if there is no operation of shaking the wrist to the left or right. Since activation of the image sensor takes approximately several hundred milliseconds to 2 seconds, this can further improve the responsiveness.

(Transmitter control by power line)
FIG. 141 shows an example of the transmitter of the present invention.

  The signal control unit 8410g controls the transmission state of the transmitter 8410a (contents of signals to be transmitted, presence / absence of transmission, intensity of luminance change used for transmission, etc.) The signal control unit 8410g sends the control content of the transmitter 8410a to the power distribution control unit 8410f. The distribution control unit 8410f changes the voltage, current, and frequency supplied to the power supply unit 8410b of the transmitter 8410a, thereby transmitting the control contents in the form of the magnitude of change and the time of change. Since the power supply unit 8410b performs a constant output without being affected by a slight change in voltage, current, or frequency, a signal that exceeds the stabilization capability of the power supply unit 8410b, for example, a signal depending on the timing or time width at which the power supply is cut off. The signal is transmitted by expressing The luminance control unit 8410d receives the content transmitted by the power distribution control unit 8410f in consideration of conversion by the power supply unit 8410b, and changes the luminance change pattern of the light emitting unit.

(Encoding method)
FIG. 142 is a diagram for explaining one of the visible light communication image encoding methods.

  In this encoding method, since the ratio of white and black is approximately the same, the average luminance of the normal phase image and the reverse phase image is approximately the same, and there is an advantage that it is difficult for humans to feel flicker.

(Encoding method that can receive light even when imaged from oblique direction)
FIG. 143 is a diagram for explaining one of the encoding methods of the visible light communication image.

  The image 1001a is an image in which white and black lines are displayed with a uniform width. When this image 1001a is imaged obliquely, in the image 1001b obtained by the imaging, the left line appears thin and the right line appears thick. When the image 1001a is projected onto a curved surface, lines with different thicknesses appear in the image 1001i obtained by the imaging.

  Therefore, a visible light communication image is created by the following encoding method. The visible light communication image 1001c includes, from the left, a white line, a black line that is three times as thick as the white line, and a white line that is one-third the thickness of the black line. In this way, the preamble is encoded as an image in which a line three times as thick as the left adjacent line and a line one third as thick as the left adjacent line are continued. Like the visible light communication images 1001d and 1001e, a line having the same thickness as the left adjacent line is encoded as “0”. Like the visible light communication images 1001f and 1001g, a line that is twice as thick as the left adjacent line or half as thick as the left adjacent line is encoded as “1”. That is, when the thickness of the encoding target line is different from the thickness of the left adjacent line, the encoding target line is encoded as “1”. As an example using this encoding method, a signal including “0101010001011” following the preamble is represented by an image such as a visible light communication image 1001h. In this example, the line having the same thickness as the left adjacent line is encoded as “0”, and the line having a different thickness from the left adjacent line is encoded as “1”, but the line having the same thickness as the left adjacent line is encoded. May be encoded as “1”, and a line having a different thickness from the left adjacent line may be encoded as “0”. Further, not only the comparison with the left adjacent line, but also the comparison with the right adjacent line may be performed. That is, “1” and “0” may be encoded depending on whether the thickness of the encoding target line is the same as or different from the thickness of the line on the right. In this way, on the transmitter side, “0” is encoded by using a color different from that of the encoding target line and by making the thickness of the adjacent line and the encoding target line the same. By doing so, “1” is encoded.

  On the receiver side, a visible light communication image is captured, and the thickness of a white or black line is detected in the captured visible light communication image. The color is different from the line to be decoded, and the thickness of the line adjacent to the line to be decoded (left adjacent or right adjacent) is compared with the thickness of the line to be decoded. The decoding target line is decoded as “0”, and when the thicknesses are different, the decoding target line is decoded as “1”. It may be decoded as “1” when the thickness is the same, and as “0” when the thickness is different. Based on the 1 and 0 data strings that have been decoded, the data is finally decoded.

  This encoding method uses a local line thickness relationship. As seen in the image 1001b and the image 1001i, the ratio of the thicknesses of the neighboring lines does not change greatly. Therefore, even if the image is taken from an oblique direction or projected onto a curved surface, it is created by this encoding method. Visible light communication images can be correctly decoded.

  In this encoding method, since the ratio of white and black is approximately the same, the average luminance of the normal phase image and the reverse phase image is approximately the same, and there is an advantage that it is difficult for humans to feel flicker. In addition, since this encoding method is not different between black and white, there is an advantage that it can be decoded by the same algorithm for any visible light communication image of a normal phase signal and a reverse phase signal.

  Moreover, there is an advantage that it is easy to add a code. For example, the visible light communication image 1001j is a combination of a four-times-thick line on the left and a quarter line on the left. In this way, there are many unique patterns such as “5x and 1/5 next to the left” and “3x and 2/3 next to the left”. It can be defined. For example, a visible light communication image can represent a single piece of data, but visible light can be used as a cancel signal to indicate that some of the data received so far has become invalid because the data to be transmitted has been changed. It is conceivable to use the communication image 1001j. The color is not limited to white and black, and any color may be used as long as it is different. For example, complementary colors may be used.

(Encoding method with different amount of information depending on distance)
144 and 145 are diagrams illustrating one of the visible light communication image encoding methods.

  As shown in (a-1) of FIG. 144, when a 2-bit signal is expressed by setting one portion of the four divided images to black and the remaining portion to white, the average luminance of the image is When white is 100% and black is 0%, it is 75%. If the white and black portions are reversed as shown in FIG. 144 (a-2), the average luminance is 25%.

  In the image 1003a, the white portion of the visible light communication image created by the encoding method of FIG. 143 is represented by the image of (a-1) in FIG. 144, and the black portion is represented by the image of (a-2) of FIG. 144. This is a visible light communication image. This visible light communication image represents a signal A encoded by the encoding method of FIG. 143 and a signal B encoded by (a-1) and (a-2) of FIG. 144. When the nearby receiver 1003b captures the visible light communication image 1003a, a fine image 1003d is obtained, and both the signal A and the signal B can be received. When the remote receiver 1003c captures the visible light communication image 1003a, a small image 1003e is obtained. In the image 1003e, the detailed part cannot be confirmed, and the part (a-1) in FIG. 144 is white and the part (a-2) in FIG. 144 is black. Therefore, only the signal A can be received. . Thereby, as the distance between the visible light communication image and the receiver is shorter, more information can be transmitted. As a method for encoding the signal B, a combination of (b-1) and (b-2) in FIG. 144 or a combination of (c-1) and (c-2) in FIG. 144 may be used.

  By using the signals A and B, the signal A can represent basic and important information, and the signal B can represent additional information. Further, when the receiver transmits the signals A and B as ID information to the server and the server transmits information corresponding to the ID information to the receiver, the information transmitted by the server may be changed depending on the presence or absence of the signal B. It becomes possible.

(Encoding method that divides data)
FIG. 146 is a diagram for explaining one of the coding methods of the visible light communication image.

  A signal 1005a to be transmitted is divided into a plurality of data pieces 1005b, 1005c, and 1005d. An address indicating the position of the data piece is added to each data piece, and further, a preamble, an error detection / correction code, a frame type description, and the like are added to form frame data 1005e, 1005f, and 1005g. Visible light communication images 1005h, 1005i, and 1005j are generated by encoding the frame data and displayed. When the display area is sufficiently large, a visible light communication image 1005k obtained by connecting a plurality of visible light communication images is displayed.

  As shown in FIG. 146, as a method for inserting a visible light communication image into a video, in the case of a display device using a solid light source, a visible light communication signal image is normally displayed and only displayed. A fixed light source is turned on, and a solid-state light source is turned off during other periods, thereby realizing a wide range of display devices, for example, projectors using DMD, projectors using liquid crystals such as LCOS, Of course, display devices using MEMS can be applied. Further, even in a display device that divides and displays an image display into subframes, for example, a display device that does not use a light source such as a backlight such as a PDP or an EL, some subframes are converted into visible light communication images. Adaptation is possible by replacing. In addition, as a solid light source, a semiconductor laser, an LED light source, etc. can be considered.

(Effect of inserting reverse phase image)
FIG. 147 and FIG. 148 are diagrams for explaining one of the visible light communication image encoding methods.

  As (1006a) in FIG. 147, the transmitter inserts a black image between the video and the visible light communication image (normal phase image). An image obtained by capturing the image with the receiver is as shown in (1006b) of FIG. Since it is easy for the receiver to search for a portion where the line of simultaneously exposed pixels is black, the position where the visible light communication image is captured is easily set as the position of the pixel exposed at the next timing. Can be specified.

  As shown in (1006a) of FIG. 147, the transmitter displays a visible light communication image (normal phase image), and then displays a reverse running visible light communication image with black and white reversed. The receiver obtains a signal-to-noise ratio that is twice that obtained when only the normal phase image is used by obtaining the difference between the pixel values of the normal phase image and the reverse phase image. Conversely, when the same S / N ratio is ensured, the luminance difference between black and white can be halved, and flickering when viewed by humans can be suppressed. Also, as shown in (1007a) and (1007b) in FIG. 148, the moving average of the expected value of the difference in luminance between the video and the visible light communication image is canceled for the normal phase image and the reverse phase image. Since the time resolution of human vision is about 1/60 second, it is possible to make humans feel that no visible light communication image is displayed by setting the time for displaying a visible light communication image to be less than this. .

  As illustrated in (1006c) of FIG. 147, the transmitter may further insert a black image between the normal phase image and the reverse phase image. In this case, an image shown in (1006d) in FIG. 147 is obtained by imaging by the receiver. In the image shown in (1006b) of FIG. 147, since the normal phase image pattern and the reverse phase image pattern are adjacent to each other, pixel values may be averaged at the boundary portion. Such an issue does not occur in the image shown in FIG.

(Super-resolution)
FIG. 149 is a diagram illustrating one method for encoding a visible light communication image.

  As shown in FIG. 149 (a), when video data and signal data to be transmitted by visible light communication are separated, super-resolution processing is performed on the video data, and visible light communication is performed on the obtained super-resolution image. Superimpose the image. That is, the super-resolution processing is not performed on the visible light communication image. When the visible light communication image is already superimposed on the video data as shown in FIG. 149 (b), (1) the edge of the visible light communication image (data such as white and black is indicated by data). (2) The super-resolution processing is performed so that the average image of the normal phase image and the reverse phase image of the visible light communication image has a uniform luminance. Thus, it is possible to perform visible light communication more appropriately (decrease the error rate) by changing the processing for the visible light communication image depending on whether or not the visible light communication image is superimposed on the video data. It becomes.

(Indication of support for visible light communication)
FIG. 150 is a diagram illustrating one operation of the transmitter.

  The transmitter 8500a superimposes and displays on the image to be projected or displayed that it is compatible with visible light communication. This display is displayed only for a predetermined time after starting the transmitter 8500a, for example.

  The transmitter 8500a transmits to the connected device 8500c that it is compatible with visible light communication. The device 8500c displays that the transmitter 8500a is compatible with visible light communication. For example, the display of the device 8500c displays that the transmitter 8500a is compatible with visible light communication. When the connected transmitter 8500a supports visible light communication, the device 8500c transmits data for visible light communication to the transmitter 8500a. The indication that the transmitter 8500a is compatible with visible light communication may be displayed when the device 8500c is connected to the transmitter 8500a, or the device 8500c may transmit visible light communication to the transmitter 8500a. It may be displayed when data is transmitted. In the case where data is displayed when visible light communication data is transmitted from the device 8500c, the transmitter 8500a acquires identification information indicating visible light communication from the data, and the identification information is included in the data. If it is included, it may be displayed that the transmitter 8500a is compatible with visible light communication.

  In this way, by displaying on the projection screen or the display of the device that the transmitter (illumination, projector, video display device) is compatible with visible light communication or whether it is compatible or not. The user can easily grasp whether the transmitter is compatible with visible light communication. Therefore, it is possible to prevent a malfunction that the visible light communication cannot be performed even though the data for visible light communication is transmitted from the device to the transmitter.

(Information acquisition using visible light communication signals)
FIG. 151 is a diagram illustrating one application example of visible light communication.

  The transmitter 8501a receives video data and signal data from the device 8501c, and displays a visible light communication image 8501b. The receiver 8501d captures the visible light communication image 8501b and receives a signal included in the visible light communication image. The receiver 8501d communicates with the device 8501c from information (address, password, etc.) included in the received signal, and the video itself displayed by the transmitter 8501a and its accompanying information (video ID, URL, password, SSID, Translation data, voice data, hash tags, product information, purchase information, coupons, vacancy information, etc.). The device 8501c may transmit the transmission status to the transmitter 8501a to the server 8501e, and the receiver 8501d may obtain the information from the server 8501e.

(data format)
FIG. 152 is a diagram illustrating one format of visible light communication data.

  The data shown in (a) of FIG. 152 has a video address table indicating the position of video data in the storage area and a position address table indicating the position of signal data transmitted by visible light communication. In a video display device that does not support visible light communication, only the video address table is referred to. Therefore, even if a signal address table and signal data are included in the input, video display is not affected. Thereby, the backward compatibility with respect to the video display apparatus which does not support visible light communication is maintained.

  In the data format shown in FIG. 152 (b), an identifier indicating that the following data is video data is arranged in front of the video data, and an identifier indicating that the subsequent data is signal data is included in the signal data. Arranged in front. By using the identifier, it is inserted into the data only when there is video data or signal data, so that the entire code amount can be reduced. Also, identification information indicating whether it is video data or signal data may be arranged. Further, the program information may include identification information indicating whether or not data for visible light communication is included. By including identification information indicating whether or not data for visible light communication is included in the program information, the user can determine whether or not visible light communication is possible when searching for a program. The program information may include an identifier indicating that data for visible light communication is included. Furthermore, by adding an identifier / identification information for each data, it is possible to switch processing such as switching of luminance for each data and switching of super-resolution, thereby reducing the error rate during visible light communication. It becomes possible.

  The data format shown in (a) of FIG. 152 is suitable for a situation in which data is read from a storage medium such as an optical disk, and the data format shown in (b) of FIG. 152 is suitable for streaming data such as television broadcasting. Yes. The signal data includes the value of the signal transmitted by visible light communication, the transmission start time, the transmission end time, the location used for transmission on the display or projection surface, the brightness of the visible light communication image, the bar of the visible light communication image. Contains information such as code orientation.

(Estimate solid shape and receive)
FIG. 153 and FIG. 154 are diagrams illustrating one application example of visible light communication.

  As shown in FIG. 153, for example, a transmitter 8503a configured as a projector projects a distance measurement image in addition to a video and a visible light communication image. The dot pattern shown in the distance measurement image is a dot pattern in which the positional relationship between a predetermined number of dots in the vicinity of an arbitrary dot is different from the positional relationship between other arbitrary combinations of dots. The receiver can identify a local dot pattern by capturing an image for distance measurement, and can estimate the three-dimensional shape of the projection plane 8503b. The receiver restores the visible light communication image distorted by the three-dimensional shape of the projection surface to a flat image and receives a signal. Note that the distance measurement image and the visible light communication image may be projected with infrared rays that cannot be perceived by humans.

  As shown in FIG. 154, for example, a transmitter 8504a configured as a projector includes an infrared projector 8504b that projects a distance measurement image using infrared rays. The receiver estimates the three-dimensional shape of the projection plane 8504c from the distance measurement image, restores the distortion of the visible light communication image, and receives the signal. Note that the transmitter 8504a may project an image with visible light and project a visible light communication image with infrared light. Note that the infrared projector 8504b may project the visible light communication image with infrared rays.

(Stereoscopic projection)
FIG. 155 and FIG. 156 are diagrams illustrating one method for displaying a visible light communication image.

  When performing stereoscopic projection or displaying a visible light communication image on a cylindrical display surface, visible light communication images 8505a to 8505f can be displayed as shown in FIG. 155 to enable reception from a wide angle. Become. By displaying the visible light communication images 8505a and 8505b, it is possible to receive from a wide angle in the horizontal direction. By combining the visible light communication images 8505a and 8505b, reception is possible even when the receiver is tilted. The visible light communication image 8505a and the visible light communication image 8505b may be displayed alternately, or the visible light communication image 8505f that is an image obtained by combining these images may be displayed. Further, by adding the visible light communication image 8505c and the visible light communication image 8505d, it becomes possible to receive from a wide angle in the vertical direction. Like the visible light communication image 8505e, the visible light communication image segment may be expressed by providing a portion that is projected with an intermediate color or a portion that is not projected. By rotating the visible light communication images 8505a to 8505f, it is possible to receive from a wider angle. In FIG. 155, the visible light communication image is displayed on the cylindrical display surface. However, the visible light communication image may be displayed on the cylindrical display surface.

  When stereoscopic projection is performed or when a visible light communication image is displayed on a spherical display surface, visible light communication images 8506a to 8506d can be displayed as shown in FIG. 156 to enable reception from a wide angle. Become. In the visible light communication image 8506a, since the receivable area in the horizontal direction is wide but the receivable area in the vertical direction is narrow, the visible light communication image 8506a is displayed in combination with the visible light communication image 8506b having the opposite property. The visible light communication image 8506a and the visible light communication image 8506b may be displayed alternately, or the visible light communication image 8506c, which is an image obtained by combining these images, may be displayed. The portion where the barcode is concentrated, such as the upper portion of the visible light communication image 8506a, is finely displayed, and there is a high possibility of receiving a signal in error. Therefore, a reception error can be prevented by displaying this portion in an intermediate color as in the visible light communication image 8506d or by projecting nothing.

(Different communication protocols for each zone)
FIG. 157 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5.

  The receiver 8420a receives zone information from the base station 8420h, recognizes in which zone it is located, and selects a reception protocol. The base station 8420h is configured as, for example, a mobile phone communication base station, a Wi-Fi access point, an IMES transmitter, a speaker, or a wireless transmitter (Bluetooth (registered trademark), ZigBee, specific low-power wireless station, etc.). Note that the receiver 8420a may specify a zone based on position information obtained from GPS or the like. As an example, it is assumed that communication is performed at a signal frequency of 9.6 kHz in zone A, and communication is performed at a signal frequency of 15 kHz for ceiling lighting and signage of 4.8 kHz in zone B. At the position 8420j, the receiver 8420a recognizes that the current location is zone A from the information of the base station 8420h, performs reception at a signal frequency of 9.6 kHz, and receives signals transmitted from the transmitters 8420b and 8420c. The receiver 8420a recognizes that the current location is zone B from the information of the base station 8420i at the position 8420l, and is further trying to receive a signal from the ceiling lighting because the in-camera is directed upward. Is received at a signal frequency of 15 kHz, and signals transmitted by the transmitters 8420e and 8420f are received. At the position 8420m, the receiver 8420a recognizes that the current location is the zone B from above the base station 8420i, and further estimates that it is trying to receive a signal transmitted by the signage from the movement of the out camera. Receive at a signal frequency of .8 kHz and receive the signal transmitted by transmitter 8420g. The receiver 8420a receives the signals of both the base station 8420h and the base station 8420i at the position 8420k, and cannot determine whether the current location is the zone A or the zone B. Therefore, the reception process is performed at both 9.6 kHz and 15 kHz. I do. It should be noted that the part where the protocol differs depending on the zone is not limited to the frequency, but may be the server that inquires about the modulation method, signal format and ID of the transmission signal. The base stations 8420h and 8420i may transmit the protocol in the zone to the receiver, or may transmit only the ID indicating the zone, and the receiver may acquire the protocol information from the server using the zone ID as a key. Good.

  Transmitters 8420b to 8420f receive zone IDs and protocol information transmitted from base stations 8420h and 8420i, and determine a signal transmission protocol. A transmitter 8420d capable of receiving signals transmitted by both base station 8420h and base station 8420i utilizes a base station zone protocol with stronger signal strength, or alternately uses both protocols.

(Zone recognition and services for each zone)
FIG. 158 is a diagram illustrating an example of operation of a receiver and a transmitter in Embodiment 5.

  The receiver 8421a recognizes the zone to which it belongs from the received signal. The receiver 8421a provides services (coupon distribution, point assignment, route guidance, etc.) determined for each zone. As an example, the receiver 8421a receives a signal transmitted from the left side of the transmitter 8421b and recognizes that it is in the zone A. Here, the transmitter 8421b may transmit different signals depending on the transmission direction. Further, the transmitter 8421b may transmit a signal such that a different signal is received according to the distance to the receiver by using a signal having a light emission pattern such as 2217a. In addition, the receiver 8421a may recognize the positional relationship with the transmitter 8421b from the direction and size in which the transmitter 8421b is imaged, and may recognize the zone in which the receiver 8421a is located.

  A part of the signals indicating that they are located in the same zone may be shared. For example, IDs representing zone A transmitted from the transmitter 8421b and the transmitter 8421c are common to the first half. Accordingly, the receiver 8421a can recognize the zone where the receiver 8421a is located only by receiving the first half of the signal.

(Summary of this embodiment)
The information communication method in the present embodiment is an information communication method for transmitting a signal by a luminance change, and a determination step for determining a plurality of luminance change patterns by modulating each of a plurality of transmission target signals; Each of the plurality of light emitters changes a luminance according to any one of the determined plurality of luminance change patterns, thereby transmitting a transmission target signal corresponding to any one of the patterns. A transmission step, wherein two or more of the plurality of light emitters each have two types of brightness different from each other for each time unit predetermined for the light emitter. The time unit predetermined for each of the two or more light emitters is set so that either one of the two lights is output. Differently, luminance changes at different frequencies to.

  Thereby, as in the operation described with reference to FIG. 113, each of two or more light emitters (for example, a transmitter configured as a lighting device) changes in luminance at a different frequency. A receiver that receives signals to be transmitted (for example, IDs of light emitters) can easily distinguish and acquire the signals to be transmitted.

  In the transmission step, each of the plurality of light emitters changes in luminance at any one of at least four frequencies, and two or more light emitters of the plurality of light emitters are: The luminance may be changed at the same frequency. For example, in the transmitting step, when the plurality of light emitters are projected onto a light receiving surface of an image sensor for receiving the plurality of transmission target signals, all the light emitters adjacent to each other on the light receiving surface. Each of the plurality of light emitters changes in luminance so that the frequency of the luminance change differs between them.

  Thus, as in the operation described with reference to FIG. 114, if there are at least four types of frequencies used for luminance change, even if there are two or more light emitters that change in luminance at the same frequency, Even if the number of frequency types is less than the number of multiple light emitters, the luminance changes between all the light emitters adjacent to each other on the light receiving surface of the image sensor based on the four-color problem or the four-color theorem. It is possible to reliably vary the frequency of the. As a result, the receiver can easily distinguish and acquire each of the transmission target signals transmitted from the plurality of light emitters.

  In the transmission step, each of the plurality of light emitters may transmit the signal to be transmitted by changing in luminance at a frequency specified by a hash value of the signal to be transmitted.

  Accordingly, as in the operation described with reference to FIG. 113, each of the plurality of light emitters changes in luminance at a frequency specified by the hash value of the signal to be transmitted (for example, the ID of the light emitter). When receiving the signal to be transmitted, the machine can determine whether the frequency specified from the actual luminance change matches the frequency specified by the hash value. That is, the receiver can determine whether or not there is an error in the received signal (for example, the ID of the light emitter).

  The information communication method may further calculate a frequency corresponding to the transmission target signal as a first frequency from the transmission target signal stored in the signal storage unit according to a predetermined function. A calculation step; a frequency determination step for determining whether or not the second frequency stored in the frequency storage unit matches the calculated first frequency; and the first frequency and the second frequency A frequency error notification step of notifying an error when it is determined that the frequency does not match, and a determination step when it is determined that the first frequency and the second frequency match. Then, a luminance change pattern is determined by modulating the transmission target signal stored in the signal storage unit, and in the transmission step, among the plurality of light emitters Any one of the light emitters according to the determined the pattern, by changing the luminance in the first frequency may transmit a signal of the transmission target stored in the signal storage unit.

  Thus, as in the operation described with reference to FIG. 120, the frequency stored in the frequency storage unit and the frequency calculated from the signal to be transmitted stored in the signal storage unit (ID storage unit) are It is determined whether or not they match, and when it is determined that they do not match, an error is notified, so that it is possible to easily detect abnormality of the signal transmission function by the light emitter.

  The information communication method further includes a check value calculation step of calculating a first check value from a transmission target signal stored in the signal storage unit according to a predetermined function, and a check value storage unit. A check value determination step for determining whether or not the stored second check value matches the calculated first check value; and the first check value and the second check value. A check value error notification step of notifying an error when it is determined that they do not match, and a determination step when it is determined that the first check value and the second check value match Then, a luminance change pattern is determined by modulating the transmission target signal stored in the signal storage unit, and in the transmission step, the plurality of light emitters are detected. Any one of the light emitters is by changing luminance in accordance with the determined the pattern, may transmit a signal of the transmission target stored in the signal storage unit.

  Thereby, like the operation described with reference to FIG. 120, the check value calculated from the check value stored in the check value storage unit and the transmission target signal stored in the signal storage unit (ID storage unit). It is determined whether or not the values match, and when it is determined that they do not match, an error is notified, so that the abnormality detection of the signal transmission function by the light emitter can be easily performed.

  The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, and corresponds to a plurality of exposure lines included in the image sensor in an image obtained by photographing the subject by an image sensor. An exposure time setting step for setting an exposure time of the image sensor so that a plurality of bright lines are generated according to a change in luminance of the subject, and the exposure time set for the subject whose luminance is changed by the image sensor. And acquiring information by demodulating data identified by the pattern of the plurality of bright lines included in the acquired bright line image, and an image acquisition step of acquiring a bright line image including the plurality of bright lines Information acquisition step and a plurality of bright line patterns included in the acquired bright line image. Te, and a frequency specifying step of specifying the frequency of the luminance change of the subject. For example, in the frequency specifying step, a plurality of header patterns, which are a plurality of predetermined patterns for indicating a header, are included in the plurality of bright line patterns, and the number of pixels between the plurality of header patterns is determined. Is determined as the frequency of luminance change of the subject.

  Thereby, since the frequency of the luminance change of the subject is specified as in the operation described with reference to FIG. 115, when a plurality of subjects with different luminance change frequencies are photographed, information from those subjects is obtained. Can be easily distinguished and acquired.

  Further, in the image acquisition step, the bright line image including a plurality of patterns each represented by a plurality of bright lines is acquired by photographing a plurality of subjects each changing in luminance, and the information acquisition step acquires the bright line image. When a part of each of the plurality of patterns included in the bright line image overlaps, by demodulating data specified by a portion excluding the part from each of the plurality of patterns, the plurality of patterns Information may be acquired from each of the patterns.

  As a result, since the data is not demodulated from a portion where a plurality of patterns (plural bright line patterns) overlap as in the operation described with reference to FIG. 117, erroneous information is acquired. Can be prevented.

  Further, in the image acquisition step, a plurality of bright line images are acquired by photographing the plurality of subjects a plurality of times at mutually different timings, and in the frequency specifying step, the bright line images are included in the bright line image. A frequency for each of a plurality of patterns is specified, and in the information acquisition step, a plurality of patterns in which the same frequency is specified is searched from a plurality of previous bright line images, and the searched plurality of patterns are combined and combined. The information may be obtained by demodulating data specified by the plurality of patterns.

  Thereby, a plurality of patterns (plural emission line patterns) in which the same frequency is specified are searched from a plurality of emission line images, the plurality of searched patterns are combined, and information is acquired from the combined patterns. Therefore, even when a plurality of subjects are moving, information from the plurality of subjects can be easily distinguished and acquired.

  The information communication method may further include: identifying information on the subject included in the information acquired in the information acquisition step; and the frequency specification for a server in which frequencies are registered for each of the identification information. A transmission step of transmitting specific frequency information indicating the frequency specified in the step; a related information acquisition step of acquiring related information associated with the identification information and the frequency indicated by the specific frequency information from the server; May be included.

  Thereby, as in the operation described with reference to FIG. 119, the identification information (ID) acquired based on the luminance change of the subject (transmitter) and the related information associated with the frequency of the luminance change are acquired. Is done. Therefore, by changing the frequency of the luminance change of the subject and updating the frequency registered in the server to the frequency after the change, the receiver that acquired the identification information before the frequency change acquires the related information from the server. Can be prevented. That is, by changing the frequency registered in the server in accordance with the change in the luminance change frequency of the subject, the receiver that has acquired the subject identification information in the past can acquire the related information from the server indefinitely. It can prevent becoming a state.

  The information communication method is further acquired in the identification information acquisition step of acquiring the identification information of the subject by extracting a part from the information acquired in the information acquisition step, and in the information acquisition step. In addition, a setting frequency specifying step of specifying a number indicated by the remaining part other than the part of the information as a setting frequency of the luminance change set for the subject may be included.

  Accordingly, as in the operation described with reference to FIG. 116, the information obtained from the plurality of bright line patterns does not depend on the identification information of the subject and the set frequency of the luminance change set on the subject. Since it can be included, the degree of freedom between the identification information and the set frequency can be increased.

(Embodiment 6)
In this embodiment, each application example using a receiver such as a smartphone in Embodiments 1 to 5 and a transmitter that transmits information as a blinking pattern of LEDs will be described.

  FIG. 159 is a diagram illustrating an example of a transmission signal according to the sixth embodiment.

  The transmission signal D is divided into data pieces Dx of a predetermined size (for example, Dx = D1, D2, D3), and a frame check sequence FCS for error detection / correction calculated from each data piece and a header Hdr are obtained for each data piece. Append to Further, a frame check sequence FCS2 for error detection / correction calculated from the original data and a header Hdr2 are added. Data composed of Hdr, Dx, and FCS is configured to be received by the image sensor. Since the image sensor is suitable for receiving continuous data in a short time, Hdr, Dx, and FCS transmit continuously. The data composed of Hdr2, Dx, and FCS2 is configured to be received by the illuminance sensor. Although it is desirable that the Hdr and FCS received by the image sensor be short, the Hdr2 and FCS2 received by the illuminance sensor can be longer signal sequences. By using a long signal sequence for Hdr2, header detection accuracy can be increased. By making FCS2 longer, a code that can detect and correct many bit errors can be adopted, and the performance of error detection and correction can be improved. Note that Hdr2 and FCS2 may not be transmitted, and instead Hdr and FCS may be received by the illuminance sensor. The illuminance sensor may receive both Hdr and Hdr2, or both FCS and FCS2.

  FIG. 160 is a diagram illustrating an example of a transmission signal according to the sixth embodiment.

  FCS2 has a long signal length, and if it is frequently inserted, the reception efficiency of the image sensor deteriorates. Therefore, the insertion frequency of FCS2 is reduced, and a signal PoFCS2 indicating the location of FCS2 is inserted instead. As an example, when 4PPM having an information amount of 2 bits per unit time is used for signal representation, if CRC32 is used for FCS2, 16 units of transmission time are required, but PoFCS2 with a range of 0 to 3 is 1 unit time Can be sent. Since the transmission time is shortened compared with the case where only the FCS 2 is inserted, the efficiency of image sensor reception can be improved. The illuminance sensor receives PoFCS2 following the transmission signal D, specifies the transmission time of FCS2 from PoFCS2, and receives FCS2. Further, the PoFCS2 following the FCS2 is received, the transmission time of the next FCS2 is specified, and the next FCS2 is received. If the FCS2 received earlier and the FCS2 received later are the same, it is estimated that the receiver is receiving the same signal.

  161A to 161C are diagrams each illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6.

  In the captured image shown in FIG. 161A, the number of bright lines is small because the image is small. Therefore, only a small amount of data can be received from this captured image at a time. The captured image illustrated in FIG. 161B is an image captured using zoom and has a large number of bright lines because the captured image is large. Therefore, if imaging is performed using the zoom, a large amount of data can be received at one time. In addition, it can receive data from a distance and can receive signals from small transmitters. As a zoom method, optical zoom or Ex zoom is used. Optical zoom is zooming by increasing the focal length of a lens. Ex zoom is a zoom method that enlarges a part of a captured image by capturing an image using only a part rather than the entire image sensor when the image is captured at a resolution lower than the capability of the image sensor. The captured image illustrated in FIG. 161C is an image captured using electronic zoom (image enlargement). Although the image becomes larger, the bright line becomes thicker when enlarged by electronic zoom, and the number of bright lines is the same as before zooming, so the reception characteristics are the same as before zooming.

  162A and 162B are diagrams illustrating an example of a captured image (bright line image) of a receiver in Embodiment 6. FIG.

  The captured image illustrated in FIG. 162A is an image captured with the subject focused, and the captured image illustrated in FIG. 162B is an image captured with the focus out. In the captured image shown in FIG. 162B, since the image is blurred, it is possible to observe bright lines up to the periphery of the actual transmitter, and more bright lines can be observed. Therefore, if the image is taken out of focus, a large amount of data can be received at one time, and data can be received from a greater distance. An image similar to the captured image shown in FIG. 162B can also be captured by capturing using the macro mode.

  163A to 163C are diagrams each illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6.

  When the exposure time is set longer than the visible light communication mode and shorter than the normal imaging mode, an image as shown in FIG. 163A is obtained. An imaging mode in which such an image is obtained is called a bright line detection mode (intermediate mode). In the image shown in FIG. 163A, the bright line of the transmitter can be observed in the left center, and a dark normal captured image appears in the other part. By displaying this image on the receiver, it is possible to make it easy for the user to capture the image of the receiver toward the target transmitter. In the bright line detection mode, an image is captured darker than in the normal imaging mode. Therefore, by capturing an image in the high sensitivity mode, it is possible to capture an image that is close to the normal imaging mode and has a brightness that is easily visible to humans. If the sensitivity is set too high, the dark part of the bright line becomes bright, so the sensitivity is set to such an extent that the bright line can be observed. The receiver shifts to the visible light communication mode and receives the transmission signal of the transmitter that is imaged at a portion designated by means such as a user touching an image. The receiver may automatically shift to the visible light communication mode and receive a signal when a bright line (transmission signal) is found in the captured image.

  The receiver finds the transmission signal from the bright line in the captured image, and highlights and displays the portion as shown in FIG. 163B, so that the portion where the signal is transmitted can be presented to the user in an easy-to-understand manner. The bright line may be observed not only by the transmission signal but also by the pattern of the subject. Therefore, instead of determining the presence or absence of a transmission signal from the bright line of one image, it may be determined that there is a transmission signal when the position of the bright line changes in a plurality of images.

  Since an image captured in the bright line detection mode is darker and less visible than an image captured in the normal imaging mode, an image with improved visibility by image processing may be displayed. The image illustrated in FIG. 163C is an example of an image in which an edge is extracted to emphasize the boundary of the imaging target.

  164 is a diagram illustrating an example of a captured image (bright line image) of a receiver in Embodiment 6. FIG. Specifically, FIG. 164 is a diagram in which a transmitter with a signal transmission period of 1/9600 seconds is imaged at the exposure time ratio shown in the lower part of the figure. When the exposure time is shorter than 1/9600 seconds, which is the transmission cycle, the captured images are almost the same and a clear bright line can be captured. As the exposure time becomes longer, the outline of the bright line becomes blurred. However, in this signal expression example, if the exposure time is up to about 1.5 times the transmission cycle, the bright line pattern can be observed and the signal can be received. In this signal representation example, the bright line can be observed if the exposure time is up to about 20 times the transmission cycle, and the exposure time in this range can be used as the exposure time in the bright line detection mode.

To what extent the signal can be received differs depending on the signal expression method. If a signal expression rule in which the number of bright lines is small and the interval between the bright lines is long, the transmission efficiency is lowered, but the signal can be received even with a longer exposure time, and the bright lines can be observed even with a longer exposure time.

(Exposure time in intermediate imaging mode)
From FIG. 164, a clear bright line can be observed if the exposure time is up to about three times the modulation period. Since the modulation frequency is 480 Hz or more, it is desirable that the exposure time in the intermediate imaging mode (intermediate mode) be 1/160 seconds or less.

  In addition, when the exposure time is 1 / 10,000 seconds or less, it is difficult to see non-light-emitting objects even in the high sensitivity mode under illumination light. Therefore, the exposure time in the intermediate imaging mode is preferably 1 / 10,000 seconds or more. However, this limitation can be relaxed by improving the sensitivity of the image sensor in the future.

  FIG. 165 is a diagram illustrating an example of a transmission signal in Embodiment 6.

  Since the receiver integrates a plurality of received data pieces and receives a series of signals, if the transmission signal is suddenly changed, the data pieces before and after the change are mixed and the signals cannot be correctly integrated. . Therefore, as shown in FIG. 165 (a), when the transmission signal is changed, the transmitter performs normal lighting for a predetermined time as a buffer band and does not transmit a signal. When the receiver cannot receive a signal for a predetermined time T2 shorter than the predetermined time T1, the receiver discards the data pieces received so far to avoid mixing the data pieces before and after the change. can do. Alternatively, as shown in FIG. 165 (b), the transmitter repeatedly transmits a signal X for notifying it when the transmission signal is changed. Repeated transmission prevents the transmission signal change notification X from being missed. Alternatively, as shown in FIG. 165 (c), the transmitter repeatedly transmits the preamble when the transmission signal is changed. When the receiver receives a preamble with a period shorter than the period in which the preamble appears in a normal signal, the receiver discards data pieces received so far.

  FIG. 166 is a diagram illustrating an example of operation of a receiver in Embodiment 6.

  The image shown in FIG. 166 (a) is an image obtained by capturing the transmitter with just focus. The receiver can take an image as shown in FIG. 166 (b) by taking an image out of focus. When the focus is further removed, the captured image looks like the image shown in FIG. In FIG. 166 (c), the bright lines of a plurality of transmitters overlap, and the receiver cannot receive signals. Therefore, the receiver takes an image by adjusting the focus so that the bright lines of the plurality of transmitters do not overlap. When there is only one transmitter in the imaging range, the receiver adjusts the focus so that the size of the transmitter is maximized in the captured image.

  The receiver may compress the captured image in a direction parallel to the bright line, but does not compress the image in a direction perpendicular to the bright line. Alternatively, the receiver reduces the degree of compression in the vertical direction. Thereby, it is possible to prevent a reception error from occurring due to blurring of bright lines due to compression.

  167 and 168 are diagrams illustrating an example of instructions to the user displayed on the screen of the receiver in the sixth embodiment.

  The receiver can estimate the position of the receiver in the manner of triangulation from the position information of each transmitter and the position, size, and angle of each transmitter in the captured image by imaging a plurality of transmitters. Therefore, when the image is captured in a form that only one transmitter can receive, the receiver changes the direction of the receiver or lowers the image to capture images of a plurality of transmitters. Indicates an imaging direction and a moving direction by displaying an image including an arrow or the like. FIG. 167 (a) is a display example of an instruction to image the right transmitter with the receiver pointing to the right, and FIG. 167 (b) is an instruction to image the transmitter in front in the back. It is a display example. FIG. 168 is a display example of an instruction for imaging the other transmitter by shaking the receiver or the like because the position of the other transmitter is unknown to the receiver. Although it is desirable to photograph a plurality of transmitters in one captured image, the positional relationship of the transmitters in the plurality of images may be estimated using image processing or sensor values of a nine-axis sensor. The receiver may use the ID received from one transmitter, inquire the position information of the surrounding transmitters from the server, and instruct the user to image the transmitter that is most easily imaged.

  The receiver detects that the user has moved the receiver from the sensor value of the 9-axis sensor, and displays the screen based on the signal received last after the movement has ended and a predetermined time has elapsed. As a result, when the receiver is directed to the transmitter intended by the user, a signal of another transmitter is received while the receiver is moving, and processing based on the transmission signal of the unintended transmitter is performed. Can be prevented.

  The receiver may continue the reception process even while moving, and may perform a process based on the received signal, for example, acquisition of information from a server using the received signal as a key. In this case, the reception process is continued after the process, and the process based on the last received signal is defined as the final process.

  The receiver may process the signal received a predetermined number of times or notify the user. The receiver may process a signal that has been received the most times while being moved.

  The receiver may include a notification unit that notifies the user when the signal is successfully received or when the presence of the signal is detected in the captured image. The notification means notifies by sound, vibration, display update (pop-up display, etc.), or the like. Thereby, the user can know the presence of the transmitter.

  FIG. 169 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

  For example, a plurality of transmitters configured as displays are arranged adjacent to each other. When transmitting the same signal, the plurality of transmitters synchronize the timing of signal transmission and transmit signals from the entire surface as shown in FIG. With this configuration, since a plurality of displays are observed as one large transmitter in the receiver, the receiver can receive signals at a higher speed and from a longer distance. When a plurality of transmitters transmit different signals, as shown in FIG. 169 (b), the plurality of transmitters transmit signals by providing a buffer band (non-transmission area) in which signals are not transmitted. With this configuration, the receiver can recognize that the plurality of transmitters are different from each other with a buffer band therebetween, and can receive different signals.

  170 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG.

  As shown in FIG. 170 (a), the liquid crystal display is provided with a backlight extinction period, and by changing the state of the liquid crystal while the backlight is extinguished, the image in the state change is made invisible and dynamic resolution is performed. A feeling can be heightened. For a liquid crystal display performing such backlight control, a signal is superimposed in accordance with the lighting cycle of the backlight, as shown in FIG. Reception efficiency can be improved by continuously transmitting a set of data (Hdr, Data, FCS). In addition, the light emitting portion is in a bright state (Hi) at the beginning and the end of the backlight lighting period. This is because if the light emitting unit is in a dark state (Lo) continuously with the backlight extinction period, the receiver cannot determine whether Lo is transmitted as a signal or whether the signal is in the dark state due to the backlight extinction period.

  During the backlight extinction period, a signal with a low average luminance may be superimposed.

  By superimposing the signal, the average luminance changes compared to the case where the signal is not superimposed. Therefore, the average luminance is adjusted to be equal by increasing / decreasing the backlight lighting period or increasing / decreasing the luminance during backlight lighting.

  FIG. 171 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

  The liquid crystal display can reduce a change in luminance of the entire screen by performing backlight control at different timings for each position. This is called backlight scanning. The backlight scan is normally performed so that the backlight is turned on in order from the end as shown in FIG. At this time, a captured image 8802a is obtained. However, in the captured image 8802a, the portion where the bright line exists is divided, and it may not be estimated that the entire screen of the display is one transmitter. Therefore, as shown in FIG. 171 (b), all the portions that emit light (the signal is superimposed) are displayed when the vertical axis represents the spatial axis in the backlight scanning division direction and the horizontal axis represents the time axis. A captured image 8802b can be obtained by setting the order of backlight scanning so as to be connected. In the captured image 8802b, all the bright line portions are connected, and it can be easily estimated that the transmission signal is from one transmitter. In addition, since the number of bright lines that can be continuously received increases, signals can be received quickly and remotely. In addition, since the size of the transmitter can be easily estimated, the position of the receiver can be accurately estimated from the position, size, and angle of the transmitter in the captured image.

  FIG. 172 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

  In a time-division backlight scan, the backlight lighting period is short, and light is emitted on the graph with the vertical axis representing the spatial axis in the backlight scan division direction and the horizontal axis representing the time axis (the signal is superimposed). ) Portions cannot be connected, a signal is superimposed on each light emitting portion in accordance with the light emission timing of the backlight as in the case of FIG. At this time, by controlling the backlight so that the distance from the other backlight lighting portion on the graph is maximized, it is possible to prevent the bright lines in the adjacent portions from being mixed.

  FIG. 173 is a diagram for describing a use case in the sixth embodiment. The system in this embodiment includes a lighting device 100 that performs visible light communication, a wearable device 101 having a visible light communication function, a smartphone 102, and a server 103.

  An object of the present embodiment is to reduce a time required for shopping by omitting a series of troubles when a user performs shopping at a store using visible light communication. Conventionally, when purchasing a product at a store, it is necessary to search the store site in advance to obtain coupon information. In addition, there is a problem that it takes time to search for a product that is the coupon target in the store.

  As shown in FIG. 173, the lighting apparatus 100 transmits its own illumination ID information using visible light communication periodically in front of a store (assuming a home appliance mass retailer as an example). When receiving the illumination ID information, the wearable device 101 of the user transmits the illumination ID information to the smartphone 102 using short-range wireless communication. The smartphone 102 transmits user information and lighting ID information to the server 103 using a mobile line or the like. The smartphone 102 receives point information of a store in front of the user, coupon information, and the like from the server 103. The user browses the information received from the server 103 with the wearable device 101 or the smartphone 102. The user can purchase the product information of the displayed store on the spot or receive guidance to an exhibition place in the store. Hereinafter, it demonstrates in detail using figures.

  FIG. 174 is a diagram illustrating an information table transmitted from the smartphone 102 to the server 103. In addition to the membership number of the store, the store ID information, the transmission time, and the location information held in the smartphone 102, the user's preference information and biometric information, search history, and action history information stored in the smartphone 102 are displayed on the smartphone 102. Will send.

  FIG. 175 is a block diagram of the server 103. The transmission / reception unit 201 receives information transmitted from the smartphone 102. The control unit 202 performs overall control. The member information DB 203 stores a member number, the name and date of birth of the user of the member number, point information, purchase history, and the like. The store DB 204 stores store information such as a store ID, product information sold at the store, store display information, and store map information. The notification information generation unit 205 generates coupon information and recommended product information according to the user's preference.

  FIG. 176 is a flowchart showing the overall processing of the system. The wearable device 101 receives the illumination ID from the luminaire 100 (S301). Next, the wearable device 101 transmits the illumination ID to the smartphone 102 using close proximity wireless communication such as Bluetooth (registered trademark) (S302). Next, the smartphone 102 transmits the history information of the user, the membership number, and the lighting ID shown in FIG. 174 to the server 103 (S303). When the server 103 receives the data, the data is first transmitted to the control unit 202 (S304). Next, the control unit 202 refers to the member information DB 203 for the member number, and acquires member information (S305). Next, the control unit 202 inquires of the store DB 204 for the illumination ID and acquires store information (S306). The store information includes product information in stock at the store, information on products that the store wants to promote, coupon information, store map information, and the like. The control unit 202 transmits member information and store information to the notification information generation unit (S307). The notification information generation unit 205 generates advertisement information suitable for the user from the member information and the store information, and transmits it to the control unit 202 (S308). The control unit 202 transmits member information and advertisement information to the transmission / reception unit 201 (S309). Member information includes user point information, expiration date information, and the like. The transmission / reception unit 201 transmits member information and advertisement information to the smartphone 102 (S310). The smartphone 102 displays the received information on the display screen (S311).

  Furthermore, the smartphone 102 transfers the information received from the server 103 to the wearable device 101 (S312). If the notification setting of the wearable device 101 is ON, the wearable device 101 displays information (S314). In addition, when a wearable device displays information, it is desirable to urge the user to be alerted by vibration or the like. This is because the user does not necessarily enter the store, and therefore the user may not notice even if coupon information is transmitted.

  FIG. 177 is a diagram illustrating an information table transmitted from the server 103 to the smartphone 102. The store map DB is in-store guidance information such as which product is displayed at which position in the store. The product information of a store includes information on products in stock at the store and price information of products. The user member information includes user point information and membership card expiration date information.

  FIG. 178 is a diagram illustrating a screen flow displayed on the wearable device 101 from when the user receives information from the server 103 in front of the store until when the user actually purchases the product. In front of the store, points given when the user visits the store and coupon information are displayed. When the user taps the coupon information, the information transmitted from the server 103 is displayed according to the user's preference. For example, when the user taps the TV, recommended TV information is displayed. When the purchase button is pressed here, a receiving method selection screen is displayed, and it is possible to select delivery to home or receiving in the store. In this embodiment, since the store where the user is located is known, there is an advantage that it can be received at the store. When the guidance to the sales floor is selected in the flow 403, the wearable device 101 shifts to the Guide Mode. In this mode, the user is guided to a specific place using an arrow or the like, and the user can be guided to a place where the selected product is actually displayed. When guided in front of the product shelf, the wearable device 101 transitions to a screen for inquiring whether purchase is necessary. The user can make a purchase decision after trying the size, color, and convenience by actually viewing the product.

  The visible light communication in the present invention can specify the position of the user with high accuracy. For this reason, for example, as shown in FIG. 179, when the user is likely to enter a dangerous area in the factory, a warning can be given. Furthermore, since it is possible to determine whether or not to issue a warning on the wearable device side, it is possible to construct a warning system with a high degree of freedom, such as giving a warning to a child under a specific age, for example.

(Embodiment 7)
FIG. 180 is a diagram showing a service providing system using the reception method described in the embodiment already described.

  First, another company B or an individual ex8001 requests the company A ex8000 managing the server ex8002 to distribute information to the mobile terminal. For example, the mobile terminal that has made visible light communication with signage is requested to distribute detailed advertisement information, coupon information, or map information. The company A ex8000 that manages the server manages information distributed to the mobile terminal in association with arbitrary ID information. The portable terminal ex8003 acquires ID information from the subject ex8004 by visible light communication, and transmits the acquired ID information to the server ex8002. The server ex8002 transmits information corresponding to the ID information to the portable terminal and counts the number of times the information corresponding to the ID information is transmitted. The company A ex8000 managing the server charges the requested company B and the individual ex8001 for a fee corresponding to the counted number. For example, the larger the count number, the larger the amount to be charged.

  FIG. 181 is a flowchart illustrating a service provision flow.

  In Step ex8000, the company A that manages the server receives a request for information distribution from another company B. Next, in Step ex8001, in the server managed by the company A, the information received in the distribution request is associated with specific ID information. In Step ex8002, the mobile terminal receives specific ID information from the subject by visible light communication, and transmits the specific ID information to a server managed by the company A. The details of the visible light communication method have already been described in other embodiments, and will be omitted. The server transmits information corresponding to the specific ID information transmitted from the mobile terminal to the mobile terminal. In Step ex8003, the server counts the number of times of information distribution. Finally, in Step ex8004, the company B is charged a fee according to the count number of information distribution. In this way, by charging according to the count number, it becomes possible to charge Company B with an appropriate fee according to the advertising effect of information distribution.

  FIG. 182 is a flowchart showing service provision in another example. The description of the same steps as those in FIG. 181 is omitted.

  In Step ex8008, it is determined whether or not a predetermined time has elapsed from the start of information distribution. If it is determined that the time is within the predetermined time, Step ex8011 does not charge the company B. On the other hand, if it is determined that the predetermined period has elapsed, Step ex8009 counts the number of times the information has been distributed. In Step ex8010, the company B is charged a fee corresponding to the information distribution count. Thus, since information is distributed free of charge within a predetermined period, company B can receive a billing service after confirming the advertising effect and the like.

  FIG. 183 is a flowchart illustrating service provision in another example. The description of the same steps as those in FIG. 182 is omitted.

  In Step ex8014, the number of times information is distributed is counted. If it is determined in Step ex8015 that the predetermined period has not elapsed since the start of information distribution, charging is not performed in Step ex8016. On the other hand, if it is determined that the predetermined period has elapsed, in Step ex8017, it is determined whether or not the number of times the information has been distributed is equal to or greater than a predetermined value. When the number of times of distributing information is less than a predetermined value, the count number is reset and the number of times of distributing information is counted again. In this case, the company B is not charged for a predetermined period in which the number of times information is distributed is less than a predetermined value. In Step ex8017, if the count number is equal to or larger than the predetermined value, the count number is reset once in Step ex8018, and the count is restarted again. In Step ex8019, the company B is charged a fee corresponding to the count number. As described above, when the number of counts during the period in which the distribution is performed free of charge is small, the company B can receive the billing service at an appropriate timing by providing a period of free distribution again. Further, when the company A has a small number of counts, the information content is analyzed. For example, when the information does not correspond to the season, the company A may propose to the company B to change the information content. It becomes possible. In addition, when providing a free information delivery period again, it is good also as a period shorter than the initial predetermined period. By making it shorter than the initial predetermined period, the burden on the company A can be reduced. Moreover, it is good also as a structure which provides a free delivery period again after a fixed period. For example, in the case of information affected by the season, a free delivery period can be provided again after a certain period until the season changes.

  The charging fee may be changed according to the amount of data regardless of the number of times of information distribution. A certain amount of data may be distributed free of charge, and a predetermined amount of data or more may be charged. Further, the billing fee may be increased as the amount of data increases. Further, a management fee may be charged when managing information in association with specific ID information. By charging as a management fee, it is possible to determine the fee when the information distribution is requested.

(Embodiment 8)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

(Modulation method that is easy to receive)
184A, 184B, and 185 are diagrams illustrating an example of signal coding in Embodiment 8. FIG.

  The transmission signal is composed of a header (H) and a body (Body). The header includes a unique signal pattern. The receiver finds this unique pattern from the received signal, recognizes which part of the received signal represents the header or body based on the position, and receives the data.

  When the transmission signal is modulated with the pattern of FIG. 184A (a), the receiver can receive data when the header and the subsequent body are received in succession. The length of time that the receiver can continuously receive the signal is determined by the size of the transmitter shown in the captured image (captured image). When the transmitter is small or when the transmitter is imaged from a distance, the time during which the receiver can continuously receive signals is short. If the time that the receiver can continuously receive a signal (continuous reception time) is the same as the transmission time of one block including the header and the body, it is received only when the transmission start time of the header is equal to the reception start time. The machine can receive data. FIG. 184A (a) shows a case where the continuous reception time is slightly longer than the transmission time of one block including the header and the body. The arrow line represents the continuous reception time, and data can be received when a signal is received at the timing indicated by the thick line, but when a signal is received at the timing indicated by the thin line, Since the header and the body are not aligned, data cannot be received.

  Therefore, by modulating the transmission signal with the pattern of (b) in FIG. 184A, it is possible to receive data with wider reception timing. The transmitter transmits a modulated signal with “body, header, body” as a set. Here, two bodies in the same set have the same signal. Even if the receiver does not receive all the signals contained in the body continuously, the body can be restored by connecting the parts of the body before and after the header, so all the signals contained in the header are continuous. Data can be received if it can be received. In FIG. 184A, the reception timing at which data can be received is indicated by a thick line. In the case of (b), it can be seen that data can be received at a wider reception timing than in the case of (a).

  In the case of the modulation scheme in FIG. 184A (b), if the signal length of the body is a fixed value, the receiver can restore the body. Alternatively, if the header includes the signal length information of the body, the receiver can restore the body.

  That is, as illustrated in FIG. 184B, the receiver first finds a header including a unique bright line pattern from a captured image (bright line image) including the bright line. Then, the receiver sequentially reads each signal of the body that follows the header (direction (1) in FIG. 184B). At this time, each time a signal is read, the receiver determines whether or not the signal of the body has been read by the amount corresponding to the signal length of the body. That is, it is determined whether all the signals included in the body have been read. When it is determined that the signal has not been read, the receiver reads the signal that follows the read signal. However, when there is no subsequent signal, the receiver sequentially reads each signal of the body that follows the front of the header (direction (2) in FIG. 184B). Thereby, all signals included in the body are read out. Here, if the signal length of the body is a fixed length, the receiver holds the signal length in advance, and makes the above determination using the signal length. Alternatively, the receiver specifies the signal length of the body from the header, and makes the above determination using the signal length.

  In addition, even when the signal length of the body is variable, by defining the modulation method so that the bodies modulated by the same transmitter have the same signal length, the receiver can determine the signal length between the two headers. The body can be restored by estimating the signal length of the body. At this time, in the modulation method of FIG. 184A (b), signals for two headers and two bodies must be received at a time. However, with the modulation scheme shown in FIG. 185, the signal length of the body can be estimated only by receiving signals for two headers and one body. Note that in FIG. 185, the modulation scheme is a set of “body header header body header 2 (H2)”, and the receiver can receive data if it can continuously receive all the signals included in the header. is there.

  As described above, the transmitter according to the present embodiment includes a second luminance change pattern corresponding to a body that is a part of a signal to be transmitted and a header for specifying the body. A luminance change pattern, and a first luminance change pattern, a second luminance change pattern, and a first luminance change pattern in that order, and the luminance changes according to the respective patterns, whereby the header and the body Send. The transmitter further determines a third luminance change pattern indicating another header different from the header, and the first luminance change pattern, the second luminance change pattern, and the first luminance change pattern. The header, the body, and other headers may be transmitted by changing the luminance according to each pattern in the order of the third luminance change pattern.

(Combination of communication using bright lines and image recognition)
FIG. 186 is a diagram illustrating an example of a captured image in the eighth embodiment.

  The receiver can read a signal from the bright line in the captured image and analyze a part other than the bright line by image processing. Thereby, as an example, the receiver receives a signal from a transmitter configured as digital signage, for example, and even if the same signal is received, it varies depending on the image shown on the transmitter screen at that time An advertisement can be displayed.

  Since bright lines become noise in image processing, image processing may be performed after interpolating the pixel values of the bright line portion from the left and right pixels of the bright line. Further, image processing may be performed on an image excluding the bright line portion.

(How to use an image sensor suitable for receiving visible light signals)
187A to 187C are diagrams each illustrating an example of a structure and operation of a receiver in Embodiment 8.

  As shown in FIG. 187A, 8910a is an image sensor of the receiver. The imaging element includes an effective pixel that is a part that captures an image, an optical black for measuring noise such as dark current, and an invalid area 8910b. Optical black further includes VOB for measuring noise in the vertical direction and HOB for measuring noise in the horizontal direction. Since the bright line is generated in the direction of 8910c (horizontal direction), the bright line cannot be obtained during the exposure time to the VOB or the invalid area 8910b, and a signal cannot be received. Therefore, at the time of visible light communication, the time during which a signal can be received can be increased by switching to an imaging mode in which the VOB and the invalid area 8910b are not used or used at a minimum.

  As shown in FIG. 187B, by not using the VOB and the invalid area 8910b, it is possible to extend the exposure time in the effective pixel area that is an area including the effective pixel. Specifically, as shown in FIG. 187B (a), during normal shooting, one shot image is acquired between time t0 to t10, time t10 to t20, and time t20 to t30. In addition, since the VOB and the invalid area 8910b are also used to acquire each captured image, the exposure time in the effective pixel area (the time during which charges are read out, the shaded portion in FIG. 187B) ) Are times t3 to t10, times t13 to t20, and times t23 to t30.

  On the other hand, in the visible light communication, by not using the VOB and the invalid area 8910b, as shown in FIG. 187B (b), the effective pixel area is only used for the time during which the VOB and the invalid area 8910b are used. The exposure time can be extended. That is, it is possible to increase the time that can be received by visible light communication. As a result, many signals can be received.

  As shown in FIG. 187C (a), during normal photographing, exposure of each exposure line in the effective pixel region is started after a predetermined time m has elapsed since the exposure of the adjacent exposure line is started. On the other hand, as shown in FIG. 187C (b), when the time of exposure in the effective pixel area is extended during visible light communication, the exposure of each exposure line in the effective pixel area is the exposure of the adjacent exposure line. Is started after a predetermined time n (n> m) has elapsed.

  As described above, when performing normal shooting, the receiver according to the present embodiment applies to each of the plurality of exposure lines in the area including the optical black of the image sensor with respect to the exposure line adjacent to the exposure line. The charge is read after a predetermined time has elapsed since the charge was read. Then, when performing the visible light communication, the receiver does not use the optical black for reading out the electric charge, and for each of the plurality of exposure lines in the area other than the optical black in the image sensor, The charge is read after a time longer than the predetermined time from the time when the charge is read to the adjacent exposure line.

  Further, during visible light communication, by setting an imaging mode in which the number of vertical pixels is not reduced by processing such as demosaicing or clipping, the time during which signals can be received can be further increased.

  When imaging is performed in a mode that does not use the VOB or the invalid area 8910b and does not reduce the vertical pixels, the exposure timings of the lower end of the captured image and the upper end of the captured image of the next frame are temporally continuous, and the signal is continuous Can be received automatically. Even when VOB or the like cannot be completely invalidated, the signal can be continuously received by modulating the transmission signal by a method capable of error correction.

  In FIG. 187A, a horizontal bright line appears because a horizontal photodiode is simultaneously exposed. During visible light communication, the horizontal and vertical emission lines can be obtained by alternately repeating this exposure mode and the exposure mode in which the vertical photodiodes are simultaneously exposed. Even if it exists, it can receive a signal stably.

(Continuous signal reception)
FIG. 187D is a diagram illustrating an example of a signal reception method in Embodiment 8.

  The imaging device includes an effective pixel that is a pixel that makes the intensity of received light an image, and an invalid pixel that does not make an image of the intensity of received light, and is used as, for example, a dark current intensity reference. For this reason, in the normal imaging mode, as shown in (a), there is a time during which only invalid pixels receive light, and during that time, signals cannot be received. Therefore, in the visible light communication mode, reception is possible by minimizing the time for which only invalid pixels receive light as shown in (b) or by setting so that effective pixels always receive light as shown in (c). Can take a long time. This also allows continuous reception. In the case of (b), there is a time during which reception is not possible, but by using an error correction code for transmission data, the entire signal can be estimated even when some signals cannot be received.

(Receiving method of signal from transmitter imaged small)
FIG. 187E is a flowchart illustrating an example of a signal reception method in Embodiment 8.

  As shown in FIG. 187E, start is made at step 9000a, a signal is received at step 9000b, and a header is detected at step 9000c. In step 9000d, it is checked whether the data size of the body following the header is known. If YES, the process proceeds to step 9000f. If NO, the process proceeds to step 9000e, the data size of the body following the header is read from the header, and the process proceeds to step 9000f. In step 9000f, it is confirmed whether or not all the signals indicating the body have been received following the header. If YES, the process proceeds to step 9000g, the body part is read from the signal received following the header, and the process ends in step 9000p. To do. If NO, the process proceeds to step 9000h, and it is confirmed whether the data length of the body received after the header and the part received before the header is sufficient. If YES, the process proceeds to step 9000i. The body part is read out by connecting the received part following the header and the received part before the header, and the process ends at step 9000p. If NO, the process proceeds to step 9000j to check whether there is a means for imaging many bright lines from the transmitter. If YES, the process proceeds to step 9000n, where the setting is changed so that many bright lines can be imaged. Return to. If NO, proceed to step 9000k, notify that the transmitter is present but the image size is insufficient, notify the direction of the transmitter in step 9000m, and receive it when approaching there. This is notified, and the process ends at step 9000p.

  By this method, a signal can be stably received even when there are few exposure lines passing through the transmitter in the captured image.

(Captured image size suitable for receiving visible light signals)
188 and 189A are diagrams illustrating an example of a reception method in Embodiment 8.

  When the effective pixel area of the image sensor is 4: 3, when the image is captured at 16: 9, the upper and lower portions of the image are clipped. When the bright line appears in the horizontal direction, the bright line is lost by this clipping, and the time during which the signal can be received is shortened. Similarly, when the effective pixel area of the image sensor is 16: 9, if the image is captured at 4: 3, the left and right portions of the image are clipped, and the time for receiving a signal is shortened when the bright line appears in the vertical direction. Therefore, the aspect ratio at which clipping does not occur, that is, 4: 3 in FIG. 188 and 16: 9 in FIG. 189A is the imaging aspect ratio in the visible light communication mode. Thereby, the receivable time can be increased.

  That is, the receiver in the present embodiment further sets the ratio between the vertical width and the horizontal width of the image obtained by the image sensor. Then, when performing the visible light communication, the receiver determines whether or not the end of the image in the direction perpendicular to the exposure line (bright line) is clipped according to the set ratio, and the end is clipped. If it is determined, the set ratio is changed to a non-clipping ratio that is a ratio at which the end is not clipped. Then, the image sensor of the receiver acquires a bright line image of the non-clipping ratio by photographing the subject whose luminance changes.

  FIG. 189B is a flowchart illustrating an example of a reception method in Embodiment 8.

  In this reception method, the reception time is extended and an imaging aspect ratio for receiving a signal from a small transmitter is set.

  That is, as shown in FIG. 189B, start is made at step 8911Ba, and the imaging mode is changed to the visible light communication mode at step 8911Bb. In step 8911Bc, it is confirmed whether the aspect ratio setting of the captured image is the setting closest to the aspect ratio of the effective pixel. In the case of YES, the process proceeds to Step 8911Bd, and the aspect ratio setting of the captured image is set to the setting closest to the aspect ratio of the effective pixel. The process ends at step 8911Be. In the case of NO, the process proceeds to Step 8911Be and ends. By setting the aspect ratio in this way in the visible light communication mode, the time during which reception is not possible can be reduced. In addition, a signal from a small transmitter or a remote transmitter can be received.

  FIG. 189C is a flowchart illustrating an example of a reception method in Embodiment 8.

  In this reception method, an imaging aspect ratio for increasing the number of samples per time is set.

  That is, as shown to FIG. 189C, it starts at step 8911Ca and changes an imaging mode to visible light communication mode at step 8911Cb. In step 8911Cc, the bright line of the exposure line can be confirmed, but it is confirmed whether the number of samples per time is small and signals cannot be received. In the case of YES, the process proceeds to Step 8911Cd, and the aspect ratio setting of the captured image is set to the setting most different from the aspect ratio of the effective pixel. In step 8911Ce, the imaging frame rate is increased, and the process returns to step 8911Cc. If NO, the process proceeds to step 8911Cf, receives a signal, and ends.

  By setting the aspect ratio in this way in the visible light communication mode, a signal having a high frequency can be received. Also, it can be received even in a noisy environment.

(Reception of visible light signal using zoom)
190 is a diagram illustrating an example of a reception method in Embodiment 8. FIG.

  The receiver finds an area where bright lines exist from the captured image 8913a, and performs zooming so that as many bright lines as possible are generated. As in the captured image 8913b, the number of bright lines can be maximized by enlarging the bright line region in the direction perpendicular to the bright line direction until the bright line region reaches the upper and lower ends of the screen.

  In addition, the receiver may find an area where the bright line is clearly displayed, and perform zooming so that the portion appears large like 8913c.

  In addition, when a some bright line area | region exists in a captured image, it is good also as performing said process sequentially about each bright line area. Further, the above processing may be performed for the bright line region designated by the user from the captured image.

(Image data size compression method suitable for visible light signal reception)
FIG. 191 is a diagram illustrating an example of a reception method in Embodiment 8.

  When the image data size needs to be compressed when the captured image (a) is transmitted from the image capturing unit to the image processing unit or transmitted from the image capturing terminal (receiver) to the server, as shown in (c) If the reduction or pixel thinning is performed in a direction horizontal to the bright line, the data size can be compressed without reducing the information amount of the bright line. When reduction or pixel thinning is performed as in (b) or (d), the number of bright lines is reduced or it is difficult to recognize bright lines. Even during image compression, reduction in reception efficiency can be prevented by not compressing in the direction perpendicular to the bright line, or by making the compression rate in the vertical direction smaller than the compression rate in the horizontal direction. Note that the moving average filter may be applied in both the vertical direction and the horizontal direction, and is effective for both data size reduction and noise reduction.

  That is, the receiver in the present embodiment further generates a compressed image by compressing the bright line image in a direction parallel to each of the multiple bright lines included in the bright line image, and transmits the compressed image. .

(Modulation method with high reception error detection accuracy)
FIG. 192 is a diagram illustrating an example of a signal modulation method in Embodiment 8.

  Since error detection using a parity bit detects a 1-bit reception error, it is impossible to detect a mistake between “01” and “10” and a difference between “00” and “11”. In the modulation method of (a), “01” and “10” are easy to be mistaken because only one position of L is different. However, in the modulation method of (b), “01”, “10”, “00” There are two different positions of L for “11”. By using the modulation scheme (b), reception errors can be detected with high accuracy. The same applies to the modulation schemes shown in FIGS.

  That is, in the present embodiment, two luminance change patterns whose timings at which a predetermined luminance value (for example, L) appears are assigned to the same parity signal unit (for example, “01” and “10”). As is not shown, luminance change patterns having different timings are assigned in advance to signal units having different timings. The transmitter according to the present embodiment determines a luminance change pattern assigned to each signal unit included in the signal to be transmitted.

(Change in receiver operation due to different circumstances)
193 is a diagram illustrating an example of operation of a receiver in Embodiment 8. FIG.

  The receiver 8920a operates differently depending on the situation where reception is started. For example, when activated in Japan, a signal modulated by a 60 kHz phase shift keying is received, and data is downloaded from the server 8920d using the received ID as a key. When activated in the United States, a signal modulated by a 50 kHz frequency shift keying is received, and data is downloaded from the server 8920e using the received ID as a key. In the situation where the operation of the receiver is changed, the location (country or building) where the receiver 8920a exists, the base station or the wireless access point (Wi-Fi, Bluetooth (registered trademark) that communicates with the receiver 8920a is registered. ), IMES, etc.) and time.

  For example, the receiver 8920a uses the location information and the last accessed wireless base station (a base station such as a carrier communication network, Wi-Fi, Bluetooth (registered trademark), or IMES) or the last received ID by visible light communication. To server 8920f. The server 8920f estimates the position of the receiver 8920a based on the received information, and receives the transmission algorithm of the transmitter near the position, and ID management that manages the ID of the transmitter near the position Send server information (URI, etc.). The receiver 8920a receives signals from the transmitters 8920b and 8920c using the received algorithm, and makes an inquiry to the received ID management servers 8920d and 8920e using the ID as a key.

  By this method, different types of communication can be performed in units such as countries, regions, and buildings. In addition, when receiving a signal, the receiver 8920a in this embodiment switches a server to be accessed or switches a reception algorithm in accordance with a frequency used for modulation of the signal, as illustrated in FIG. The signal modulation method may be switched.

(Well-known communication of visible light to humans)
194 is a diagram illustrating an example of operation of a transmitter in Embodiment 8. FIG.

  As shown in FIG. 194 (a), the light emitting unit of the transmitter 8921a repeats blinking visible to humans and visible light communication. By performing blinking that is visible to humans, it is possible to inform humans that visible light communication is possible. The user notices that visible light communication is possible because the transmitter 8921a is blinking, performs visible light communication with the receiver 8921b directed to the transmitter 8921a, and performs user registration of the transmitter 8921a.

  That is, the transmitter according to the present embodiment alternately and repeatedly performs a step in which the light emitter transmits a signal according to a change in luminance and a step in which the light emitter blinks so as to be visually recognized by human eyes.

  The transmitter may separately provide a visible light communication unit and a blinking unit (communication status display unit) as shown in FIG.

  The transmitter operates as shown in (c) of FIG. 194 using the modulation method shown in FIG. 77 or 78, so that the light emitting unit blinks for humans while performing visible light communication. Can show. That is, for example, the transmitter repeatedly performs high luminance visible light communication with a brightness of 75% and low luminance visible light communication with a brightness of 1% alternately. For example, when an abnormality or the like occurs in the transmitter and a signal different from usual is transmitted, the operation shown in (c) of FIG. 194 is performed to alert the user without stopping the visible light communication. Can do.

(Expansion of reception range with diffuser)
FIG. 195 is a diagram illustrating an example of a receiver in Embodiment 8.

  FIG. 195 (a) shows the normal mode of the receiver 8922a, and FIG. 195 (b) shows the visible light communication mode of the receiver 8922a. The receiver 8922a includes a diffuser plate 8922b in front of the imaging unit. The receiver 8922a moves the diffuser plate 8922b in front of the imaging unit in the visible light communication mode so that the light source spreads and images are taken. At this time, the position of the light diffusing plate 8922b is adjusted so that light from a plurality of light sources does not overlap. Note that a macro lens or a zoom lens may be used instead of the diffuser plate 8922b. Thereby, the signal of a distant transmitter and a small transmitter can be received.

  Note that the imaging direction of the imaging unit may be moved instead of moving the diffuser plate 8922b.

  Note that the area of the image sensor in which the diffuser plate 8922b is reflected may not be used in the normal imaging mode but may be used only in the visible light communication mode. As a result, the above-described effect can be obtained without moving the diffuser plate 8922b and the imaging unit.

(Synchronization method of signal transmission from multiple transmitters)
196 and 197 are diagrams illustrating an example of a transmission system according to the eighth embodiment.

  When a plurality of projectors are used for projection mapping or the like, in order to avoid interference, only one projector needs to transmit a signal for one projection or the signal transmission timings of a plurality of projectors must be synchronized. FIG. 196 shows a mechanism for synchronization of transmission timing.

  Projector A and projector B that project onto the same projection surface transmit signals as shown in FIG. The receiver captures and receives this, calculates the time difference between the signal a and the signal b, and adjusts the signal transmission timing of each projector.

  Since the projector A and the projector B are not synchronized at the start of operation, a time during which neither of the signals is transmitted (total pause time) is provided to prevent the signals a and b from being overlapped and cannot be received. As the timing adjustment of the projector proceeds, the signal transmitted by the projector may be changed. For example, the timing can be adjusted efficiently by increasing the total pause time at the start of the operation and shortening the total pause time as the timing adjustment proceeds.

  In order to accurately adjust the timing, it is desirable that the signal a and the signal b are included in one captured image. The imaging frame rate of the receiver is often from 60 fps to 7.5 fps. By setting the signal transmission period to 1 / 7.5 second or less, the signal a and the signal b can be contained in an image captured at 7.5 fps. Further, by setting the signal transmission cycle to 1/60 second or less, the signal a and the signal b can be reliably contained in an image captured at 30 fps.

  FIG. 197 is a diagram illustrating a case where transmitters configured by a plurality of displays are synchronized. An image is captured so that the synchronized display fits in one image, and the timing is adjusted.

(Reception of visible light signal by illuminance sensor and image sensor)
198 is a diagram illustrating an example of operation of a receiver in Embodiment 8. FIG.

  Since the image sensor has higher power consumption than the illuminance sensor, the receiver 8940a activates the image sensor 8940b to receive the signal when the illuminance sensor 8940c detects the signal. As shown in FIG. 198 (a), the receiver 8940a receives the signal transmitted from the transmitter 8940d by the illuminance sensor 8940c. Thereafter, the receiver 8940a activates the image sensor 8940b, receives the transmission signal of the transmitter 8940d by the image sensor, and recognizes the position of the transmitter 8940d. At this time, if the image sensor 8940b receives a part of the signal and if the part of the signal is the same as the signal received by the illuminance sensor 8940c, the receiver 8940a temporarily determines that the same signal has been received. For example, display of the current position. The determination is completed when all the signals are received by the image sensor 8940b.

  At the time of provisional determination, it may be displayed that the determination is not completed. For example, the display of the current position is made translucent, or the position error is displayed.

  The part of the signal can be determined as, for example, 20% of the entire signal length or an error detection code part.

  In the situation shown in FIG. 198 (b), the illuminance sensor 8940c cannot receive the signal due to interference, but can recognize that the signal exists. For example, when the sensor value of the illuminance sensor 8940c is Fourier-transformed, if a peak appears in the modulation frequency of the transmission signal, it can be estimated that the signal exists. When the receiver 8940a estimates that a signal is present from the sensor value of the illuminance sensor 8940c, the receiver 8940a activates the image sensor 8940b and receives signals from the transmitters 8940e and 8940f.

(Reception start trigger)
199 is a diagram illustrating an example of operation of a receiver in Embodiment 8. FIG.

  Since power is consumed when an image sensor or illuminance sensor (hereinafter collectively referred to as a light receiving sensor) is activated, it is stopped when not needed and activated when necessary to improve power consumption efficiency. Can be improved. Since the power consumption of the illuminance sensor is less than that of the image sensor, the illuminance sensor may be always activated and control only the image sensor.

  In FIG. 199 (a), the receiver 8941a detects movement from the sensor value of the 9-axis sensor, activates the light receiving sensor, and starts reception.

  In FIG. 199 (b), the operation of tilting the receiver is detected from the sensor values of the 9-axis sensor, and the light receiving sensor on the side facing upward is activated to start reception.

  In FIG. 199 (c), the operation of protruding the receiver is detected from the sensor values of the 9-axis sensor, and the light receiving sensor in the protruding direction is activated to start reception.

  In (d) of FIG. 199, from the sensor values of the 9-axis sensor, an operation of turning the receiver upward or a motion of shaking is detected, and the light receiving sensor on the side directed upward is activated to start reception.

  That is, the receiver in the present embodiment further determines whether or not the receiver (receiving device) has been moved in a predetermined manner, and determines that the receiver has been moved in a predetermined manner. Start the sensor.

(Reception start gesture)
FIG. 200 is a diagram illustrating an example of a gesture operation for starting reception according to the communication method.

  For example, the receiver 8942a configured as a smartphone detects, from the sensor value of the 9-axis sensor, an operation of standing the receiver and sliding it in the horizontal direction or an operation of repeating the sliding in the horizontal direction. Thereafter, the receiver 8942a starts reception, and acquires the position of the transmitter 8942b based on the received ID. Thereafter, the receiver 8942a acquires its position from the relative positional relationship between the plurality of transmitters 8942b and itself. By sliding the receiver 8942a, the plurality of transmitters 8942b can be stably imaged, and the self-position can be estimated with high accuracy by the triangulation method.

  This operation may be performed only when the home screen is in the foreground. Thereby, it is possible to prevent this communication from being performed against the user's intention while using another application.

(Application examples for car navigation systems)
201 and 202 are diagrams illustrating an example of an application example of the transmission and reception system in the eighth embodiment.

  For example, the transmitter 8950b configured as a car navigation system transmits information for wireless connection to the transmitter 8950b, such as Bluetooth (registered trademark) pairing information, a Wi-Fi SSID and password, and an IP address. For example, the receiver 8950a configured as a smartphone establishes a wireless connection with the transmitter 8950b based on the received information, and subsequent exchanges are performed via this wireless connection.

  For example, the user inputs a destination, shop information to be searched, or the like to the smartphone 8950a, the smartphone 8950a transmits the input information to the car navigation 8950b via a wireless connection, and the car navigation 8950b displays route information. For example, the smartphone 8950a operates as a controller of the car navigation 8950b, and controls music and video that the car navigation 8950b reproduces. Further, for example, music and video held by the smartphone 8950a are reproduced by the car navigation system 8950b. Further, for example, the car navigation system 8950b acquires peripheral store information and road congestion information and displays the information on the smartphone 8950a. For example, when an incoming call is received, the smartphone 8950a performs a call process using a microphone and a speaker of the car navigation 8950b that are wirelessly connected. Note that the smartphone 8950a may perform the above operation by establishing a wireless connection when an incoming call is received.

  The car navigation system 8950b automatically establishes a wireless connection with a registered terminal when the wireless connection is set to the automatic connection mode. When not in the automatic connection mode, the car navigation system 8950b transmits connection information using visible light communication and waits for a connection. Note that the car navigation system 8950b may also wait for a connection by transmitting connection information using visible light communication even in the automatic connection mode. When the car navigation system is manually connected, the automatic connection mode may be canceled, and further, the connection with the automatically connected terminal may be disconnected.

(Application examples for content protection)
FIG. 203 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  For example, the transmitter 8951b configured as a television transmits content protection information held by itself or a connected device 8951c. For example, the receiver 8951a configured as a smartphone receives the content protection information, and then protects the content so that the content protected by the content protection information of the transmitter 8951b or the device 8951c can be played for a predetermined time. I do. Thereby, the content held in another device owned by the user can be reproduced by the receiver.

  The transmitter 8951b may store the content protection information in the server, and the receiver 8951a may acquire the content protection information from the server using the received ID of the transmitter 8951b as a key.

  Note that the receiver 8951a may transmit the acquired content protection information to another device.

(Application example as an electronic lock)
FIG. 204A is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8952a receives the ID transmitted from the transmitter 8952b and transmits it to the server 8952c. When the ID of the transmitter 8952b is sent from the receiver 8952a, the server 8952c unlocks the door 8952d, opens the automatic door, or moves to the floor registered in the receiver 8952a. Call to the floor with 8952a. Accordingly, the receiver 8952a serves as a key, and the user can perform unlocking or the like before reaching the door 8952d.

  That is, the receiver in the present embodiment acquires a first bright line image that is an image including a plurality of bright lines by photographing a subject whose luminance changes (for example, the above-described transmitter), and acquires the first bright line image acquired. First transmission information (for example, subject ID) is acquired by demodulating data specified by a plurality of bright line patterns included in the bright line image. Then, after the first transmission information is acquired, the receiver causes the door opening / closing drive device to open the door by transmitting a control signal (for example, a subject ID).

  In order to prevent malicious operation, the server 8952c may confirm that the communication partner is the receiver 8952a by using security protection such as a secure element of the receiver 8952a. Also, to confirm that the receiver 8952a is indeed near the transmitter 8952b, when the server 8952c receives the ID of the transmitter 8952b, it issues a command to send a different signal to the transmitter 8952b, The unlocking may be performed when the signal is sent from the receiver 8952a.

  When a plurality of transmitters 8952b configured as lighting devices are arranged along the path toward the door 8952d, the receiver 8952a receives the ID from the transmitters 8952b, thereby receiving the receivers. It can be determined whether 8952a is approaching door 8952d. For example, when the values indicated by the IDs are reduced in the order in which the IDs are acquired, the receiver determines that the door is approaching. Or a receiver pinpoints the position of each transmitter 8952b based on those ID, Furthermore, based on the position of those transmitters 8952b, and the position of those transmitters 8952b reflected in a picked-up image Estimate your position. Then, the receiver determines whether or not the door 8952d is approaching by comparing the position of the door 8952d held in advance with the estimated position of the door 8952d as needed. When the receiver determines that the door 8952d is approaching, the receiver transmits one of the acquired IDs to the server 8952c. As a result, the server 8952c performs processing for opening the door 8952d.

  That is, the receiver in the present embodiment acquires a second bright line image that is an image including a plurality of bright lines by photographing another subject whose luminance changes, and is included in the acquired second bright line image. Second transmission information (for example, IDs of other subjects) is acquired by demodulating data specified by the plurality of bright line patterns. The receiver determines whether or not the receiver is approaching the door based on the acquired first and second transmission information, and determines that the receiver is approaching the door. , ID of any subject).

  FIG. 204B is a flowchart of the information communication method in this embodiment.

  The information communication method in the present embodiment is an information communication method for acquiring information from a subject, and includes steps SK21 to SK24.

  That is, in this information communication method, a plurality of bright lines corresponding to each exposure line included in the image sensor are displayed on the image obtained by photographing the first subject, which is the subject, by the image sensor. A first exposure time setting step SK21 for setting a first exposure time of the image sensor so as to occur according to a change, and the first subject for which the image sensor has changed in luminance is set as the first exposure time. A first bright line image acquisition step SK22 for acquiring a first bright line image, which is an image including the plurality of bright lines, by photographing with one exposure time, and the first bright line image included in the acquired first bright line image A first information acquisition step SK23 for acquiring first transmission information by demodulating data specified by a plurality of bright line patterns; After the transmission information is acquired, and by sending a control signal, and a door control step SK24 to open the door against the opening and closing devices of the door.

  FIG. 204C is a block diagram of the information communication apparatus in this embodiment.

  The information communication device K20 in the present embodiment is an information communication device that transmits a signal by a change in luminance, and includes constituent elements K21 to K24.

  That is, the information communication device K20 is configured so that a plurality of bright lines corresponding to each exposure line included in the image sensor are generated according to a change in luminance of the subject in an image obtained by photographing the subject by an image sensor. The exposure time setting unit K21 that sets the exposure time of the image sensor, and the bright-line image that is an image including the plurality of bright lines is acquired by photographing the subject that changes in brightness at the set exposure time. Bright line image acquisition unit K22 having an image sensor, information acquisition unit K23 that acquires transmission information by demodulating data specified by the plurality of bright line patterns included in the acquired bright line image, and the transmission information Is acquired, the door is opened with respect to the door opening and closing drive equipment by sending a control signal And a that door control unit K24.

  In such an information communication method and information communication apparatus K20 shown in FIGS. 204B and 204C, for example, as shown in FIG. 204A, a receiver including an image sensor can be used like a door key, An electronic lock can be dispensed with. As a result, it is possible to perform communication between various devices including a device having a small computing power.

  In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. For example, the program causes the computer to execute the information communication method shown by the flowchart in FIG. 204B.

(Application example for store visit information transmission)
FIG. 205 is a diagram illustrating an example of application of the transmission and reception system in Embodiment 8.

  The receiver 8953a transmits the ID transmitted from the transmitter 8953b to the server 8953c. The server 8953c notifies the clerk 8953d of the order information associated with the receiver 8953a. The store clerk 8953d prepares goods and the like based on the order information. Thereby, since an order is prepared when a user enters a store etc., the user can receive goods etc. quickly.

(Application example of order control according to location)
FIG. 206 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8954a displays a screen on which an order can be placed only when the transmission signal of the transmitter 8954b is received. Thereby, the store can avoid the order of the customer who is not near.

  Alternatively, the receiver 8954a places an order by adding the ID of the transmitter 8954b to the order information. Thereby, the store can grasp the position of the orderer and grasp the position where the product is delivered. Alternatively, the store can estimate the time until the orderer visits the store. Note that the receiver 8954a may add the travel time to the store calculated from the travel speed to the order information. Further, the receiver may not accept the purchase for an unnatural purchase (for example, purchase of a boarding pass that departs from other than the station at the current location) as judged from the current location.

(Example of application to directions)
FIG. 207 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8955a receives, for example, the transmission ID of the transmitter 8955b configured as a guide board, acquires map data displayed on the guide board from the server, and displays the map data. At this time, the server may transmit an advertisement suitable for the user of the receiver 8955a, and the receiver 8955a may also display this advertisement information. The receiver 8955a displays a route from the current location to a location designated by the user.

(Application example for location communication)
FIG. 208 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8956a receives, for example, an ID transmitted from a transmitter 8956b configured as home or school lighting, and transmits position information acquired using the ID as a key to the terminal 8956c. Thus, the guardian with the terminal 8906c can know that the child with the receiver 8956a has gone home or has arrived at school. Further, the supervisor having the terminal 8956c can grasp the current position of the worker having the receiver 8956a.

(Application log storage and application examples)
FIG. 209 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8957a receives, for example, an ID transmitted from the transmitter 8957b configured as a signboard, acquires coupon information from the server, and displays the coupon information. The receiver 8957a stores subsequent user actions such as saving a coupon, moving to a store displayed on the coupon, shopping at the store, and leaving without saving the coupon. Save to 8957c. As a result, the subsequent behavior of the user who has acquired information from the sign 8957b can be analyzed, and the advertising value of the sign 8957b can be estimated.

(Application example for screen sharing)
210 and 211 are diagrams illustrating an example of application examples of the transmission and reception system in the eighth embodiment.

  For example, the transmitter 8960b configured as a projector or a display transmits information (SSID, wireless connection password, IP address, password for operating the transmitter) for wireless connection to itself. Alternatively, an ID serving as a key for accessing these pieces of information is transmitted. For example, the receiver 8960a configured as a smartphone, a tablet, a laptop computer, or a camera receives the signal transmitted from the transmitter 8960b, acquires the information, and establishes a wireless connection with the transmitter 8960b. This wireless connection may be connected via a router, or may be directly connected by Wi-Fi Direct, Bluetooth (registered trademark), Wireless Home Digital Interface, or the like. The receiver 8960a transmits a screen displayed by the transmitter 8960b. Thereby, the image of the receiver can be easily displayed on the transmitter.

  When the transmitter 8960b is connected to the receiver 8960a, the transmitter 8960b notifies the receiver 8960a that a password is required in addition to the information transmitted by the transmitter in order to display the screen. If is not sent, the transmitted screen may not be displayed. At this time, the receiver 8960a displays a password input screen such as 8960d and allows the user to input the password.

  FIG. 211 is a diagram illustrating an example in which the screen of the transmitter 8961c is displayed on the transmitter 8960b via the receiver 8960a. For example, the transmitter 8961c configured as a notebook personal computer transmits information for connection to itself or an ID associated with the information. The receiver 8960a receives the signal transmitted from the transmitter 8960b and the signal transmitted from the transmitter 8961c, establishes a connection with each transmitter, and displays an image to be displayed on the transmitter 8960b from the transmitter 8961c. Let it transmit. The transmitter 8960b and the transmitter 8961c may directly communicate with each other, or may communicate with each other via the receiver 8960a or a router. Accordingly, even when the signal transmitted from the transmitter 8960b cannot be received by the transmitter 8961c, an image of the transmitter 8961c can be easily displayed on the transmitter 8960b.

  Note that the receiver 8960a may perform the above operation only when the difference between the time when the signal transmitted by the transmitter 8960b is received and the time when the signal transmitted by the transmitter 8961c is received is equal to or less than a predetermined time. Good.

  Note that the transmitter 8961c may transmit an image to the transmitter 8960b only when the correct password is received from the receiver 8960a.

(Application example of position estimation using wireless access point)
FIG. 212 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  For example, the receiver 8963a configured as a smartphone receives the ID transmitted by the transmitter 8963b. The receiver 8963a acquires position information of the transmitter 8963b using the received ID as a key, and estimates its own position from the position and direction of the transmitter 8963b in the captured image. The receiver 8963a receives a signal from a wireless transmitter 8963c configured as, for example, a Wi-Fi access point. The receiver 8963a estimates its own position from the position information of the wireless transmitter 8963c and the radio wave transmission direction information included in the signal. The receiver 8963a can estimate the self-position with high accuracy by estimating the self-position using a plurality of means.

  A method for estimating the self-location using information of the wireless transmitter 8963c will be described. The wireless transmitter 8963c transmits synchronized signals in different directions from a plurality of antennas. In addition, the wireless transmitter 8963c sequentially changes the signal transmission direction. The receiver 8963a estimates that the radio wave transmission direction when the radio wave intensity is the strongest is the direction from the radio transmitter 8963c to itself. Further, a path difference is calculated from a difference in arrival times of radio waves transmitted from different antennas and respectively passing through paths 8963d, 8963e, and 8963f, and the radio transmitter 8963c and itself are calculated from the radio wave transmission angle differences θ12, θ13, θ23. Calculate the distance. Further, by using the surrounding electric field information and radio wave reflector information, the self-position can be estimated with higher accuracy.

(Position estimation by visible light communication and wireless communication)
FIG. 213 is a diagram illustrating a configuration for performing position estimation using visible light communication and wireless communication. That is, FIG. 213 shows a configuration for performing terminal position estimation using visible light communication and wireless communication.

  The portable terminal (smart phone terminal) acquires the ID of the light emitting unit by performing visible light communication with the light emitting unit. The server inquires of the acquired ID for the position information of the light emitting unit. Thereby, the mobile terminal acquires an actual distance L1 and an actual distance L2, which are distances in the x direction and the y direction, respectively, between the MIMO (multiple-input and multiple-output) access point and the light emitting unit. Further, as already described in other embodiments, the mobile terminal detects the inclination θ1 of the mobile terminal using a gyro sensor or the like.

  On the other hand, when beam forming is performed from the MIMO access point toward the mobile terminal, the beam form angle θ2 is set by the MIMO access point and is a known value. Therefore, the mobile terminal acquires the beam forming angle θ2 by wireless communication or the like.

  As a result, the portable terminal uses the actual distance L1 and the actual distance L2, the inclination θ1 of the portable terminal, and the beam forming angle θ2 to determine the coordinate position (x1, y1) of the portable terminal relative to the MIMO access point. Can be calculated. Since a MIMO access point can form a plurality of beams, it is possible to estimate a position with higher accuracy by using a plurality of beam forming.

  As described above, according to the present embodiment, it is possible to further improve the accuracy of position estimation by using both position estimation by visible light communication and position estimation by wireless communication.

  As described above, the information communication method according to one or more aspects has been described based on the embodiment. However, the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

  In addition, as illustrated in FIGS. 214, 215, and 216, an information communication method according to one embodiment of the present invention may be applied.

  FIG. 214 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  A camera configured as a receiver for visible light communication performs imaging in a normal imaging mode (Step 1). By this imaging, the camera acquires an image file configured in a format such as EXIF (Exchangeable image file format). Next, the camera performs imaging in the visible light communication imaging mode (Step 2). Based on the bright line pattern in the image obtained by this imaging, the camera acquires a signal (visible light communication information) transmitted by the visible light communication from the transmitter that is the subject (Step 3). Furthermore, the camera obtains information corresponding to the key from the server by using the signal (reception information) as a key and accessing the server (Step 4). The camera transmits a signal (visible light reception data) transmitted from the subject by visible light communication, information acquired from the server, and data indicating a position where the transmitter as the subject is projected in the image indicated by the image file. And the data indicating the time when the signal transmitted by the visible light communication is received (the time in the moving image) are stored as metadata in the above-described image file. Note that when a plurality of transmitters are projected as subjects in an image (image file) obtained by imaging, the camera stores, for each transmitter, some metadata corresponding to the transmitter. Save to file.

  When a display or projector configured as a transmitter for visible light communication displays an image indicated by the above-described image file, the display or projector transmits a signal corresponding to metadata included in the image file by visible light communication. For example, the display or the projector may transmit the metadata itself by visible light communication, or may transmit a signal associated with the transmitter displayed in the image as a key.

  A portable terminal (smart phone) configured as a receiver for visible light communication receives a signal transmitted by visible light communication from the display or projector by capturing an image of the display or projector. If the received signal is the above-described key, the portable terminal uses the key to acquire the transmitter metadata associated with the key from the display, projector, or server. In addition, if the received signal is a signal (visible light reception data or visible light communication information) transmitted from an actual transmitter by visible light communication, the portable terminal receives the visible light from a display, a projector, or a server. Information corresponding to received light data or visible light communication information is acquired.

  FIG. 215 is a flowchart illustrating operation of the camera (receiver) of the transmission / reception system according to the eighth embodiment.

  First, when the camera detects that the imaging button is pressed (step S901), the camera performs imaging in the normal imaging mode (step S902). Then, the camera performs imaging in the visible light imaging mode by increasing the shutter speed to a certain speed or higher, that is, by setting an exposure time shorter than that in the normal imaging mode (step S903). Thereby, the camera acquires a signal transmitted from the subject by visible light communication.

  Thereafter, the camera treats a signal (information) obtained by visible light communication as a key, thereby obtaining information associated with the key from the server (step S905). Next, the camera stores the signal, each piece of information, and each piece of data in a metadata area (such as an area in which EXIF meter data is stored) of an image file obtained by imaging in the normal imaging mode (step S905). . That is, the camera captures a signal acquired by visible light communication, information acquired from a server, and an image of a transmitter that is a subject that has transmitted the signal by visible light communication (an image obtained by imaging in a normal imaging mode). Stores position data indicating the position inside.

  Then, the camera determines whether or not to capture a moving image (step S906). If it is determined to capture a moving image (Y in step S906), the process from step S902 is repeatedly executed. On the other hand, if the camera determines not to capture a moving image (N in step S906), the imaging process ends.

  FIG. 216 is a flowchart illustrating operation of the display (transmitter) of the transmission / reception system according to the eighth embodiment.

  First, the display checks the metadata area of the image file to determine whether there is one or more transmitters shown in the image indicated by the image file (step S911). Here, when the display determines that there are a plurality of transmitters (a plurality of steps S911), the display further determines whether or not the split transmission mode is set as the visible light communication mode (step S912). When the display determines that the divided transmission mode is set (Y in step S912), the display divides the display area (transmission portion) of the display and transmits a signal from each display area (step S914). Specifically, for each transmitter, the display treats the area in which the transmitter is projected, or the area in which the transmitter and its surroundings are projected, as a display area. A signal corresponding to the machine is transmitted by visible light communication.

  On the other hand, when the display determines in step S912 that the divided transmission mode is not set (N in step S912), signals corresponding to each of the plurality of transmitters are transmitted by visible light communication from the entire display area of the display. Transmit (step S913). That is, the display transmits keys associated with a plurality of pieces of information from the entire screen.

  When the display determines that there is one transmitter in step S911 (one of step S911), the display transmits a signal corresponding to the single transmitter from the entire display area of the display through visible light communication. (Step S915). That is, the display transmits from the entire screen.

  Furthermore, when the display accepts, for example, from a portable terminal (smartphone) an access that uses a signal (transmission information) transmitted by visible light communication as a key after any of steps S913 to S915, the display receives the key in the image file. Corresponding metadata is passed to the mobile terminal that is the access source (step S916).

(Summary of this embodiment etc.)
The information communication method in the present embodiment is an information communication method for acquiring information from a subject, and each exposure included in the image sensor is included in an image obtained by photographing the first subject that is the subject by an image sensor. A first exposure time setting step for setting a first exposure time of the image sensor so that a plurality of bright lines corresponding to a line are generated according to a change in luminance of the first subject; A first bright line image acquisition step of acquiring a first bright line image that is an image including the plurality of bright lines by photographing the first subject that changes with the set first exposure time; The first transmission information is acquired by demodulating data specified by the pattern of the plurality of bright lines included in the acquired first bright line image. An information obtaining step of, after said first transmission information is obtained by sending a control signal, and a door control steps to open the door against the opening and closing devices of the door.

  Thus, for example, as shown in FIG. 204A, a receiver including an image sensor can be used like a door key, and a special electronic lock can be dispensed with. As a result, it is possible to perform communication between various devices including a device having a small computing power.

  The information communication method may further include a second image in which the image sensor includes a plurality of bright lines by photographing the second subject whose luminance changes with the set first exposure time. Second transmission information is acquired by demodulating data specified by a pattern of the plurality of bright lines included in the acquired second bright line image and a second bright line image acquiring step of acquiring a bright line image A second information acquisition step; and an approach determination step of determining whether or not a receiving device including the image sensor is approaching the door based on the acquired first and second transmission information. In the door control step, the control signal may be transmitted when it is determined that the receiving device is approaching the door.

  Thereby, for example, as shown in FIG. 204A, the door can be opened only when the receiving device (receiver) approaches the door, that is, only at an appropriate timing.

  Further, in the information communication method, a second exposure time setting step for setting a second exposure time longer than the first exposure time, and the image sensor sets a third subject. A normal image acquisition step of acquiring a normal image on which the third subject is projected by photographing at the second exposure time, and the normal image acquisition step includes an area including optical black of the image sensor For each of the plurality of exposure lines, the charge is read after a predetermined time has elapsed from the time when the charge is read for the exposure line adjacent to the exposure line, and the first bright line image obtaining step is performed. Then, other than the optical black in the image sensor, without using the optical black for reading out charges. For each of the plurality of exposure lines in the region, the charge may be read after elapse of a time longer than the predetermined time from the time when the charge is read for the exposure line adjacent to the exposure line. Good.

  As a result, for example, as shown in FIGS. 187A to 187E, when the first bright line image is acquired, the readout (exposure) of the charge from the optical black is not performed, and thus the area other than the optical black in the image sensor. It is possible to lengthen the time required for reading (exposure) the charge with respect to the effective pixel region. As a result, it is possible to increase the time for receiving a signal in the effective pixel region, and it is possible to acquire many signals.

  Further, in the information communication method, the length in the direction perpendicular to each of the plurality of bright lines in the plurality of bright line patterns included in the first bright line image is less than a predetermined length. A length determination step for determining whether or not there is a frame rate of the image sensor when the pattern length is determined to be less than the predetermined length; A frame rate changing step of changing to a second frame rate that is slower than the first frame rate when the image is acquired; and the image sensor detects the first subject whose luminance changes at the second frame rate. And a third bright line image acquisition step of acquiring a third bright line image that is an image including a plurality of bright lines by photographing at the set first exposure time, and acquisition May include a third information obtaining step of obtaining the first transmission information by demodulating the data identified by said plurality of emission lines pattern included in the third bright line image.

  Thus, for example, as shown in FIG. 224A, when the signal length indicated by the bright line pattern (bright line region) included in the first bright line image is less than one block of the transmitted signal, for example, The rate is lowered, and the bright line image is acquired again as the third bright line image. As a result, the length of the bright line pattern included in the third bright line image can be increased, and the transmitted signal can be acquired for one block.

  The information communication method further includes a ratio setting step for setting a ratio between a vertical width and a horizontal width of an image obtained by the image sensor, and the first bright line image acquisition step includes the set ratio. A clipping determination step for determining whether or not an end in a direction perpendicular to each exposure line in the image is clipped, and when it is determined that the end is clipped, the ratio set in the ratio setting step Changing the ratio to a non-clipping ratio which is a ratio at which the edge is not clipped, and the image sensor captures the first bright line with the non-clipping ratio by photographing the first subject whose luminance changes. An acquisition step of acquiring an image.

  Accordingly, as shown in FIGS. 188 and 189A to 189C, for example, the ratio of the horizontal width to the vertical width of the effective pixel region of the image sensor is 4: 3, and the ratio of the horizontal width to the vertical width of the image is 16: When the bright line along the horizontal direction appears, that is, when the exposure line is along the horizontal direction, it is determined that the upper end and the lower end of the image are clipped. That is, it is determined that the end of the first bright line image is missing. In this case, the ratio of the image is changed to 4: 3, which is a ratio that is not clipped. As a result, it is possible to prevent the end of the first bright line image from being lost, and a large amount of information can be acquired from the first bright line image.

  The information communication method further includes a compression step of generating a compressed image by compressing the first bright line image in a direction parallel to each of the plurality of bright lines included in the first bright line image. And a compressed image transmission step of transmitting the compressed image.

  Thereby, as shown in FIG. 191, for example, the first bright line image can be appropriately compressed without missing information indicated by a plurality of bright lines.

  The information communication method further determines that the receiving device including the image sensor has been moved in a predetermined manner, and a gesture determination step for determining whether or not the receiving device has been moved in a predetermined manner. In some cases, an activation step of activating the image sensor may be included.

  Accordingly, for example, as shown in FIG. 199, the image sensor can be easily activated only when necessary, and the power consumption efficiency can be improved.

(Embodiment 9)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  FIG. 217 is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9.

  The robot 8970 has, for example, a function as a self-propelled cleaner and a function as a receiver in each of the above embodiments. The lighting devices 8971a and 8971b each have a function as a transmitter in each of the above embodiments.

  For example, the robot 8970 performs cleaning while moving in the room and photographs the lighting device 8971a that illuminates the room. The lighting device 8971a transmits the ID of the lighting device 8971a by changing the luminance. As a result, the robot 8970 receives the ID from the lighting device 8971a and estimates its own position (self-position) based on the ID as in the above embodiments. That is, the robot 8970 moves itself based on the detection result by the 9-axis sensor, the relative position of the lighting device 8971a reflected in the image obtained by photographing, and the absolute position of the lighting device 8971a specified by the ID. Is estimated.

  Further, when the robot 8970 moves away from the lighting device 8971a, the robot 8970 transmits a signal (turn-off command) to instruct the lighting device 8971a to turn off. For example, when the robot 8970 leaves the lighting device 8971a by a predetermined distance, the robot 8970 transmits a turn-off command. Alternatively, the robot 8970 transmits a turn-off command to the lighting device 8971a when the lighting device 8971a does not appear in the image obtained by shooting or when another lighting device appears in the image. When the lighting device 8971a receives a turn-off command from the robot 8970, the lighting device 8971a turns off according to the turn-off command.

  Next, the robot 8970 detects that the lighting device 8971b has been approached based on the estimated self-position while moving and cleaning. That is, the robot 8970 holds information indicating the position of the lighting device 8971b, and when the distance between the self position and the position of the lighting device 8971b is equal to or less than a predetermined distance, the lighting device 8971b. Detecting that you are approaching. Then, the robot 8970 transmits a signal (lighting command) for instructing lighting to the lighting device 8971b. When the lighting device 8971b receives the lighting command, the lighting device 8971b lights up in accordance with the lighting command.

  Thus, the robot 8970 can easily perform cleaning by brightening only the surroundings of the robot 8970 while moving.

  FIG. 218 is a diagram illustrating an example of application of the transmitter in Embodiment 9.

  For example, as shown in FIG. 218 (a), a plurality of light emitting areas A to F are displayed side by side on the display, and a signal is transmitted by changing the luminance of each of the light emitting areas A to F. Here, in the example shown to (a) of FIG. 218, the light emission area | regions AF are each rectangular shape, and are arranged along the horizontal direction and the vertical direction. In such a case, a non-luminance changing area that does not change in luminance crosses the display through the light emitting areas A, B, and C and the light emitting areas D, E, and F along the horizontal direction of the display. Yes. Further, another non-luminance changing area that does not change in luminance crosses the display along the vertical direction of the display, passing between the light emitting areas A and D and the light emitting areas B and E. Further, other non-luminance changing areas that do not change in luminance cross the display through the light emitting areas B and E and the light emitting areas C and F along the vertical direction of the display.

  Therefore, when the receiver in each of the above embodiments photographs the display with the exposure line of the receiver oriented in the horizontal direction, the non-luminance change along the horizontal direction of the image (captured image) obtained by the imaging A bright line does not appear in the portion corresponding to the region. That is, in the captured image, the region where the bright line appears (bright line region) becomes discontinuous. Further, when the receiver captures the display with the exposure line oriented in the vertical direction, no bright line appears in the portion corresponding to the two non-luminance change regions along the vertical direction in the captured image. That is, even at this time, the bright line region becomes discontinuous in the captured image. When the bright line region becomes discontinuous in this way, it becomes difficult to receive a signal transmitted due to a change in luminance.

  Therefore, the display 8972 in this embodiment has a function as a transmitter in each of the above embodiments, and each of the plurality of light emitting areas A to F is arranged so as to be shifted so that the bright line areas are continuous.

  For example, as shown in FIG. 218 (b), the display 8972 arranges the upper light emitting areas A, B, and C and the lower light emitting areas D, E, and F in the horizontal direction. Or as shown to (c) of FIG. 218, the display 8972 displays the light emission area | region AF of parallelogram or a rhombus, respectively. Accordingly, it is possible to eliminate the non-luminance change region that crosses the display 8972 through the light emitting regions A to F along the vertical direction of the display 8972. As a result, even if the receiver captures the display 8972 with the exposure line oriented in the vertical direction, the bright line region can be continued in the captured image.

  Furthermore, as shown in (d) and (e) of FIG. 218, the display 8972 may be arranged by shifting each of the light emitting areas A to F in the vertical direction. Thereby, it is possible to eliminate the non-luminance changing region that crosses the display 8972 through the light emitting regions A to F along the horizontal direction of the display 8972. As a result, even if the receiver captures the display 8972 with the exposure line oriented in the horizontal direction, the bright line region can be continued in the captured image.

  As shown in FIG. 218 (f), the display 8972 may display hexagonal light emitting areas A to F so that the sides of the respective areas are parallel to each other. Even in such a case, the non-luminance change region that crosses the display 8972 through the light emitting regions A to F along the horizontal direction and the vertical direction of the display 8972 can be eliminated in the same manner as described above. As a result, even if the receiver shoots the display 8972 with the exposure line oriented in the horizontal direction or the display 8972 shoots with the exposure line oriented in the vertical direction, The region can be continuous.

  FIG. 219 is a flowchart of the information communication method in this embodiment.

  The information communication method in the present embodiment is an information communication method for transmitting a signal according to a luminance change, and includes steps SK11 and SK12.

  That is, in this information communication method, by modulating the signal to be transmitted, a determination step SK11 for determining a luminance change pattern, and by changing the luminance of the plurality of light emitters according to the determined luminance change pattern, And a transmission step SK12 for transmitting the signal to be transmitted. In the surface on which the plurality of light emitters are arranged, a non-luminance changing region that is outside the plurality of light emitters and does not change in luminance is along the at least one of the vertical direction and the horizontal direction of the surface. The plurality of light emitters are disposed on the surface so as not to cross the surface through the plurality of light emitters.

  FIG. 220 is a block diagram of the information communication apparatus in this embodiment.

  The information communication device K10 in the present embodiment is an information communication device that transmits a signal by a change in luminance, and includes constituent elements K11 and K12.

  In other words, the information communication device K10 modulates the signal to be transmitted to determine a luminance change pattern, and a plurality of light emitters change in luminance according to the determined luminance change pattern. And a transmission unit K12 for transmitting the transmission target signal. In the surface on which the plurality of light emitters are arranged, a non-luminance changing region that is outside the plurality of light emitters and does not change in luminance is along the at least one of the vertical direction and the horizontal direction of the surface. The plurality of light emitters are disposed on the surface so as not to cross the surface through the plurality of light emitters.

  In the information communication method and the information communication apparatus K10 shown in FIGS. 219 and 220 as described above, for example, as shown in FIG. 218, the information is acquired by photographing the surface (display) by the image sensor provided in the receiver. In the captured image, the bright line region can be made continuous. As a result, it is possible to easily receive a signal to be transmitted, and it is possible to perform communication between various devices including a device having a small calculation power.

  In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. For example, the program causes the computer to execute the information communication method shown by the flowchart in FIG.

  FIG. 221A is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9.

  The receiver 8973 is configured as a smartphone having a function as a receiver in each of the above embodiments. As shown in FIG. 221A (a), the receiver 8973 captures the display 8972 and tries to read the bright line appearing in the captured image. Here, when the display 8972 is dark, the receiver 8973 may not be able to read the bright line and may not be able to receive a signal from the display 8972. At this time, the receiver 8973 emits a flash at a predetermined rhythm, as shown in FIG. 221A (b). When the display 8972 receives the flash, as shown in FIG. 221A (c), the display 8972 raises the luminance and performs a bright display. As a result, the receiver 8973 can read the bright line appearing in the photographed image and can receive a signal from the display 8972.

  FIG. 221B is a flowchart illustrating operation of receiver 8973 in Embodiment 9.

  The receiver 8973 first determines whether or not an operation or gesture by the user for starting reception has been received (step S831). Here, when the receiver 8973 determines that it has been received (Y in step S831), the receiver 8973 starts reception by photographing using an image sensor (step S832). Next, the receiver 8973 determines whether or not a predetermined time has elapsed from the start of reception without completion of reception (step S833). Here, when the receiver 8973 determines that the predetermined time has elapsed (Y in step S833), the flashing light is blinked at a predetermined rhythm (step S834), and the processing from step S833 is repeatedly executed. Note that in the case where the process of step S833 is repeated, the receiver 8973 determines whether or not a predetermined time has elapsed since the flashing was blinked without completion of reception. In step S834, instead of flashing the flash, the receiver 8973 generates a predetermined sound having a frequency that is difficult for humans to hear, or transmits a signal to the display 8972 to inform that it is waiting for reception. You may send to the machine.

  FIG. 222 is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9.

  The lighting device 8974 has a function as a transmitter in each of the above embodiments. The lighting device 8974 illuminates a route bulletin board 8975 at a railway station, for example, while changing in luminance. The receiver 8973 pointed to the route bulletin board 8975 by the user photographs the route bulletin board 8975. As a result, the receiver 8973 acquires the ID of the route bulletin board 8975, and acquires detailed information about each route described in the route bulletin board 8975, which is information associated with the ID. The receiver 8973 displays a guide image 8973a indicating the detailed information. For example, the guidance image 8973a indicates the distance to the route described on the route bulletin board 8975, the direction toward the route, and the time when the next train arrives on the route.

  Here, when the guide image 8973a is touched by the user, the receiver 8973 displays a supplementary guide image 8973b. This supplementary guide image 8973b is, for example, a user's selection operation of any one of a railway timetable, information on another route different from the route indicated by the guide image 8973a, and detailed information on the station. It is an image for displaying accordingly.

  FIG. 223 is a diagram illustrating an example of application of the transmitter in Embodiment 9.

  Each of the lighting devices 8976a to 8976c has a function as a transmitter in each of the above embodiments, and illuminates a store sign 8977. Here, as shown to (a) of FIG. 223, the illuminating devices 8976a-8976c may each transmit the same ID by changing a brightness | luminance synchronously. Further, as shown in FIG. 223 (b), only the lighting devices 8976a and 8976c arranged at both ends transmit the same ID by changing the luminance in synchronization, and are arranged between the lighting devices. The illuminated lighting device 8976b may illuminate the sign 8977 without transmitting an ID due to a change in luminance. Further, as shown in FIG. 223 (c), in a state where the lighting device 8976b does not transmit the ID, the lighting device 8976a and the lighting device 8976c arranged at both ends transmit different IDs by changing the luminance. May be. In this case, since the lighting device 8976b between the lighting devices 8976a and 8976c does not change the luminance for transmitting the ID, signals from the lighting device 8976a and the lighting device 8976c interfere with each other. Can be prevented. Note that the ID transmitted by the lighting device 8976a and the ID transmitted by the lighting device 8976c are different from each other, but the same information may be associated with these IDs.

  FIG. 224A is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9.

  The lighting device 8978 has a function as a transmitter in each of the above embodiments, and constantly transmits a signal by changing the luminance as illustrated in (1) of FIG. 224A.

  The receiver in this embodiment images the lighting device 8978. At this time, as illustrated in FIG. 224A, the imaging range 8979 of the receiver includes the lighting device 8978 and a portion other than the lighting device 8978. That is, the upper range “a” and the lower range “c” in the shooting range 8979 each include a portion other than the lighting device 8978, and the central range “b” in the shooting range 8979 includes the lighting device 8978.

  As shown in (2) and (3) of FIG. 224A, the receiver acquires a photographed image (bright line image) including a plurality of bright lines generated by the luminance change of the lighting device 8978 by photographing with the lighting device 8978. In this bright line image, no bright line appears in the portion corresponding to the upper range a and lower range c, and a bright line appears only in the portion corresponding to the central range b.

  Here, when the receiver captures the lighting device 8978 at a frame rate of 30 fps, for example, as shown in (2) of FIG. 224A, the length b of the bright line area in the bright line image is short, and the receiver has, for example, 15 fps. When the illumination device 8978 is photographed at the frame rate of 1, the length b of the bright line region in the bright line image is long as shown in (3) of FIG. The length of the bright line region (the bright line pattern) is a length in a direction perpendicular to each bright line included in the bright line region.

  Therefore, the receiver in the present embodiment captures the lighting device 8978 at a frame rate of 30 fps, for example, and determines whether or not the length b of the bright line area in the bright line image is less than a predetermined length. The predetermined length is, for example, a length corresponding to one block of a signal transmitted by the luminance change of the lighting device 8978. When the receiver determines that the length is less than the predetermined length, the receiver changes the frame rate to, for example, 15 fps. Accordingly, the receiver can collectively receive signals for one block from the lighting device 8978.

  FIG. 224B is a flowchart illustrating operation of the receiver in Embodiment 9.

  First, the receiver determines whether or not a bright line is included in the photographed image, that is, whether or not a stripe by the exposure line is photographed (step S841). Here, if the receiver determines that the image is being captured (Y in step S841), the receiver determines which imaging mode (imaging mode) is set (step S842). When the receiver determines that the imaging mode is the intermediate imaging mode (intermediate mode) or the normal imaging mode (normal imaging mode), the receiver changes the imaging mode to the visible light imaging mode (visible light communication mode) (step S843). .

  Next, the receiver determines whether or not the length in the direction perpendicular to the bright line in the bright line region (bright line pattern) is equal to or longer than a predetermined length (step S844). That is, the receiver determines whether or not there is a stripe region having a predetermined size or more in a direction perpendicular to the exposure line. If it is determined that the length is not greater than or equal to the predetermined length (N in step S844), the receiver determines whether the optical zoom is available (step S845). If it is determined that the optical zoom can be used (Y in step S845), the receiver performs the optical zoom so that the bright line area becomes longer, that is, the fringe area is enlarged (step S846). ). On the other hand, if it is determined that the optical zoom cannot be used (N in step S845), the receiver determines whether or not the Ex zoom (Ex optical zoom) can be used (step S847). If it is determined that the Ex zoom can be used (Y in step S847), the Ex zoom is performed so that the bright line region becomes longer, that is, the fringe region is enlarged (step S848). On the other hand, if it is determined that the Ex zoom cannot be used (N in step S847), the receiver decreases the imaging frame rate (step S849). Then, the receiver receives the signal by photographing the lighting device 8978 at the set frame rate (step S850).

  In the example shown in FIG. 224B, the frame rate is decreased when the optical zoom and the Ex zoom cannot be used. However, the frame rate may be decreased when the zoom can be used. Ex zoom is a function that limits the use area of the image sensor, narrows the angle of view, and makes the apparent focal length telephoto.

  225 is a diagram illustrating operation of a receiver in Embodiment 9. FIG.

  When the illumination device 8978 which is a transmitter is small in the captured image 8980a, the receiver can acquire a captured image 8980b in which the illumination device 8978 is greatly projected by using optical zoom or Ex zoom. it can. That is, the receiver can acquire a bright line image (photographed image) having a bright line region that is long in a direction perpendicular to the bright line by using optical zoom or Ex zoom.

  226 is a diagram illustrating an example of application of a transmitter in Embodiment 9. FIG.

  The transmitter 8981 has a function as a transmitter in each of the above embodiments, and communicates with the operation panel 8982, for example. The operation panel 8982 includes a transmission switch 8982a and a power switch 8982b.

  When the transmission switch 8982a is turned on, the operation panel 8982 instructs the transmitter 8981 to perform visible light communication. Upon receiving the instruction, the transmitter 8981 transmits a signal by changing the luminance. When transmission switch 8982a is turned off, operation panel 8982 instructs transmitter 8981 to stop visible light communication. When the transmitter 8981 accepts the instruction, the transmitter 8981 stops the signal transmission without changing the luminance.

  When the power switch 8982b is turned on, the operation panel 8982 instructs the transmitter 8981 to turn on the transmitter 8981. Upon receiving the instruction, transmitter 8981 turns on its own power supply. For example, when the transmitter 8981 is configured as a lighting device, the transmitter 8981 illuminates the surroundings when the power is turned on, and when the transmitter 8981 is configured as a television, the transmitter 8981 is imaged when the power is turned on. Etc. are displayed. Further, when the power switch 8982b is turned off, the operation panel 8982 instructs the transmitter 8981 to turn off the power of the transmitter 8981. When the transmitter 8981 receives the instruction, the transmitter 8981 turns off its own power supply and enters a standby state.

  FIG. 227 is a diagram illustrating an example of application of the receiver in Embodiment 9.

  A receiver 8973 configured as a smartphone has a function as a transmitter in each of the above embodiments, for example, and acquires an authentication ID and an expiration date from the server 8983. If the current time is within the expiration date, the receiver 8773 transmits the authentication ID to the peripheral device 8984 by changing the brightness of the display, for example. The peripheral device 8984 is, for example, a camera, a barcode reader, or a personal computer.

  When the peripheral device 8984 receives the authentication ID from the receiver 8973, the peripheral device 8984 transmits the authentication ID to the server 8983 to request verification. The server 8983 compares the authentication ID transmitted from the peripheral device 8984 with the authentication ID held by itself and transmitted to the receiver 8973, and if they match, notifies the peripheral device 8984 of the match. When the peripheral device 8984 receives a notification of matching from the server 8983, the peripheral device 8984 performs processing such as releasing the lock set for itself, performing payment processing for electronic payment, and logging in.

  FIG. 228A is a flowchart illustrating an example of operation of a transmitter in Embodiment 9.

  The transmitter in the present embodiment has a function as the transmitter in each of the above embodiments, and is configured as, for example, a lighting device or a display. Such a transmitter determines, for example, whether or not the dimming level (brightness level) is below a predetermined level (step S861a). Here, if the transmitter determines that the level is lower than the predetermined level (Y in step S861a), the transmitter stops transmitting a signal due to a luminance change (step S861b).

  FIG. 228B is a flowchart illustrating an example of operation of a transmitter in Embodiment 9.

  The transmitter in this embodiment determines whether or not the dimming level (brightness level) exceeds a predetermined level (step S862a). If the transmitter determines that the predetermined level is exceeded (Y in step S862a), the transmitter starts transmitting a signal due to a change in luminance (step S862b).

  FIG. 229 is a flowchart illustrating an example of operation of a transmitter in this embodiment.

  The transmitter in the present embodiment determines whether or not a predetermined mode is selected (step S863a). For example, the predetermined mode is an eco mode or a power saving mode. Here, when the transmitter determines that the predetermined mode is selected (Y in step S863a), the transmitter stops transmitting the signal due to the luminance change (step S863b). On the other hand, when the transmitter determines that the predetermined mode is not selected (N in step S863a), the transmitter starts transmitting a signal due to a luminance change (step S863c).

  FIG. 230 is a flowchart illustrating an example of operation of the imaging device according to the ninth embodiment.

  The imaging device in the present embodiment is, for example, a video camera, and determines whether or not recording is in progress (step S864a). If the imaging device determines that recording is in progress (Y in step S864a), the imaging device transmits a visible light transmission stop command to the transmitter that transmits a signal due to a change in luminance (step S864b). The transmitter that has received this visible light transmission stop command stops signal transmission (visible light transmission) due to luminance change. On the other hand, when the imaging device determines that the recording is not in progress (N in step S864a), the imaging device further determines whether the recording is stopped, that is, whether the recording is immediately after the recording (step S864c). Here, when the imaging device determines that the recording is stopped (Y in step S864c), the imaging device transmits a visible light transmission start command to the above-described transmitter (step S864d). Upon receiving this visible light transmission start command, the transmitter starts signal transmission (visible light transmission) due to luminance change.

  FIG. 231 is a flowchart illustrating an example of operation of the imaging device according to the ninth embodiment.

  The imaging device in the present embodiment is, for example, a digital still camera, and determines whether the imaging button (shutter button) is half-pressed or whether focus is being performed (step S865a). . Next, the imaging device determines whether or not light appears in a direction along the exposure line of the image sensor provided in the imaging device (step S865b). Here, when it is determined that light and shade appear (Y in step S865b), there is a possibility that a transmitter that transmits a signal due to a luminance change is near the imaging device. Therefore, the imaging device transmits a visible light transmission stop command to the transmitter (step S865c). Next, the imaging device acquires a captured image by performing imaging (step S865d). Then, the imaging device transmits a visible light transmission start command to the above-described transmitter (step S865e). Thereby, the imaging device can acquire a captured image without being affected by the luminance change by the transmitter. In addition, the period in which the transmission of the signal due to the luminance change is stopped is limited to a very short period in which imaging is performed by the imaging device, and thus it is possible to suppress a period during which visible light communication is not possible.

  FIG. 232 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9.

  The transmitter in this embodiment has a function as the transmitter in each of the above embodiments, and transmits a signal by outputting high luminance light (Hi) or low luminance light (Lo) for each slot. To do. Specifically, the slot is a time unit of 104.2 μs. Further, the transmitter transmits a signal indicating 1 by outputting Hi, and transmits a signal indicating 0 by outputting Lo.

  FIG. 233 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9.

  The transmitter described above sequentially outputs PHY (physical layer) frames as signal units by outputting Hi or Lo for each slot. The PHY frame includes an 8-slot preamble, a 2-slot FCS (Frame Check Sequence), and a 20-slot body. Each part included in the PHY frame is transmitted in the order of preamble, FCS, and body.

  The preamble corresponds to a PHY frame header, and includes, for example, “010110111”. The preamble may be composed of 7 slots. In this case, the preamble includes “01011011”. The FCS includes “01” when the number of 1 included in the body is an even number, and includes “11” when the number of 1 included in the body is an odd number. The body includes five symbols consisting of four slots. The symbol includes “0111”, “1011”, “1101”, or “1110” in the case of 4PPM modulation.

  234 is a diagram illustrating an example of signals transmitted by a transmitter in Embodiment 9. FIG.

  The above symbols are converted into 2-bit values by the receiver. For example, the symbols “0111”, “1011”, “1101”, and “1110” are converted into “00”, “01”, “10”, and “11”, respectively. Therefore, the body (20 slots) of the PHY frame is converted into a 10-bit signal. This 10-bit body includes a 3-bit TYPE indicating the type of the PHY frame, a 2-bit ADDR indicating the address of the PHY frame or the body, and a 5-bit DATA indicating the substance of the data. For example, when the type of the PHY frame is TYPE1, TYPE indicates “000”. ADDR indicates “00”, “01”, “10”, or “11”.

  In the receiver, DATA included in the bodies of the four PHY frames is combined. The above ADDR is used for this connection. That is, the receiver includes the DATA included in the body of the PHY frame having ADDR “00”, the DATA included in the body of the PHY frame having ADDR “01”, and the body of the PHY frame having ADDR “10”. 20 bits of data is generated by combining the DATA included in the body and the DATA included in the body of the PHY frame having ADDR “11”. Thereby, four PHY frames are decoded. The generated data includes 16-bit valid data and 4-bit CRC (Cyclic Redundancy Check).

  FIG. 235 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9.

  The types of PHY frames described above include TYPE1, TYPE2, TYPE3, and TYPE4. The length of the body, the length of ADDR, the length of DATA, the number of concatenated DATA (number of concatenations), the length of effective DATA, and the type of CRC differ for each type.

  For example, in the case of TYPE1, TYPE (TYPEBIT) indicates “000”, the body length is 20 slots, the ADDR length is 2 bits, and the DATA length is 5 bits. The number is 4, the length of effective DATA is 16 bits, and the type of CRC is CRC-4. On the other hand, in the case of TYPE2, TYPE (TYPEBIT) indicates “001”, the body length is 24 slots, the ADDR length is 4 bits, and the DATA length is 5 bits. The number is 8, the length of effective DATA is 32 bits, and the type of CRC is CRC-8.

  Visible light communication can be appropriately performed by such signals shown in FIGS. 232 to 235.

  FIG. 236 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9.

  The system in this embodiment includes a transmitter 8991 having the same function as the transmitter in each of the above embodiments, a receiver 8973 configured as, for example, a smartphone, a content sharing server 8992, and an ID management server 8993. With.

  For example, the content creator uploads content such as audio video data indicating a still image or moving image for introducing a product, and product information indicating a manufacturer, a production area, a raw material, or a specification of the product to the content sharing server 8992. To do. Then, the content sharing server 8992 registers the product information in the ID management server 8993 in association with the content ID for identifying the content.

  Next, the transmitter 8991 downloads the content and the content ID from the content sharing server 8992 in accordance with the operation by the user, displays the content, changes the luminance, that is, by visible light communication, that is, sets the content ID. Send. When the user views the content and is interested in the product introduced by the content, the user points the receiver 8973 toward the transmitter 8991 and performs shooting. The receiver 8973 receives the content ID by photographing the content displayed on the transmitter 8991.

  Next, the receiver 8973 accesses the ID management server 8993 and makes an inquiry about the content ID to the ID management server 8993. As a result, the receiver 8973 acquires the product information associated with the content ID from the ID management server 8993 and displays the product information. Here, when receiving an operation prompting the purchase of a product corresponding to the product information, the receiver 8973 accesses the manufacturer of the product and executes a process for purchasing the product.

  Next, the ID management server notifies inquiry information indicating the number of inquiries made to the content ID or the number of accesses to the manufacturer indicated by the product information associated with the content ID. Upon receiving the inquiry information, the manufacturer electronically sends an affiliate reward according to the number of inquiries indicated by the inquiry information to the content creator specified by the content ID via the ID management server 8993 and the content sharing server 8992. Pay by settlement.

  FIG. 237 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9.

  In the case of the example shown in FIG. 236, when content and product information are uploaded, the content sharing server 8992 registers product information in the ID management server 8993 in association with the content ID, but performs such registration. It does not have to be. For example, as shown in FIG. 237, the content sharing server 8992 searches the ID management server for a product ID for identifying the product of the uploaded product information, and embeds the product ID in the uploaded content.

  Next, the transmitter 8991 downloads the content ID and the content ID embedded with the product ID from the content sharing server 8992 in accordance with the operation by the user, displays the content, and changes the luminance, that is, visible light. The content ID and the product ID are transmitted by communication. When the user views the content and is interested in the product introduced by the content, the user points the receiver 8973 toward the transmitter 8991 and performs shooting. The receiver 8973 receives the content ID and the product ID by photographing the content displayed on the transmitter 8991.

  Next, the receiver 8973 accesses the ID management server 8993 and makes an inquiry about the content ID and the product ID to the ID management server 8993. As a result, the receiver 8973 acquires the product information associated with the product ID from the ID management server 8993 and displays the product information. Here, when receiving an operation prompting the purchase of a product corresponding to the product information, the receiver 8973 accesses the manufacturer of the product and executes a process for purchasing the product.

  Next, the ID management server notifies the maker indicated by the product information associated with the product ID of the inquiry information indicating the number of inquiries made for the content ID and the product ID or the number of accesses. To do. Upon receiving the inquiry information, the manufacturer electronically sends an affiliate reward according to the number of inquiries indicated by the inquiry information to the content creator specified by the content ID via the ID management server 8993 and the content sharing server 8992. Pay by settlement.

  238 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. In FIG.

  The system in the present embodiment includes a content sharing server 8992a instead of the content sharing server 8992 shown in FIG. 237, and further includes an SNS server 8994. This SNS server 8994 is a server that performs a social networking service, and performs part of the processing performed by the content sharing server 8992 shown in FIG.

  Specifically, the SNS server 8994 acquires the content and product information uploaded from the content creator, searches for the product ID corresponding to the product information, and embeds the product ID in the content. Then, the SNS server 8994 transfers the content in which the product ID is embedded to the content sharing server 8992a. The content sharing server 8992a receives the content transferred from the SNS server 8994, and transmits the content in which the product ID is embedded and the content ID to the transmitter 8991.

  That is, in the example shown in FIG. 238, a unit including the SNS server 8994 and the content sharing server 8992a plays a role as the content sharing server 8992 shown in FIG.

  In the systems shown in FIGS. 236 to 238, an appropriate affiliate reward can be paid appropriately for an advertisement (content) inquired using visible light communication.

  As described above, the information communication method according to one or more aspects has been described based on the embodiment. However, the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

  Hereinafter, the present embodiment will be supplemented.

(Mixed modulation method)
FIG. 239 and FIG. 240 are diagrams illustrating an example of operation of a transmitter in Embodiment 9.

  As shown in FIG. 239, the transmitter modulates the transmission signal with a plurality of modulation schemes, and transmits the modulated signals alternately or simultaneously.

  By modulating and transmitting the same signal with a plurality of modulation schemes, a receiver that supports only one of the modulation schemes can be received. In addition, for example, by using a modulation method with a high transmission rate, a modulation method resistant to noise, or a modulation method with a long communication distance, reception can be performed in an optimum manner according to the environment on the reception side.

  When the receiver supports reception of a plurality of modulation schemes, the receiver receives a signal modulated by a plurality of methods. When the same signal is modulated, the transmitter assigns the same signal ID and transmits the modulated signal. Thereby, the receiver can recognize that the same signal is modulated by different modulation schemes by checking the signal ID, and synthesizes a signal having the same signal ID from a plurality of types of modulation signals. Thus, a signal can be received quickly and accurately.

  For example, the transmitter includes a signal division unit and modulation units 1 to 3. The signal dividing unit divides the transmission signal into the partial signal 1 and the partial signal 2, associates the partial signal 1 with the signal ID, and associates the partial signal 2 with another signal ID. The modulation unit 1 generates a signal indicating a sine wave by performing frequency modulation on the partial signal 1 accompanied by the signal ID. The modulation unit 2 generates a signal indicating a rectangular wave by performing frequency modulation different from that of the modulation unit 1 on the partial signal 1 accompanied by the signal ID. On the other hand, the modulation unit 3 generates a signal indicating a rectangular wave by performing pulse position modulation on the partial signal 2 accompanied by another signal ID.

  As shown in FIG. 240, the transmitter transmits signals modulated by a plurality of modulation schemes together. In the example of FIG. 240, the receiver can receive only the signal modulated by the frequency modulation method using a low frequency by setting the exposure time to be long. The receiver can receive a pulse position modulation method using a high frequency band by setting the exposure time short. At this time, the receiver averages the luminance in the direction perpendicular to the bright line, thereby averaging the received light intensity over time, and can obtain a signal when the exposure time is long. .

(Transmission signal verification and digital modulation)
241 and 242 are diagrams illustrating an example of a structure and operation of a transmitter in Embodiment 9.

  As illustrated in FIG. 241, the transmitter includes a signal storage unit, a signal verification unit, a signal modulation unit, a light emitting unit, an abnormality notification unit, an original key storage unit, and a key generation unit. The signal storage unit stores a transmission signal and a signal conversion value obtained by converting the transmission signal using a verification key described later. A one-way function is used for this conversion. The original key storage unit stores an original key, which is a key source value, for example, as a circuit constant such as a resistance value or a time constant. The key generation unit generates a verification key from the original key.

  The signal verification unit obtains a signal conversion value by converting the transmission signal stored in the signal storage unit using the verification key. Whether or not the signal has been tampered with is verified based on whether or not the signal conversion value obtained here is equal to the signal conversion value stored in the signal storage unit. As a result, simply copying the signal stored in the signal storage unit to another transmitter prevents the transmitter from being forged because the verification key is different in the other transmitter, and the transmitter can be prevented from forgery. I can do it.

  If the signal has been tampered with, the abnormality notification unit displays that fact. As the method, for example, there is a method of blinking the light emitting unit at a period that can be visually recognized by a human or sounding a sound. By limiting the abnormality notification to a predetermined time immediately after the power is turned on, the transmitter can be used for purposes other than transmission even when there is an abnormality in the signal.

  If the signal has not been tampered with, the signal modulator modulates the signal into a light emission pattern. Various modulation methods can be used for this modulation method. Modulation schemes that can be used here include, for example, amplitude shift keying (ASK), phase shift keying (PSK), frequency shift keying (FSK), quadrature amplitude modulation (QAM), and delta modulation (DM). , Minimum deviation modulation (MSK), complementary code modulation (CCK), orthogonal frequency division multiplexing (OFDM), amplitude modulation (AM), frequency modulation (FM), phase modulation (PM), pulse width modulation (PWM) , Pulse amplitude modulation (PAM), pulse density modulation (PDM), pulse position modulation (PPM), pulse code modulation (PCM), frequency hopping spread spectrum (FHSS), direct sequence spread spectrum (DSSS), etc. Select according to the characteristics (analog or digital, whether continuous data transmission, etc.) and required characteristics (transmission speed, noise resistance, transmission distance) . Furthermore, a modulation scheme combining these can be used.

  In the first to ninth embodiments, the same effect can be obtained even when a signal modulated by the modulation method described here is used.

  As illustrated in FIG. 242, the transmitter may include a signal demodulation unit instead of the signal verification unit. In this case, the signal storage unit holds the encrypted transmission signal obtained by encrypting the transmission signal using the encryption key that is paired with the decryption key generated by the key generation unit. The signal demodulation unit decrypts the encrypted transmission signal using the decryption key. With this configuration, it is possible to make it difficult to forge the transmitter, that is, to create a transmitter that transmits an arbitrary signal.

(Embodiment 10)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

(Reception of signals from multiple directions by multiple light receiving units)
FIG. 243 is a diagram illustrating a timepiece equipped with an optical sensor.

  This timepiece is configured as a receiver for visible light communication, and includes a plurality of optical sensors and a condensing lens corresponding to each of the plurality of optical sensors. Specifically, as shown in the cross-sectional view of FIG. 243, a condensing lens is disposed on the upper surface of the optical sensor. In FIG. 243, the condenser lens is disposed with a predetermined inclination. The shape of the condensing lens is not limited to this, and may be other shapes as long as the condensing lens can be condensed. With such a configuration, the optical sensor can collect and receive light from an external light source by the lens. Therefore, even a small optical sensor mounted on a watch can perform visible light communication. In FIG. 243, it is divided into 12 regions, 12 photosensors are mounted, and a condensing lens is arranged on the upper surface of each photosensor. Thus, by dividing the inside of the watch into a plurality of regions and arranging a plurality of photosensors, information from a plurality of light sources can be acquired. For example, in FIG. 243, the light from the light source 1 can be received by the first optical sensor, and the light from the light source 2 can be received by the second optical sensor. Moreover, it is also possible to use a solar cell as an optical sensor. By using a solar battery as an optical sensor, it is possible to perform visible light communication at the same time as performing photovoltaic power generation with a single optical sensor, thereby reducing costs and making it compact. It becomes possible. Furthermore, when a plurality of optical sensors are arranged, information from a plurality of light sources can be acquired at the same time, so that the position estimation accuracy can be improved. In this embodiment, the optical sensor is provided in the timepiece. However, the present invention is not limited thereto, and the optical sensor may be provided in another device as long as the terminal is movable, such as a mobile phone or a mobile terminal.

  FIG. 244 is a diagram illustrating an example of a receiver in Embodiment 10.

  For example, the receiver 9020a configured as a wristwatch includes a plurality of light receiving units. For example, as shown in FIG. 244, the receiver 9020a is arranged in the vicinity of the character indicating 12 o'clock at the light receiving portion 9020b arranged at the upper end portion of the rotating shaft that supports the long hand and the short hand of the watch. Light receiving portion 9020c. The light receiving unit 9020b receives light traveling toward the light receiving unit 9020b along the direction of the rotation axis described above, and the light receiving unit 9020c transmits light traveling toward the light receiving unit 9020c along the direction connecting the rotation axis and the character indicating 12:00. Receive. Accordingly, when the user holds the receiver 9020a in front of the chest as when checking the time, the light receiving unit 9020b can receive light from above. As a result, the receiver 9020a can receive a signal from the ceiling lighting. Further, when the user holds the receiver 9020a in front of the chest as when checking the time, the light receiving unit 9020c can receive light from the front direction. As a result, the receiver 9020a can receive a signal from a signage or the like at the front.

  These light receiving units 9020b and 9020c have directivity, so that signals can be received without interference even when there are a plurality of transmitters at close positions.

  FIG. 245 is a diagram illustrating an example of a receiver in Embodiment 10.

  For example, as shown in FIG. 245 (a), a receiver 9021 configured as a wristwatch includes 17 light receiving elements (light receiving units). These light receiving elements are arranged on the dial. Of these light receiving elements, twelve light receiving elements are arranged at positions corresponding to 1 to 12 o'clock on the dial, and the remaining five are arranged in the central portion on the dial. Yes. These 17 light receiving elements have directivities different from each other, and receive light (signals) in directions corresponding to the respective light receiving elements. Thus, by arranging a plurality of light receiving elements having directivity, the receiver 9021 can estimate the direction of the received signal. Further, as shown in FIG. 245 (b), a prism for guiding light to the light receiving element may be arranged in front of the light receiving element. In other words, the receiver 9021 includes eight light receiving elements arranged at equal intervals on the periphery of the dial and a plurality of prisms that guide light to at least one of the light receiving elements. By providing such a prism, the fine direction of the transmitter can be estimated even with a small number of light receiving elements. For example, when only the light receiving element 9021d among the eight light receiving elements receives light, the receiver 9021 can estimate that there is a transmitter in the direction connecting the center of the dial and the prism 9021a, and the light receiving element 9021d and the light receiving element When 9021e receives the same signal, it can be estimated that there is a transmitter in the direction connecting the center of the dial and the prism 9021b. The watch windshield may be provided with a directivity function or a prism function.

  FIG. 246A is a flowchart of an information communication method according to an aspect of the present invention.

  An information communication method according to an aspect of the present invention is an information communication method in which a mobile terminal acquires information, and includes steps SE11 and SE12.

  That is, in this information communication method, at least one solar battery among the plurality of solar batteries each having directivity provided in the mobile terminal is in a direction according to the directivity of the solar battery. A light receiving step (SE11) for receiving visible light emitted along the line, and an information acquiring step (SE12) for acquiring information by demodulating a signal specified by the received visible light.

  FIG. 246B is a block diagram of a mobile terminal according to one embodiment of the present invention.

  A mobile terminal E10 according to an aspect of the present invention is a mobile terminal that acquires information, and includes a plurality of solar batteries E11 and an information acquisition unit E12 each having directivity. The information acquisition unit E12 receives visible light emitted along a direction according to the directivity of the solar battery E11 by at least one of the solar batteries E11. In some cases, information is obtained by demodulating a signal specified by the received visible light.

  In such an information communication method and portable terminal E10 shown in FIGS. 246A and 246B, the solar battery E11 can be used for power generation while being used as an optical sensor for visible light communication. The cost of the mobile terminal E10 to be acquired can be reduced, and the mobile terminal E10 can be made compact. Moreover, since each of the plurality of solar cells E11 has directivity, the direction in which the transmitter emitting the visible light is present can be estimated based on the directivity of the solar cells E11 that have received visible light. . Further, since each of the plurality of solar cells E11 has directivity, visible light emitted from the plurality of transmitters can be distinguished and received, and information is appropriately acquired from each of the plurality of transmitters. be able to.

  Further, in the light receiving step (SE11), as shown in FIG. 245 (b), the solar battery E11 (9021d, 9021e) is a prism (9021a, 9021b or 9021c) provided in the portable terminal E10 (9021). ) May be received. Thereby, it is possible to estimate with high accuracy the direction in which the transmitter emitting visible light is present while suppressing the number of solar cells E11 provided in the portable terminal E10. Furthermore, as shown in FIG. 245, the portable terminal E10 is a wristwatch, and a plurality of solar cells E11 (light receiving elements) are arranged along the periphery of the dial of the wristwatch, and the plurality of solar cells E11. The directions of visible light received by each of these may be different from each other. Thereby, information can be appropriately acquired by a wristwatch.

(Cooperation between watch-type receiver and smartphone)
FIG. 247 is a diagram illustrating an example of a reception system in Embodiment 10.

  For example, a receiver 9022b configured as a wristwatch is connected to a smartphone 9022a and a glasses-type display 9022c via wireless communication such as Bluetooth (registered trademark). When receiving a signal or confirming the presence of the signal, the receiver 9022b displays information indicating that the signal has been received on the display 9022c. The receiver 9022b transmits the received signal (reception signal) to the smartphone 9022a. The smartphone 9022a acquires data associated with the received signal from the server 9022d, and displays the acquired data on the glasses-type display 9022c.

(Wayway guidance using a watch-type display)
FIG. 248 is a diagram illustrating an example of a reception system in Embodiment 10.

  For example, the receiver 9023b configured as a wristwatch is connected to the smartphone 9022a via wireless communication such as Bluetooth (registered trademark). The receiver 9023b has a dial made up of a display such as a liquid crystal display, and can display information other than the time. The smartphone 9022a recognizes the current location from the signal received by the receiver 9023b, and displays the route and distance to the destination on the display surface of the receiver 9023b.

(Frequency shift keying and frequency multiplexing)
249A, 249B, and 249C are diagrams each illustrating an example of a modulation scheme in Embodiment 10.

  FIG. 249A (a) expresses a specific signal as a specific modulation frequency. The receiving side performs frequency analysis of the light pattern (light source luminance change pattern) to obtain a dominant modulation frequency, and restores the signal.

  As shown in FIG. 249C (a), many values can be expressed by temporally changing the modulation frequency. Since the imaging frame rate of a general image sensor is 30 fps, it can be reliably received by continuing one modulation frequency for 1/30 second or more. Further, as shown in FIG. 249C (b), when changing the frequency, by providing a time during which no signal is superimposed, the receiver can easily recognize the change in the modulation frequency. The light pattern during which no signal is superimposed can be distinguished from the signal superimposed portion by making the brightness constant or setting a specific modulation frequency. If the specific modulation frequency used here is an integer multiple of 30 Hz, the difference image is less likely to appear and the reception process is not hindered. Reception is facilitated by setting the length of time during which no signal is superimposed to be equal to or longer than the signal having the longest period in the optical pattern used for the signal. As an example, if the light pattern with the lowest modulation frequency is 100 Hz, the length of time during which no signal is superimposed is set to 1/100 second or more.

  (B) of FIG. 249A is an example (1) in which a specific bit and a specific modulation frequency are associated with each other, and an optical pattern is expressed as a waveform obtained by superimposing modulation frequencies having a corresponding bit of 1. Specifically, when transmitting information whose first bit is 1, the transmitter changes in luminance with a light pattern having a frequency f1 = 1000 Hz. Further, when transmitting information whose second bit is 1, the transmitter changes in luminance with a light pattern having a frequency f2 = 1100 Hz. Further, when transmitting information whose third bit is 1, the transmitter changes in luminance with a light pattern having a frequency f3 = 1200 Hz. Therefore, for example, when transmitting information consisting of a bit string of “110”, the transmitter changes in luminance with the optical pattern of frequency f2 at time T2, and with the optical pattern of frequency f1 at time T1 longer than time T2. Change. For example, when transmitting information consisting of a bit string of “111”, the transmitter changes in luminance with an optical pattern of frequency f2 at time T2, and with an optical pattern of frequency f3 at time T3 shorter than time T2. Furthermore, the luminance changes with the light pattern of frequency f1 at time T1. In this case, a higher CN ratio (Carrier to Noise Ratio) is required as compared with the modulation method of (a), but more values can be expressed. In example (1), when the number of bits that are turned on is large, that is, when the waveform includes many frequencies, the energy per frequency is reduced and a higher CN ratio is required. There is a problem.

  Therefore, in the example (2) expressing the light pattern, the number of frequencies included in the waveform is limited to a predetermined number or less, that is, the number of frequencies can be varied below the predetermined number. Alternatively, in the example (3) expressing the light pattern, the number of frequencies included in the waveform is limited to a predetermined number. Thereby, the above-mentioned problem can be avoided. In Example (3), since the number of included frequencies is determined, the signal and noise can be separated more easily than in Examples (1) and (2), and the method is most resistant to noise. ing.

When a signal is expressed using n types of frequencies, in the example (1), 2 ⁿ −1 signals can be expressed. Further, in Example (2) to limit the types of frequency by m different, represent a _{(Σ (k = 1~m) n} C k) -1 ways, in Example _(3), signals as n _{C m} be able to.

As a method of superimposing a plurality of modulation frequencies, (i) a method of simply adding the respective waveforms, (ii) a method of performing weighted averaging with weights of the respective waveforms, and (iii) a waveform of each frequency. There is a method to repeat in order. When performing frequency analysis such as discrete cosine series expansion on the receiving side, the peak tends to be smaller at higher frequencies. Therefore, in (ii), weighting is performed by adjusting the peak of each frequency to the same size. Do the average. That is, the higher the frequency, the better. In (iii), instead of repeating outputting the waveform of each frequency once (one cycle at a time), the size of the frequency peak at the time of reception is adjusted by adjusting the ratio of the number of outputs (cycle number). Can be adjusted. The number of cycles to be output may be increased as the frequency is higher, or the output time may be increased as the frequency is higher. By this adjustment, it is possible to make the reception processing easier by aligning the sizes of the frequency peaks, and it is possible to express additional information by giving meaning to the difference in the size of the frequency peaks. For example, when the order of the size of the frequency peak is given, if the number of included frequencies is n, the information amount of log ₂ (n!) Bits can be added. The frequency may be changed every cycle, the frequency may be changed every cycle or every half cycle, the frequency may be changed every constant multiple of a half cycle, or at regular intervals. The frequency may be changed. The timing of changing the frequency may be when the luminance is the highest, when the luminance is the lowest, or when the value is an arbitrary value. Flickering can be suppressed by equalizing the luminance before and after changing the frequency (= changing the luminance continuously). For that purpose, it is only necessary to output the frequency for a time that is an integral multiple of the half wavelength of each frequency to be transmitted. At this time, the time for outputting each frequency is different. In addition, by outputting a signal of a certain frequency for a time that is an integral multiple of a half cycle, it is possible to easily recognize that the frequency is included in the signal on the receiving side by frequency analysis even in the case of digital output. Can do. Flickering is less likely to be perceived by the human eye or camera if the same frequency is sent continuously rather than continuously. For example, when outputting the cycle T ₁ twice, T ₂ twice, and T ₃ once, T ₁ T ₂ T ₃ T ₂ T rather than T ₁ T ₁ T ₂ T ₂ T _{3 1} is better. Instead of repeating output in a predetermined order, it may be output while changing the order. By giving meaning in this order, additional information can also be expressed. This order does not appear in the frequency peak, but this information can be acquired by analyzing the order of the frequencies. When analyzing the frequency order rather than analyzing the frequency peak, it is necessary to set the exposure time short. Therefore, the exposure time may be set short only when additional information is required, or the exposure time may be shortened. Only the receiver that can be set may acquire this additional information.

  FIG. 249B shows a case where the signal of FIG. 249A is expressed by a binary light pattern. As a method of overlapping frequencies, the methods (i) and (ii) have a complicated analog waveform, and even if the analog waveform is binarized, a complicated shape cannot be expressed. For this reason, the receiver cannot obtain an accurate frequency peak, and reception errors increase. In the method (iii), since the analog waveform does not have a complicated shape, the influence of binarization is small, and a relatively accurate frequency peak can be obtained. Therefore, the method (iii) is excellent when using a light pattern digitized with two values or a small number of values. This modulation method can be interpreted as a kind of frequency modulation, focusing on the fact that the signal is expressed by the frequency of the light pattern, and the signal is expressed by adjusting the length of the pulse width. Focusing on this point, it can be interpreted as a type of PWM modulation.

  By setting the unit of time when the luminance changes to a discrete value, transmission and reception can be performed in the same manner as pulse modulation. The average luminance can be increased by setting the interval where the luminance is low as the shortest time unit, not limited to the length of the frequency cycle to be transmitted. At this time, the longer the frequency of the transmission frequency is, the higher the average luminance is. Therefore, the average luminance can be increased by increasing the number of times of output of the frequency having the longer frequency. When the frequency analysis is performed by setting the length of the high luminance section to the length of the transmission frequency cycle minus the length of the low luminance section even if the low luminance section has the same length Shows a frequency peak in the transmission frequency. Therefore, a signal can be received by using a frequency analysis method such as discrete cosine transform without setting the exposure time of the receiver so short.

  As shown in (c) of FIG. 249C, many values can be expressed by temporally changing the superposition of modulation frequencies in the same manner as (a) of FIG. 249C.

  A signal with a high modulation frequency cannot be received unless the exposure time is set short, but a modulation frequency with a certain height can be used without setting the exposure time. In the case of a terminal in which all terminals can receive a signal expressed at a low modulation frequency by transmitting a signal modulated using a frequency from a low modulation frequency to a high modulation frequency, and the exposure time can be set short. By receiving signals up to a high modulation frequency, more information can be received quickly from the same transmitter. Alternatively, when a low frequency modulation signal is found in the normal imaging mode, the entire transmission signal including the high frequency modulation signal may be received in the visible light communication mode.

  Since frequency shift keying and frequency division multiplexing have the effect of not causing flickering in the human eye even when using a lower modulation frequency than expressing the signal depending on the pulse position, use many frequency bands. Can do.

  In the first to tenth embodiments, the same effect can be obtained even when a signal modulated by the reception method / modulation method described here is used.

(Mixed signal separation)
249D and 249E are diagrams illustrating an example of mixed signal separation in the tenth embodiment.

  The receiver has the function (a) of FIG. The light receiving unit receives the light pattern. The frequency analysis unit maps a signal to the frequency domain by performing a Fourier transform on the light pattern. The peak detector detects the peak of the frequency component of the light pattern. If no peak is detected by the peak detector, the subsequent processing is interrupted. The peak time change analysis unit analyzes the time change of the peak frequency. The signal source specifying unit specifies which combination of modulation frequencies of signals transmitted from the same transmitter is detected when a plurality of frequency peaks are detected.

  Thereby, even when a plurality of transmitters are arranged close to each other, reception can be performed while avoiding signal interference. In addition, when light from a transmitter receives light reflected from a floor, wall, ceiling, etc., light from multiple transmitters is often mixed, but even in this case, avoid signal interference. Can be received.

  As an example, when the receiver receives an optical pattern in which the signal from the transmitter A and the signal from the transmitter B are mixed, a frequency peak as shown in FIG. 249D (b) is obtained. Since fA1 disappears and fA2 appears, it can be specified that fA1 and fA2 are signals from the same transmitter. Similarly, it can be specified that fA1, fA2, and fA3 are signals from the same transmitter, and fB1, fB2, and fB3 are signals from the same transmitter.

  By making the time interval at which one transmitter changes the modulation frequency constant, it is possible to easily identify signals from the same transmitter.

  When the timings at which the modulation frequencies of a plurality of transmitters change are equal, the above method cannot identify signals from the same transmitter. Therefore, by changing the time interval for changing the modulation frequency of the transmitter for each individual transmitter, it is possible to avoid the situation where the timings at which the modulation frequencies of the multiple transmitters change are always the same. It becomes possible to specify the signal.

  As shown in (c) of FIG. 249D, by setting the time from when the transmitter changes the modulation frequency to the next change to a value obtained from the current modulation frequency and the modulation frequency before the change, Even when a plurality of transmitters change the modulation frequency at the same timing, it is possible to specify which modulation frequency signal is transmitted from the same transmitter.

  The transmitter may recognize a transmission signal of another transmitter and adjust the modulation frequency change timing so as not to be equal.

  The above method is the same in the same way not only in the case of frequency shift key modulation where one transmission signal is composed of one modulation frequency, but also when one transmission signal is composed of a plurality of modulation frequencies. The effect is obtained.

  As shown in (a) of FIG. 249E, when the optical pattern is not changed in time by the frequency multiplexing modulation method, signals from the same transmitter cannot be specified, but as shown in (b) of FIG. 249E. In addition, it is possible to specify a signal from the same transmitter from the time change of the peak by including a section without a signal or changing to a specific modulation frequency.

  FIG. 249F is a flowchart illustrating processing of the information processing program in the tenth embodiment.

  This information processing program is a program for changing the luminance of the light emitter (or light emitting unit) of the transmitter described above with the light pattern shown in FIG. 249A (b) or FIG. 249B (b).

  In other words, this information processing program is an information processing program for causing a computer to process information to be transmitted in order to transmit the information to be transmitted by a luminance change. Specifically, the information processing program encodes information to be transmitted to determine the luminance change frequency SA11 and to change the luminance according to the luminance change frequency determined by the light emitter. The computer executes an output step SA12 that outputs a signal indicating the determined frequency of the luminance change so as to transmit the information to be transmitted. In the determination step SA11, a first frequency (for example, frequency f1) and a second frequency (for example, frequency f2) different from the first frequency are respectively determined as the luminance change frequencies. In the output step SA12, the luminous body changes in luminance according to the first frequency at a first time (for example, time T1), and after the first time has elapsed, a second time (for example, time) that is different from the first time. In T2), the signals indicating the first and second frequencies are output as signals indicating the determined frequency of the luminance change so that the luminance changes according to the second frequency.

  Thereby, the information to be transmitted can be appropriately transmitted by the visible light signals having the first and second frequencies. Further, by making the first time different from the second time, transmission according to various situations can be performed. As a result, communication between various devices can be enabled.

  For example, as shown in FIGS. 249A and 249B, the first time is a time corresponding to one cycle of the first frequency. The second time is a time corresponding to one cycle of the second frequency.

  Further, in the output step SA12, the signal indicating the first frequency and the signal indicating the second frequency so that the number of outputs of the signal indicating the first frequency and the signal indicating the second frequency are different. At least one of and may be repeatedly output. Thereby, transmission according to various situations can be performed.

  In addition, in the output step SA12, the signal indicating the first frequency so that the number of outputs of the low frequency signal among the signals indicating the first and second frequencies is larger than the number of outputs of the high frequency signal. And at least one of the signals indicating the second frequency may be repeatedly output.

  Thereby, when the luminous body changes in luminance according to the frequency indicated by each output signal, the luminous body can transmit information to be transmitted with bright luminance. For example, it is assumed that the luminance change time according to the first frequency, which is a low frequency, and the luminance change value according to the second frequency, which is a high frequency, have the same duration of dark luminance. In this case, the luminance change according to the first frequency (that is, the low frequency) takes a longer time for the bright luminance to continue than the luminance change according to the second frequency (that is, the high frequency). Therefore, when many signals indicating the first frequency are output, the light emitter can transmit information to be transmitted with bright luminance.

  In addition, in the output step SA12, the signal indicating the first frequency so that the number of outputs of the high-frequency signal among the signals indicating the first and second frequencies is larger than the number of outputs of the low-frequency signal. And at least one of the signals indicating the second frequency may be repeatedly output. For example, as shown in FIGS. 249A and 249B, the number of outputs of the signal of frequency f2 is greater than the number of outputs of the signal of frequency f1.

  As a result, when the luminous body changes in luminance according to the frequency indicated by each output signal, the reception efficiency of the information transmitted by the luminance change can be increased. For example, when information to be transmitted is transmitted to a receiver by a visible light signal expressed by a plurality of frequencies, the receiver performs frequency analysis such as Fourier transform on an image obtained by imaging. The frequency peak contained in the visible light signal is detected. At this time, peak detection is more difficult as the frequency becomes higher. Therefore, as described above, each signal is output so that the number of outputs of the high-frequency signal among the signals indicating the first and second frequencies is greater than the number of outputs of the low-frequency signal. , High frequency peak detection can be facilitated. As a result, reception efficiency can be improved.

  In the output step SA12, at least one of the signal indicating the first frequency and the signal indicating the second frequency may be repeatedly output so that signals indicating the same frequency are not continuously output. Good. For example, as shown in FIGS. 249A and 249B, the signal indicating the frequency f1 is not continuously output, and the signal indicating the frequency f2 is not continuously output.

  Accordingly, when the luminance of the light emitter changes according to the frequency indicated by each output signal, flickering of light from the light emitter can be made difficult to be caught by human eyes or a camera.

  FIG. 249G is a block diagram of an information processing device in Embodiment 10.

  This information processing apparatus A10 is an apparatus for changing the luminance of the light emitter of the transmitter described above with the light pattern shown in FIG. 249A (b) or FIG. 249B (b).

  That is, the information processing apparatus A10 is an apparatus that processes information to be transmitted in order to transmit the information to be transmitted by a change in luminance. Specifically, the information processing apparatus A10 encodes information to be transmitted, thereby changing the luminance according to the frequency change unit A11 that determines the frequency of the luminance change and the luminance change frequency for which the light emitter is determined. And an output unit A12 that outputs a signal indicating the determined luminance change frequency so as to transmit information to be transmitted. Here, the frequency determination unit A11 determines the first frequency and the second frequency different from the first frequency as the luminance change frequencies. In the output unit A12, the luminance of the light emitter changes in accordance with the first frequency in the first time, and the luminance in accordance with the second frequency in a second time different from the first time after the first time has elapsed. In order to change, a signal indicating the first and second frequencies is output as a signal indicating the frequency of the determined luminance change. Such an information processing apparatus A10 can achieve the same effects as the information processing program described above.

(Operation of home appliances via illumination by visible light communication)
FIG. 250A is a diagram illustrating an example of a visible light communication system in Embodiment 10.

  For example, a transmitter configured as ceiling lighting (illumination equipment) has a wireless communication function such as Wi-Fi or Bluetooth (registered trademark). The transmitter transmits information (such as a light emitting device ID and an authentication ID) for connecting to the transmitter by wireless communication by visible light communication. For example, the receiver A configured as a smartphone (mobile terminal) performs wireless communication with the transmitter based on the received information. The receiver A may be connected to the transmitter using other information, and in that case, the receiver A may not have a reception function. The receiver B is configured as an electronic device (control target device) such as a microwave oven. The transmitter transmits the information of the paired receiver B to the receiver A. Receiver A displays information of receiver B as an operable device. The receiver A transmits an operation command (control signal) of the receiver B to the transmitter through wireless communication, and the transmitter transmits the operation command to the receiver B through visible light communication. Thereby, the user can operate the receiver B via the receiver A. A device connected to the receiver A via the Internet or the like can operate the receiver B via the receiver A.

  Since the receiver B has a transmission function and the transmitter has a reception function, bidirectional communication can be performed. The transmission function may be realized as visible light by light emission, or communication by sound may be performed. For example, the transmitter includes a sound collection unit and can recognize the state of the receiver B by recognizing the sound emitted by the receiver B. For example, the operation end sound of the receiver B is recognized and transmitted to the receiver A, and the receiver A can notify the user by displaying the operation end of the receiver B on the display.

  The receiver A and the receiver B are provided with NFC. The receiver A receives the signal from the transmitter, and then communicates with the receiver B via NFC. The receiver B can receive the signal from the transmitter that transmitted the signal received immediately before. It is registered in the receiver A and the transmitter. This is called pairing of the transmitter and the receiver B. The receiver A registers cancellation of pairing with the transmitter when the receiver B is moved. When receiver B is paired with another transmitter, the newly paired transmitter communicates that information to the previously paired transmitter and releases the previous pairing.

  FIG. 250B is a diagram for describing a use case in Embodiment 10. With reference to FIG. 250B, an embodiment in which the receiving unit 1028 using the modulation method of the PPM method, the FDM method, the FSK method, or the frequency allocation method of the present invention is used will be described.

  First, the light emission operation of the light emitter 1003 which is a lighting device will be described. In a light emitting device 1003 such as a lighting fixture or a TV monitor attached to a ceiling or a wall, an authentication ID is first generated by an authentication ID generation unit 1010 using a random number generation unit 1012 that changes with time. When there is no interrupt process (step 1011) for the ID of the light emitter 1003 and the authentication ID 1004, the light emitter 1003 determines that there is no “transmission data string” sent from the portable terminal 1020, and (1) light emission A machine ID, (2) an authentication ID, and an identifier for identifying whether there is a transmission data string 1009 sent from the electronic device 1040 that is a control target device via the portable terminal 1020, that is, (3) a transmission data string Three flags = 0 are sent from the light emitting unit 1016 such as an LED to the outside continuously or intermittently.

  The transmitted optical signal is received by the photo sensor 1041 of the electronic device 1040 (step 1042), and the electronic device 1040 receives the device ID and authentication ID (device authentication ID and light emitter ID) of the electronic device 1040 in step 1043. Check if it is genuine. If the confirmation result is YES (regular), the electronic device 1040 checks whether the transmission data string flag = 1 (step 1051). Only when the result of the check is YES (transmission data string flag = 1), the electronic device 1040 executes a data command of the transmission data string, for example, a user command such as cooking recipe execution (step 1045).

  Here, a mechanism for optical transmission using the optical modulation method of the present invention of the electronic device 1040 will be described. The electronic device 1040 displays the device ID, the authentication ID for authenticating the device, and the light emitting device ID of the light emitting device 1003 received by the electronic device 1040 as described above, that is, reliably received, on the display unit 1047. It is sent using the LED backlight unit 1050 or the like (step 1046).

  Since the optical signal of the present invention is sent from a display unit 1047 such as a liquid crystal of a microwave oven or a POS machine at a modulation frequency of 60 Hz or more without flickering, it is transmitted to a general consumer with light. I don't know the signal is being sent. Therefore, the display unit 1047 can independently display a menu of a microwave oven, for example.

(ID detection method of light emitting device 1003 that electronic device 1040 can receive)
A user who intends to use a microwave oven or the like receives an optical signal from the light emitter 1003 by the in-camera unit 1017 of the portable terminal 1020, and receives a light-emitting device ID and a light-emitting device authentication ID by the in-camera processing unit 1026. (Step 1027). The light-emitting device ID that can be received by the electronic device 1040 includes the position information using 3G mobile phone radio waves and Wi-Fi, and the light-emitting device ID existing at the position recorded in the cloud 1032 or the mobile terminal. It may be detected (step 1025).

  When the user points the out camera 1019 of the portable terminal 1020 toward the display unit 1047 of the microwave oven (electronic device) 1040, for example, the optical signal 1048 of the present invention can be demodulated using a MOS camera. Faster data can be received by increasing the shutter speed. The receiving unit 1028 receives the device ID, authentication ID, service ID, or service ID of the electronic device 1040 converted from the service URL or the device status.

  In step 1029, the 3G Wi-Fi communication unit 1031 is used to connect to the external cloud 1032 inside the terminal or using the received URL, and send the service ID and device ID. In the cloud 1032, data corresponding to the device ID and service ID in the database 1033 is searched and sent to the mobile terminal 1020. Based on this data, video data, command buttons, etc. are displayed on the screen of the portable terminal. The user who sees this inputs a desired command by an input method such as pressing a button on the screen (step 1030). In the case of YES (input), the transmitter 1022 of the BTLE (Blue Tooth (registered trademark) Low Energy) transmission / reception unit 1021 includes a device ID of the electronic device 1040, a device authentication ID, a light emitter ID, a light emitter authentication ID, and a step. A transmission data string including 1030 user commands and the like is transmitted.

  The light emitting device 1003 receives the “transmission data string” by the reception unit 1007 of the BTLE transmission / reception unit 1004 and detects that the “transmission data string” is received by the interrupt processing unit 1011 (YES in Step 1013). Data of “transmission data string + ID + transmission data flag = 1” is modulated by the modulation unit of the present invention, and light is transmitted from the light emitting unit 1016 such as an LED. When it is not detected that the “transmission data string” has been received (NO in step 1013), the light emitter 1003 continuously transmits the light emitter ID and the like.

  As described above, since the electronic device 1040 has actually received and confirmed that the signal from the light emitter 1003 can be received, the electronic device 1040 can reliably receive the signal.

  In this case, since the light emitting device ID is included in the transmission data string in the interrupt processing unit 1011, it can be seen that there is an electronic device to be transmitted within the light irradiation range of the light emitting device with that ID. Therefore, since the electronic device is sent only from the light emitting device in a very narrow position without sending a signal from another light emitting device, the radio wave space can be used efficiently.

  If this method is not adopted, the Bluetooth signal reaches far away, so that an optical signal is sent from a light emitter located at a different position from the electronic device. Therefore, since a certain light emitter cannot transmit light to other electronic devices to transmit during the light emission period, or interferes with it, this type of solution is effective.

  Next, countermeasures against malfunction of electronic devices are described.

  The photosensor 1041 receives an optical signal in step 1042. First, since the light emitting device ID is checked, the light emission signal of another light emitting device ID can be removed, thereby reducing malfunction.

  In the present invention, the transmission data string 1009 includes the device ID and device authentication ID of the electronic device to be received. Accordingly, since it is checked in step 1043 whether the device authentication ID and the device ID are the ID of the electronic device 1040, no malfunction occurs. There is an effect that malfunction of a microwave oven or the like due to erroneous processing of a signal transmitted from the electronic device 1040 to another electronic device can be prevented.

  A method for preventing a malfunction in execution of a user command will be described.

  In step 1044, when the transmission data flag = 1, it is determined that there is a user command, and when the transmission data flag = 0, the process stops. When the transmission data flag = 1, the device ID and the authentication ID of the user data string are authenticated, and then the transmission data string such as a user command is executed. For example, the electronic device 1040 takes out the recipe and displays it on the screen. If the user presses the button, the recipe, i.e., 600w for 3 minutes, 200w for 1 minute, and steam cooking for 2 minutes are started without malfunction.

  When the user command is executed, electromagnetic noise of 2.4 GHz is generated in the case of the microwave oven. In order to reduce this, when receiving a command via Bluetooth or Wi-Fi via a smartphone and operating, the intermittent drive unit 1061 stops the output of the microwave oven intermittently, for example, about 100 ms in 2 seconds. During this time, communication such as Bluetooth or Wi-Fi 802.11n becomes possible. For example, when the range is not stopped, sending a stop command from the smartphone to the light emitter 1003 by BTLE is hindered. On the other hand, in the present invention, it is possible to send without being affected by the jamming wave, and the range can be stopped by the optical signal, or the recipe of the range can be changed.

  Since the feature of this embodiment is that a photosensor 1041 with a cost of several yen is added to an electronic device with a display unit, bidirectional communication with a smartphone linked with the cloud can be achieved. There is an effect that it can be installed in white goods and converted into smart appliances. However, although white goods were used as an embodiment, a POS terminal with a display unit may be used, and a similar effect can be obtained with a supermarket electronic price tag or a personal computer.

  Further, in this embodiment, the light emitting device ID can be received only from the illuminator at the top of the electronic device. Since the reception area is narrow, by defining a small zone ID such as Wi-Fi for each light-emitting device and assigning the following ID of the position in each zone, there is also an effect of reducing the number of digits of the ID of the light emitting unit. is there. In this case, the number of digits of the ID of the light emitting device that transmits using the aforementioned PPM, FSK, or FDM of the present invention is reduced, so that an optical signal can be received from a small light source, an ID can be quickly acquired, Can be received.

  FIG. 250C is a diagram illustrating an example of a signal transmission / reception system in Embodiment 10.

  The signal transmission / reception system includes a smartphone (smartphone) that is a multi-function mobile phone, an LED light-emitting device that is a lighting device, a home appliance such as a refrigerator, and a server. The LED light emitter performs communication using BTLE (Bluetooth (registered trademark) Low Energy) and visible light communication using an LED (Light Emitting Diode). For example, the LED light emitter controls a refrigerator or communicates with an air conditioner by BTLE. In addition, the LED light emitter controls a power source of a microwave oven, an air purifier, a television (TV), or the like by visible light communication.

  The television includes a solar power generation element, for example, and uses the solar power generation element as an optical sensor. That is, when a signal is transmitted when the brightness of the LED light emitter changes, the television detects a change in the brightness of the LED light emitter based on a change in the power generated by the solar power generation element. Then, the television acquires the signal transmitted from the LED light emitter by demodulating the signal indicated by the detected luminance change. When the signal is a command indicating that the power is on, the television switches its main power source to ON, and when the signal is a command indicating that the power source is OFF, the television switches its main power source to OFF.

The server can communicate with the air conditioner via the router and the specific low-power radio station (extra-small). Furthermore, since the air conditioner can communicate with the LED light emitter via BTLE, the server can communicate with the LED light emitter. Therefore, the server can switch the power of the TV ON and OFF via the LED light emitter. In addition, the smartphone can control the power supply of the TV via the server by communicating with the server via, for example, Wi-Fi (Wireless Fidelity).

  As shown in FIGS. 250A to 250C, the information communication method according to the present embodiment is such that the mobile terminal (smartphone) uses a control signal (transmission data string) by wireless communication (such as BTLE or Wi-Fi) different from visible light communication. Or a wireless communication step for transmitting a user command) to a lighting device (light emitter), a visible light communication step in which the lighting device performs visible light communication by changing the luminance according to the control signal, and a control target device (electronic An execution step of detecting a luminance change of the lighting device, acquiring a control signal by demodulating a signal specified by the detected luminance change, and executing a process according to the control signal; Including. Thereby, even if the portable terminal cannot change the luminance for visible light communication, the luminance of the lighting device can be changed instead of the portable terminal by wireless communication, and the control target device is appropriately controlled. can do. The mobile terminal may be a wristwatch instead of a smartphone.

(Reception without interference)
FIG. 251 is a flowchart illustrating a reception method in which interference is eliminated in the tenth embodiment.

  First, start is performed in step 9001a, and it is confirmed whether there is a periodic change in the intensity of light received in step 9001b. If YES, the process proceeds to step 9001c. In the case of NO, the process proceeds to Step 9001d to receive a wide range of light by setting the lens of the light receiving unit to a wide angle, and returns to Step 9001b. In step 9001c, it is confirmed whether the signal can be received. If YES, the process proceeds to step 9001e, the signal is received, and the process ends in step 9001g. In the case of NO, the process proceeds to Step 9001f, the lens of the light receiving unit is telephoto, and light in a narrow range is received, and the process returns to Step 9001c.

  By this method, signals from transmitters in a wide direction can be received while eliminating interference of signals from a plurality of transmitters.

(Estimation of transmitter orientation)
FIG. 252 is a flowchart illustrating a transmitter direction estimation method according to the tenth embodiment.

  First, start is performed in step 9002a, the lens of the light receiving unit is set to the maximum telephoto in step 9002b, and it is checked whether there is a periodic change in the intensity of light received in step 9002c. Proceed to 9002d. In the case of NO, the process proceeds to Step 9002e, where a wide range of light is received by setting the lens of the light receiving unit to a wide angle, and the process returns to Step 9002c. In step 9002d, the signal is received, in step 9002f, the lens of the light receiving unit is set to the maximum telephoto position, the light receiving direction is changed along the boundary of the light receiving range, and the direction in which the light receiving intensity is maximized is detected. Estimating that it is in the direction, the process ends at step 9002d.

  With this method, the direction in which the transmitter is present can be estimated. The maximum wide angle may be set first, and the telephoto may be gradually increased.

(Start receiving)
FIG. 253 is a flowchart illustrating a reception start method in Embodiment 10.

  First, start is performed in step 9003a, and it is confirmed in step 9003b whether a signal from a base station such as Wi-Fi, Bluetooth (registered trademark) or IMES is received. If YES, the process proceeds to step 9003c. If NO, the process returns to step 9003b. In step 9003c, it is confirmed whether or not the base station is registered in the receiver or server as a trigger for starting reception. If YES, the process proceeds to step 9003d, starts receiving a signal, and ends in step 9003e. . If NO, the process returns to step 9003b.

  By this method, reception can be started without the user performing an operation to start reception. Further, power consumption can be suppressed as compared with the case where reception is always performed.

(Generation of ID using information from other media)
FIG. 254 is a flowchart illustrating an ID generation method using information of another medium according to the tenth embodiment.

  First, start in step 9004a, ID of carrier communication network, Wi-Fi, Bluetooth (registered trademark), etc. connected in step 9004b, or position information obtained from the ID, position information obtained from GPS, etc. Is transmitted to the upper bit ID index server. In step 9004c, the upper bits of the visible light ID are received from the upper bit ID index server, and in step 9004d, the signal from the transmitter is received as the lower bits of the visible light ID. In step 9004e, the upper and lower bits of the visible light ID are combined and transmitted to the ID resolution server, and the process ends in step 9004f.

  With this method, it is possible to obtain upper bits that are commonly used in the vicinity of the receiver, and to reduce the amount of data transmitted by the transmitter. In addition, the receiving speed of the receiver can be increased.

  Note that the transmitter may transmit both the upper and lower bits. In this case, the receiver using this method can synthesize the ID when the lower bit is received, and the receiver not using this method obtains the ID by receiving the entire ID from the transmitter. .

(Selection of reception method by frequency separation)
FIG. 255 is a flowchart illustrating a reception method selection method based on frequency separation in the tenth embodiment.

  First, in step 9005a, the optical signal received in step 9005b is applied to a frequency filter circuit, or discrete Fourier series expansion is performed to perform frequency decomposition. In step 9005c, it is confirmed whether or not a low frequency component exists. If YES, the process proceeds to step 9005d, a signal expressed in a low frequency region such as frequency modulation is decoded, and the process proceeds to step 9005e. If NO, the process proceeds to step 9005e. In step 9005e, it is confirmed whether or not the base station is registered in the receiver or server as a trigger for starting reception. If YES, the process proceeds to step 9005f, and a signal expressed in a high frequency region such as pulse position modulation. And the process proceeds to Step 9005g. If NO, the process advances to step 9005g. In step 9005g, signal reception is started, and in step 9005h, the process ends.

  By this method, a signal modulated by a plurality of modulation schemes can be received.

(Signal reception when exposure time is long)
FIG. 256 is a flowchart showing a signal reception method when the exposure time is long in the tenth embodiment.

  First, start is made in step 9030a, and if the sensitivity can be set in step 9030b, the sensitivity is set to the maximum. If the exposure time can be set in step 9030c, the exposure time is set shorter than that in the normal shooting mode. In step 9030d, two images are picked up and a difference in luminance is obtained. If the position or direction of the imaging unit changes during the imaging of two images, the change is canceled and an image as if it was captured from the same position and direction is generated to obtain the difference. In Step 9030e, a value obtained by averaging the luminance values in the direction parallel to the difference image or the exposure line of the captured image is obtained. The averaged values in step 9030f are arranged in the direction perpendicular to the exposure line, and discrete Fourier transform is performed. In step 9030g, it is recognized whether there is a peak near a predetermined frequency, and the process ends in step 9030h.

  With this method, a signal can be received even when the exposure time is long, such as when the exposure time cannot be set or when a normal image is taken simultaneously.

  When the exposure time is automatically set, when the camera is directed to a transmitter configured as illumination, the exposure time is set to about 1/60 second to about 1/480 second by the automatic exposure correction function. If the exposure time cannot be set, a signal is received under this condition. In the experiment, when the illumination is blinked periodically, if the time of one cycle is about 1/16 or more of the exposure time, stripes can be visually recognized in the direction perpendicular to the exposure line. I was able to recognize it. At this time, since the brightness is too high in the portion where the illumination is reflected and it is difficult to confirm the stripe, it is preferable to obtain the period of the signal from the portion where the illumination light is reflected.

  When using a method of periodically turning on / off the light emitting unit, such as a frequency shift keying method or a frequency multiplexing modulation method, even if the modulation frequency is the same as that of the pulse position modulation method, It is difficult for the viewer to see the flicker, and it is difficult for the flicker to appear in the video shot with the video camera. Therefore, a low frequency can be used as the modulation frequency. Since the temporal resolution of human vision is about 60 Hz, a frequency higher than this frequency can be used as the modulation frequency.

  When the modulation frequency is an integral multiple of the imaging frame rate of the receiver, pixels at the same position in the two images are imaged when the light pattern of the transmitter is in the same phase, so a bright line appears in the difference image. It is difficult to receive. Since the imaging frame rate of the receiver is usually 30 fps, reception is easy if the modulation frequency is set to a value other than an integral multiple of 30 Hz. Also, since there are various imaging frame rates of the receiver, two disjoint modulation frequencies are assigned to the same signal, and the transmitter receives signals by alternately using the two modulation frequencies for transmission. The machine can easily restore the signal by receiving at least one signal.

  FIG. 257 is a diagram illustrating an example of a transmitter dimming (adjusting brightness) method.

By adjusting the ratio between the high luminance section and the low luminance section, the average luminance changes and the brightness can be adjusted. At this time, to keep the period T ₁ which repeats high and low brightness constant, it is possible to keep the frequency peak constant. For example, in all of (a), (b), and (c) of FIG. 257, transmission is performed while the time T1 between the first luminance change that becomes brighter than the average luminance and the second luminance change is kept constant. When dimming the machine darkly, the time for lighting brighter than the average brightness is shortened. On the other hand, when the transmitter is dimmed brightly, the illumination time is set longer than the average luminance. (B) and (c) in FIG. 257 are dimmed darker than (a), and (c) in FIG. 257 is dimmed the darkest. Thereby, dimming can be performed while transmitting signals having the same meaning.

  The average brightness may be changed by changing the brightness value of the section with high brightness, the brightness of the section with low brightness, or both brightness values.

  FIG. 258 is a diagram illustrating an example of a method of configuring the dimming function of the transmitter.

  Since the accuracy of the component parts is limited, even if the same dimming setting is performed, the brightness is slightly different from that of another transmitter. However, when transmitters are arranged side by side, if the brightness of adjacent transmitters is different, unnaturalness can be felt. Therefore, the user adjusts the brightness of the transmitter by operating the dimming correction operation unit. The dimming correction unit holds the correction value, and the dimming control unit controls the brightness of the light emitting unit according to the correction value. When the degree of dimming is changed by the user operating the dimming operation unit, the dimming control unit uses the changed dimming setting value and the correction value held in the dimming correction unit. In addition, the brightness of the light emitting unit is controlled. The dimming control unit transmits the dimming setting value to another transmitter through the interlocking dimming unit. When the dimming setting value is transmitted from another device through the linked dimming unit, the dimming control unit, based on the dimming setting value and the correction value held in the dimming correction unit, To control the brightness.

  According to one embodiment of the present invention, there is provided a control method for controlling an information communication device that transmits a signal by changing luminance of a light emitter, and includes a plurality of different signals for a computer of the information communication device. A determination step for determining a luminance change pattern of a different frequency for each different signal by modulating a signal to be transmitted; and a luminance change obtained by modulating a single signal at a time corresponding to a single frequency. A control method may include a transmission step of transmitting a signal to be transmitted by changing the luminance of the light emitter so as to include only the pattern.

  For example, when a time corresponding to a single frequency includes a luminance change pattern obtained by modulating a plurality of signals, the waveform of the luminance change over time becomes complicated, and it is difficult to receive it appropriately. However, by performing control so that only a luminance change pattern obtained by modulating a single signal is included at a time corresponding to a single frequency, reception can be performed more appropriately.

  According to one embodiment of the present invention, the determining step is configured such that the number of transmissions for transmitting one signal among a plurality of different signals is different from the number of transmissions for transmitting other signals within a predetermined time. In addition, the number of transmissions may be determined.

  Since the number of transmissions for transmitting one signal is different from the number of transmissions for transmitting another signal, flickering during transmission can be prevented.

  According to one embodiment of the present invention, the determining step may increase the number of transmissions of a signal corresponding to a high frequency within a predetermined time, compared to the number of transmissions of other signals.

  When frequency conversion is performed on the receiving side, a signal corresponding to a high frequency has low luminance, but by increasing the number of transmissions, it is possible to increase the luminance value when performing frequency conversion.

  According to one embodiment of the present invention, the luminance change pattern may be a pattern in which the waveform of the luminance change over time is any one of a rectangular wave, a triangular wave, and a sawtooth wave.

  By using a rectangular wave or the like, reception can be performed more appropriately.

  According to one embodiment of the present invention, when increasing the value of the average luminance of the illuminant, the time during which the luminance of the illuminant is greater than a predetermined value at a time corresponding to a single frequency, You may lengthen with respect to the case where the value of the average luminance of the said light-emitting body is made small.

  By adjusting the time during which the luminance of the light emitter is greater than a predetermined value at the time corresponding to a single frequency, it is possible to transmit a signal and adjust the average luminance of the light emitter. For example, when the light emitter is used as illumination, it is possible to transmit a signal while making the overall brightness darker or brighter.

  The receiver can set the exposure time to a predetermined value by using an API for setting the exposure time (which stands for application programming interface and indicates a means for using the function of the OS). The visible light signal can be received stably. Further, the receiver can set the sensitivity to a predetermined value by using an API for setting the sensitivity, and stably receives a visible light signal even when the brightness of the transmission signal is dark or bright. be able to.

(Embodiment 11)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

(Exposure time setting)
259A to 259D are flowcharts illustrating an example of operation of a receiver in Embodiment 11.

  In order to receive a visible light signal by the image sensor using the method of the present invention, it is necessary to set the exposure time to be shorter than a predetermined time. The predetermined time is determined by the modulation method and the modulation frequency of the visible light signal. Generally, it is necessary to shorten the exposure time as the modulation frequency increases.

  As the exposure time is shortened, the bright line can be clearly observed. On the other hand, when the exposure time is shortened, the amount of received light is reduced and the entire captured image becomes dark. That is, the signal intensity is attenuated. Therefore, the reception performance (reception speed and error rate) can be improved by shortening the exposure time within a range in which the presence of the visible light signal can be detected.

  As illustrated in FIG. 259A, the receiver first sets the imaging mode to the visible light imaging mode (step S9201). At this time, the receiver determines whether or not to receive a signal having a monochrome imaging function and modulated only with luminance information (step S9202). Here, when it is determined that there is a monochrome imaging function and a signal modulated only with luminance information is received (Y in step S9202), the receiver sets the mode related to the color among the imaging modes to the monochrome imaging function. Is set to a monochrome imaging mode using (step S9203). As a result, when receiving a signal modulated only with luminance information, that is, when receiving a visible light signal representing information only by luminance change, the processing speed can be improved by not handling color information. it can. On the other hand, when it is determined in step S9202 that there is a monochrome imaging function and a signal modulated only with luminance information is not received (N in step 9202), that is, visible light is used using color information. When the signal is expressed, the receiver sets the color-related mode in the imaging mode to the color imaging mode (step S9204).

  Next, the receiver determines whether or not the function for specifying the exposure time is provided in the imaging unit including the above-described image sensor (step S9205). If it is determined that the image is provided (Y in step S9205), the receiver sets the exposure time to a time shorter than the predetermined time using the function so that a bright line appears in the captured image. (Step S9206). Note that the receiver may set the exposure time so that the exposure time is as short as possible in the range where the transmitter transmitting the visible light signal can be seen in the captured image.

  On the other hand, if it is determined in step S9205 that the function for specifying the exposure time is not provided (N in step S9205), the receiver further determines whether the imaging unit has a function for setting sensitivity (step S9205). Step S9207). If it is determined that the function for setting the sensitivity is provided (Y in step S9207), the receiver sets the sensitivity to the maximum by the function (step S9208). As a result, when a captured image is obtained by performing imaging with the maximized sensitivity, the captured image becomes bright. Therefore, in the receiver, if the automatic exposure is set to be effective, the exposure time is set to be short so that the exposure is within a certain range by the automatic exposure. In automatic exposure, every time imaging is performed, a captured image obtained by the imaging is used for input of automatic exposure. Based on the captured image, the exposure time is set so that the exposure falls within a certain range. Adjusted from time to time. Details of the automatic exposure will be described later.

  Further, the receiver determines whether or not the imaging unit has a function of setting the F value (aperture) (step S9209). If it is determined that the function for setting the F value is provided (Y in step S9209), the receiver sets the F value to the minimum (open the aperture) by using the function (step S9210). As a result, when a captured image is obtained by performing imaging with the minimized F value, the captured image becomes bright. Therefore, in the receiver, if the automatic exposure is set to be effective, the exposure time is set to be short so that the exposure is within a certain range by the automatic exposure.

  Further, the receiver determines whether or not the imaging unit has a function for designating an exposure correction value (step S9211). If it is determined that the function for specifying the value of exposure correction is provided (Y in step S9211), the receiver sets the value of exposure correction to the minimum by the function (step S9212). As a result, in the receiver, when the automatic exposure is set to be effective, the exposure time is set short to reduce the exposure by the automatic exposure.

  A scene mode (high-speed scene mode) for imaging a subject that moves at high speed is generally defined by the names “sports” and “action”.

  As shown in FIG. 259B, the receiver has a function for setting the high-speed scene mode after step S9211 or step S9212 and the sensitivity is set to the scene mode by setting the high-speed scene mode. Judgment whether or not the condition that the F value is set lower than before, the F value is set higher than before the scene mode setting, or the exposure correction value is not set higher than before the scene mode setting is satisfied. (Step S9213). If it is determined that the above condition is satisfied (Y in step S9213), the receiver sets the scene mode to the high-speed scene mode (step S9214). As a result, in the receiver, when the automatic exposure is set to be effective, the exposure time is set to be short by the automatic exposure in order to capture an image of a subject moving at high speed without blurring.

  Next, the receiver sets automatic exposure to be effective (step S9215), and captures an image of the subject (step S9216).

  As illustrated in FIG. 259C, the receiver determines whether or not the image capturing unit has a zoom function after step S9215 (step S9217). If it is determined that the zoom function is provided (Y in step S9217), the receiver can further specify whether or not the center position of the zoom can be designated, that is, the center position can be arbitrarily set in the captured image. It is determined whether or not the position can be set (step S9218). If it is determined that the center position can be designated (Y in step S9218), the receiver designates the center position of the zoom in the bright part in the captured image, so that the subject corresponding to the bright part becomes large at the center. Zooming is performed so that an image is captured (step S9219). On the other hand, if it is determined that the center position cannot be specified (N in step S9218), the receiver determines whether the center of the captured image is brighter than a predetermined value of brightness, It is determined whether or not the brightness is brighter than the average brightness at the location (step S9220). If it is determined that the image is bright (Y in step S9220), the receiver performs zooming (step S9221). That is, also at this time, the subject corresponding to the bright part is imaged largely at the center.

  Generally, many devices with an imaging unit use center-weighted metering for photometry, so if there is a bright part in the center of the captured image, the bright part even if the photometric position is not designated as another position The exposure is adjusted with reference to. Thereby, the exposure time is set short. In addition, the area of the bright portion is increased by zooming, and the exposure is adjusted based on a brighter screen, so the exposure time is set short.

Next, the receiver determines whether or not there is a function for designating a photometric position or a focus position (step S9222). If it is determined that the function is present (Y in step S9222), the receiver performs processing for finding a bright place in the captured image. That is, the receiver performs processing for finding out the location of an area having a predetermined shape and size that is brighter than the predetermined brightness from the captured image. Specifically, the receiver first determines whether or not an exposure evaluation calculation formula for automatic exposure is known (step S9224). If it is determined that it is known (Y in step S9224), the receiver evaluates the brightness of each region in the captured image using the same calculation formula as the known calculation formula, thereby A bright area is found (step S9226). On the other hand, when it is determined that the calculation formula for the exposure evaluation is unknown (N in step S9224), the receiver calculates the average value of the brightness of each pixel in a region having a predetermined shape and size. By using the calculation formula, the brightness of each area in the captured image is evaluated to find the location of the above-described bright area (step S9225). The predetermined shape is, for example, a rectangle, a circle, or a cross shape. Moreover, the area | region may be comprised from the non-continuous several area | region. In addition, the above average value may be calculated not by a simple average but by a weighted average with a greater weight at the center.

  The receiver determines whether the total area of all the bright areas found is smaller than a predetermined area (step S9227). If it is determined that the image is small (Y in step S9227), the receiver performs zooming so that the total area of the bright regions is imaged with a size equal to or larger than a predetermined area (step S9228). Next, the receiver determines whether or not the photometric position can be designated (step S9229). If it is determined that the photometric position can be designated (Y in step S9229), the receiver designates the brightest area as the photometric position (step S9230). In automatic exposure, the exposure is adjusted according to the brightness of the photometric position. Therefore, by specifying the place of the brightest area as the photometric position, the exposure time is set short by automatic exposure. On the other hand, if it is determined in step S9229 that the photometric position cannot be designated (N in step S9229), that is, if the focus position can be designated, the receiver designates the location of the brightest area as the focus position. Various imaging units can be mounted on the receiver, and in the automatic exposure of some imaging units among the various imaging units, the exposure is adjusted by the brightness of the focus position. Therefore, by specifying the brightest area as the focus position, the exposure time is set short by automatic exposure. The location specified here may be different from the location of the region used for the brightness evaluation or calculation, and is set in accordance with the setting method of the imaging unit. As an example, if the specified format is a format that specifies a center point, the receiver specifies the center of the brightest area, and if the specified format is a format that specifies a rectangular area, the receiver Specifies a rectangular area that includes the center of the bright area.

  Next, the receiver determines whether there is an area brighter than the area of the place designated as the photometric position or the focus position in the captured image (step S9232). Here, if it is determined that the area is present (Y in step S9232), the receiver repeatedly executes the processing from step S9217. On the other hand, if it is determined that there is no such area (N in step S9232), the receiver images the subject (step S9233).

  Next, the receiver determines whether or not the automatic exposure should be terminated based on the captured image obtained by the imaging in step S9233, or whether or not a certain time has elapsed since the automatic exposure was set to be effective. (Step S9234). Here, for example, when it is determined that the automatic exposure should not be terminated (N in Step S9234), the receiver further determines the position or the imaging direction of the imaging unit based on the captured image obtained by the imaging in Step S9233. It is determined whether or not there has been a change (step S9235). If it is determined that the position of the imaging unit or the imaging direction has changed (Y in step S9235), the receiver performs the processing from step S9217 again. Thereby, even when the place designated as the photometric position or the focus position moves in the captured image, the place of the brightest region at that time can be designated again. On the other hand, if it is determined in step S9235 that the position of the imaging unit or the imaging direction has not changed (N in step S9235), the receiver repeatedly performs the processing from step S9232. Note that the receiver may search for the brightest area every time a single captured image is acquired by imaging, and respecify the photometric position and focus position.

  If it is determined in step S9234 that the automatic exposure should be terminated, if the exposure time does not change, or if it is determined that a certain time has elapsed (Y in step S9234), or exposure is performed in step S9206. When the time is set, the receiver sets the automatic exposure to invalid (step S9236) and images the subject (step S9237). Then, the receiver determines whether a visible light signal has been received by the imaging (step S9238). If it is determined that no visible light signal has been received (N in step S9238), the receiver further determines whether or not a predetermined time has elapsed (step S9239). If it is determined that the predetermined time has not elapsed (N in step S9239), the receiver repeatedly executes the processing from step S9237. On the other hand, when it is determined that the predetermined time has elapsed (Y in step S9239), that is, when the visible light signal cannot be received even after the predetermined time has elapsed, the receiver By repeatedly executing the processing from step S9232, the search for the brightest region is performed again.

  Note that when the receiver uses the zoom function, the receiver may turn off the zoom at an arbitrary timing. That is, when the zoom is off, the receiver may detect whether there is a bright subject in the range where the image is not captured during the zoom but is not zoomed. Note that this bright subject is likely to be a transmitter that transmits a visible light signal by a change in luminance. Thereby, the signal from the transmitter which exists in a wide range can be received.

  Here, the automatic exposure and photometry method will be described.

  Hereinafter, automatic exposure in FIGS. 259A to 259D will be described. Automatic exposure is an operation, process, or function in which the imaging unit of the receiver adjusts the exposure time, sensitivity, and aperture, and automatically adjusts the photometric result to a predetermined value.

  Photometric methods for obtaining a photometric result include methods such as average photometry (entire photometry), center-weighted photometry, spot photometry (partial photometry), and split photometry. In average photometry, the average brightness of the entire image obtained by imaging is calculated. In center-weighted metering, a weighted average value of brightness is calculated by increasing the weight closer to the center (or designated portion) of the image. In spot photometry, the average value (or weighted average value) of the brightness in a predetermined range (or several locations) defined around the center or specified portion of the image is calculated. In split photometry, an image is divided into a plurality of parts, and photometry is performed on each part to calculate a total brightness value.

  In an imaging unit having an automatic exposure function, the exposure time can be indirectly set by the automatic exposure function even if the exposure time cannot be set to be shortened directly. For example, if the sensitivity is set high (for example, at the maximum value), the image obtained by imaging becomes bright if the other conditions are the same. Therefore, the exposure time can be set short by automatic exposure. Further, when the aperture is set to be opened (that is, opened), similarly, the exposure time can be set short. If a value indicating the level of exposure correction is set low (for example, to a minimum value), an image obtained by imaging becomes dark due to automatic exposure. That is, the exposure time is set short. By specifying the brightest place in the image as the photometry position, the exposure time can be set short. If the photometric method can be specified, the exposure time can be set short by specifying spot photometry. If the photometric range can be designated, the exposure time can be set short by designating the photometric range to the minimum. When the area of the bright part in the image is large, the exposure time can be set short by specifying the photometric range as large as possible so as not to exceed that part. If a plurality of photometric positions can be designated, the exposure time can be set short by designating the same place as the photometric position several times. By zooming in on a bright place in the image and designating that place as the photometric position, the exposure time can be set short.

  Here, the EX zoom will be described.

  FIG. 260 is a diagram for explaining the EX zoom.

  The zoom in FIG. 259C, that is, a method for obtaining a large image includes optical zoom for changing the size of the image captured on the image sensor by adjusting the focal length of the lens, and digital processing for interpolating the image captured on the image sensor. There are a digital zoom that obtains a large image and an EX zoom that obtains a large image by changing a plurality of image sensors used for imaging. The EX zoom can be used when the number of image sensors included in the image sensor is larger than the resolution of the captured image.

  For example, in the image sensor 10080a shown in FIG. 260, 32 × 24 imaging elements are arranged in a matrix. That is, 32 image sensors are arranged horizontally and 24 elements are arranged vertically. When an image having a horizontal 16 × longitudinal resolution is obtained by imaging by the image sensor 10080a, as shown in FIG. 260 (a), among the 32 × 24 imaging elements included in the image sensor 10080a, the image sensor Only 16 × 12 image sensors (for example, image sensors indicated by black squares in the image sensor 1080a in FIG. 260A) that are uniformly distributed throughout 10080a are used for imaging. That is, only an odd-numbered or even-numbered image sensor is used for imaging among a plurality of image sensors arranged in the vertical direction and the horizontal direction. As a result, an image 10080b having a desired resolution is obtained. In FIG. 260, a subject appears on the image sensor 1008a, but this is for easy understanding of the correspondence between each image sensor and an image obtained by imaging.

  The receiver including the image sensor 10080a captures a wide range, and searches for a transmitter or receives information from many transmitters in the image sensor 10080a. An image is picked up using only a part of the image pickup elements arranged in a distributed manner.

  When the receiver performs the EX zoom, as shown in FIG. 260 (b), in the image sensor 10080a, some image pickup devices (for example, FIG. 260 (b)) arranged locally densely. Only 16 × 12 image sensors indicated by black squares in the image sensor 1080a in FIG. As a result, a part of the image 10080b corresponding to a part of the imaging elements is zoomed, and an image 10080d is obtained. By such an EX zoom, it is possible to receive a visible light signal for a long time by capturing a large image of the transmitter, the reception speed is improved, and a visible light signal can be received from a distance.

  In digital zoom, the number of exposure lines that receive visible light signals cannot be increased, and the reception time of visible light signals does not increase. Therefore, it is better to use other zooms as much as possible. The optical zoom requires a physical movement time of the lens and the image sensor. However, since the EX zoom is performed only by electronic setting change, there is an advantage that the time required for the zoom is short. From this viewpoint, the priority order of each zoom is (1) EX zoom, (2) optical zoom, and (3) digital zoom. The receiver may select and use any one or a plurality of zooms according to the priority order and the necessity of the zoom magnification. Note that in the imaging methods shown in FIGS. 260A and 260B, image noise can be suppressed by using an unused imaging device.

  FIG. 261A is a flowchart illustrating processing of a reception program in the tenth embodiment.

  This reception program is a program that causes a computer provided in the receiver to execute, for example, the processes shown in FIGS. 259A to 260.

  That is, this reception program is a reception program for receiving information from the light emitter. Specifically, the reception program sets the exposure time of the image sensor using automatic exposure, and causes the image sensor to image the illuminant whose luminance changes with the set exposure time. A bright line image acquisition step SA22 for acquiring a bright line image that is an image including a bright line corresponding to each of a plurality of exposure lines included in the image sensor; and decoding a plurality of bright line patterns included in the acquired bright line image. The information acquisition step SA23 for acquiring information is executed by the computer. In exposure time setting step SA21, as in step S9208 of FIG. 259A, the sensitivity of the image sensor is set to the maximum value in a predetermined range for the image sensor, and the sensitivity is set according to the sensitivity of the maximum value. Set the exposure time by automatic exposure.

  As a result, even if the exposure time of the image sensor cannot be set directly, the exposure time is short enough to obtain an appropriate bright line image using the automatic exposure function installed in a general camera. Can be set. That is, in the automatic exposure, the exposure is adjusted based on the brightness of the image acquired by imaging by the image sensor. Therefore, if the sensitivity of the image sensor is set to a large value, the image becomes bright. Therefore, in order to suppress exposure, the exposure time of the image sensor is set short. If the sensitivity of the image sensor is set to the maximum value, the exposure time can be set shorter and an appropriate bright line image can be acquired. That is, information from the light emitter can be appropriately received. As a result, communication between various devices can be enabled. The sensitivity is, for example, ISO sensitivity.

  In the exposure time setting step SA21, as in step S9212 in FIG. 259A, the value indicating the exposure correction level of the image sensor is set to the minimum value in the range preset for the image sensor. The exposure time corresponding to the maximum sensitivity and the minimum level exposure correction is set by automatic exposure.

  As a result, since the value indicating the level of exposure correction is set to the minimum value, the exposure time can be set shorter by automatic exposure processing that attempts to suppress exposure, and a more appropriate bright line image is acquired. be able to. The unit of the value indicating the exposure correction level is, for example, EV.

  In the exposure time setting step SA21, as shown in FIG. 259C, a brighter part than the other part is specified in the first image acquired by imaging the subject including the light emitter by the image sensor. Then, a part of the subject corresponding to the bright part is enlarged by optical zoom. Furthermore, an exposure time is set by using a second image acquired by imaging a part of the enlarged subject by the image sensor for input of automatic exposure. In the bright line image acquisition step SA22, a bright line image is acquired by causing the image sensor to capture a part of the enlarged subject with the set exposure time.

  Thereby, a part of the subject corresponding to the bright part of the first image is enlarged by the optical zoom, that is, the bright light emitter is enlarged by the optical zoom. Can also be brightened overall. Then, since the bright second image is used for the input of automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can.

  Further, in the exposure time setting step SA21, as shown in FIG. 259C, the central portion of the first image acquired by the imaging of the subject including the illuminant is brightness at a plurality of positions in the first image. It is judged whether it is brighter than the average of. If it is determined that the central portion is bright, a part of the subject corresponding to the central portion is enlarged by optical zoom. Furthermore, an exposure time is set by using a second image acquired by imaging a part of the enlarged subject by the image sensor for input of automatic exposure. In the bright line image acquisition step SA22, a bright line image is acquired by causing the image sensor to capture a part of the enlarged subject with the set exposure time.

  Accordingly, a part of the subject corresponding to the bright central portion of the first image is enlarged by the optical zoom, that is, a bright light-emitting body is enlarged by the optical zoom, so that the second image is changed to the first image. Can be brighter overall. Then, since the bright second image is used for the input of automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can. Further, in the optical zoom, if the enlargement center position cannot be arbitrarily set, the angle of view or the central portion of the image is enlarged. Therefore, even if the center position cannot be arbitrarily set, if the central portion of the first image is bright, the second image can be brightened as a whole by using the optical zoom. it can. Further, even when the central portion of the first image is dark, if the enlargement by the optical zoom is performed, the second image becomes dark and the exposure time becomes long. Therefore, as described above, it is possible to prevent the exposure time from becoming longer by enlarging with the optical zoom only when it is determined that the central portion is bright.

  Further, in the exposure time setting step SA21, as shown in FIG. 260, among the K imaging elements (K is an integer of 3 or more) included in the image sensor, N that are uniformly distributed in the image sensor. In a first image acquired by imaging a subject including a light emitter using only N (N is an integer of 2 or more and an integer of 2 or more), a brighter part than the other part is specified. Furthermore, by using a second image obtained by imaging with only N dense image pickup elements corresponding to a bright portion among K image pickup elements included in the image sensor for input of automatic exposure, exposure is performed. Set the time. Also, in the bright line image acquisition step SA22, a bright line image is acquired by causing only N dense image pickup elements included in the image sensor to capture images with a set exposure time.

  Thereby, even if the bright part in the first image is not at the center by so-called EX zoom, the second image can be brightened as a whole, and the exposure time can be set shorter.

  In the exposure time setting step SA21, as shown in FIG. 259C, the photometric position in the image acquired by the image sensor capturing the subject is set, and the exposure time according to the brightness of the set photometric position. Is set by automatic exposure.

  As a result, if a bright part in the image acquired by imaging is set as the photometric position, the exposure time can be set shorter by the automatic exposure process to suppress exposure, and an appropriate bright line image can be obtained. Can be acquired.

  The reception program further includes an imaging mode setting step for switching the imaging mode of the image sensor from a color imaging mode for acquiring a color image by imaging to a monochrome imaging mode for acquiring a monochrome image by imaging. It may be executed by a computer. In this case, in the exposure time setting step SA21, the exposure time is set by using an image acquired in the monochrome imaging mode for input of automatic exposure.

  Accordingly, since an image acquired in the monochrome imaging mode is used for automatic exposure input, an appropriate exposure time can be set without being influenced by color information. Further, when the exposure time is set in the monochrome imaging mode, the bright line image is acquired by imaging according to the mode. Therefore, when information is transmitted from the light emitter only by a luminance change, the information can be appropriately acquired.

  In the exposure time setting step SA21, the exposure time of the image sensor is updated by using the acquired image for automatic exposure every time an image is acquired by imaging the light emitter with the image sensor. Here, for example, as shown in step S9234 in FIG. 259D, when the fluctuation range of the exposure time updated as needed is less than or equal to a predetermined range, the exposure time is updated by ending the update of the exposure time by automatic exposure. Set.

  Thereby, when the fluctuation of the exposure time is stabilized, that is, when the brightness of the image acquired by imaging falls within the target brightness by the automatic exposure, the exposure time at that time is the bright line image. It is used for imaging for acquisition. Therefore, an appropriate bright line image can be acquired.

  FIG. 261B is a block diagram of a receiving apparatus in Embodiment 10.

  The receiving device A20 is the above-described receiver that executes the processes shown in FIGS. 259A to 260, for example.

  That is, the receiving device A20 is a device for receiving information from a light emitter, and sets an exposure time setting unit A21 that sets the exposure time of the image sensor using automatic exposure, and a light emitter that changes in luminance. An image sensor A22 that acquires a bright line image that is an image including a bright line corresponding to each of a plurality of exposure lines included in the image sensor by causing the image sensor to capture an image with the exposure time that is included, and included in the acquired bright line image And a decoding unit A23 that acquires information by decoding a plurality of bright line patterns. The exposure time setting unit A21 sets the sensitivity of the image sensor to the maximum value in a range predetermined for the image sensor, and sets the exposure time according to the sensitivity of the maximum value by automatic exposure. This receiving apparatus A20 can achieve the same effects as the above receiving program.

  A reception program according to an aspect of the present invention is a reception program for receiving information from a light emitter that changes in luminance according to a signal output by the information processing program, and automatically exposes an exposure time of an image sensor. Each of a plurality of exposure lines included in the image sensor by causing the image sensor to take an image of an object including the light-emitting body that changes in luminance with the set exposure time. A bright line image acquisition step of acquiring a bright line image that is an image including a bright line corresponding to the information, and an information acquisition step of acquiring information by decoding the patterns of the plurality of bright lines included in the acquired bright line image In the exposure time setting step, the sensitivity of the image sensor is Set to the maximum value of the predetermined range for Mejisensa, it sets the exposure time corresponding to the sensitivity of the maximum value by the automatic exposure.

  As a result, as shown in FIGS. 259A to 261B, even if the exposure time of the image sensor cannot be set directly, an appropriate bright line can be obtained using the automatic exposure function installed in a general camera. An exposure time that is short enough to acquire an image can be set. That is, in the automatic exposure, the exposure is adjusted based on the brightness of the image acquired by imaging by the image sensor. Therefore, if the sensitivity of the image sensor is set to a large value, the image becomes bright. Therefore, in order to suppress exposure, the exposure time of the image sensor is set short. If the sensitivity of the image sensor is set to the maximum value, the exposure time can be set shorter and an appropriate bright line image can be acquired. That is, information from the light emitter can be appropriately received. As a result, communication between various devices can be enabled. The sensitivity is, for example, ISO sensitivity.

  In the exposure time setting step, a value indicating a level of exposure correction of the image sensor is set to a minimum value in a range preset for the image sensor, and the sensitivity of the maximum value and the minimum value are set. The exposure time corresponding to the value level exposure correction may be set by the automatic exposure.

  As a result, since the value indicating the level of exposure correction is set to the minimum value, the exposure time can be set shorter by automatic exposure processing that attempts to suppress exposure, and a more appropriate bright line image is acquired. be able to. The unit of the value indicating the exposure correction level is, for example, EV.

  In the exposure time setting step, a brighter part than the other part is identified in the first image acquired by imaging the subject including the light emitter by the image sensor, and one of the subjects corresponding to the bright part is identified. The exposure time is set by using a second image obtained by enlarging a part by optical zoom and using the second image acquired by imaging the part of the enlarged subject by the image sensor, In the bright line image acquisition step, the bright line image may be acquired by causing the image sensor to capture a part of the enlarged subject with the set exposure time.

  Thereby, a part of the subject corresponding to the bright part of the first image is enlarged by the optical zoom, that is, the bright light emitter is enlarged by the optical zoom. Can also be brightened overall. Then, since the bright second image is used for the input of automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can.

  Further, in the exposure time setting step, the central portion of the first image acquired by imaging the subject including the light emitter by the image sensor is more than the average brightness of a plurality of positions in the first image. When it is determined whether the central portion is bright, a part of the subject corresponding to the central portion is enlarged by optical zoom, and the enlarged image of the subject by the image sensor is determined. The exposure time is set by using a second image acquired by a part of imaging for the input of the automatic exposure. In the bright line image acquisition step, a part of the enlarged subject is set. The bright line image may be acquired by causing the image sensor to take an image with the exposure time.

  Accordingly, a part of the subject corresponding to the bright central portion of the first image is enlarged by the optical zoom, that is, a bright light-emitting body is enlarged by the optical zoom, so that the second image is changed to the first image. Can be brighter overall. Then, since the bright second image is used for the input of automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can. Further, in the optical zoom, if the enlargement center position cannot be arbitrarily set, the angle of view or the central portion of the image is enlarged. Therefore, even if the center position cannot be arbitrarily set, if the central portion of the first image is bright, the second image can be brightened as a whole by using the optical zoom. it can. Further, even when the central portion of the first image is dark, if the enlargement by the optical zoom is performed, the second image becomes dark and the exposure time becomes long. Therefore, as described above, it is possible to prevent the exposure time from becoming longer by enlarging with the optical zoom only when it is determined that the central portion is bright.

  Further, in the exposure time setting step, among K (K is an integer of 3 or more) image pickup elements included in the image sensor, N (N is K) that are uniformly distributed in the image sensor. In a first image acquired by imaging an object including the light emitter using only an imaging element of less than 2 (an integer greater than or equal to 2), a portion brighter than other portions is specified, and K images included in the image sensor are identified. Among the elements, the exposure time is set by using a second image acquired by imaging with only the dense N imaging elements corresponding to the bright part for the input of the automatic exposure, and the bright line image In the acquisition step, the bright line image is acquired by causing only the N dense image pickup elements included in the image sensor to capture images with the set exposure time. It may be.

  Thereby, even if the bright part in the first image is not at the center by so-called EX zoom, the second image can be brightened as a whole, and the exposure time can be set shorter.

  In the exposure time setting step, a photometric position in an image acquired by the image sensor capturing the subject is set, and the exposure time according to the brightness of the set photometric position is automatically set. You may set by exposure.

  As a result, if the bright part in the image acquired by imaging is set as the photometry position, the exposure time can be set shorter by the automatic exposure process to suppress exposure, and an appropriate bright line image is acquired. can do.

  Further, the reception program further sets an imaging mode setting for switching the imaging mode of the image sensor from a color imaging mode for acquiring a color image by imaging to a monochrome imaging mode for acquiring a monochrome image by imaging. The step may be executed by the computer, and in the exposure time setting step, the exposure time may be set by using an image acquired in the monochrome imaging mode as an input for the automatic exposure.

  Accordingly, since an image acquired in the monochrome imaging mode is used for automatic exposure input, an appropriate exposure time can be set without being influenced by color information. Further, when the exposure time is set in the monochrome imaging mode, the bright line image is acquired by imaging according to the mode. Therefore, when information is transmitted from the light emitter only by a luminance change, the information can be appropriately acquired.

  In the exposure time setting step, the exposure time of the image sensor is updated by using the acquired image for input of the automatic exposure every time an image is acquired by imaging the light emitter by the image sensor. Then, the exposure time may be set by ending the update of the exposure time by the automatic exposure when the fluctuation range of the exposure time updated as needed falls below a predetermined range.

  Thereby, when the fluctuation of the exposure time is stabilized, that is, when the brightness of the image acquired by imaging falls within the target brightness by the automatic exposure, the exposure time at that time is the bright line image. It is used for imaging for acquisition. Therefore, an appropriate bright line image can be acquired.

(Embodiment 12)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  In the present embodiment, the exposure time is set for each exposure line or each image sensor.

  262, 263, and 264 are diagrams illustrating an example of a signal reception method in Embodiment 12.

  As shown in FIG. 262, in an image sensor 10010a that is an imaging unit provided in the receiver, an exposure time is set for each exposure line. That is, a long exposure time for normal imaging is set for a predetermined exposure line (white exposure line in FIG. 262), and a visible light imaging for other exposure lines (black exposure line in FIG. 262). A short exposure time is set. For example, a long exposure time and a short exposure time are alternately set for each exposure line arranged in the vertical direction. Thereby, when imaging the transmitter which transmits a visible light signal by a luminance change, normal imaging and visible light imaging (visible light communication) can be performed almost simultaneously. The two exposure times may be alternately set for each line, may be set for every several lines, or different exposure times may be set for the upper part and the lower part of the image sensor 10010a. By using two exposure times in this way, when data obtained by imaging a plurality of exposure lines set to the same exposure time are collected, a normal captured image 10010b and a bright line image showing a plurality of bright line patterns are obtained. A visible light captured image 10010c is obtained. In the normal captured image 10010b, since a portion that has not been captured with a long exposure time (that is, an image corresponding to a plurality of exposure lines set to a short exposure time) is missing, the preview image is obtained by interpolating that portion. 10010d can be displayed. Here, information obtained by visible light communication can be superimposed on the preview image 10010d. This information is information associated with a visible light signal obtained by decoding a plurality of bright line patterns included in the visible light captured image 10010c. The receiver stores the normal captured image 10010b or an image obtained by performing interpolation on the normal captured image 10010b as a captured image, and stores the received visible light signal or information associated with the visible light signal. As additional information, it can also be added to the stored captured image.

  As shown in FIG. 263, an image sensor 10011a may be used instead of the image sensor 10010a. In the image sensor 1011a, the exposure time is set not for each exposure line but for each column (hereinafter, referred to as a vertical line) composed of a plurality of imaging elements arranged along a direction perpendicular to the exposure line. That is, a long exposure time for normal imaging is set for a predetermined vertical line (white vertical line in FIG. 263), and a visible light imaging for other vertical lines (black vertical line in FIG. 263). A short exposure time is set. In this case, in the image sensor 10011a, exposure is started at different timings for each exposure line, as in the image sensor 10010a. However, the exposure time of each image sensor included in the exposure line is different for each exposure line. . The receiver obtains a normal captured image 10011b and a visible light captured image 10011c by imaging with the image sensor 10011a. Further, the receiver generates and displays a preview image 10011d based on the normal captured image 10011b and information associated with the visible light signal obtained from the visible light captured image 10011c.

  In the image sensor 10011a, unlike the image sensor 10010a, all exposure lines can be used for visible light imaging. As a result, since the visible light captured image 10011c obtained by the image sensor 10011a includes more bright lines than the visible light captured image 10010c, the reception accuracy of the visible light signal can be increased.

  As shown in FIG. 264, an image sensor 10012a may be used instead of the image sensor 10010a. In the image sensor 10012a, the exposure time is set for each image sensor so that the same exposure time is not set continuously for each image sensor along the horizontal direction and the vertical direction. That is, the exposure time for each image sensor is such that a plurality of image sensors with a long exposure time set and a plurality of image sensors with a short exposure time are distributed like a grid or checkered pattern. Is set. Also in this case, similarly to the image sensor 10010a, the exposure is started at different timings for each exposure line, but the exposure time of each image sensor included in the exposure line is different for each exposure line. The receiver obtains a normal captured image 10012b and a visible light captured image 10012c by imaging with the image sensor 10012a. Further, the receiver generates and displays a preview image 10012d based on the normal captured image 10012b and information associated with the visible light signal obtained from the visible light captured image 10012c.

  Since the normal captured image 10012b obtained by the image sensor 10012a has data of a plurality of imaging elements arranged in a grid pattern or uniformly, interpolation or more accurately than the normal captured image 10010b and the normal captured image 10011b is performed. You can resize. The visible light captured image 10012c is generated by imaging using all exposure lines of the image sensor 10012a. That is, in the image sensor 10012a, unlike the image sensor 10010a, all exposure lines can be used for visible light imaging. As a result, the visible light captured image 10012c obtained by the image sensor 10012a includes a larger number of bright lines than the visible light captured image 10010c, as in the visible light captured image 10011c. Can be done with precision.

  Here, the interlaced display of the preview image will be described.

  FIG. 265 is a diagram illustrating an example of a screen display method of a receiver in Embodiment 12.

  The receiver including the image sensor 10010a shown in FIG. 262 described above is set to an exposure time set for an odd-numbered exposure line (hereinafter referred to as an odd-numbered line) and an even-numbered exposure line (hereinafter referred to as an even-numbered line). The exposure time is changed every predetermined time. For example, as shown in FIG. 265, at time t1, the receiver sets a long exposure time for each image sensor on the odd lines and sets a short exposure time on each image sensor on the even lines. Imaging is performed using the set exposure time. Further, at time t2, the receiver sets a short exposure time for each image sensor of the odd line, sets a long exposure time for each image sensor of the even line, and sets the set exposure time. The used imaging is performed. Then, at time t3, the receiver performs imaging using each exposure time set similarly to time t1, and uses each exposure time set similarly to time t2 at time t4. Take an image.

  Further, at time t1, the receiver captures an image obtained from each of a plurality of odd lines by imaging (hereinafter referred to as an odd line image) and an image obtained from each of a plurality of even lines by imaging (hereinafter referred to as an even line image). Image1 including the above. At this time, since the exposure time is short in each of the plurality of even lines, the subject is not clearly displayed in each of the even line images. Therefore, the receiver generates a plurality of interpolated line images by interpolating pixel values for the plurality of odd line images. Then, the receiver displays a preview image including a plurality of interpolation line images instead of the plurality of even line images. That is, odd-numbered line images and interpolated line images are alternately arranged in the preview image.

  At time t2, the receiver acquires Image2 including a plurality of odd line images and even line images by imaging. At this time, since the exposure time is short in each of the plurality of odd lines, the subject is not clearly displayed in each of the odd line images. Therefore, the receiver displays a preview image including the odd line image of Image1 instead of the odd line image of Image2. That is, the odd line image of Image1 and the even line image of Image2 are alternately arranged in the preview image.

  Further, at time t3, the receiver acquires Image3 including a plurality of odd line images and even line images by imaging. At this time, similarly to the time t1, since the exposure time is short in each of the plurality of even lines, the subject is not clearly displayed in each of the even line images. Therefore, the receiver displays a preview image including an even line image of Image2 instead of an even line image of Image3. That is, in the preview image, the even line image of Image2 and the odd line image of Image3 are alternately arranged. Then, at time t4, the receiver acquires Image4 including a plurality of odd line images and even line images by imaging. At this time, similarly to the time t2, since the exposure time is short in each of the plurality of odd lines, the subject is not clearly displayed in each of the odd line images. Therefore, the receiver displays a preview image including the odd line image of Image3 instead of the odd line image of Image4. That is, the odd line image of Image3 and the even line image of Image4 are alternately arranged in the preview image.

  In this way, the receiver performs so-called interlaced display that displays an image including even-numbered line images and odd-numbered line images that have different acquired timings.

  Such a receiver can display a fine preview image while performing visible light imaging. Note that the plurality of image sensors set with the same exposure time may be a plurality of image sensors arranged along the horizontal direction of the exposure line as in the image sensor 10010a, or the exposure line as in the image sensor 10011a. A plurality of image sensors arranged along a direction perpendicular to the image sensor may be used, or a plurality of image sensors arranged according to a checkered pattern like the image sensor 10012a. Further, the receiver may save the preview image as imaging data.

  Next, the spatial ratio between normal imaging and visible light imaging will be described.

  266 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG.

  In the image sensor 10014b provided in the receiver, a long exposure time or a short exposure time is set for each exposure line as in the image sensor 10010a described above. In this image sensor 10014b, the ratio of the number of image sensors for which a long exposure time is set to the number of image sensors for which a short exposure time is set is 1: 1. This ratio is a ratio between normal imaging and visible light imaging, and is hereinafter referred to as a spatial ratio.

  However, in the present embodiment, the space ratio is not necessarily 1: 1. For example, the receiver may include an image sensor 10014a. In this image sensor 10014a, the number of image sensors with a short exposure time is larger than that of images with a long exposure time, and the spatial ratio is 1: N (N> 1). The receiver may include an image sensor 10014c. In this image sensor 10014c, the image sensor with a short exposure time is smaller than the image sensor with a long exposure time, and the spatial ratio is N (N> 1): 1. In addition, instead of the image sensors 10014a to 10014c, the receiver sets the exposure time for each of the above-described vertical lines, and the image sensors 10015a to 10015c have a spatial ratio of 1: N, 1: 1, or N: 1, respectively. Any of these may be provided.

  In such image sensors 10014a and 10015a, since there are many image sensors with a short exposure time, the reception accuracy or reception speed of a visible light signal can be increased. On the other hand, since the image sensors 10014c and 10015c have many image sensors with a long exposure time, a fine preview image can be displayed.

  Further, the receiver may perform interlaced display as illustrated in FIG. 265 using the image sensors 10014a, 10014c, 10015a, and 10015c.

  Next, the time ratio between normal imaging and visible light imaging will be described.

  FIG. 267 is a diagram illustrating an example of a signal reception method in Embodiment 12.

  The receiver may switch the imaging mode between the normal imaging mode and the visible light imaging mode for each frame, as illustrated in FIG. The normal imaging mode is an imaging mode in which a long exposure time for normal imaging is set for all imaging elements of the image sensor of the receiver. The visible light imaging mode is an imaging mode in which a short exposure time for visible light imaging is set for all imaging elements of the image sensor of the receiver. In this way, by switching between long and short exposure times, a preview image can be displayed by imaging with a long exposure time while receiving a visible light signal by imaging with a short exposure time.

  Note that when the long exposure time is determined by automatic exposure, the receiver ignores the image obtained by imaging with a short exposure time, and only determines the brightness of the image obtained by imaging with a long exposure time. Automatic exposure may be performed with reference. Thereby, a long exposure time can be determined as an appropriate time.

  Further, as illustrated in FIG. 267 (b), the receiver may switch the imaging mode between the normal imaging mode and the visible light imaging mode for each set of a plurality of frames. When it takes time to switch the exposure time or when it takes time to stabilize the exposure time, as shown in FIG. 267 (b), by changing the exposure time for each set of a plurality of frames, Visible light imaging (reception of visible light signals) and normal imaging can be made compatible. Further, since the number of exposure time switching is reduced as the number of frames included in the set is increased, power consumption and heat generation in the receiver can be suppressed.

  Here, the number of at least one frame continuously generated by imaging with a long exposure time in the normal imaging mode and the at least one frame continuously generated by imaging with a short exposure time in the visible light imaging mode The ratio (hereinafter referred to as the time ratio) with the number of s is not necessarily 1: 1. That is, in the cases shown in FIGS. 267 (a) and (b), the time ratio is 1: 1, but the time ratio may not be 1: 1.

  For example, as illustrated in (c) of FIG. 267, the receiver may increase the number of frames in the visible light imaging mode than the frames in the normal imaging mode. Thereby, the receiving speed of the visible light signal can be increased. If the frame rate of the preview image is equal to or higher than a predetermined rate, the difference in the preview image depending on the frame rate is not recognized by human eyes. When the imaging frame rate is sufficiently high, for example, when the frame rate is 120 fps, the receiver sets the visible light imaging mode for three consecutive frames, and then the visible light imaging for one frame. Set the mode. Thereby, the receiver can receive a visible light signal at high speed while displaying a preview image at a frame rate of 30 fps, which is sufficiently higher than the above-described predetermined rate. Further, since the number of times of switching is reduced, the effect described with reference to FIG.

  Further, as illustrated in (d) of FIG. 267, the receiver may increase the number of frames in the normal imaging mode than the frames in the visible light imaging mode. Thus, the preview image can be smoothly displayed by increasing the number of frames in the normal imaging mode, that is, the frames obtained by imaging with a long exposure time. In addition, since the number of times of receiving the visible light signal is reduced, there is a power saving effect. Further, since the number of times of switching is reduced, the effect described with reference to FIG.

  Further, as shown in FIG. 267 (e), the receiver first switches the imaging mode for each frame as in the case shown in FIG. 267 (a), and then receives a visible light signal. When completed, the number of frames in the normal imaging mode may be increased as in the case shown in FIG. Thereby, after the reception of the visible light signal is completed, the search for a new visible light signal can be continued while the preview image is displayed smoothly. Further, since the number of times of switching is reduced, the effect described with reference to FIG.

  FIG. 268 is a flowchart illustrating an example of a signal reception method in Embodiment 12.

  The receiver starts visible light reception, which is a process of receiving a visible light signal (step S10017a), and sets the exposure time length setting ratio to a value designated by the user (step S10017b). The exposure time length short / high setting ratio is at least one of the above-described space ratio and time ratio. The user may specify only the space ratio, only the time ratio, or both the space ratio and the time ratio, or the receiver may automatically set regardless of the user's specification.

  Next, the receiver determines whether or not the reception performance is equal to or less than a predetermined value (step S10017c). If it is determined that the value is equal to or less than the predetermined value (Y in step S10017c), the receiver sets a high ratio of the short exposure time (step S10017d). Thereby, reception performance can be improved. Note that the ratio of the short exposure time is the ratio of the number of image sensors set with a short exposure time to the number of image sensors set with a long exposure time in the case of a spatial ratio. It is a ratio of the number of frames generated continuously in the visible light imaging mode to the number of frames generated continuously in the mode.

  Next, the receiver receives at least a part of the visible light signal and determines whether or not a priority is set for at least a part of the received visible light signal (hereinafter referred to as a received signal) ( Step S10017e). When priority is set, an identifier indicating the priority is included in the received signal. When the receiver determines that the priority is set (Y in step S10017e), the receiver sets the exposure time length / short ratio according to the priority (step S10017f). That is, if the priority is high, the receiver sets the ratio of the short exposure time high. For example, an emergency light configured as a transmitter emits an identifier indicating a high priority when the luminance changes. In this case, the receiver can increase the reception speed by increasing the ratio of the short exposure time, and can promptly display the evacuation route and the like.

  Next, the receiver determines whether or not reception of all visible light signals has been completed (step S10017g). Here, when it is determined that it has not been completed (N in step S10017g), the receiver repeatedly executes the processing from step S10017c. On the other hand, when it is determined that the process is completed (Y in step S10017g), the receiver sets the ratio of the long exposure time to a high value and shifts to the power saving mode (step S10017h). Note that the ratio of the long exposure time is the ratio of the number of image sensors set with a long exposure time to the number of image sensors set with a short exposure time in the case of a spatial ratio. This is the ratio of the number of frames generated continuously in the normal imaging mode to the number of frames generated continuously in the imaging mode. As a result, the preview image can be displayed smoothly without receiving unnecessary visible light.

  Next, the receiver determines whether another visible light signal has been found (step S10017i). Here, when it is determined that it has been found (Y in step S10017i), the receiver repeatedly executes the processing from step S10017b.

  Next, simultaneous execution of visible light imaging and normal imaging will be described.

  FIG. 269 is a diagram illustrating an example of a signal reception method in Embodiment 12.

  The receiver may set two or more exposure times for the image sensor. That is, as shown in FIG. 269 (a), each of the exposure lines included in the image sensor is continuously exposed for the longest exposure time among the two or more set exposure times. For each exposure line, the receiver reads out the imaging data obtained by exposure of the exposure line when the above-described two or more set exposure times have elapsed. Here, the receiver does not reset the read image data until the longest exposure time has elapsed. Therefore, the receiver can obtain image data for a plurality of exposure times only by exposure with the longest exposure time by recording the accumulated value of the read image data. Note that the image sensor may or may not record the cumulative value of the imaging data. When the image sensor is not used, the components of the receiver that read data from the image sensor perform accumulation calculation, that is, recording the accumulated value of the imaging data.

  For example, when two exposure times are set, as shown in FIG. 269 (a), the receiver captures visible light imaging data including a visible light signal generated by exposure with a short exposure time. Read out, followed by normal imaging data generated by exposure with a long exposure time.

  Thereby, visible light imaging that is imaging for receiving a visible light signal and normal imaging can be performed simultaneously, and normal imaging can be performed while receiving a visible light signal. Further, by using data of a plurality of exposure times, a signal frequency higher than the sampling theorem can be recognized, and a high-frequency signal or a high-density modulation signal can be received.

  Further, when outputting the imaging data, the receiver outputs a data string including the imaging data as an imaging data body, as shown in FIG. 269 (b). That is, the receiver has an imaging mode identifier indicating an imaging mode (visible light imaging or normal imaging), an imaging element identifier for specifying an imaging element or an exposure line to which the imaging element belongs, and an exposure number of an imaging data body. By adding additional information including an imaging data number indicating whether it is time-based imaging data and an imaging data length indicating the size of the imaging data body to the imaging data body, the above-described data string is generated and output. . In the method for reading image data described with reference to FIG. 269 (a), the respective image data is not necessarily output in the order of exposure lines. Therefore, by adding the additional information shown in FIG. 269 (b), it is possible to specify which exposure line the imaging data is.

  FIG. 270A is a flowchart showing processing of a reception program in the twelfth embodiment.

  This reception program is a program that causes a computer provided in the receiver to execute, for example, the processes illustrated in FIGS. 262 to 269.

  That is, this reception program is a reception program for receiving information from a light emitter that changes in luminance. Specifically, this reception program causes the computer to execute step SA31, step SA32, and step SA33. In step SA31, a first exposure time is set for some of the K image sensors (K is an integer of 4 or more) included in the image sensor, and the K image sensors are set. A second exposure time shorter than the first exposure time is set for the remaining plurality of image sensors. In step SA32, the image sensor captures the subject, which is a light-emitting body that changes in luminance, with the set first and second exposure times, and outputs from the plurality of image sensors having the first exposure time set. And acquiring bright lines corresponding to the plurality of exposure lines included in the image sensor, the images corresponding to the outputs from the plurality of imaging elements set with the second exposure time. A bright line image that is an image to be included is acquired. In step SA33, information is acquired by decoding a plurality of bright line patterns included in the acquired bright line image.

  Thereby, since imaging is performed by the plurality of imaging elements in which the first exposure time is set and the plurality of imaging elements in which the second exposure time is set, a normal image can be obtained by one imaging by the image sensor. And bright line images can be acquired. That is, it is possible to simultaneously capture a normal image and acquire information by visible light communication.

  In the exposure time setting step SA31, the first exposure time is set for a part of a plurality of image sensor rows in the L (L is an integer of 4 or more) image sensor rows included in the image sensor. Then, the second exposure time is set for the remaining plurality of image sensor rows in the L image sensor rows. Here, each of the L image sensor rows is composed of a plurality of image sensors arranged in a row included in the image sensor.

  Accordingly, it is possible to set the exposure time for each image pickup device array which is a large unit without individually setting the exposure time for each image pickup device which is a small unit, and to reduce the processing load. .

  For example, each of the L imaging element rows is an exposure line included in the image sensor as shown in FIG. Alternatively, as shown in FIG. 263, each of the L image pickup device arrays includes a plurality of image pickup devices arranged along a direction perpendicular to an exposure line included in the image sensor.

  Further, as shown in FIG. 265, in the exposure time setting step SA31, the same exposure time is used for each of the odd-numbered image sensor rows of the L image sensor rows included in the image sensor. One of the first exposure time and the second exposure time is set, and the first exposure time and the second exposure time are the same exposure time for each of even-numbered image sensor rows of the L image sensor rows. You may set the other of time. When the exposure time setting step SA31, the image acquisition step SA32, and the information acquisition step SA33 are repeated, the repeated exposure time setting step SA31 is set for each of the odd-numbered image sensor rows in the previous exposure time setting step SA31. The exposure time that has been set may be interchanged with the exposure time that has been set for each even-numbered imaging element array.

  As a result, each time a normal image is acquired, the plurality of image sensor arrays used for the acquisition can be switched between an odd number of image sensor arrays and an even number of image sensor arrays. As a result, each of the sequentially acquired normal images can be displayed by interlace. Further, by complementing two consecutively acquired normal images with each other, a new normal image including an image by an odd-numbered plurality of imaging element arrays and an image by an even-numbered plurality of imaging element arrays is generated. be able to.

  As shown in FIG. 266, in the exposure time setting step SA31, the first exposure time is set when the setting mode is switched between the normal priority mode and the visible light priority mode and can be switched to the normal priority mode. The number of image sensors may be larger than the number of image sensors for which the second exposure time is set. In addition, when the mode is switched to the visible light priority mode, the number of image sensors for which the first exposure time is set may be smaller than the number of image sensors for which the second exposure time is set.

  As a result, when the setting mode is switched to the normal priority mode, the image quality of the normal image can be improved. When the setting mode is switched to the visible light priority mode, the reception efficiency of information from the light emitter is improved. can do.

Further, as shown in FIG. 264, in the exposure time setting step SA31, a plurality of image sensors for which the first exposure time is set and a plurality of image sensors for which the second exposure time is set are in a checkered pattern ( For each image sensor included in the image sensor, the exposure time of the image sensor may be set so as to be distributed as in Checkered pattern.

  As a result, the plurality of image pickup devices for which the first exposure time is set and the plurality of image pickup devices for which the second exposure time is set are uniformly distributed. Not normal images and bright line images can be acquired.

  FIG. 270B is a block diagram of a receiving apparatus in Embodiment 12.

  The receiving device A30 is the above-described receiver that executes, for example, the processes shown in FIGS.

  That is, the receiving device A30 is a receiving device that receives information from a light-emitting body that changes in luminance, and includes a multiple exposure time setting unit A31, an imaging unit A32, and a decoding unit A33. The multiple exposure time setting unit A31 sets the first exposure time for some of the plurality of image pickup elements (K is an integer of 4 or more) included in the image sensor, and K pieces. A second exposure time shorter than the first exposure time is set for the remaining plurality of image sensors. The imaging unit A32 causes the image sensor to capture an image of a subject, which is a light-emitting body that changes in luminance, with the set first and second exposure times, so that a plurality of imaging elements with the first exposure time are set. A bright image corresponding to each of the plurality of exposure lines included in the image sensor, which is an image corresponding to the output from the plurality of imaging elements for which the second exposure time is set, while acquiring a normal image according to the output A bright line image that is an image including The decoding unit A33 acquires information by decoding a plurality of bright line patterns included in the acquired bright line image. Such a receiving apparatus A30 can achieve the same effects as the above-described receiving program.

  Next, the display of the content regarding the received visible light signal is demonstrated.

  FIGS. 271 and 272 are diagrams illustrating an example of display on the receiver when a visible light signal is received.

  As illustrated in FIG. 271 (a), when the receiver captures an image of the transmitter 10020d, the receiver displays an image 10020a on which the transmitter 10020d is projected. Further, the receiver generates and displays an image 10020b by superimposing the object 10020e on the image 10020a. The object 10020e is an image indicating that the image of the transmitter 10020d is present and that a visible light signal is received from the transmitter 10020d. The object 10020e may be an image that varies depending on the reception state of the visible light signal (the state of reception, the state of searching for a transmitter, the degree of progress of reception, the reception speed, or the error rate). For example, the receiver changes the color of the object 1020e, the thickness of the line, the type of line (single line, double line, dotted line, or the like), or the interval between dotted lines. Thereby, a user can be made to recognize a receiving state. Next, the receiver generates and displays an image 10020c by superimposing an image indicating the content of the acquired data on the image 10020a as an acquired data image 10020f. The acquired data is data associated with the received visible light signal or the ID indicated by the received visible light signal.

  When the receiver displays the acquired data image 10020f, as shown in FIG. 271 (a), the receiver displays the acquired data image 10020f like a balloon from the transmitter 10020d or near the transmitter 10020d. The acquired data image 10020f is displayed. Further, as shown in FIG. 271 (b), the receiver may display the acquired data image 10020f so that the acquired data image 10020f gradually approaches the receiver side from the transmitter 10020d. Thereby, the user can recognize which transmitter the received data image 10020f is based on the visible light signal received from. Further, as shown in FIG. 272, the receiver may display the acquired data image 10020f so that the acquired data image 10020f gradually emerges from the end of the receiver display. This makes it possible for the user to easily recognize that the visible light signal has been acquired at that time.

  Next, AR (Augmented Reality) will be described.

  FIG. 273 is a diagram illustrating an example of display of the acquired data image 10020f.

  When the image of the transmitter moves in the display, the receiver moves the acquired data image 10020f in accordance with the movement of the image of the transmitter. This allows the user to recognize that the acquired data image 10020f corresponds to the transmitter. The receiver may display the acquired data image 10020f in association with another image instead of the image of the transmitter. Thereby, AR display can be performed.

  Next, saving and discarding of acquired data will be described.

  FIG. 274 is a diagram illustrating an example of an operation for saving or discarding acquired data.

  For example, as shown in FIG. 274 (a), when the user performs a swipe down on the acquired data image 10020f, the receiver stores the acquired data indicated by the acquired data image 10020f. . The receiver places an acquired data image 10020f indicating the stored acquired data at the end of the acquired data image indicating one or more other already stored acquired data. This allows the user to recognize that the acquisition data indicated by the acquisition data image 10020f is the acquisition data stored last. For example, as shown in FIG. 274 (a), the receiver arranges the acquired data image 10020f in the forefront among the plurality of acquired data images.

  In addition, as illustrated in (b) of FIG. 274, when the user performs a right swipe on the acquired data image 10020f, the receiver discards the acquired data indicated by the acquired data image 10020f. Alternatively, the receiver may discard the acquired data indicated by the acquired data image 10020f when the image of the transmitter is framed out of the display by the user moving the receiver. Note that the same effect as described above can be obtained regardless of whether the swipe direction is up, down, left, or right. The receiver may display the swipe direction corresponding to the save or discard. As a result, the user can recognize that the data can be saved or discarded by the operation.

  Next, browsing of acquired data will be described.

  FIG. 275 is a diagram illustrating a display example when browsing acquired data.

  As shown in FIG. 275 (a), the receiver displays the acquired data images of a plurality of acquired data in a small manner overlapping the lower end of the display. At this time, when the user taps a part of the displayed acquired data image, the receiver displays each of the plurality of acquired data images in a large size as shown in FIG. 275 (b). Thereby, only when it is necessary to view each piece of acquired data, those acquired data images are displayed in a large size, and when not necessary, the display can be used effectively for other displays.

  When the user taps the acquired data image that the user wants to display in the state shown in FIG. 275 (b), the receiver displays the tapped acquired data image in a larger size as shown in FIG. 275 (c). A lot of information is displayed in the acquired data image. When the user taps the back surface display button 10024a, the receiver displays the back surface of the acquired data image, and displays other data related to the acquired data.

  Next, turning off camera shake correction at the time of accident position estimation will be described.

  The receiver disables camera shake correction (off), or converts the captured image according to the correction direction and correction amount of camera shake correction, thereby acquiring the correct imaging direction and accurately performing self-position estimation. Can be done. The captured image is an image obtained by imaging by the imaging unit of the receiver. Self-position estimation means that the receiver estimates its own position. In the self-position estimation, specifically, the receiver specifies the position of the transmitter based on the received visible light signal, and receives based on the size, position, or shape of the transmitter reflected in the captured image. Identify the relative positional relationship between the transmitter and transmitter. Then, the receiver estimates the position of the receiver based on the position of the transmitter and the relative positional relationship between the receiver and the transmitter.

  In addition, when partial reading is performed using only a part of the exposure lines shown in FIG. 262, that is, when the imaging shown in FIG. 262 is performed, the transmitter is out of frame with a slight blurring of the receiver. End up. In such a case, the receiver can continuously receive the signal by enabling the camera shake correction.

  Next, self-position estimation using an asymmetrical light emitting unit will be described.

  FIG. 276 is a diagram illustrating an example of a transmitter in Embodiment 12.

  The above-described transmitter includes a light emitting unit, and transmits a visible light signal by changing the luminance of the light emitting unit. In the above-described self-position estimation, the receiver determines the relative positional relationship between the receiver and the transmitter based on the shape of the transmitter (specifically, the light emitting unit) in the captured image. Find the relative angle to the transmitter. Here, for example, as shown in FIG. 276, when the transmitter includes a light-emitting unit 10090a having a rotationally symmetric shape, as described above, based on the shape of the transmitter in the captured image, The relative angle with the receiver cannot be determined accurately. Therefore, it is desirable that the transmitter includes a light emitting unit having a shape that is not rotationally symmetric. Thereby, the receiver can obtain | require correctly the above-mentioned relative angle. That is, since the error of the measurement result is large in the azimuth sensor for acquiring the angle, the receiver can perform accurate self-position estimation by using the relative angle obtained by the above method.

  Here, as shown in FIG. 276, the transmitter may include a light emitting unit 10090b that is not in a completely rotationally symmetric shape. The shape of the light emitting unit 10090b is symmetric with respect to 90 ° rotation, but is not completely rotationally symmetric. In this case, the receiver obtains a rough angle with the azimuth sensor, and further uses the shape of the transmitter in the captured image to uniquely limit the relative angle between the receiver and the transmitter. And accurate self-position estimation can be performed.

  Further, the transmitter may include a light emitting unit 10090c shown in FIG. The shape of the light emitting unit 10090c is basically a rotationally symmetric shape. However, since a light guide plate or the like is provided in a part of the light emitting unit 10090c, the shape of the light emitting unit 10090c is not rotationally symmetric.

  Further, the transmitter may include a light emitting unit 10090d shown in FIG. Each of the light emitting units 10090d includes rotationally symmetric illumination. However, the overall shape of the light emitting unit 10090d configured by combining them is not a rotationally symmetric shape. Therefore, the receiver can perform accurate self-position estimation by imaging the transmitter. In addition, it is not necessary for all illumination included in the light emitting unit 10090d to be illumination for visible light communication that changes in luminance in order to transmit a visible light signal, and only some illumination is illumination for visible light communication. May be.

  Further, the transmitter may include a light emitting unit 10090e and an object 10090f illustrated in FIG. Here, the object 10090f is an object (for example, a fire alarm or a pipe) configured so that the positional relationship with the light emitting unit 10090e does not change. Since the shape of the combination of the light emitting unit 10090e and the object 10090f is not a rotationally symmetric shape, the receiver can accurately perform self-position estimation by imaging the light emitting unit 10090e and the object 10090f.

  Next, a time series process for self-position estimation will be described.

  Each time the receiver captures an image, the receiver can perform self-position estimation from the position and shape of the transmitter in the captured image. As a result, the receiver can estimate the moving direction and distance of the receiver being imaged. Further, the receiver can perform more accurate self-position estimation by performing triangulation using a plurality of frames or images. By integrating estimation results using a plurality of images and estimation results using a plurality of different combinations of images, the receiver can perform self-position estimation more accurately. At this time, the receiver can perform self-position estimation more accurately by emphasizing and summing up the results estimated from recent captured images.

  Next, skipping of optical black will be described.

  FIG. 277 is a diagram illustrating an example of a reception method in Embodiment 12. Note that the horizontal axis of the graph shown in FIG. 277 indicates time, and the vertical axis indicates the position of each exposure line in the image sensor. Furthermore, a solid line arrow in the graph indicates a time (exposure timing) at which exposure of each exposure line in the image sensor is started.

  During normal imaging, the receiver reads the horizontal optical black signal in the image sensor as shown in FIG. 277 (a), but skips the horizontal optical black signal as shown in FIG. 277 (b). May be. Thereby, a continuous visible light signal can be received.

  Horizontal optical black is optical black in the horizontal direction on the exposure line. The vertical optical black is a portion of the optical black other than the horizontal optical black.

  Since the receiver adjusts the black level based on the signal read from the optical black, the black level can be adjusted using the optical black at the start of the visible light imaging similarly to the normal imaging. When vertical optical black can be used, the receiver can perform continuous reception and black level adjustment by performing black level adjustment using only vertical optical black. When visible light imaging continues, the receiver may adjust the black level using horizontal optical black every predetermined time. In the case where the normal imaging and the visible light imaging are alternately performed, the receiver skips the horizontal optical black signal when continuously performing the visible light imaging, and reads the horizontal optical black signal otherwise. Then, the receiver can adjust the black level while continuously receiving the visible light signal by adjusting the black level based on the read signal. The receiver may adjust the black level by setting the darkest part of the visible light captured image as black.

  As described above, the optical black from which the signal is read out is only vertical optical black, so that continuous visible light signals can be received. In addition, by providing a mode for skipping horizontal optical black signals, black level adjustment can be performed during normal imaging, and continuous communication can be performed as necessary during visible light imaging. In addition, by skipping the horizontal optical black signal, the difference in timing of starting exposure between exposure lines is increased, so that a visible light signal from a transmitter that is only small can be received.

  Next, an identifier indicating the type of transmitter will be described.

  The transmitter may transmit the transmitter identifier indicating the type of transmitter added to the visible light signal. In this case, when the receiver receives the transmitter identifier, the receiver can perform a receiving operation according to the type of the transmitter. For example, in the case where the transmitter identifier indicates digital signage, the transmitter displays a content ID indicating which content is currently displayed in addition to the transmitter ID for performing individual identification of the transmitter as a visible light signal. As sending. The receiver can display information according to the content currently displayed by the transmitter by separately handling these IDs based on the transmitter identifier. For example, when the transmitter identifier indicates digital signage or an emergency light, the receiver can reduce reception errors by imaging with increased sensitivity.

(Embodiment 13)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  First, the header pattern in the present embodiment will be described.

  FIG. 278 is a diagram illustrating an example of a header pattern in the present embodiment.

  The transmitter divides data to be transmitted into packets and transmits the packets. The packet is composed of a header and a body, for example. The luminance change pattern of the header, that is, the header pattern needs to be a luminance change pattern that does not exist in the body. Thereby, the position of the packet can be specified in the data transmitted continuously.

  For example, data to be transmitted is modulated by a 4PPM modulation scheme. Specifically, in the 4PPM, data to be transmitted is modulated into a luminance change pattern in which one of the four slots indicates “0” and the remaining three slots indicate “1”. Therefore, when continuously transmitted data is modulated, the number of consecutive slots indicating “0” is 2 or less, and the number of slots indicating “0” among the 4 slots and the next 4 slots. Is 2 or less.

  Therefore, in the present embodiment, the header pattern is “111111111000” shown in (a) of FIG. 278, “111111110001” shown in (b), “111111101001” shown in (c), or “111111100101” shown in (d). ". Thereby, a header and the data (namely, body) transmitted continuously can be identified. In other words, in the header pattern shown in FIG. 278 (a), if only the four slots “1000” after the header pattern are viewed, it can be identified that the four slots are a part of the header. In this case, since the number of slots indicating “0” in the header is 3, and the maximum number of consecutive slots indicating “0” is 4, the receiver can easily recognize the luminance change. That is, the receiver can easily receive data from a small transmitter or a remote transmitter.

  Also, in the header pattern shown in FIG. 278 (b), if only 5 slots “10001” after the header pattern are seen, it can be identified that the 5 slots are a part of the header. In this case, since the maximum number of consecutive slots indicating “0” is 3, which is smaller than that in the case of FIG. 278 (a), flicker due to a change in luminance can be suppressed. As a result, the burden on the circuit of the transmitter or the design requirement can be suppressed. That is, the capacitor can be reduced, and power consumption, heat generation, or a load on the power source can be suppressed.

  Also, in the header pattern shown in (c) or (d) of FIG. 278, if only the six slots “101001” or “100101” after the header pattern are seen, the six slots are identified as part of the header. can do. In this case, the maximum number of consecutive slots indicating “0” is 2, which is smaller than that in the case of FIG. 278 (b), and therefore flicker due to the change in luminance can be further suppressed.

  FIG. 279 is a diagram for describing an example of a packet configuration of a communication protocol in this embodiment.

  The transmitter divides data to be transmitted into packets and transmits the packets. The packet includes a header, an address part, a data part, and an error correction code part. By setting the luminance change pattern of the header to a luminance change pattern that does not exist in other portions, the position of the packet in the continuous data can be specified. A part of the divided data is stored in the data part, and an address indicating which part of the data of the data part is stored in the address part. The error correction code unit is a code for detecting or correcting a reception error of a part or all of a packet (specifically, ECC1, ECC2 or ECC3 shown in FIG. 279, which are collectively referred to as an error correction code). ) Is stored.

  ECC1 is an error correction code of the address part. By providing an error correction code for the address portion instead of an error correction code for the entire packet, the reliability of the address portion can be made higher than the overall reliability. Thereby, when a plurality of packets are received, the data portion can be verified by comparing the data portions of the packets having the same address, and the reception error rate can be reduced. Even if the error correction code in the address part is made longer than the error correction code in the data part, the same effect can be obtained.

  ECC2 and ECC3 are error correction codes of the data part. The number of error correction code units may be other than two. By dividing the error correction code of the data part into multiple parts, error correction can be performed using the error correction codes that have been received, and reliable data can be received without receiving the entire packet. Can do.

  The transmitter may transmit a number of error correction codes equal to or less than a specified number. Thereby, the receiver can receive data at high speed. This transmission method is effective for a transmitter having a small light emitting unit and high luminance, such as a downlight. This is because when the luminance is high, the error occurrence probability is low, so that many error correction codes are not necessary. When ECC3 is not sent, since the next header is transmitted from ECC2, the state where the luminance is high continues for four slots or more, and the receiver can recognize that the part is not ECC3.

  As shown in (b) of FIG. 279, the header, the address part, and ECC1 are transmitted at a lower frequency than the data part, ECC2, and ECC3. In other words, the data part, ECC2 and ECC3 are transmitted at a higher frequency than the header, address part and ECC1. As a result, it is possible to reduce the reception error rate for data such as a header, and to transmit large data in the data portion quickly.

  As described above, in the present embodiment, the packet includes the first error correction code (ECC2, ECC3) for the data part and the second error correction code (ECC1) for the address part. Then, when receiving the packet, the receiver receives the address part and the second error correction code transmitted by the luminance change according to the second frequency from the transmitter. Further, the receiver receives from the transmitter the data part and the first error correction code transmitted by the luminance change according to the first frequency higher than the second frequency.

  Here, a receiving method for comparing data portions having the same address will be described.

  FIG. 280 is a flowchart illustrating an example of a reception method in this embodiment.

  The receiver receives the packet (step S10101) and performs error correction (step S10102). Then, the receiver determines whether or not a packet having the same address as that of the received packet has already been received (step S10103). If it is determined that the data is received (Y in Step 10103), the receiver compares the data. That is, the receiver determines whether the data parts are equal (step S10104). Here, when it is determined that they are not equal (N in step S10104), the receiver further determines whether the difference in the plurality of data parts is equal to or greater than a predetermined number, specifically, the number of different bits or the luminance. It is determined whether or not the number of slots having different states is equal to or greater than a predetermined number (step S10105). If it is determined that the number is equal to or greater than the predetermined number (N in step S10105), the receiver discards the packet that has already been received (step S10106). Thereby, when a packet is started to be received from another transmitter, interference with a packet received from a previous transmitter can be avoided. On the other hand, if it is determined that the number is not equal to or greater than the predetermined number (N in step S10105), the receiver sets the data of the data part with the most packets having the same data part as the data of the address (step S10107). Alternatively, the receiver takes the most equal bit as the value of that bit at that address. Alternatively, the receiver sets the luminance state having the largest number of equal luminance states as the luminance state of the slot at the address, and demodulates the data at the address.

  Thus, in the present embodiment, the receiver first acquires the first packet including the data portion and the address portion from the plurality of bright line patterns. Next, the receiver has at least one second packet which is a packet including the same address part as the address part of the first packet among at least one packet already acquired before the first packet. It is determined whether or not there is a packet. Next, when the receiver determines that the at least one second packet exists, the receiver determines whether the data parts of the at least one second packet and the first packet are all equal. judge. If it is determined that the respective data parts are not all equal, the receiver, in each of the at least one second packet, out of the parts included in the data part of the second packet, the first packet It is determined whether the number of parts different from each part included in the data part is greater than or equal to a predetermined number. Here, when there is a second packet in which the number of different portions is determined to be greater than or equal to a predetermined number among the at least one second packet, the receiver includes the at least one second packet. Is discarded. On the other hand, if there is no second packet in which the number of different portions of the at least one second packet is determined to be greater than or equal to a predetermined number, the receiver may use the first packet and at least one second packet. Among these packets, a plurality of packets having the largest number of packets having the same data part are specified. The receiver then decodes at least a part of the visible light identifier (ID) by decoding the data part included in each of the plurality of packets as a data part corresponding to the address part included in the first packet. get.

  Thereby, when a plurality of packets having the same address part are received, even if the data part of those packets is different, the appropriate data part can be decoded, and at least a part of the visible light identifier is You can get it correctly. That is, a plurality of packets having the same address part transmitted from the same transmitter basically have the same data part. However, when the receiver switches the transmitter that is the transmission source of the packet, the receiver may receive a plurality of packets having different data parts even though they have the same address part. In such a case, in the present embodiment, as in step S10106 in FIG. 280, the already received packet (second packet) is discarded, and the data portion of the latest packet (first packet). Can be decoded as a correct data portion corresponding to the address portion. Furthermore, even when there is no transmitter switching as described above, the data portions of a plurality of packets having the same address portion may be slightly different depending on the transmission / reception state of the visible light signal. In such a case, in the present embodiment, an appropriate data part can be decoded by so-called majority decision as in step S10107 of FIG.

  Here, a reception method for demodulating data in the data portion from a plurality of packets will be described.

  FIG. 281 is a flowchart illustrating an example of a reception method in this embodiment.

  First, the receiver receives the packet (step S10111) and performs error correction of the address part (step S10112). At this time, the receiver does not demodulate the data part and holds the pixel value obtained by imaging as it is. Then, the receiver determines whether or not there are a predetermined number or more of packets having the same address among the plurality of packets that have already been received (step S10113). If it is determined that the packet exists (Y in step S10113), the receiver performs demodulation processing by combining pixel values of portions corresponding to data portions of a plurality of packets having the same address (step S10114).

  As described above, in the reception method according to the present embodiment, the first packet including the data portion and the address portion is acquired from the plurality of bright line patterns. Then, among at least one packet that has already been acquired before the first packet, there are a predetermined number or more of second packets that include the same address part as the address part of the first packet. It is determined whether or not. If it is determined that there are more than the predetermined number of second packets, the pixel values of the partial areas of the bright line image corresponding to the respective data portions of the second packet more than the predetermined number, The pixel values of a part of the bright line image corresponding to the data portion of the packet are matched. That is, the pixel values are added. By the addition, a composite pixel value is calculated, and at least a part of the visible light identifier (ID) is obtained by decoding the data portion including the composite pixel value.

  Since the timings at which the plurality of packets are received are different, the pixel values in the data portion are values that reflect the luminance of the transmitter at slightly different times. Therefore, the portion to be demodulated as described above includes a larger amount of data (number of samples) than the data portion of a single packet. Thereby, a data part can be demodulated more correctly. Further, by increasing the number of samples, a signal modulated with a higher modulation frequency can be demodulated.

  As shown in FIG. 279 (b), the data part and its error correction code part are modulated at a higher frequency than the header part, the address part, and the error correction code part of the address part. By the above demodulation method, since the data portion and the subsequent part can be demodulated even if modulated at a high modulation frequency, the transmission time of the whole packet can be shortened by this configuration, and from a smaller light source even from a longer distance. However, a visible light signal can be received faster.

  Next, a reception method for receiving variable length address data will be described.

  FIG. 282 is a flowchart illustrating an example of a reception method in this embodiment.

  The receiver receives the packet (step S10121), and determines whether or not a packet in which all the bits of the data part are 0 (hereinafter referred to as a 0 termination packet) is received (step S10122). Here, if it is determined that it has been received, that is, if it is determined that there is a zero-termination packet (Y in step S10122), the receiver determines whether or not all packets having addresses below the zero-termination packet address are available, that is, It is determined whether or not it has been received (step S10123). Note that the address is set to a value that increases in accordance with the order of transmission of each packet generated by dividing the transmitted data. If it is determined that the receiver is complete (Y in step S10123), the receiver determines that the address of the 0-termination packet is the last address of the packet transmitted from the transmitter. Then, the receiver restores the data by connecting the data of the packets of each address up to the 0 terminal packet (step S10124). Further, the receiver performs an error check on the restored data (step S10125). This makes it possible to send and receive variable-length address data even when the data to be transmitted is not divided into several parts, that is, when the address is not fixed length but variable length. More IDs can be transmitted and received with high efficiency.

  Thus, in the present embodiment, the receiver acquires a plurality of packets each including a data part and an address part from a plurality of bright line patterns. Then, the receiver determines whether or not there is a 0-termination packet that is a packet in which all bits included in the data part indicate 0 among the plurality of acquired packets. If it is determined that there is a zero-termination packet, the receiver, among a plurality of packets, N packets (N is an integer equal to or greater than one) including an address part associated with the address part of the zero-termination packet. It is determined whether or not all related packets are present. Next, when the receiver determines that all the N related packets exist, the receiver acquires a visible light identifier (ID) by arranging and decoding the data portions of the N related packets. Here, the address part associated with the address part of the 0-termination packet is an address part that indicates an address that is smaller than the address shown in the address part of the 0-termination packet and is 0 or more.

  Next, a reception method using an exposure time longer than the modulation frequency period will be described.

  283 and 284 are diagrams for describing a reception method in which the receiver according to the present embodiment uses an exposure time longer than the period of the modulation frequency (modulation period).

  For example, as shown in FIG. 283 (a), if the exposure time is set to a time equal to the modulation period, the visible light signal may not be received correctly. The modulation period is the time of one slot described above. That is, in such a case, there are few exposure lines (exposure lines indicated by black in FIG. 283) reflecting the luminance state of a certain slot. As a result, it is difficult to estimate the luminance of the transmitter when a lot of noise is accidentally included in the pixel values of these exposure lines.

  On the other hand, for example, as shown in FIG. 283 (b), when the exposure time is set to a time longer than the modulation period, a visible light signal can be correctly received. That is, in such a case, since there are many exposure lines reflecting the brightness of a certain slot, the brightness of the transmitter can be estimated from the pixel values of many exposure lines, and it is resistant to noise.

  On the other hand, if the exposure time is too long, the visible light signal cannot be received correctly.

  For example, as shown in FIG. 284 (a), when the exposure time is equal to the modulation period, the luminance change (that is, the change in the pixel value of each exposure line) received by the receiver is used for transmission. Follow changes in brightness. However, as shown in FIG. 284 (b), when the exposure time is three times the modulation period, the luminance change received by the receiver can sufficiently follow the luminance change used for transmission. Can not. Further, as shown in FIG. 284 (c), when the exposure time is 10 times the modulation period, the luminance change received by the receiver cannot follow the luminance change used for transmission at all. That is, the longer the exposure time, the higher the noise resistance because the luminance can be estimated from many exposure lines. However, the longer the exposure time, the lower the identification margin or the smaller the identification margin. Due to these balances, the noise resistance can be maximized by setting the exposure time to about 2 to 5 times the modulation period.

  Next, the number of packet divisions will be described.

  FIG. 285 is a diagram illustrating an efficient division number with respect to the size of transmission data.

  When a transmitter transmits data by a change in luminance, if all transmitted data (transmission data) is included in one packet, the data size of the packet is large. However, as described with reference to FIG. 279, when the transmission data is divided into a plurality of partial data and the partial data is included in each packet, the data size of each packet is reduced. Here, the receiver receives the packet by imaging. However, the larger the data size of the packet, the more difficult it is for the receiver to receive the packet by one imaging, and it is necessary to repeat imaging.

  Therefore, as shown in (a) and (b) of FIG. 285, it is desirable for the transmitter to increase the division number of the transmission data as the data size of the transmission data increases. However, if the number of divisions is too large, the transmission data cannot be restored unless all of the partial data is received.

  Therefore, as shown in FIG. 285 (a), the data size of the address (address size) is variable, and the data size of the transmission data is 2-16 bits, 16-24 bits, 24-64 bits, 66- In the case of 78 bits, 78-128 bits, 128 bits or more, transmission data into 1-2 pieces, 2-4 pieces, 4 pieces, 4-6 pieces, 6-8 pieces, 7 pieces or more, respectively. Is divided, transmission data can be efficiently transmitted by a visible light signal. As shown in FIG. 285 (b), the data size of the address (address size) is fixed to 4 bits, and the data size of the transmission data is 2-8 bits, 8-16 bits, 16-30 bits, If 30-64 bits, 66-80 bits, 80-96 bits, 96-132 bits, 132 bits or more, 1-2, 2-3, 2-4, 4-5, When the transmission data is divided into 4-7 pieces, 6 pieces, 6-8 pieces, and 7 or more partial data pieces, the transmission data can be efficiently transmitted by a visible light signal.

  Further, the transmitter sequentially changes the luminance based on each packet including each of the plurality of partial data. For example, the transmitter changes the luminance based on the packets in the order of the addresses of the packets. Further, the transmitter may perform the luminance change based on the plurality of partial data again in an order different from the address order. Thereby, each partial data can be reliably received by the receiver.

  Next, a notification operation setting method by the receiver will be described.

  FIG. 286A is a diagram illustrating an example of a setting method in this embodiment.

  First, the receiver acquires a notification operation identifier for identifying the notification operation and a priority of the notification operation identifier (specifically, an identifier indicating the priority) from a server near the receiver. (Step S10131). Here, in the notification operation, when each packet including each of a plurality of partial data is transmitted due to a change in luminance and received by the receiver, reception is performed to notify the user of the receiver that the packets have been received. The operation of the machine. For example, the operation is sounding, vibration, or screen display.

  Next, the receiver receives a packetized visible light signal, that is, each packet including each of a plurality of partial data (step S10132). Here, the receiver acquires the notification operation identifier and the priority of the notification operation identifier (specifically, an identifier indicating the priority) included in the visible light signal (step S10133).

  Further, the receiver sets the current notification operation setting of the receiver, that is, the notification operation identifier preset in the receiver and the priority of the notification operation identifier (specifically, an identifier indicating the priority). ) Is read out (step S10134). Note that the notification operation identifier set in advance in the receiver is set by a user operation, for example.

  Then, the receiver selects the identifier having the highest priority among the preset notification operation identifiers and the notification operation identifiers acquired in steps S10131 and S10133 (step S10135). Next, the receiver resets the selected notification operation identifier to itself, thereby performing the operation indicated by the selected notification operation identifier, and notifies the user of reception of the visible light signal (step S10136).

  Note that the receiver may select a notification operation identifier having a high priority from the two notification operation identifiers without performing any one of steps S10131 and S10133.

  Note that the priority of notification action identifiers transmitted from servers installed in theaters or museums, or the priority of notification action identifiers included in visible light signals transmitted within those facilities are set high. Also good. Thereby, it is possible to prevent the sound for the reception notification from being sounded in the facility regardless of the setting of the user. In other facilities, by setting the priority of the notification operation identifier low, the receiver can notify the reception by the operation according to the user's setting.

  FIG. 286B is a diagram illustrating another example of the setting method according to the present embodiment.

  First, the receiver acquires a notification operation identifier for identifying the notification operation and a priority of the notification operation identifier (specifically, an identifier indicating the priority) from a server near the receiver. (Step S10141). Next, the receiver receives a packetized visible light signal, that is, each packet including each of a plurality of partial data (step S10142). Here, the receiver acquires the notification operation identifier and the priority of the notification operation identifier (specifically, an identifier indicating the priority) included in the visible light signal (step S10143).

  Further, the receiver sets the current notification operation setting of the receiver, that is, the notification operation identifier preset in the receiver and the priority of the notification operation identifier (specifically, an identifier indicating the priority). ) Is read (step S10144).

  The receiver includes an operation notification identifier indicating an operation for prohibiting the generation of the notification sound in the notification operation identifier set in advance and the notification operation identifier acquired in each of steps S10141 and S10143. It is determined whether or not it is (step S10145). Here, if it is determined that it is included (Y in step S10145), the receiver sounds a notification sound for notifying completion of reception (step S10146). On the other hand, if it is determined that it is not included (N in step S10145), the receiver notifies the user of the completion of reception by, for example, vibration (step S10147).

  Note that the receiver does not perform either one of steps S10141 and S10143, and determines whether or not the two notification operation identifiers include an operation notification identifier indicating an operation that prohibits the generation of the notification sound. May be.

  In addition, the receiver may perform self-position estimation based on an image obtained by imaging, and notify the user of reception through an operation associated with the estimated position or a facility at the position.

  FIG. 287A is a flowchart illustrating processing of the information processing program in the thirteenth embodiment.

  This information processing program is a program for changing the luminance of the light emitter of the transmitter described above according to the number of divisions shown in FIG.

  That is, this information processing program is an information processing program that causes a computer to process information to be transmitted in order to transmit the information to be transmitted by a change in luminance. Specifically, the information processing program includes an encoding step SA41 that generates an encoded signal by encoding information to be transmitted, and the number of bits of the generated encoded signal is in a range of 24 to 64 bits. In some cases, the computer executes a division step SA42 for dividing the encoded signal into four partial signals and an output step SA43 for sequentially outputting the four partial signals. These partial signals are output as packets shown in FIG. 279 (a). The information processing program may cause the computer to specify the number of bits of the encoded signal and determine the number of partial signals based on the specified number of bits. In this case, the information processing program causes the computer to generate the determined number of partial signals by dividing the encoded signal.

  Thereby, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and output. As a result, when the luminance of the illuminant changes according to the four partial signals that are output, the four partial signals are transmitted as visible light signals and received by the receiver. Here, the larger the number of bits of the output signal, the more difficult it is for the receiver to properly receive the signal by imaging, and the reception efficiency decreases. Therefore, it is desirable to divide the signal into signals having a small number of bits, that is, small signals. However, if the signal is too finely divided into many small signals, the receiver cannot receive the original signal unless each small signal is individually received, resulting in a decrease in reception efficiency. Therefore, as described above, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and sequentially output to indicate the information to be transmitted. The encoded signal can be transmitted as a visible light signal with the best reception efficiency. As a result, communication between various devices can be enabled.

  In the output step SA43, the four partial signals may be output in accordance with the first order, and the four partial signals may be output again in accordance with a second order different from the first order.

  As a result, the four partial signals are repeatedly output in a different order. Therefore, when each output signal is transmitted to the receiver as a visible light signal, the reception efficiency of the four partial signals is increased. It can be further increased. That is, even if the four partial signals are repeatedly output in the same order, the same partial signal may not be received by the receiver. However, by changing the order, the occurrence of such a case can be suppressed.

  Also, as shown in FIGS. 286A and 286B, in the output step SA43, a notification operation identifier may be further attached to the four partial signals. The notification operation identifier is used to identify the operation of the receiver notifying the user of the receiver that the four partial signals are received when the four partial signals are transmitted by the luminance change and received by the receiver. Identifier.

  Thus, when the notification operation identifier is transmitted as a visible light signal and received by the receiver, the receiver receives the four partial signals to the user according to the operation identified by the notification operation identifier. You can be notified. That is, the notification operation by the receiver can be set on the side that transmits the information to be transmitted.

  Further, as shown in FIGS. 286A and 286B, in the output step SA43, a priority identifier for identifying the priority of the notification operation identifier may be further output in association with the four partial signals.

  Thus, when the priority identifier and the notification operation identifier are transmitted as a visible light signal and received by the receiver, the receiver handles the notification operation identifier according to the priority identified by the priority identifier. be able to. That is, when the receiver acquires another notification operation identifier, the receiver is identified by the notification operation identified by the notification operation identifier transmitted as the visible light signal and the other notification operation identifier. One of the notification operations can be selected based on the priority.

  FIG. 287B is a block diagram of an information processing device in Embodiment 13.

  This information processing apparatus A40 is an apparatus for changing the luminance of the light emitter (light emitting unit) of the transmitter described above according to the number of divisions shown in FIG.

  That is, the information processing apparatus A40 is an apparatus that processes the information to be transmitted in order to transmit the information to be transmitted with a luminance change. Specifically, the information processing apparatus A40 includes an encoding unit A41 that generates an encoded signal by encoding information to be transmitted, and a range in which the number of bits of the generated encoded signal is 24 to 64 bits. In this case, a dividing unit A42 that divides the encoded signal into four partial signals and an output unit A43 that sequentially outputs the four partial signals are provided. Such an information processing apparatus A40 can achieve the same effects as the information processing program described above.

  An information processing program according to an aspect of the present invention is an information processing program that causes a computer to process information on a transmission target in order to transmit the information on the transmission target with a luminance change, and encodes the information on the transmission target An encoding step for generating an encoded signal by dividing the encoded signal into four partial signals when the number of bits of the generated encoded signal is in a range of 24 to 64 bits; And causing the computer to execute an output step of sequentially outputting the four partial signals.

  As a result, as shown in FIGS. 284 to 287B, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and output. As a result, when the luminance of the illuminant changes according to the four partial signals that are output, the four partial signals are transmitted as visible light signals and received by the receiver. Here, the larger the number of bits of the output signal, the more difficult it is for the receiver to properly receive the signal by imaging, and the reception efficiency decreases. Therefore, it is desirable to divide the signal into signals having a small number of bits, that is, small signals. However, if the signal is too finely divided into many small signals, the receiver cannot receive the original signal unless each small signal is individually received, resulting in a decrease in reception efficiency. Therefore, as described above, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and sequentially output to indicate the information to be transmitted. The encoded signal can be transmitted as a visible light signal with the best reception efficiency. As a result, communication between various devices can be enabled.

  In the output step, the four partial signals may be output according to a first order, and the four partial signals may be output again according to a second order different from the first order.

  As a result, the four partial signals are repeatedly output in a different order. Therefore, when each output signal is transmitted to the receiver as a visible light signal, the reception efficiency of the four partial signals is increased. It can be further increased. That is, even if the four partial signals are repeatedly output in the same order, the same partial signal may not be received by the receiver. However, by changing the order, the occurrence of such a case can be suppressed.

  In the output step, the four partial signals are further output with a notification operation identifier attached thereto, and the notification operation identifier is transmitted when the four partial signals are transmitted by a luminance change and received by a receiver. Furthermore, an identifier for identifying the operation of the receiver that notifies the user of the receiver that the four partial signals have been received may be used.

  Thus, when the notification operation identifier is transmitted as a visible light signal and received by the receiver, the receiver receives the four partial signals to the user according to the operation identified by the notification operation identifier. You can be notified. That is, the notification operation by the receiver can be set on the side that transmits the information to be transmitted.

  Further, in the output step, a priority identifier for identifying a priority of the notification operation identifier may be output along with the four partial signals.

  Thus, when the priority identifier and the notification operation identifier are transmitted as a visible light signal and received by the receiver, the receiver handles the notification operation identifier according to the priority identified by the priority identifier. be able to. That is, when the receiver acquires another notification operation identifier, the receiver is identified by the notification operation identified by the notification operation identifier transmitted as the visible light signal and the other notification operation identifier. One of the notification operations can be selected based on the priority.

  Next, registration of network connection of the electronic device will be described.

  FIG. 288 is a diagram for describing an application example of the transmission / reception system in this embodiment.

  The transmission / reception system includes a transmitter 10131b configured as an electronic device such as a washing machine, a receiver 10131a configured as a smartphone, and a communication device 10131c configured as an access point or a router.

  FIG. 289 is a flowchart showing processing operations of the transmission / reception system according to the present embodiment.

  When the start button is pressed (step S10165), the transmitter 10131b transmits Wi-Fi, Bluetooth (registered trademark) information such as an SSID, a password, an IP address, a MAC address, or an encryption key to connect to the transmitter 10131b. ) Or Ethernet (registered trademark) or the like (step S10166), and waits for connection. The transmitter 10131b may transmit these pieces of information directly or indirectly. When transmitting indirectly, the transmitter 10131b transmits an ID associated with the information. For example, the receiver 10131a that has received the ID downloads information associated with the ID from a server or the like.

  The receiver 10131a receives the information (step S10151), connects to the transmitter 10131b, and connects to the communication device 10131c configured as an access point or router (SSID, password, IP address, MAC address, Alternatively, the encryption key or the like is transmitted to the transmitter 10131b (step S10152). The receiver 10131a registers information (MAC address, IP address, encryption key, etc.) for the transmitter 10131b to connect to the communication device 10131c in the communication device 10131c, and makes the communication device 10131c wait for connection. Further, the receiver 10131a notifies the transmitter 10131b that preparation for connection from the transmitter 10131b to the communication device 10131c is completed (step S10153).

  The transmitter 10131b disconnects from the receiver 10131a (step S10168) and connects to the communication device 10131c (step S10169). If the connection is successful (Y in step S10170), the transmitter 10131b notifies the receiver 10131a of the connection success via the communication device 10131c, and notifies the user of the connection success with a screen display, LED status, voice, or the like. (Step S10171). If the connection fails (N in step S10170), the transmitter 10131b notifies the receiver 10131a of the connection failure through visible light communication, and notifies the user in the same way as when it is successful (step S10172). The connection success may be notified by visible light communication.

  The receiver 10131a connects to the communication device 10131c (step S10154), and if there is no notification of a connection success or failure (N in step S10155 and N in step S10156), the transmitter 10131b can be accessed via the communication device 10131c. It is confirmed whether or not (step S10157). If not (N in step S10157), the receiver 10131a determines whether or not the connection to the transmitter 10131b using the information received from the transmitter 10131b has been made a predetermined number of times or more (step S10158). If it is determined that the predetermined number of times has not been performed (N in step S10158), the receiver 10131a repeats the process from step S10152. On the other hand, if it is determined that the predetermined number of times has been performed (Y in step S10158), the receiver 10131a notifies the user of the processing failure (step S10159). If the receiver 10131a determines in step S10156 that there has been a notification of a successful connection (Y in step S10156), the receiver 10131a notifies the user of the processing success (step S10160). That is, the receiver 10131a notifies the user whether or not the transmitter 10131b has been able to connect to the communication device 10131c by screen display, voice, or the like. Accordingly, the transmitter 10131b can be connected to the communication device 10131c without requiring complicated input by the user.

  Next, registration of a network connection of an electronic device (when connecting via another electronic device) will be described.

  FIG. 290 is a diagram for describing an application example of the transmission and reception system in this embodiment.

  This transmission / reception system includes an air conditioner 10133b, a transmitter 10133c configured as an electronic device such as a wireless adapter connected to the air conditioner 10133b, a receiver 10133a configured as, for example, a smartphone, and a communication device configured as an access point or a router. 10133d and another electronic device 10133e configured as, for example, a wireless adapter, a wireless access point, or a router.

  FIG. 291 is a flowchart showing processing operations of the transmission / reception system in the present embodiment. Hereinafter, the air conditioner 10133b or the transmitter 10133c is referred to as an electronic device A, and the electronic device 10133e is referred to as an electronic device B.

  First, when the start button is pressed (step S10188), the electronic device A transmits information (individual ID, password, IP address, MAC address, encryption key, etc.) for connection to itself (step S10189). The connection is awaited (step S10190). The electronic device A may transmit these pieces of information directly or indirectly as described above.

  The receiver 10133a receives the information from the electronic device A (step S10181), and transmits the information to the electronic device B (step S10182). When electronic device B receives the information (step S10196), it connects to electronic device A according to the received information (step S10197). Then, the electronic device B determines whether or not the connection with the electronic device A is established (step S10198), and notifies the receiver 10133a of the success or failure (step S10199 or step S10200).

  If the electronic device A is connected to the electronic device B during a predetermined time (Y in step S10191), the electronic device A notifies the receiver 10133a of the connection success via the electronic device B (step S10192) and is not connected (step S10192). N in step S10191), the receiver 10133a is notified of the connection failure by visible light communication (step S10193). In addition, the electronic device A notifies the user of the success or failure of the connection through a screen display, a light emission state, sound, or the like. Accordingly, the electronic device A (transmitter 10133c) can be connected to the electronic device B (electronic device 10133e) without requiring complicated input by the user. Note that the air conditioner 10133b and the transmitter 10133c illustrated in FIG. 290 may be configured integrally, and similarly, the communication device 10133d and the electronic device 10133e may be configured integrally.

  Next, transmission of appropriate imaging information will be described.

  FIG. 292 is a diagram for describing an example of application of the transmission and reception system in this embodiment.

  This transmission / reception system includes a receiver 10135a configured as, for example, a digital still camera or a digital video camera, and a transmitter 10135b configured as, for example, illumination.

  FIG. 293 is a flowchart illustrating a processing operation of the transmission / reception system according to the present embodiment.

  First, the receiver 10135a sends an imaging information transmission command to the transmitter 10135b (step S10211). Next, the transmitter 10135b receives the imaging information transmission command, the imaging information transmission button is pressed, the imaging information transmission switch is turned on, or the power is turned on (in step S10221). Y), imaging information is transmitted (step S10222). The imaging information transmission command is a command for transmitting imaging information, and the imaging information indicates, for example, the color temperature of illumination, spectral distribution, illuminance, or light distribution. The transmitter 10135b may transmit the imaging information directly or indirectly as described above. When transmitting indirectly, the transmitter 10135b transmits an ID associated with the imaging information. The receiver 10135a that has received the ID downloads, for example, imaging information associated with the ID from a server or the like. At this time, the transmitter 10135b is a method for transmitting a transmission stop command to itself (frequency of radio waves, infrared rays, or sound waves for transmitting the transmission stop command, or an SSID, a password or an IP address for connection to self-confidence) ) May be sent.

  When receiving the imaging information (step S10212), the receiver 10135a transmits a transmission stop command to the transmitter 10135b (step S10213). Here, when the transmitter 10135b receives a transmission stop command from the receiver 10135a (step S10223), the transmitter 10135b stops transmission of imaging information and emits light uniformly (step S10224).

  Furthermore, the receiver 10135a sets the imaging parameter according to the imaging information received in step S10212 (step S10214), or notifies the user of the imaging information. The imaging parameter is, for example, white balance, exposure time, focal length, sensitivity, or scene mode. Thereby, it is possible to take an image with an optimum setting according to the illumination. Next, the receiver 10135a captures an image after the transmission of imaging information from the transmitter 10135b is stopped (Y in step S10215) (step S10216). Thereby, it is possible to perform imaging without changing the brightness of the subject due to signal transmission. Note that, after step S10216, the receiver 10135a may transmit a transmission start command for prompting the start of transmission of imaging information to the transmitter 10135b (step S10217).

  Next, the display of a charge state is demonstrated.

  FIG. 294 is a diagram for describing an example of application of a transmitter in this embodiment.

  For example, the transmitter 10137b configured as a charger includes a light emitting unit, and transmits a visible light signal indicating a charged state of the battery from the light emitting unit. Thus, the state of charge of the battery can be notified without an expensive display device. When a small LED is used as the light emitting unit, a visible light signal cannot be received unless the LED is imaged from nearby. In addition, in the transmitter 10137c having a protrusion near the LED, it is difficult to close-up the LED due to the protrusion. Therefore, the visible light signal from the transmitter 10137b having no protrusion near the LED can be received more easily than the visible light signal from the transmitter 10137c.

(Embodiment 14)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  First, transmission in the demo mode and failure will be described.

  FIG. 295 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  When an error has occurred, the transmitter transmits a signal indicating that the error has occurred, or a signal corresponding to the error code. Can tell error details. The receiver can correct the error or appropriately report the error content to the service center by indicating an appropriate response according to the error content.

  If the transmitter is in demo mode, it will send a demo code. As a result, for example, when a demo of a transmitter that is a product is performed at a storefront, the store visitor can receive the demo code and acquire the product description associated with the demo code. Whether or not the demo mode is selected is determined by whether the transmitter operation setting is in the demo mode, the storefront CAS card is inserted, the CAS card is not inserted, or the recording medium is not inserted. Judging from the point.

  Next, signal transmission from the remote controller will be described.

  FIG. 296 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  For example, when a transmitter configured as an air conditioner remote controller receives main body information, the transmitter transmits the main body information, so that the receiver receives information on a distant main body from a nearby transmitter. can do. The receiver can also receive information from a main body that exists in a place where visible light communication is impossible, such as over a network.

  Next, a process of transmitting only when the place is bright is described.

  FIG. 297 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  The transmitter transmits if the ambient brightness is above a certain level, and stops transmitting if the brightness is below a certain level. Thereby, for example, a transmitter configured as an advertisement for a train can automatically stop its operation when the vehicle enters the garage, and battery consumption can be suppressed.

  Next, content distribution (association change / scheduling) in accordance with the display of the transmitter will be described.

  FIG. 298 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  The transmitter associates the content desired to be acquired by the receiver with the transmission ID in accordance with the display timing of the content to be displayed. Each time the display content is changed, the association change is registered with the server.

  When the display timing of the display content is known, the transmitter sets the server so that another content is delivered to the receiver in accordance with the change timing of the display content. When the request for the content associated with the transmission ID is received from the receiver, the server transmits the content according to the set schedule to the receiver.

  Thereby, for example, when a transmitter configured as digital signage changes display contents one after another, the receiver can acquire content that matches the content displayed by the transmitter.

  Next, content distribution (synchronization by time) according to the display of the transmitter will be described.

  FIG. 299 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  In response to a content acquisition request associated with a predetermined ID, it is registered in advance in the server so that different content is passed according to the time.

  The transmitter synchronizes the time with the server, adjusts the timing so that a predetermined part is displayed at a predetermined time, and displays the content.

  Thereby, for example, when a transmitter configured as digital signage changes display contents one after another, the receiver can acquire content that matches the content displayed by the transmitter.

  Next, content distribution (transmission of display time) in accordance with the display of the transmitter will be described.

  FIG. 300 is a diagram illustrating an example of operation of a transmitter and a receiver in this embodiment.

  The transmitter transmits the display time of the content being displayed in addition to the ID of the transmitter. The content display time is information that can identify the currently displayed content, and can be expressed by, for example, an elapsed time from the start time of the content.

  The receiver acquires content associated with the received ID from the server, and displays the content in accordance with the received display time. Thereby, for example, when a transmitter configured as digital signage changes display contents one after another, the receiver can acquire content that matches the content displayed by the transmitter.

  Further, the receiver changes the content to be displayed as time elapses. As a result, even if a signal is not received again when the display content of the transmitter changes, the content that matches the display content is displayed.

  Next, data uploading according to the user's permission status will be described.

  FIG. 301 is a diagram illustrating an example of operation of a receiver in this embodiment.

  If the user has registered for an account, the receiver is authorized to access information such as the location of the receiver, phone number, ID, installed application, and user (Age, sex, occupation, preference, etc.) are transmitted to the server together with the received ID.

  If the account is not registered, if the user permits uploading of information as described above, it is similarly sent to the server. If not permitted, only the received ID is sent to the server. .

  As a result, the user can receive the content according to the situation at the time of reception and his / her personality, and the server can be used for data analysis by obtaining user information.

  Next, activation of the content reproduction application will be described.

  FIG. 302 is a diagram illustrating an example of operation of a receiver in this embodiment.

  The receiver acquires content associated with the received ID from the server. When the active application can handle (can display or play) the acquired content, the acquired content is displayed / reproduced by the active application. If it cannot be handled, it is confirmed whether or not an app that can be handled is installed in the receiver. If it is installed, the app is activated to display / play the acquired content. If it is not installed, it automatically installs, displays a message prompting installation, displays a download screen, and displays and plays the acquired content after installation.

  As a result, the acquired content can be appropriately handled (displayed / reproduced).

  Next, activation of the designated application will be described.

  FIG. 303 is a diagram illustrating an example of operation of a receiver in this embodiment.

  The receiver acquires the content associated with the received ID and information (application ID) for specifying the application to be activated from the server. When the running application is the designated application, the acquired content is displayed / reproduced. When the designated application is installed in the receiver, the designated application is activated to display / play the acquired content. If it is not installed, it automatically installs, displays a message prompting installation, displays a download screen, and displays and plays the acquired content after installation.

  The receiver may acquire only the application ID from the server and start the designated application.

  The receiver may perform specified settings. The receiver may set the designated parameter and activate the designated application.

  Next, selection between streaming reception and normal reception will be described.

  FIG. 304 is a diagram illustrating an example of operation of a receiver in this embodiment.

  The receiver determines that the signal is being streamed when the value of the predetermined address of the received data is a predetermined value or the received data includes a predetermined identifier, and receives the streaming data. Receive with. Otherwise, it is received by a normal receiving method.

  Thus, reception can be performed regardless of whether the signal is transmitted by the streaming distribution method or the normal distribution method.

  Next, private data will be described.

  FIG. 305 is a diagram illustrating an example of operation of a receiver in this embodiment.

  When the received ID value is within a predetermined range or includes a predetermined identifier, the receiver refers to the table in the application, and if the reception ID exists in the table, the receiver is designated in the table. Get the content. Otherwise, the content set as the reception ID is acquired from the server.

  As a result, content can be received without registering with the server. Further, since communication with the server is not performed, a quick response can be obtained.

  Next, setting of the exposure time according to the frequency will be described.

  FIG. 306 is a diagram illustrating an example of operation of a receiver in this embodiment.

  The receiver detects the signal and recognizes the modulation frequency of the signal. The receiver sets the exposure time according to the period of the modulation frequency (modulation period). For example, the signal can be easily received by setting the exposure time to be approximately equal to the modulation period. Further, for example, by setting the exposure time to an integral multiple of the modulation period or a value close to it (approximately ± 30%), it is possible to easily receive a signal by convolutional decoding.

  Next, the optimum parameter setting of the transmitter will be described.

  FIG. 307 is a diagram illustrating an example of operation of a receiver in this embodiment.

  In addition to the data received from the transmitter, the receiver transmits current location information and information related to the user (address, sex, age, preference, etc.) to the server. The server transmits parameters for optimal operation of the transmitter to the receiver in accordance with the received information. The receiver sets the received parameter if it can be set in the transmitter. If it cannot be set, the parameter is displayed and the user is prompted to set the parameter.

  This allows, for example, operating the washing machine to optimize the water properties of the area where the transmitter is used, or operating the rice cooker to cook rice in a way that is optimal for the type of rice used by the user You can make it.

  Next, an identifier indicating the data structure will be described.

  FIG. 308 is a diagram for describing an example of the structure of transmission data in this embodiment.

  The transmitted information includes an identifier, and the receiver can know the configuration of the subsequent part by its value. For example, it is possible to specify the data length, the type and length of the error correction code, the data division point, and the like.

  Thereby, the transmitter can change the type and length of the data body and the error correction code according to the nature of the transmitter and the communication path. Also, the transmitter can cause the receiver to acquire an ID corresponding to the content ID by transmitting the content ID in addition to the ID of the transmitter.

(Embodiment 15)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  FIG. 309 is a diagram for describing operation of a receiver in this embodiment.

  The receiver 1210a in this embodiment switches the shutter speed between a high speed and a low speed, for example, in units of frames when performing continuous shooting with the image sensor. Furthermore, the receiver 1210a switches the process for the frame to a barcode recognition process and a visible light recognition process based on the frame obtained by the shooting. Here, the barcode recognition process is a process for decoding a barcode reflected in a frame obtained by a low shutter speed. The visible light recognition process is a process of decoding the above-described bright line pattern reflected in a frame obtained with a high shutter speed.

  Such a receiver 1210 a includes a video input unit 1211, a barcode / visible light identification unit 1212, a barcode recognition unit 1212 a, a visible light recognition unit 1212 b, and an output unit 1213.

  The video input unit 1211 includes an image sensor, and switches a shutter speed for photographing by the image sensor. That is, the video input unit 1211 switches the shutter speed alternately between low speed and high speed, for example, in units of frames. More specifically, the video input unit 1211 switches the shutter speed to high speed for odd-numbered frames and switches the shutter speed to low speed for even-numbered frames. Shooting at a low shutter speed is shooting in the above-described normal shooting mode, and shooting at a high shutter speed is shooting in the above-described visible light communication mode. That is, when the shutter speed is low, the exposure time of each exposure line included in the image sensor is long, and a normal captured image on which the subject is projected is obtained as a frame. Further, when the shutter speed is high, the exposure time of each exposure line included in the image sensor is short, and a visible light communication image in which the above-described bright line is projected is obtained as a frame.

  The barcode / visible light identification unit 1212 switches processing for the image by determining whether a barcode appears or an emission line appears in the image obtained by the video input unit 1211. For example, if a barcode appears in a frame obtained by shooting at a low shutter speed, the barcode / visible light identification unit 1212 causes the barcode recognition unit 1212a to perform processing on the image. On the other hand, if a bright line appears in an image obtained by shooting at a high shutter speed, the barcode / visible light identifying unit 1212 causes the visible light recognizing unit 1212b to execute processing on the image.

  The barcode recognition unit 1212a decodes a barcode appearing in a frame obtained by shooting at a low shutter speed. The barcode recognition unit 1212a acquires barcode data (for example, a barcode identifier) by decoding, and outputs the barcode identifier to the output unit 1213. The barcode may be a one-dimensional code or a two-dimensional code (for example, a QR code (registered trademark)).

  The visible light recognizing unit 1212b decodes a bright line pattern appearing in a frame obtained by photographing at a high shutter speed. The visible light recognizing unit 1212b obtains visible light data (for example, a visible light identifier) by the decoding, and outputs the visible light identifier to the output unit 1213. The visible light data is the above-described visible light signal.

  The output unit 1213 displays only frames obtained by shooting at a low shutter speed. Therefore, when the subject imaged by the video input unit 1211 is a barcode, the output unit 1213 displays the barcode. When the subject photographed by the video input unit 1211 is a digital signage that transmits a visible light signal, the output unit 1213 displays an image of the digital signage without displaying the bright line pattern. And when the output part 1213 acquires a barcode identifier, it acquires the information matched with the barcode identifier from a server etc., for example, and displays the information. Further, when the output unit 1213 acquires a visible light identifier, the output unit 1213 acquires information associated with the visible light identifier from, for example, a server and displays the information.

  That is, the receiver 1210a as a terminal device includes an image sensor, and the image sensor is switched while alternately switching the shutter speed of the image sensor between a first speed and a second speed higher than the first speed. Perform continuous shooting with. (A) When the subject imaged by the image sensor is a barcode, the receiver 1210a obtains an image showing the barcode by photographing when the shutter speed is the first speed, A barcode identifier is obtained by decoding the barcode reflected in the image. In addition, (b) when the subject to be photographed by the image sensor is a light source (for example, digital signage), the receiver 1210a includes a plurality of images included in the image sensor by photographing when the shutter speed is the second speed. A bright line image that is an image including a bright line corresponding to each of the exposure lines is acquired. Then, the receiver 1210a acquires a visible light signal as a visible light identifier by decoding a plurality of bright line patterns included in the acquired bright line image. Further, the receiver 1210a displays an image obtained by photographing when the shutter speed is the first speed.

  The receiver 1210a in this embodiment can decode a barcode and receive a visible light signal by switching between the barcode recognition process and the visible light recognition process. Furthermore, power consumption can be suppressed by switching.

  The receiver in this embodiment may perform image recognition processing simultaneously with visible light processing instead of barcode recognition processing.

  FIG. 310A is a diagram for describing another operation of the receiver in this embodiment.

  The receiver 1210b in this embodiment switches the shutter speed between a high speed and a low speed, for example, in units of frames when performing continuous shooting with the image sensor. Further, the receiver 1210b simultaneously performs the image recognition process and the above-described visible light recognition process on the image (frame) obtained by the photographing. The image recognition process is a process for recognizing a subject appearing in a frame obtained with a low shutter speed.

  Such a receiver 1210 b includes a video input unit 1211, an image recognition unit 1212 c, a visible light recognition unit 1212 b, and an output unit 1215.

  The video input unit 1211 includes an image sensor, and switches a shutter speed for photographing by the image sensor. That is, the video input unit 1211 switches the shutter speed alternately between a low speed and a high speed, for example, in frame units. More specifically, the video input unit 1211 switches the shutter speed to high speed for odd-numbered frames and switches the shutter speed to low speed for even-numbered frames. Shooting at a low shutter speed is shooting in the above-described normal shooting mode, and shooting at a high shutter speed is shooting in the above-described visible light communication mode. That is, when the shutter speed is low, the exposure time of each exposure line included in the image sensor is long, and a normal captured image on which the subject is projected is obtained as a frame. Further, when the shutter speed is high, the exposure time of each exposure line included in the image sensor is short, and a visible light communication image in which the above-described bright line is projected is obtained as a frame.

  The image recognition unit 1212c recognizes a subject appearing in a frame obtained by shooting at a low shutter speed and specifies the position of the subject in the frame. As a result of recognition, the image recognition unit 1212c determines whether or not the subject is an AR (Augmented Reality) target (hereinafter referred to as an AR target). When the image recognition unit 1212c determines that the subject is an AR object, the image recognition unit 1212c generates image recognition data that is data for displaying information about the subject (for example, the position of the subject and the AR marker). The AR marker is output to the output unit 1215.

  The output unit 1215 displays only frames obtained by shooting at a low shutter speed, as with the output unit 1213 described above. Therefore, when the subject photographed by the video input unit 1211 is digital signage that transmits a visible light signal, the output unit 1213 displays the image of the digital signage without displaying the bright line pattern. Further, when acquiring the image recognition data from the image recognition unit 1212c, the output unit 1215 superimposes a white frame-shaped indicator surrounding the subject on the frame based on the position of the subject in the frame indicated by the image recognition data. .

  FIG. 310B is a diagram illustrating an example of an indicator displayed by the output unit 1215.

  The output unit 1215 superimposes a white frame-shaped indicator 1215b surrounding a subject image 1215a configured as digital signage on a frame, for example. That is, the output unit 1215 displays the indicator 1215b indicating the subject whose image has been recognized. Furthermore, when the output unit 1215 acquires the visible light identifier from the visible light recognition unit 1212b, the output unit 1215 changes the color of the indicator 1215b from white to red, for example.

  FIG. 310C is a diagram illustrating a display example of AR.

  The output unit 1215 further acquires information related to the subject associated with the visible light identifier as related information from, for example, a server. The output unit 1215 describes related information in the AR marker 1215c indicated by the image recognition data, and displays the AR marker 1215c in which the related information is described in association with the subject image 1215a in the frame.

  In such a receiver 1210b in this embodiment, AR using visible light communication can be realized by simultaneously performing image recognition processing and visible light recognition processing. Note that the receiver 1210a illustrated in FIG. 310A may display the indicator 1215b illustrated in FIG. 310B in the same manner as the receiver 1210b. In this case, when a barcode is recognized in a frame obtained by shooting at a low shutter speed, the receiver 1210a displays a white frame-shaped indicator 1215b surrounding the barcode. Then, when the barcode is decoded, the receiver 1210a changes the color of the indicator 1215b from white to red. Similarly, when a bright line pattern is recognized in a frame obtained by imaging at a high shutter speed, the receiver 1210a identifies a part in the low-speed frame corresponding to the part where the bright line pattern is located. For example, when the digital signage is transmitting a visible light signal, an image of the digital signage in the low-speed frame is specified. Note that the low speed frame is a frame obtained by shooting at a low shutter speed. Then, the receiver 1210a displays a white frame-shaped indicator 1215b surrounding a specified portion (for example, the above-mentioned digital signage image) in the low-speed frame so as to be superimposed on the low-speed frame. Then, when the bright line pattern is decoded, the receiver 1210a changes the color of the indicator 1215b from white to red.

  FIG. 311A is a diagram for describing an example of a transmitter in this embodiment.

  Transmitter 1220a in the present embodiment transmits a visible light signal in synchronization with transmitter 1230. That is, the transmitter 1220a transmits the same visible light signal as the visible light signal at the timing when the transmitter 1230 transmits the visible light signal. Note that the transmitter 1230 includes a light emitting unit 1231, and transmits a visible light signal when the luminance of the light emitting unit 1231 changes.

  Such a transmitter 1220 a includes a light receiving unit 1221, a signal analyzing unit 1222, a transmission clock adjusting unit 1223 a, and a light emitting unit 1224. The light emitting unit 1224 transmits a visible light signal that is the same as the visible light signal transmitted from the transmitter 1230 by changing the luminance. The light receiving unit 1221 receives a visible light signal from the transmitter 1230 by receiving visible light from the transmitter 1230. The signal analysis unit 1222 analyzes the visible light signal received by the light receiving unit 1221 and transmits the analysis result to the transmission clock adjustment unit 1223a. The transmission clock adjustment unit 1223a adjusts the timing of the visible light signal transmitted from the light emitting unit 1224 based on the analysis result. That is, the transmission clock adjustment unit 1223a uses the light emitting unit 1224 so that the timing at which the visible light signal is transmitted from the light emitting unit 1231 of the transmitter 1230 matches the timing at which the visible light signal is transmitted from the light emitting unit 1224. Adjust the brightness change timing.

  Thereby, the waveform of the visible light signal transmitted by the transmitter 1220a and the waveform of the visible light signal transmitted by the transmitter 1230 can be matched in timing.

  FIG. 311B is a diagram for describing another example of the transmitter in this embodiment.

  The transmitter 1220b in this embodiment transmits a visible light signal in synchronization with the transmitter 1230, similarly to the transmitter 1220a. That is, the transmitter 1200b transmits the same visible light signal as the visible light signal at the timing when the transmitter 1230 transmits the visible light signal.

  Such a transmitter 1220b includes a first light receiving unit 1221a, a second light receiving unit 1221b, a comparing unit 1225, a transmission clock adjusting unit 1223b, and a light emitting unit 1224.

  Similar to the light receiving unit 1221, the first light receiving unit 1221 a receives visible light from the transmitter 1230 by receiving visible light from the transmitter 1230. The second light receiving unit 1221 b receives visible light from the light emitting unit 1224. The comparison unit 1225 compares the first timing at which visible light is received by the first light receiving unit 1221a and the second timing at which visible light is received by the second light receiving unit 1221b. Then, the comparison unit 1225 outputs the difference (that is, the delay time) between the first timing and the second timing to the transmission clock adjustment unit 1223b. The transmission clock adjusting unit 1223b adjusts the timing of the visible light signal transmitted from the light emitting unit 1224 so that the delay time is shortened.

  Thereby, the waveform of the visible light signal transmitted by the transmitter 1220b and the waveform of the visible light signal transmitted by the transmitter 1230 can be matched more accurately in terms of timing.

  In the example shown in FIGS. 311A and 311B, two transmitters transmit the same visible light signal, but different visible light signals may be transmitted. That is, when the two transmitters transmit the same visible light signal, they are transmitted in synchronization as described above. When two transmitters transmit different visible light signals, only one of the two transmitters transmits a visible light signal, while the other transmitter is uniformly lit or extinguished. To do. Thereafter, one transmitter is uniformly lit or extinguished, while only the other transmitter transmits a visible light signal. Two transmitters may transmit different visible light signals simultaneously.

  FIG. 312A is a diagram for describing an example of synchronous transmission by a plurality of transmitters in this embodiment.

  A plurality of transmitters 1220 in the present embodiment are arranged, for example, in a line as shown in FIG. 312A. Note that these transmitters 1220 have the same configuration as the transmitter 1220a shown in FIG. 311A or the transmitter 1220b shown in FIG. 311B. Each of the plurality of transmitters 1220 transmits a visible light signal in synchronization with one of the transmitters 1220 on both sides.

  Thereby, many transmitters can transmit a visible light signal synchronously.

  FIG. 312B is a diagram for describing an example of synchronous transmission by a plurality of transmitters in this embodiment.

  One transmitter 1220 among the plurality of transmitters 1220 in this embodiment is a reference for synchronizing the visible light signal, and the remaining plurality of transmitters 1220 are visible light signals so as to match the reference. Send.

  Thereby, many transmitters can transmit visible light signals more accurately and synchronously.

  FIG. 313 is a diagram for describing another example of synchronous transmission by a plurality of transmitters in this embodiment.

  Each of the plurality of transmitters 1240 in the present embodiment receives a synchronization signal and transmits a visible light signal according to the synchronization signal. Thereby, a visible light signal is transmitted from each of the plurality of transmitters 1240 in synchronization.

  Specifically, each of the plurality of transmitters 1240 includes a control unit 1241, a synchronization control unit 1242, a photocoupler 1243, an LED drive circuit 1244, an LED 1245, and a photodiode 1246.

  Control unit 1241 receives the synchronization signal and outputs the synchronization signal to synchronization control unit 1242.

  The LED 1245 is a light source that emits visible light, and blinks (that is, changes in luminance) in accordance with control by the LED drive circuit 1244. Thus, a visible light signal is transmitted from the LED 1245 to the outside of the transmitter 1240.

  The photocoupler 1243 transmits a signal between the synchronization control unit 1242 and the LED drive circuit 1244 while being electrically insulated. Specifically, the photocoupler 1243 transmits a transmission start signal described later transmitted from the synchronization control unit 1242 to the LED drive circuit 1244.

  When the LED drive circuit 1244 receives the transmission start signal from the synchronization control unit 1242 via the photocoupler 1243, the LED drive circuit 1244 causes the LED 1245 to start transmitting the visible light signal at the timing when the transmission start signal is received.

  The photodiode 1246 detects visible light emitted from the LED 1245, and outputs a detection signal indicating that the visible light has been detected to the synchronization control unit 1242.

  When receiving the synchronization signal from the control unit 1241, the synchronization control unit 1242 transmits a transmission start signal to the LED drive circuit 1244 via the photocoupler 1243. By transmitting this transmission start signal, transmission of the visible light signal is started. When the synchronization control unit 1242 receives the detection signal from the photodiode 1246 by transmitting the visible light signal, the synchronization control unit 1242 is a delay that is a difference between the timing at which the detection signal is received and the timing at which the synchronization signal is received from the control unit 1241. Calculate time. When the synchronization control unit 1242 receives the next synchronization signal from the control unit 1241, the synchronization control unit 1242 adjusts the timing for transmitting the next transmission start signal based on the calculated delay time. That is, the synchronization control unit 1242 adjusts the timing of transmitting the next transmission start signal so that the delay time for the next synchronization signal becomes a predetermined set delay time. Thus, the synchronization control unit 1242 transmits the next transmission start signal at the adjusted timing.

  FIG. 314 is a diagram for explaining signal processing in the transmitter 1240.

  When receiving the synchronization signal, the synchronization control unit 1242 generates a delay time setting signal that generates a delay time setting pulse at a predetermined timing. Note that receiving the synchronization signal specifically means receiving a synchronization pulse. That is, the synchronization control unit 1242 generates the delay time setting signal so that the delay time setting pulse rises at the timing when the set delay time has passed since the falling edge of the synchronization pulse.

  Then, the synchronization control unit 1242 transmits a transmission start signal to the LED drive circuit 1244 via the photocoupler 1243 at a timing delayed by the correction value N obtained last time from the falling edge of the synchronization pulse. As a result, a visible light signal is transmitted from the LED 1245 by the LED drive circuit 1244. Here, the synchronization control unit 1242 receives the detection signal from the photodiode 1246 at a timing delayed by the sum of the intrinsic delay time and the correction value N from the falling edge of the synchronization pulse. That is, transmission of a visible light signal is started from that timing. Hereinafter, this timing is referred to as transmission start timing. Note that the above-described intrinsic delay time is a delay time caused by a circuit such as the photocoupler 1243, and is a delay time that occurs even when the synchronization control unit 1242 receives the synchronization signal and immediately transmits the transmission start signal. .

  The synchronization control unit 1242 specifies the time difference from the transmission start timing to the rise of the delay time setting pulse as the correction correction value N. Then, the synchronization control unit 1242 calculates and holds the correction value (N + 1) by the correction value (N + 1) = the correction value N + the correction correction value N. Thus, when the synchronization control unit 1242 receives the next synchronization signal (synchronization pulse), it transmits a transmission start signal to the LED drive circuit 1244 at a timing delayed by the correction value (N + 1) from the falling edge of the synchronization pulse. To do. The correction correction value N can be a negative value as well as a positive value.

  Accordingly, each of the plurality of transmitters 1240 transmits the visible light signal after the set delay time has elapsed after receiving the synchronization signal (synchronization pulse). it can. That is, even if there is a variation in the inherent delay time caused by a circuit such as the photocoupler 1243 in each of the plurality of transmitters 1240, the visible light from each of the plurality of transmitters 1240 is not affected by the variation. The transmission of the optical signal can be accurately synchronized.

  The LED drive circuit consumes a large amount of power, and is electrically insulated from the control circuit that handles the synchronization signal using a photocoupler or the like. Therefore, when such a photocoupler is used, it is difficult to synchronize the transmission of visible light signals from a plurality of transmitters due to the variation in the inherent delay time described above. However, in the plurality of transmitters 1240 in this embodiment, the light emission timing of the LED 1245 is detected by the photodiode 1246, the delay time from the synchronization signal is detected by the synchronization control unit 1242, and the delay time is set in advance. The time is adjusted to be the time (the set delay time described above). As a result, even if there are individual variations in the photocouplers provided in a plurality of transmitters each configured as, for example, LED lighting, a visible light signal (for example, visible light ID) is synchronized with high accuracy from the plurality of LED lighting. Can be sent.

  Note that the LED illumination may be turned on or off during periods other than the visible light signal transmission period. In the case of lighting other than the first visible light signal transmission period, the first falling edge of the visible light signal may be detected. In the case of turning off the light other than the visible light signal transmission period, the first rising edge of the visible light signal may be detected.

  In the above-described example, the transmitter 1240 transmits a visible light signal every time a synchronization signal is received. However, the transmitter 1240 may transmit a visible light signal without receiving the synchronization signal. That is, if the transmitter 1240 transmits a visible light signal once in response to reception of the synchronization signal, the transmitter 1240 may sequentially transmit the visible light signal without receiving the synchronization signal. Specifically, the transmitter 1240 may sequentially transmit the visible light signal 2 to several thousand times with respect to one reception of the synchronization signal. The transmitter 1240 may transmit a visible light signal corresponding to the synchronization signal at a rate of once every 100 milliseconds or once every few seconds.

  Further, when the visible light signal is repeatedly transmitted according to the synchronization signal, the continuity of light emission of the LED 1245 may be lost due to the above-described set delay time. That is, a slightly longer blanking period may occur. As a result, blinking of the LED 1245 is visually recognized by a person, and so-called flicker may occur. Therefore, the transmitter 1240 may transmit a visible light signal corresponding to the synchronization signal at a period of 60 Hz or more. Thereby, blinking is performed at high speed, and the blinking is difficult to be visually recognized by a person. As a result, the occurrence of flicker can be suppressed. Alternatively, the transmitter 1240 may transmit a visible light signal corresponding to the synchronization signal at a sufficiently long cycle such as once every few minutes. Thereby, although blinking is visually recognized by a person, it is possible to prevent the blinking from being viewed repeatedly and continuously, and to reduce the discomfort that flicker gives to the person.

(Preprocessing for receiving method)
FIG. 315 is a flowchart illustrating an example of a reception method in this embodiment. FIG. 316 is an explanatory diagram for describing an example of a reception method in this embodiment.

  First, the receiver calculates the average value of the pixel values of a plurality of pixels arranged in a direction parallel to the exposure line (step S1211). If the pixel values of N pixels are averaged according to the central limit theorem, the expected value of the noise amount becomes N minus ½ power, and the SN ratio is improved.

  Next, the receiver leaves only the portion where the pixel value changes in the vertical direction in all the colors, and removes the change in the pixel value where the pixel value changes differently (step S1212). When the transmission signal (visible light signal) is expressed by the luminance of the light emitting unit provided in the transmitter, the luminance of the illumination of the transmitter or the backlight of the display changes. At this time, as shown in the part (b) of FIG. 316, the pixel values change in the same direction for all the colors. In the parts (a) and (c) of FIG. 316, the pixel values change differently for each color. In these portions, the pixel value fluctuates due to reception noise or a picture of the display or signage. Therefore, by removing these fluctuations, the SN ratio can be improved.

  Next, the receiver obtains a luminance value (step S1213). Since the luminance is not easily changed by color, the influence of the display or signage picture can be eliminated, and the SN ratio can be improved.

  Next, the receiver applies a luminance value to a low-pass filter (step S1214). In the receiving method according to the present embodiment, since a moving average filter based on the length of exposure time is applied, almost no signal is included in the high frequency region, and noise is dominant. Therefore, the S / N ratio can be improved by using a low-pass filter that cuts a high frequency region. Since there are many signal components up to the frequency up to the reciprocal of the exposure time, the effect of improving the S / N ratio can be increased by blocking the higher frequency. When the frequency component included in the signal is finite, the S / N ratio can be improved by blocking a frequency higher than that frequency. A filter (such as a Butterworth filter) that does not include a frequency vibration component is suitable for the low-pass filter.

(Receiving method by convolution maximum likelihood decoding)
FIG. 317 is a flowchart illustrating another example of the reception method in this embodiment. FIG. 318 and FIG. 319 are diagrams for explaining another example of the reception method in this embodiment. Hereinafter, the reception method when the exposure time is longer than the transmission cycle will be described with reference to these drawings.

  When the exposure time is an integral multiple of the transmission period, reception can be performed with the highest accuracy. Even if it is not an integral multiple, it can be received within a range of (N ± 0.33) times (N is an integer).

  First, the receiver sets the transmission / reception offset to 0 (step S1221). The transmission / reception offset is a value for correcting a difference between the transmission timing and the reception timing. Since this deviation is unknown, the receiver gradually changes the value that is a candidate for the transmission / reception offset, and adopts the most suitable value as the transmission / reception offset.

  Next, the receiver determines whether or not the transmission / reception offset is less than the transmission cycle (step S1222). Here, since the reception cycle and the transmission cycle are not synchronized with each other, a reception value matching the transmission cycle is not always obtained. For this reason, when the receiver determines in step S1222 that it is less than the transmission cycle (Y in step S1222), the reception value (for example, pixel value) matched to the transmission cycle is used for each transmission cycle using the reception value in the vicinity thereof. ) Is calculated by interpolation (step S1223). As the interpolation method, linear interpolation, nearest neighbor value, spline interpolation, or the like can be used. Next, the receiver obtains a difference between the received values obtained for each transmission cycle (step S1224).

  FIG. 318 shows an example in which the exposure time is three times the transmission cycle and the transmission signal is a binary value of 0 or 1. The received value at a certain point in time is a value obtained by adding three transmission signals. By obtaining the difference from the received value at the next time point, the value of the newly received signal can be obtained. At this time, since the difference in the reception values includes noise, it is not clear which signal is received. Therefore, the receiver calculates the probability (estimated likelihood) of which signal it was (step S1225). If the transmission signal is x and the difference between the reception values is y, this probability can be expressed by a conditional probability P (x | y). However, since it is difficult to obtain P (x | y), the receiver performs calculation using the value on the right side of P (x | y) ∝P (y | x) P (x) using Bayes' law. .

  It is conceivable to perform this calculation for all received values. When the number of received values is N, the convolution state transition pattern exists as the Nth power of 2 and NP is difficult, but it can be efficiently calculated by using the Viterbi algorithm.

  Most of the state transition paths in FIG. 319 are paths that do not conform to the transmission format. Therefore, the format check is performed every time the state transition is performed, and when the route does not conform to the transmission format, the accuracy of estimating a correct received signal can be improved by setting the likelihood of the route to 0. .

  The receiver adds a predetermined value to the transmission / reception offset (step S1226), and repeatedly executes the processing from step S1222. If the receiver determines in step S1222 that it is not less than the transmission cycle (N in step S1222), the receiver specifies the highest likelihood among the likelihoods of the received signals calculated for each transmission / reception offset. Then, the receiver determines whether or not the highest likelihood is greater than or equal to a predetermined value (step S1227). If it is determined that the value is equal to or greater than the predetermined value (Y in step S1227), the receiver uses the received signal with the highest likelihood as the final estimation result. Alternatively, the receiver uses, as a received signal candidate, a received signal having a likelihood equal to or higher than a value obtained by subtracting a predetermined value from the highest likelihood (step S1228). On the other hand, if it is determined in step S1227 that the highest likelihood is less than the predetermined value (N in step S1227), the receiver discards the estimation result (step S1229).

  When there is too much noise, it is often impossible to properly estimate the received signal, and the likelihood decreases at the same time. Therefore, when the likelihood is low, the reliability of the received signal can be improved by discarding the estimation result. In addition, even when a valid signal is not included in the input image, there is a problem in that the maximum likelihood decoding outputs a valid signal as an estimation result. However, since the likelihood also decreases in this case, this problem can be avoided by discarding the estimation result when the likelihood is low.

(Embodiment 16)
[1 Introduction]
Conventional visible light communication systems include a system using a general-purpose image sensor as a light receiving device and a system using a photo sensor or a special high-speed image sensor. The former method includes, for example, “Pikapi Camera (registered trademark)” of Casio Computer Co., Ltd. Since the imaging frame rate of many general-purpose image sensors has an upper limit of 30 fps, the luminance change of the light source to be transmitted needs to be a frequency lower than this. Since such a slow frequency luminance change is visible to human eyes, illumination cannot be used as a transmitter, and a dedicated transmitter must be used. The latter scheme includes IEEE 802.15.7 and CP1223. Since the modulation frequency used in these methods is as high as 9.6 kHz or more and it appears to be lit uniformly to the human eye, it is possible to use illumination as a transmitter. However, since a dedicated receiving device is necessary, the point that it cannot be received by a smartphone is an obstacle to popularization.

  We have developed a method that can receive a signal modulated at a speed that is not visible to the human eye, using a general-purpose image sensor installed in today's smartphone as a receiving device. CMOS image sensors are superior to CCD image sensors in terms of high-speed response, high integration, low power consumption, and low voltage drive, and are used in almost all smartphones and digital cameras. The imaging method of the CMOS type image sensor is a line scanning method in which exposure is sequentially performed for each line of an image, and is known as a cause of distortion in an image when a moving object is imaged. We have developed a method (Line Scan Sampling, LSS) that performs sampling at a frequency of 30 kHz or more, which is 1000 times that of the prior art, by appropriately setting the exposure time using the characteristics of this line scan method. In addition, we devised a modulation method suitable for this method and implemented it in lighting and displays. A reception system using LSS was installed in a commercially available smartphone, and it was confirmed that a signal modulated at 10 kHz or higher can be received.

[2 line scan sampling]
The CMOS image sensor converts the received light into a pixel value by the following procedure and reads it as one-dimensional data.

  1. When the photodiode in the pixel is exposed, a charge corresponding to the exposure amount is generated, and this charge is converted into a voltage by the amplifier.

  2. The voltage is sent to the vertical signal line by the row selection switch. Here, the fixed pattern noise is removed and temporarily stored.

  3. The data is sequentially sent to the horizontal signal line by the column selection switch and read as one-dimensional data.

  Image sensors used in smartphones and digital cameras today are highly miniaturized, and each pixel does not have a memory. For this reason, the exposure at 1 is not performed simultaneously for all the pixels, but is performed sequentially in accordance with the timing of 2, which is performed sequentially for each row. That is, since the start and end of exposure are performed at slightly different timings for each row, the image captured by the CMOS image sensor shows images at different times for each row. By utilizing this mechanism, it is possible to sample changes in the brightness of the transmitter much faster than sampling for each imaging frame. A line of pixels that are exposed simultaneously is called an exposure line.

  FIG. 320 is a diagram in which a signal modulated at a modulation frequency of 10 kHz is imaged with exposure times changed to 1/100, 1/1000, and 1 / 10,000. The pixel value of the captured image is obtained as a value obtained by multiplying the integral value of the brightness of the imaging target within the exposure time by a value determined by the brightness of the lens and the sensitivity setting value. Usually, when photographing is performed indoors, an exposure time of about 1/30 to 1/200 is used. When the exposure time Te is sufficiently longer than the modulation period Ts, the difference in luminance between the exposure line that captures the brightest period and the exposure line that captures the darkest period can be approximated as Ts / Te. When Te = 1/100 seconds and Ts = 1 / 10,000 seconds (10 kHz), the pixel value difference is only 1%. Therefore, this blinking is not recognized in photographs taken under normal conditions. However, as shown in FIG. 320C, when the exposure time is shortened, the blinking state clearly appears as the pixel value of the exposure line. By shortening the exposure time, it is possible to capture a luminance change at a high frequency.

  Not all photodiodes in a CMOS image sensor are used directly for imaging. The optical black portion is shielded from light and works to cancel dark current due to thermal noise by subtracting the output potential of this portion from the output potential of the effective pixel. There is also an invalid part that exists for design reasons. In addition, effective pixels are often configured with an aspect ratio close to 4: 3, but when the size of the captured image is set to 16: 9, the effective pixel portion is cut off from top and bottom, resulting in invalidity. It becomes the same treatment as the part. Since the image sensor sequentially reads out each row including not only effective pixels but also optical black and invalid portions, it takes time to capture the top line of the next image after capturing the bottom line of the image. There is a time difference corresponding to the time for exposing the optical black and the ineffective portion. This time is called blanking time.

  The time when the luminance change of the light source can be sampled by the LSS is only the time when the exposure line in which the light source is imaged is exposed. This situation is shown in FIG. Further, even if the light source is captured on the entire screen, the sample is discontinuous because of the blanking time described above. Therefore, it is necessary to perform signal transmission with a protocol suitable for LSS on the assumption that signal reception is discontinuous. Although today's smartphones do not have such a function, as shown in FIG. 322, if the light source position can be specified and set so that only that portion can be imaged, continuous reception is possible, and communication efficiency is improved. Can be dramatically increased.

  If the sampling frequency imaging frequency is 30 fps and the vertical size of the image is 1080 pixels, the number of samples by LSS is 30 × 1,080 = 32,400 times per second, but there is a period during which sampling is not possible due to the blanking time. The sampling frequency is faster than this. The blanking time varies depending on the setting for each model and the imaging conditions such as the frame rate and the image resolution, but is approximately 1 to 10 milliseconds, so the sampling frequency is approximately 33 to 46 kHz.

[3 Transmitter conditions]
In order to use illumination light as a light source for visible light communication, it is necessary that the luminance change for signal expression is at a level that cannot be perceived by humans. For this purpose, the average luminance (effective luminance) needs to be constant regardless of the signal to be transmitted. Further, it is necessary that the frequency of luminance change is sufficiently high or the rate of change is sufficiently small. The human perceptual limit frequency is called Critical Flicker Frequency (CFF), which is about 60 Hz although there is a difference depending on conditions. However, this is a limit in periodic flashing, and a higher modulation frequency is required for irregular changes for signal representation. In addition, even when the image is taken with a camera or a video camera, it should not be affected by a change in luminance. As described above, when the exposure time is set in a range used in normal shooting, the influence of the luminance change is small, and there is no problem with a still image. However, in the case of moving image shooting, a shadow like a scanning line may be perceived even if the luminance changes at a frequency higher than that of CFF. This is due to an alias between the frame frequency and the signal frequency of moving image shooting. In order to remove this influence, it is necessary to use a frequency considerably higher than CFF or reduce the rate of change.

  There are two types of dimming of illumination: control based on the amount of current flowing through the light source (current control) and control based on the length of light emission (PWM control). Since a change in luminance is used for signal representation, PWM control cannot be used. However, in order to change the current backlight control such as a display backlight or the like to the current control, a large circuit change is required, which becomes a barrier to the introduction of visible light communication. Therefore, it is desirable that the modulation system includes a function that can adjust the average luminance.

  As the original function of lighting, higher brightness is desirable. Since the number and withstand voltage of the LEDs serving as the light sources are determined by the maximum luminance, a modulation method in which the ratio of the effective luminance to the maximum luminance (effective luminance rate (Effective Luminance Rate (ELR))) is desirable is desirable.

  A signal can be transmitted from the display by controlling the brightness of the backlight of the display. However, it is necessary to pay attention to the following points as compared with the case where illumination is used as a transmitter. Since the luminance of the light source is low, the SN ratio is low. The screen picture is noisy, and if the picture is dark, the signal-to-noise ratio is further reduced. To improve the resolution of moving images, the backlight must be turned off while changing the liquid crystal transmittance. is there. The screen refresh rate is higher for higher grade products and is up to 240 Hz for current products. In this case, the signal is transmitted intermittently in units of 1/240 seconds.

[4 Modulation method suitable for LSS]
The biggest feature of LSS is that reception is discontinuous. As modulation methods corresponding to non-continuous reception, there are a small symbol method and a large symbol method.

[4.1 Large symbol system]
In the large symbol system, a uniform symbol having a symbol transmission time longer than the image capturing period is used. A uniform symbol refers to a symbol that can decode a signal if any part of the symbol is received, such as a frequency modulation symbol. The receiver receives one symbol per image and reconstructs communication data by concatenating symbols received from a plurality of images. The method of receiving one symbol per image is similar to the conventional image sensor reception method, but this method has a much larger amount of information per symbol and humans cannot see the blinking of the light source. There is a difference. It is possible to restore the communication data by connecting the received data in the order received, but for example, if the processing of the imaging frame is dropped due to the processing load of the receiver, it can be restored normally It lacks reliability. By using a part of the signal for expressing the address, data can be normally received even in the above case.

An encoded signal by frequency modulation has a uniform signal and a large amount of information per symbol, and is suitable as a large symbol code. An example of frequency modulation by on / off control is shown in FIG. In simple frequency modulation, the effective luminance rate is 50%. However, the effective luminance rate can be increased by fixing the time of one period and increasing the time when the luminance is high. FIGS. 323 (b) and (c) are examples of frequency analysis of different ELR signals at the same frequency, and it can be seen that the frequency represented by the signal can be recognized from the fundamental frequency.

  Since the sampling of the signal by the LSS is the average value of the luminance during the exposure period, a moving average filter of the length of the exposure time is applied. The frequency characteristics of this filter are shown in FIG. Therefore, it is necessary to keep in mind that the exposure time of the receiver needs to be kept constant, and the frequency cut off here cannot be used.

[4.2 Small symbol system]
In the Small symbol system, the receiver receives a plurality of symbols during a series of reception times, and restores communication data by connecting portions received in a plurality of image frames. If the repetition cycle of the transmission signal is constant, it is possible to calculate the length of the non-reception period from the imaging frame rate and concatenate the reception part, but most of today's smartphones are processing load on the processor and Since control is performed to change the imaging frame rate depending on the temperature, this method has low reliability. Therefore, by dividing the communication data into a plurality of packets and adding a header indicating the packet boundary and an address indicating the packet number, the received data can be connected regardless of the length of the non-reception period. In the former method, when the ratio of the reception cycle (imaging frame rate) and the transmission cycle is represented by a small integer, a situation occurs in which only the same part of the communication data can be received, but in the latter method, This problem can be solved by making the packet transmission order random.

  Pulse position modulation and frequency modulation are suitable for the small symbol system because the symbol transmission time is short and the ELR can be increased.

The encoding method for keeping the luminance constant in the pulse position modulation includes Manchester code and four
There is a pulse-position modu-lation (4PPM) code (FIGS. 325 and 326). In both cases, the coding efficiency is 50%, but the effective luminance rate is 50% for the Manchester code and 75% for the 4PPM code, and the 4PPM code is excellent. FIG. 325 is a coding scheme (variable) based on 4PPM code and corresponding to luminance adjustment.
4PPM, V4PPM). With this encoding method, the effective luminance rate can be continuously changed from 25% to 75%. In addition, since the rising position of the signal is constant regardless of the luminance, there is a feature that the receiving side can receive without being aware of the luminance setting value. There is a variable PPM (VPPM) method as a luminance adjustment compatible coding method based on Manchester code, but when the effective luminance rate can be changed between 25% and 75% in the VPPM method, the recognizable shortest pulse width is set. Considering the standard, as shown in FIG. 327, the pulse width of 25% of the symbol length is the same as the width of one pulse of 4 PPM. At this time, the encoding efficiency of V4PPM is twice that of VPPM, and it can be said that V4PPM is excellent.

  By performing frequency analysis such as discrete cosine transform, a frequency-modulated symbol can be received. This method has an advantage that it can be received even with a long exposure time. However, since the information on the order of symbols is lost, combinations of frequencies that can be used in consideration of harmonics are limited. In the following experiment, V4PPM is used as a symbol modulation system of the Small symbol system.

[4.3 Performance evaluation]
We evaluate the performance of the two modulation methods. The smartphone P-03E is a receiver, and the liquid crystal television TH-L47DT5 is a transmitter. When the liquid crystal is rewritten, the backlight is turned off. The rewrite frequency of the liquid crystal is 240 Hz, and the backlight is turned on for 75% in the standard mode. Therefore, the time for continuously transmitting signals is 1,000,000 / 240 × 0.75 = 3,125 microseconds. Become. When the exposure time is 1 / 10,000, a 50% gray screen is displayed on the display screen, and a 1 kHz on / off signal is transmitted, the signal noise power measured by the above transmitter and receiver is shown in FIG. Shown in (a). The following experiment was performed using a simulation signal simulating this S / N ratio ((b) in FIG. 328). The received signal used was a value obtained by averaging the pixel values of 256 pixels in the direction horizontal to the exposure line. The following results are the results of 1,000 trials under each condition.

  A single frequency symbol is used as a symbol of the large symbol system. The closer the effective luminance rate is to 50%, the lower the reception error. The received signal is calculated by discrete cosine transform of the pixel value in the direction perpendicular to the exposure line. FIG. 329A shows the reception error (difference between the transmission signal frequency and the reception signal frequency). The reception error increased rapidly around 9 kHz. This is because the signal power is reduced by the LSS moving average filter shown in FIG. 324 and is buried in noise. The reason why the reception error is large in the low frequency region is that only a signal having a small period can be transmitted during the transmission period. The reception error rate at each frequency margin is shown in FIGS. 329B to 329F. For example, if the allowable error rate is 5%, a value in the range from 1.6 kHz to 8 kHz can be assigned in 50 Hz steps, so (8,000-1,600) / 50 = 128 = 7 bits. Information can be expressed. For example, if 2 bits are an address and 5 bits are data, 20-bit information can be expressed. Since communication data can be decoded from an image of 4 frames at the fastest speed, the effective communication speed when imaged at 30 fps is 150 bps. In an actual application, it is necessary to include an error check code in order to detect a reception error.

  V4PPM was used as a symbol of the Small symbol system. FIG. 330 shows the reception success rate at each symbol rate. However, this reception success rate indicates the rate at which all symbols in one packet have been correctly received. Further, the modulation frequency here indicates the number of time slots of luminance change included in one second. That is, with a modulation frequency of 10 kHz, 2,500 V4PPM symbols are included. If the allowable error rate is 5%, the modulation frequency can be 10 kHz. If the entire continuous transmission period is one packet, the boundary can be determined if the beginning of the transmission period is in the ON state (high luminance state), so the header indicating the packet boundary can be indicated by one slot. Therefore, one packet includes the number of V4PPM symbols shown in (Equation 1) below.

  That is, 14-bit information is included. If 2 bits are assigned to an address and 12 bits are assigned to data, 48 bits of information can be expressed. Since a plurality of packets can be received if the transmitter in the captured image is sufficiently large, the effective communication speed becomes maximum when all the packets can be received from one image, and becomes 1,440 bps when capturing at 30 fps.

  Since the number of bits that can be expressed by the Small symbol method is larger, this was implemented and the operation was confirmed. The packet configuration is as described above, and a 4-bit CRC code is included in the synthesized 48-bit data. In addition, when different data packets are received at the same address, data having a large number of receptions of the same data packet is employed. Further, when the same number of packets of the same data is received, the reception is continued until any data reaches the maximum number independently. If an error is detected by CRC, all received packets are discarded. The distance between the transceivers was 4 m. At this distance, an image of at least one packet is included in one image. In 200 trials, the average reception time was 351 milliseconds, and no error remained after the CRC error check. The expected value of the number of packet receptions required to collect N types of packets can be calculated by the following (Equation 2).

  Therefore, when N = 4, the expected value is 8.33. For this reason, there is no error in reception, and the expected reception time when one packet is received from one image is 8,33 × 33 = 275 milliseconds. The average reception time is slower than this because it is necessary to receive multiple packets due to a reception error, and the reception time is improved by reducing the reception error by taking measures such as including an error detection code in the packet. Can be made.

[5 Conclusion]
Visible light communication is a type of wireless communication using electromagnetic waves in the visible light band visible to the human eye. It attracts attention from the social application aspect that lighting becomes a communication infrastructure. Features include no need for radio wave certification, safety without impact on living organisms, no influence on other devices due to electromagnetic waves, visible source / communication path, and communication range at a glance, preventing unauthorized communication Easy to shield, highly directional, can communicate only with a specific partner, and can share the energy of communication with lighting. Further, in addition to the use as bidirectional communication similar to conventional wireless communication typified by WiFi or the like, a use method as a sign using one-way communication is considered. For example, application of specifying the position indoors where GPS does not reach by transmitting position information from ceiling lighting is expected.

  In this paper, we proposed high-speed sampling using the line scan characteristics of a CMOS image sensor and confirmed that signals modulated with a modulation frequency of 10 kHz can be received by current smartphones.

  Various applications can be considered because a visible light signal with illumination as a transmitter can be received by a smartphone. For example, it can be expected that the position information is transmitted from the ceiling lighting and the position is specified indoors where GPS does not reach. In addition, applications such as acquiring a coupon with a smartphone or confirming vacant seat information using a signboard as a transmitter can be considered.

  The visible light communication method proposed in this paper has the following advantages over the illuminance sensor reception method in addition to being able to use a smartphone as a receiver. Since the light to be received can be spatially separated, even when a plurality of transmitters exist nearby, signals can be received individually without interference. In addition, since the light receiving direction can be specified, the relative position with respect to the light source can be calculated. That is, by acquiring the absolute position of the light source from the received signal, the absolute position of the receiver can be obtained with an accuracy of several centimeters. In addition, communication using a display or a sign as a transmitter can be performed. Display and signboards have lower brightness and illuminance than illumination, so it is difficult to receive them with a photosensor, but the image sensor reception method can receive signals regardless of environmental illuminance. In the display, the movement of the screen becomes a noise source. In the image sensor reception method, a flat part with less noise can be selected and a signal can be received from that part.

  In the future, we plan to improve the reception algorithm and investigate further improvements in communication performance. In addition, the application example of this visible light communication system will be examined, and the use in the industrial use aspect will be verified.

(Embodiment 17)
In this embodiment, a display system that is configured as a transmitter in each of the above embodiments and that displays a video and transmits a visible light signal will be described.

  FIG. 331 is a block diagram illustrating a configuration of the display system in this embodiment.

  The display system in this embodiment includes a video signal transmitter 1250 that generates and transmits a video signal, and a video display 1270 that displays a video and transmits a visible light signal.

  The video signal transmitter 1250 includes a video signal generation unit 1251, a visible light signal generation unit 1252, a visible light synchronization signal generation unit 1253, and a video standard signal transmission unit 1254.

  The video signal generation unit 1251 generates a video signal and outputs it to the video standard signal transmission unit 1254. The visible light signal generation unit 1252 generates a visible light signal as an electrical signal and outputs it to the video standard signal transmission unit 1254. The visible light synchronization signal generation unit 1253 generates a visible light synchronization signal and outputs it to the video standard signal transmission unit 1254.

  The video standard signal transmission unit 1254 outputs the video signal, visible light signal, and visible light synchronization signal generated as described above to the video display 1270 via the video standard transmission path group 1260.

  The video display 1270 includes a video standard signal receiving unit 1271, a video display unit 1272, and a visible light signal transmission unit 1273.

  The video standard signal receiving unit 1271 receives the video signal, the visible light signal, and the visible light synchronization signal from the video standard signal sending unit 1254 via the video standard transmission path group 1260. Then, the video standard signal receiving unit 1271 outputs the video signal to the video display unit 1272, and outputs the visible light signal and the visible light synchronization signal to the visible light signal transmission unit 1273.

  The video display unit 1272 includes, for example, a liquid crystal display, an organic EL display, or a plasma display. When the video signal is received from the video standard signal receiving unit 1271, a video corresponding to the video signal is displayed. When the video display 1270 is a projector or the like, the video display unit 1272 includes a light projection mechanism including a light source and an optical system. When the video signal is received from the video standard signal receiving unit 1271, the video signal is displayed. The image corresponding to is projected on the screen.

  The visible light signal transmitter 1273 acquires the visible light signal and the visible light synchronization signal from the video standard signal receiver 1271. In addition, when the visible light signal transmission unit 1273 receives the visible light synchronization signal, the visible light signal transmission unit 1273 causes the video display unit 1272 to start blinking according to the already acquired visible light signal at the reception timing. Accordingly, the video display unit 1272 displays a video and transmits a visible light signal as a light signal by changing the luminance. Note that the visible light signal transmission unit 1273 may include a light source such as an LED and change the luminance of the light source.

  FIG. 332 is a diagram illustrating a transmission / reception mode between the video standard signal transmission unit 1254 and the video standard signal reception unit 1271.

  The video standard signal sending unit 1254 sends the video signal, the visible light signal, and the visible light synchronization signal to the video standard signal receiving unit 1271 using a plurality of video standard transmission channels included in the video standard transmission channel group 1260.

  When receiving the video signal, the visible light signal, and the visible light synchronization signal, the video standard signal receiving unit 1271 outputs the visible light synchronization signal to the visible light signal transmission unit 1273 in preference to the interpretation of the video signal and the visible light signal. To do. Thereby, it can suppress that the output of a visible light synchronizing signal is delayed by the interpretation of a video signal and a visible light signal.

  FIG. 333 is a diagram illustrating an example of a specific transmission / reception mode between the video standard signal transmission unit 1254 and the video standard signal reception unit 1271.

  The video standard signal sending unit 1254 sends the video signal, the visible light signal, and the visible light synchronization signal to the video standard signal receiving unit 1271 using a plurality of video standard transmission channels included in the video standard transmission channel group 1260. At this time, the video standard signal transmission unit 1254 receives the video signal and the visible light signal through the video standard transmission path used in the video standard among the plurality of video standard transmission paths included in the video standard transmission path group 1260. The image is sent to the video standard signal receiving unit 1271. In addition, the video standard signal transmission unit 1254 sends the visible light synchronization signal to the video standard signal via the video standard transmission path that is not used in the video standard among the plurality of video standard transmission paths included in the video standard transmission path group 1260. Send to receiver 1271.

  FIG. 334 is a diagram illustrating another example of a specific transmission / reception mode between the video standard signal transmission unit 1254 and the video standard signal reception unit 1271.

  The video standard signal sending unit 1254 sends the video signal and the visible light signal to the video standard signal receiving unit 1271 through the video standard transmission path used in the video standard, as described above. On the other hand, for the visible light synchronization signal, the video standard signal transmission unit 1254 may transmit the visible light synchronization signal to the video standard signal reception unit 1271 via a video standard transmission path for future expansion. The video standard transmission path for future expansion is a video standard transmission path that is provided for future expansion according to the standard.

  FIG. 335 is a diagram illustrating another example of a specific transmission / reception mode between the video standard signal transmission unit 1254 and the video standard signal reception unit 1271.

  The video standard signal sending unit 1254 sends the video signal and the visible light signal to the video standard signal receiving unit 1271 through the video standard transmission path used in the video standard, as described above. On the other hand, for the visible light synchronization signal, the video standard signal sending unit 1254 passes through a video standard transmission path (hereinafter referred to as a power transmission transmission path) used for sending power consumed by the video display 1270. The visible light synchronization signal may be sent to the video standard signal receiving unit 1271. As a result, the visible light synchronization signal is transmitted together with the power. That is, the video standard signal sending unit 1254 sends the visible light synchronization signal superimposed on the power.

  FIG. 336A and FIG. 336B are diagrams illustrating the power transmitted in the power transmission line.

  When the visible light synchronization signal is not transmitted through the power transmission line, as shown in FIG. 336A, the voltage defined in the video standard is continuously applied to the power transmission line. The On the other hand, when the visible light synchronization signal is transmitted via the power transmission transmission line, the visible light synchronization signal is transmitted to the voltage defined by the video standard as shown in FIG. 336B. Is superimposed. In this case, the visible light synchronization signal is such that the maximum voltage of the visible light synchronization signal is less than or equal to the rated voltage upper limit of the video standard transmission line and the minimum voltage of the visible light synchronization signal is greater than or equal to the rated voltage lower limit of the video standard transmission line Is superimposed on the power. In this case, the visible light synchronization signal is superimposed on the power so that the average value of the voltage during the period in which the visible light synchronization signal is superimposed is equivalent to the voltage defined in the video standard.

  FIG. 337 is a diagram illustrating another example of a specific transmission / reception mode between the video standard signal transmission unit 1254 and the video standard signal reception unit 1271.

  The video standard signal sending unit 1254 sends the video signal and the visible light signal to the video standard signal receiving unit 1271 through the video standard transmission path used in the video standard, as described above. On the other hand, for the visible light synchronization signal, the video standard signal transmission unit 1254 sends the visible light synchronization signal to the video standard signal reception unit 1271 via the video standard transmission path used for transmission of the vertical synchronization signal in the video standard. It may be sent out. The vertical synchronization signal is a signal for synchronizing the video in the vertical direction. The video standard signal transmission unit 1254 transmits a visible light synchronization signal as a vertical synchronization signal.

  FIG. 338 is a diagram illustrating another example of a specific transmission / reception mode between the video standard signal transmission unit 1254 and the video standard signal reception unit 1271.

  The video standard signal sending unit 1254 sends the video signal and the visible light signal to the video standard signal receiving unit 1271 through the video standard transmission path used in the video standard, as described above. On the other hand, for the visible light synchronization signal, the video standard signal transmission unit 1254 passes through a video standard transmission line (hereinafter referred to as a mixed transmission line) used for transmission of the video signal, the control signal, and the vertical synchronization signal in the video standard. The visible light synchronization signal may be sent to the video standard signal receiving unit 1271. The video standard signal transmission unit 1254 transmits a visible light synchronization signal as a vertical synchronization signal.

  In this case, the video standard signal receiving unit 1271 extracts the visible light synchronization signal from the signal transmitted and received via the mixed transmission path, prior to the interpretation of the video signal and the control signal, The visible light synchronization signal is output to the visible light signal transmission unit 1273.

  As described above, in this embodiment, since the visible light synchronization signal is extracted prior to the interpretation of the video signal and the visible light signal, the output of the visible light synchronization signal is output by the interpretation of the video signal and the visible light signal. The delay can be suppressed.

(Embodiment 18)
The present disclosure relates to a display device capable of outputting a visible light communication signal and a control method thereof.

  For example, Japanese Patent Application Laid-Open No. 2007-43706 and Japanese Patent Application Laid-Open No. 2009-212768 disclose a communication technique using visible light. In Japanese Patent Application Laid-Open No. 2007-43706 and Japanese Patent Application Laid-Open No. 2009-212768, in a video display device including a display, a projector, etc., communication is performed by superimposing communication information by visible light on normal video display. Technology is disclosed.

  The present disclosure provides a display device capable of outputting a visible light communication signal without greatly degrading the image quality of a display image, and reducing a reception error of the output visible light communication signal, and a control method thereof.

  A display device according to the present disclosure is a display device that is capable of outputting a visible light communication signal, and displays a video on a display panel having a display surface for displaying video, and the display panel based on the video signal. A display control unit for controlling the display panel, a backlight having a light emitting surface for illuminating the display surface of the display panel from the back, and a backlight control generated based on the video signal for the visible light communication signal A signal processing unit to be superimposed on the signal, and a light emission surface of the backlight is divided into a plurality of regions, and the light emission is controlled in each of the plurality of regions according to a backlight control signal output by the signal processing unit; and A backlight control unit that provides a period for controlling turning off at different timings in each of the plurality of regions, and the signal processing unit includes: The viewing optical communication signal when superimposed on the backlight control signal, for the signal indicating the extinction of the backlight of the backlight control signal, does not overlap the visible light communication signal.

  The display device according to the present disclosure can output a visible light communication signal without greatly degrading the image quality of a display image, and can reduce reception errors of the output visible light communication signal.

[Knowledge that became the basis of this disclosure]
In the fields of recent display devices, particularly liquid crystal displays and projectors using liquid crystals, a technique called backlight scanning has been adopted to improve image quality. Here, the backlight scan is a backlight control method that improves the slow response speed of the liquid crystal and the motion blur due to the hold type. The display screen is divided into several areas (backlight areas). The light emission of the backlight is controlled so that it is sequentially turned on periodically every time. More specifically, the backlight scan is a control method in which a backlight extinction period is provided and the timing of the extinction period (blanking period) that is periodically performed in each backlight region is different. In general, the blanking period is often controlled so as to match the liquid crystal scanning timing.

  However, as disclosed in JP 2007-43706 A, in visible light communication, a method of superimposing a visible light communication signal by blinking a backlight is used. Therefore, the visible light communication signal cannot be transmitted during the backlight turn-off time, and this stop period causes a signal transmission error. There wasn't.

  Therefore, the present disclosure provides a display device that can output a visible light communication signal without greatly degrading the image quality of a display image and reduce reception errors of the output visible light communication signal.

  Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

  In addition, the applicant provides the accompanying drawings and the following description in order for those skilled in the art to fully understand the present disclosure, and is not intended to limit the claimed subject matter. .

  The eighteenth embodiment will be described below with reference to FIGS. 339 to 346.

[1. Constitution]
FIG. 339 is a schematic diagram illustrating an example of a visible light communication system according to the eighteenth embodiment.

[1.1. Configuration of visible light communication system]
A visible light communication system 1300 illustrated in FIG. 339 includes a display device 1400 and a smartphone 1350.

  The display device 1400 is a television, for example, and can display an image on the display surface 1410. The display device 1400 can also superimpose a visible light communication signal on the display surface 1410.

  The smartphone 1350 is an example of an electronic device that receives a visible light communication signal, and can receive the visible light communication signal transmitted from the display device 1400. Thus, the user of the smartphone 1350 can receive information related to the video displayed on the display device 1400.

  In the present embodiment, the display device 1400 will be described using a monitor that displays an image such as a television or a display as an example. However, the present invention is not limited to this. The display device 1400 may be a device that projects an image like a projector. Moreover, although the smart phone 1350 is mentioned as an example as an electronic device which receives the visible light communication signal which the display apparatus 1400 output, if it is an electronic device which can receive a visible light communication signal, it will not be restricted to a smart phone. For example, the electronic device may be a receiving device compliant with JEITA-CP1222. Moreover, this electronic device is not limited to a smartphone but may be a general mobile terminal. The electronic device may receive the visible light communication signal and obtain the information by decoding the received visible light communication signal.

  An information transmission method for transmitting a visible light communication signal may be based on a standard such as JEITA-CP-1223, which has been developed to an international standard, or IEEE-P802.15 already created. In other words, the electronic device may be configured using a receiving device that supports these standards.

[1.2. Configuration of display device]
FIG. 340 is a block diagram illustrating an example of a schematic configuration of the display apparatus according to the eighteenth embodiment.

  A display device 1400 illustrated in FIG. 340 is a display device that can output a visible light communication signal, and includes a first input unit 1420, a first processing unit 1430, a first control unit 1440, and a display panel 1450. A second input unit 1460, a second processing unit 1470, a second control unit 1480, and a backlight 1490.

  The first input unit 1420 receives a video signal related to a video displayed on the display panel 1450. The video signal is sent from the first input unit 1420 through, for example, an antenna cable, a video signal line, a composite cable, an HDMI (registered trademark) cable, a PJLink cable, a LAN cable, etc. Is input. Here, the video signal may be stored in various recording media using a video recorder or a player.

  The first processing unit 1430 receives a video signal from the first input unit 1420. The first processing unit 1430 performs general image processing such as image quality processing on the video signal. The first processing unit 1430 transmits the video signal subjected to the image processing to the first control unit 1440. In addition, the first processing unit 1430 transmits information indicating the size of the subframe and the video signal, display timing, brightness, and the like to the first control unit 1440 and the second processing unit 1470.

  Note that the first processing unit 1430 may output a duty ratio calculated based on the video signal and a backlight control signal (hereinafter referred to as a BL control signal) for each region to the second processing unit. Good.

  The display panel 1450 is a liquid crystal panel, for example, and has a display surface 1410 for displaying an image.

  The first control unit 1440 is an example of a display control unit, and controls the display panel 1450 to display a video on the display surface 1410 of the display panel 1450 based on the video signal. In the present embodiment, the first control unit 1440 controls to display a video on the display panel 1450 based on the video signal transmitted from the first processing unit 1430. More specifically, the first control unit 1440 performs liquid crystal opening control of the display panel 1450 based on the video signal transmitted from the first processing unit 1430.

  The second input unit 1460 receives a signal used for visible light communication (hereinafter referred to as a visible light communication signal), and transmits the input visible light communication signal to the second processing unit 1470. In the present embodiment, a visible light communication signal is input to the second input unit 1460 through a dedicated cable or a LAN cable, for example, a visible light communication signal created by a PC or the like.

  Note that the visible light communication signal may be superimposed on part of the broadcast radio wave and input to the second input unit 1460 through the antenna cable. In addition, a visible light communication signal stored in various recording media by a video recorder or a player may be input to the second input unit 1460. For example, a video recorder that records a visible light communication signal may be placed on a part of a line such as an HDMI (registered trademark) cable or a PJLink cable and input to the second input unit 1460. In addition, a visible light communication signal separately generated by a PC or the like may be superimposed on the video signal and input to the second input unit 1460 from the video recorder or the playback device.

  In addition to receiving from the outside, the second input unit 1460 uses the information built in the display device, such as the ID of the display device, and reads the server information through the Internet or the like, thereby receiving the visible light communication signal. It may be acquired.

  The second processing unit 1470 encodes the visible light communication signal input from the second input unit 1460 to generate an encoded signal, calculate a duty based on the video signal and / or the visible light communication signal, and the like. In addition, the second processing unit 1470 receives the B.B signal input from the first processing unit 1430. The encoded signal is superimposed on the L control signal.

  In the present embodiment, the encoded signal is described as a signal in which an ON period and an OFF period are mixed at a certain rate. Further, the description will be made assuming that the encoded signal is encoded by the I-4PPM system in principle. Note that the encoded signal may be encoded by a Manchester method or the like. The modulated signal is described as ON / OFF with a modulation rate of 100%, but is not limited thereto. When using High / Low modulation with a modulation rate not 100%, the ON / OFF in the following description may be read as High / Low. The duty of the visible light communication signal not only treats the ON period as a value normalized for the entire signal transmission period, but also corresponds to (High level × High period + Low level × Low period) / (Signal transmission period × High level). May be read and read.

  More specifically, the second processing unit 1470 is an example of a signal processing unit, and performs a process of superimposing a visible light communication signal on a backlight control signal generated based on a video signal. However, when the second processing unit 1470 superimposes the visible light communication signal on the backlight control signal, the second processing unit 1470 does not superimpose the visible light communication signal on the signal indicating that the backlight is turned off in the backlight control signal. . An encoded visible light communication signal (encoded signal) may also be described as a visible light communication signal.

  The second control unit 1480 is an example of a backlight control unit, divides the light emitting surface of the backlight 1490 into a plurality of regions, and outputs a backlight control signal (BL control) output from the second processing unit 1470. In accordance with the signal), light emission is controlled in each of the plurality of regions, and control is performed to provide periods for performing the light-off control at different timings in each of the plurality of regions. In the present embodiment, second control section 1480 controls the brightness and timing of backlight 1490 based on the backlight control signal (BL control signal) transmitted from second processing section 1470.

  The backlight 1490 irradiates the display panel 1450 with light from the back side. More specifically, the backlight 1490 has a light emitting surface that illuminates the display surface 1410 of the display panel 1450 from the back. Accordingly, the viewer can view the video displayed on the display panel 1450.

  In this embodiment mode, the light emission surface of the backlight 1490 is divided into a plurality of regions, and backlight scanning can be realized by sequentially performing light emission control for each region. Note that the plurality of regions on the light-emitting surface of the backlight 1490 correspond to the plurality of regions on the display surface 1410.

[2. Operation of display device]
Next, the operation of the display device 1400 configured as described above will be described.

  The display device 1400 performs backlight scanning in which the entire screen of the display panel 1450 is sequentially scanned with a backlight, and is sequentially turned off in accordance with the writing of the video signal.

  In general, in a display panel composed of liquid crystal, the phase change speed of liquid crystal is slow, and it takes time until the video is switched even if the video signal is switched to display different gradations. Therefore, a backlight scan is performed in order to improve moving image characteristics such as bleeding caused by displaying a video being switched by temporarily turning off the backlight of the display panel while scanning. On the other hand, the scanning speed for switching has been improved year by year, and it has become possible from a normal scanning speed of 60 frames per second to a high scanning speed such as twice or four times as high as 60 frames per second. When scanning at a high speed, smooth moving image characteristics can be obtained by performing frame interpolation between normal frames and switching the images little by little.

  For this reason, the backlight scan that turns off while scanning the backlight is very important for improving the moving image characteristics, and when the visible light communication signal is turned off by the backlight scan, it is better not to emit a signal for moving image characteristics. Good.

  For the above reason, the display device 1400 does not output a visible light communication signal during a light extinction period (hereinafter referred to as a blanking period) by backlight scanning.

  In the following, even when the display device 1400 does not output a visible light communication signal during the blanking period of the backlight control signal (BL control signal), for example, the receiver side such as the smartphone 1350 has a high probability of the visible light communication signal. A method (operation) that can be received by the method will be described.

Example 1
[2.1.1. Example of operation of second processing unit]
FIG. 341A is a diagram illustrating B. FIG. 341B is a diagram illustrating an example of a state before a visible light communication signal is superimposed on an L control signal, and FIG. It is a figure which shows an example of the state which superimposed the visible light communication signal on L control signal.

  In FIG. 341A and FIG. 341B, the backlight 1490 is controlled for each of the eight areas A to H which are display areas of the divided display surface 1410. An example in which L control signals A to H are input is shown. A hatched portion indicates an area where an encoded signal (visible light communication signal) exists.

  The encoded signal shown in FIG. When the L control signals A to H are superposed at different phases and encoded signals having different phases are mixed within the range received by the receiver side, an error (visible light) is detected when decoding at the receiver side. Communication signal reception error).

  Therefore, in this embodiment, as shown in FIG. 341B, the phase of the encoded signal (visible light communication signal) is superimposed in a certain area of the display area.

  Here, the expression of matching the phases has been described with an example of synchronizing the rising timing, but is not limited thereto. Any time may be used as long as the rise time is determined from the state before the rise start to the rise completion state. In addition, since a delay time in the control signal voltage path occurs, it does not mean that synchronization is only achieved when the timings match, and the phase can be changed even when there is a certain delay time within a certain range. Included in the expression to match. The same applies to the following embodiments (Embodiments 18 to 23).

  Here, since the backlight is sequentially turned off for each region by the sequential scanning, it is difficult to superimpose the encoded signal without including any extinguishing period (blanking period). Therefore, in this embodiment, in a specific area (hereinafter referred to as a reference area) among the divided display areas, the encoded signal is superimposed at the same timing immediately after the extinguishing period (blanking period). Note that in other regions other than the specific region (reference region), the encoded signal is superimposed in the same phase as the encoded signal of the reference region, but in the extinguishing period (blanking period) when the backlight is extinguished. The encoded signal is not superimposed.

  In the example shown in FIG. A region C to which the L control signal C is input is set as a reference region, and B.B shown in FIG. After adjusting the superimposed timing of the encoded signal so that the leading (rising timing) P2 of the encoded signal comes to the rising timing P1 of the L control signal C, the encoded signal is changed to B.B. The L control signals A to H are superimposed on the same phase. Then, the second processing unit 1470 is configured such that the B.I. When superimposing the encoded signal on the L control signals A to H, The code signal is superimposed in the ON period of the L control signal, but is not superimposed in the OFF period.

  The reference area is not limited to the area C described above. Hereinafter, examples of reference areas that can be set in the present embodiment will be described. For example, the reference region may be set to the brightest region (region with a short blanking period or region with the highest transmittance of the display panel) among the divided display regions.

  Even when the brightest area is set as the reference area, further measures are required when the position of the reference area is changed for each frame. This is because the position of the encoded signal superimposed for each frame changes, and the center of gravity of the image changes extremely for each frame, which may cause flickering. Alternatively, if the period of the encoded signal to be overlapped overlaps between the regions and one of the overlapping encoded signals is cut off halfway or not overlapped in the first predetermined period, a reception error will occur on the receiving side. This is because it is also possible to occur. Therefore, when the position of the reference region is changed for each frame, a period during which the encoded signal is not superimposed may be provided for at least one frame period.

  When the bright area is set as the reference area, the brightness of the image is determined by the first processing unit 1430 based on the video signal, instead of determining the bright area based on the brightness of the display area for each frame. A bright area may be determined based on the transition of the center of the height.

  If the brightness of the entire display area does not change more than a certain amount, such as when there is no scene change, the area including the brightest part of the display area based on the average of the video signal over a certain period of time It may be set as. Note that the reference area may be determined in advance.

[2.1.2. Effect]
As described above, in this embodiment, the display device is a display device (1400) capable of outputting a visible light communication signal, and is based on the display panel (1450) having a display surface for displaying an image and the image signal. A display control unit (first control unit 1440) for controlling the display panel to display an image on the display surface of the display panel, and a light emitting surface for illuminating the display surface of the display panel (1450) from the back side. A backlight (1490), a signal processing unit (second processing unit 1470) for superimposing the visible light communication signal on a backlight control signal generated based on the video signal, and a backlight (1490) The light emitting surface is divided into a plurality of regions, and each of the plurality of regions is in accordance with a backlight control signal output by the signal processing unit (second processing unit 1470). A backlight control unit (second control unit 1480) for performing light control and providing a period for performing light extinction control at different timings in each of the plurality of regions, the signal processing unit (second processing) Unit 1470), when superimposing the visible light communication signal on the backlight control signal, for the signal indicating that the backlight (1490) is extinguished among the backlight control signals, Do not overlap.

  With this configuration, it is possible to provide a display device that can output a visible light communication signal without greatly degrading the image quality of a display image and reduce reception errors of the output visible light communication signal.

  In addition, the signal processing unit (second processing unit 1470) superimposes the visible light communication signal on the backlight control signal of each of the plurality of regions, and superimposes each of the plurality of regions. The visible light communication signals may be in phase with each other.

  Thereby, the reception error of the visible light communication signal can be suppressed.

  Here, for example, the signal processing unit may match the phase of the visible light communication signal superimposed on each of the plurality of regions with reference to the backlight control signal of a predetermined region among the plurality of regions. Good.

  Thereby, the period of the visible light communication signal that is not superimposed on the blanking period can be minimized.

  Further, the predetermined area may be a brightest area among the plurality of areas, and the predetermined area may be an area corresponding to an end portion of the display surface among the plurality of areas. Good.

  Thereby, the influence of the brightness | luminance fall by the extinction of the backlight by a visible light communication signal can be suppressed.

(Example 2)
Hereinafter, description will be made assuming that the time length of the blanking period is the same in each area of the display area.

The total time during which the backlight 1490 is turned off (total turn-off period) is O of L control signal
This is a period obtained by adding the blanking period, which is the FF period, and the OFF period of the encoded signal.

  Therefore, even if the encoded signal is superimposed immediately after the blanking period ends in the reference region and the encoded signal is completely included from this blanking period to the next blanking period, The period during which the backlight 1490 is turned off increases by the OFF period of the encoded signal superimposed on the L control signal. That is, in the reference area, when the encoded signal is superimposed, the reference area becomes darker than before the encoded signal is superimposed.

  On the other hand, for example, in an area other than the reference area, since the encoded signal is not superimposed on the blanking period, only the OFF period of the encoded signal period on which the overlapping encoded signal is not overlapped with the blanking period is backed from the reference area. The time during which the light 1490 is turned off is shortened. That is, for example, in an area other than the reference area, when the encoded signal is superimposed, the area may become brighter than the reference area.

  In order to improve this, two methods for providing an adjustment period during which the backlight 1490 is turned on or off can be considered. That is, in the first method, since the total light extinction period of the reference region becomes the longest, there is a method of matching the total light extinction period of other regions with the total light extinction period of the reference region. As a second method, there is a method in which the total light extinction period of all regions is combined with the total light extinction period determined based on the original video signal.

[2.2.1. Example of operation of second processing unit according to first method]
First, the operation of the second processing unit 1470 according to the first method will be described with reference to FIGS. 342 and 343.

  FIG. 342 and FIG. 343 are timing charts for explaining the first method in the second example of the eighteenth embodiment. In FIG. 342 (a), B. of the reference area before the encoded signal is superimposed. L control signal is shown, and FIG. 342 (b) shows B.B of the reference region after the encoded signal is superimposed. The L control signal is shown. In FIG. 343 (a), B. of another area before the encoded signal is superimposed. L control signal is shown, and FIG. 343 (b) shows B.B in another region after the encoded signal is superimposed. The L control signal is shown.

  More specifically, in FIG. 342, the second processing unit 1470 displays the B.D. After adjusting the leading edge (rising timing) of the encoded signal at the rising timing (time t12) of the L control signal, An example of superimposing on the L control signal is shown. In FIG. 343, the second processing unit 1470 transmits the encoded signal having the same phase as the encoded signal superimposed on the reference region to the B.D. An example of superimposing on the L control signal is shown.

  That is, in FIG. 342 and FIG. 343, the second processing unit 1470 displays the B.D. An example is shown in which the encoded signal is superimposed on the L control signal in the same phase as the other regions simultaneously with the end of the blanking period of the reference region. Note that priority is given to the blanking period of each region as in the first embodiment, and the encoded signal is not superimposed in the blanking period.

  As shown in FIG. 342 (b), in the reference region, for example, in addition to the blanking period B1 shown as the period from time t11 to time t12, for example, the encoded signal period C1 shown as the period from time t12 to time t14. There is a coded signal extinguishing period T1 in which the OFF periods of the coded signal in the middle are totaled.

  Therefore, in the reference region shown in FIG. 342 (b), for example, the sum of the OFF period of the encoded signal (encoded signal extinction period) in one frame shown as the period from time t11 to time t13 is the encoded signal. When duty is used, it can be expressed as encoded signal extinction period T1 = encoded signal period C1 × (1-Duty).

  As shown in FIG. 342 (b), in the reference region, there is basically no period in which the encoded signal period C1 and the blanking period B1 overlap, so that the total extinction period T2 of one frame = blanking period B1 + The encoded signal extinction period T1. That is, the total extinguishing period of the reference area is the longest compared to other areas.

  On the other hand, in areas other than the reference area, the encoded signal period and the blanking period may overlap. As described above, the blanking period is B.B. Since the L control signal has priority over the encoded signal, the encoded signal is not superimposed.

  Therefore, as shown in (b) of FIG. 343, in other regions other than the reference region, for example, as a period from time t22 to time t23 in an encoded signal period C1 indicated as a period from time t21 to time t24. The total turn-off period is shorter than the reference area by the OFF period of the encoded signal of the encoded signal period C1 that overlaps the blanking period B2 shown.

  Here, the total of the OFF periods of the encoded signal in the encoded signal period C1 (encoded signal extinction period) is (encoded signal extinction period) = B2 where the overlapping period of the encoded signal and the blanking period is B2. (Encoded signal period C1-Blanking period B2) × (1-Duty).

  By the way, as described above, if the total light extinction period is different for each area of the screen (display area), the luminance varies depending on the area, so that the image quality is deteriorated.

  Therefore, the second processing unit 1470 operates according to the first method that provides an adjustment period during which the backlight 1490 is turned on or off, thereby matching the total extinction period of each region in the screen.

  More specifically, the second processing unit 1470 performs total extinction per frame of the reference area in the other area according to the first method of matching the total extinction period of the other area with the total extinction period of the reference area. An adjustment period for adjusting the difference from the period is provided. As described above, in this embodiment, the time length of the blanking period in each region is the same as a premise.

  Here, in FIG. 343 (b), the adjustment period A1 shown as the period from time t24 to time t26 can be expressed as a blanking period B2 × (1-Duty). That is, the adjustment period in each region other than the reference region can be calculated from the blanking period, the encoded signal period, and the phase of the encoded signal in each region including the reference region. FIG. 343 (b) shows an example where the adjustment period is arranged in one frame indicated as a period from time t21 to time t25.

  As described above, the display device 1400 according to the present embodiment causes the second processing unit 1470 to provide the adjustment period using the first method. Thereby, the display device 1400 converts the encoded signal into the B.P. Although superimposing on the L control signal reduces the brightness of the entire screen (display area) by a certain amount, the encoded signal can be output without greatly changing the image quality itself.

  Note that when the adjustment period is provided immediately after the encoded signal period, the second processing unit 1470 is preferable because it can be stably disposed in a place as close as possible to the blanking period where the phase change of the liquid crystal of the display panel 1450 is large. This is not a limitation. The second processing unit 1470 may provide the adjustment period before the time when the next encoded signal is superimposed.

[2.2.2. Example of operation of second processing unit by second method]
Next, the operation of the second processing unit 1470 according to the second method will be described.

  In general, the adjustment period during which the backlight 1490 is turned on or off in order to adjust the total extinction period can be defined as follows, for example. That is, the extinguishing period (blanking period and black video period) of the original backlight 1490 based on the video signal is T4, and the encoded signal period in the encoded signal period that does not overlap the blanking period in the encoded signal period. If the total OFF period is T5 and the blanking period after the visible light communication signal is superimposed is T6, the adjustment period can be expressed as T4-T5-T6. As described above, it is desirable to install the adjustment period as close as possible to the blanking period.

  For example, in the reference region, T5 first calculates the total OFF period of the encoded signal period in the encoded signal period, and then subtracts the total OFF period of the encoded signal period that overlaps the blanking period. Can be calculated.

  Hereinafter, a specific example of the operation of the second processing unit 1470 according to the second method will be described with reference to FIGS. 344A to 345D.

  344A to 345D are timing charts for explaining the second method in example 2 of embodiment 18. FIG.

  First, an operation in which the second processing unit 1470 provides an adjustment period by the second method when the encoded signal period and the blanking period do not overlap will be described with reference to FIGS. 344A to 344D.

  In FIG. 344A to FIG. 344D, B. before the encoded signal is superimposed on (a) in the upper stage. L control signal is shown, and B. to B.e. L control signal and B.B adjusted by the second method. L control signal is shown. In the figure, the blanking period is shown as B1, and the encoded signal period is shown as C1.

  B. a coded signal superimposed by the second method; There are four methods shown in FIGS. 344A to 344D for adjusting the L control signal depending on the sum (time sum) of the adjustment period, the encoded signal period, and the blanking period, and the positive / negative relationship of the adjustment period. Divided. Hereinafter, each case will be described.

[Adjustment method when encoded signal period and blanking period do not overlap (Case 1)]
FIG. 344A shows an example where the adjustment period is 0 or more and (adjustment period + encoded signal period + blanking period) is 1 frame or less.

  As shown in the upper part of FIG. 344A (b), a part of the adjustment period is arranged in the period from the end time P2 of the blanking period B1 to the start time P3 of the encoded signal period C1, and the rest of the adjustment period Is placed after the encoded signal period, preferably immediately after (time P5).

  The second processing unit 1470 provides the adjustment period shown in the upper part of FIG. 344A (b), so that the encoded signal superimposed on the B.B. Adjust the L control signal.

  In this way, the second control unit 1480 can adjust the adjusted B.P. In accordance with the L control signal, the backlight 1490 is turned off after the blanking period B1 and before the start of the encoded signal period C1, and after the encoded signal period C1 and the encoded signal period C1, the adjustment period P2 to P3 The light is turned off until the period is reduced.

  When the adjustment period is shorter than the period between P2 and P3, the adjustment period may be provided only between P2 and P3. When P2 = P3, the entire adjustment period may be provided after the end of the encoded signal period C.

[Adjustment method when encoded signal period and blanking period do not overlap (Case 2)]
FIG. 344B shows an example where the adjustment period is 0 or more and (adjustment period + encoded signal period + blanking period) is longer than the length of one frame.

  As shown in the upper part of (c) of FIG. 344B, a part of the adjustment period is arranged in a period from the end time P2 of the blanking period B1 to the time P3 when the encoded signal period C1 starts, and the rest of the adjustment period Are arranged starting from the end time P4 of one frame.

  The second processing unit 1470 provides an adjustment period shown in the upper part of (c) of FIG. 344B, so that the encoded signal is superimposed on the B.B. Adjust the L control signal.

  In this way, the second control unit 1480 can adjust the adjusted B.P. According to the L control signal, after the blanking period B1, the backlight 1490 is turned off until the start time P3 of the encoded signal period C1, and the period from the time P5 to the time P4 before the end of the encoded signal period C1 is also turned off. In other words, the adjusted B.D. is adjusted so that the encoded signal is not transmitted from the time P5 that overlaps the rest of the adjustment period and the encoded signal period C1 to the end time P10 of the encoded signal period C1. It is not superimposed on the L control signal (or the signal is turned off).

  When P2 = P3 (same time), the entire adjustment period is provided after the encoded signal period.

[Adjustment method when encoded signal period and blanking period do not overlap (Case 3)]
FIG. 344C shows an example where the adjustment period is less than 0 and (adjustment period + encoded signal period + blanking period) is equal to or shorter than the length of one frame. Here, the adjustment period smaller than 0 means an adjustment period during which the backlight 1490 is turned on.

  As shown in the upper part of (d) of FIG. 344C, the adjustment period starts from the end time P2 of the blanking period B1, and the time period corresponding to the absolute value of the adjustment period (the period from time P6 to time P2) ).

  The second processing unit 1470 provides the adjustment period shown in the upper part of (d) of FIG. 344C, so that the encoded signal is superimposed on the B.D. Adjust the L control signal.

  In this way, the second control unit 1480 can adjust the adjusted B.P. In accordance with the L control signal, the backlight 1490 is turned on during a period from time P6 to time P2 in the blanking period B1.

  If P2 = P3, the entire adjustment period may be arranged after the encoded signal period C1. When the adjustment period is longer than the blanking period, the duty ratio of the encoded signal is considered, and the encoded signal is not superimposed on the end of the encoding period C1 until the insufficient lighting time can be secured. It is good also as a light extinction period.

[Adjustment method when encoded signal period and blanking period do not overlap (Case 4)]
FIG. 344D shows an example in which the adjustment period is smaller than 0 and (adjustment period + encoded signal period + blanking period) is longer than the length of one frame.

  As shown in the upper part of (e) of FIG. 344D, the adjustment period starts from the time P2 when the blanking period B1 ends, and the period (from time P7 to time P2) goes back by the time corresponding to the absolute value of the adjustment period. Period). Thereby, the backlight 1490 in the period from time P7 to time P2 in the blanking period B1 is turned on.

  Although the blanking period and the encoded signal period do not overlap, the absolute value of the adjustment period may be longer than the blanking period in view of the negative adjustment period. . In this case, if all the adjustment periods are arranged starting from the time P2 at which the blanking period B1 ends, the time P7 becomes the time P1 or the time before the time P1, and the blanking period does not exist. In addition to lighting all during the blanking period, when the backlight 1490 still needs to be lit (brighten the area), the encoded signal is used as the remaining period excluding the blanking period in the adjustment period. What is necessary is just to light the OFF period of the encoding signal of a period. In other words, the remaining period of the adjustment period may be arranged until a period (time P8) retroactive from time P9, and the lighting of the encoded signal may be stopped to continue lighting.

  Here, the time P8 needs to be determined because the original blanking period B1 is equal to the sum of the OFF periods obtained by subtracting the period between the times P8 and P9 from the encoded signal period C1. Specifically, the time P8 can be calculated based on the relationship of blanking period B1 = (encoded signal period C1- (time P9-time P8)) × (1-Duty).

  Thereby, the second processing unit 1470 allows the second control unit 1480 to continue to turn on the backlight 1490 from time P8 until the start of the next blanking period in addition to the blanking period B1. . The L control signal can be adjusted.

  When P2 = P3, all the adjustment periods may be arranged after the encoded signal period C1.

  Next, an operation in which the second processing unit 1470 provides an adjustment period by the second method when the encoded signal period and the blanking period overlap will be described using FIGS. 345A to 344D.

  345A to 345D, the upper stage (a) B. L control signal is shown, and in the lower (b) to (e), B. L control signal and B.B adjusted by the second method. L control signal is shown. In the figure, a blanking period is shown as B1, an encoded signal period is shown as C1, and time Q1 to time Q6 are shown as one frame period.

  B. a coded signal superimposed by the second method; The method for adjusting the L control signal is divided into four methods shown in FIGS. 345A to 345D depending on the sum of the adjustment period, the encoded signal period, and the blanking period, and the positive / negative relationship of the adjustment period. Hereinafter, each case will be described.

[Adjustment method when encoded signal period and blanking period overlap (case 1)]
FIG. 345A shows an example in which the adjustment period is 0 or more and (adjustment period + encoded signal period + blanking period) is 1 frame or less.

  As shown in the upper part of FIG. 7A (b), the adjustment period is arranged starting from the time Q4 when the encoded signal period C1 ends.

  The second processing unit 1470 provides the adjustment period shown in the upper part of (b) of FIG. 344A, so that the period from time Q4 to time Q5, which is the adjustment period, as shown in the lower part of (b) of FIG. 345A. In addition, the encoded signal is not superimposed in the period from time Q2 to time Q3 that overlaps the blanking period B1. Adjust the L control signal.

  In this way, the second control unit 1480 can adjust the adjusted B.P. In accordance with the L control signal, the backlight 1490 is turned off during the period from time Q4 to time Q5 together with the period from time Q2 to time Q3 overlapping with the blanking period B1. Note that, during the period from time Q4 to time Q5, the backlight 1490 is turned off and the encoded signal is not transmitted.

[Adjustment method when the encoded signal period and the blanking period overlap (case 2)]
FIG. 345B shows an example in which the adjustment period is 0 or more and (adjustment period + encoded signal period + blanking period) is 1 frame or more.

  As shown in the upper part of (c) of FIG. 345B, the adjustment period is arranged in the period from the time Q8 to the time Q6, which is the adjustment period, starting from the time Q6 when the encoded signal of the next frame starts. .

  The second processing unit 1470 provides the adjustment period shown in the upper part of (c) of FIG. 345B, so that the period from time Q8 to time Q6, which is the adjustment period, as shown in the lower part of (c) of FIG. 345B. In addition, the encoded signal is not superimposed in the period from time Q2 to time Q3 that overlaps the blanking period B1. Adjust the L control signal.

  In this way, the second control unit 1480 can adjust the adjusted B.P. In accordance with the L control signal, the backlight 1490 is turned off during the period from time Q8 to time Q6 together with the period from time Q2 to time Q3 overlapping with the blanking period B1. Note that, during the period from time Q8 to time Q6, the backlight 1490 is turned off and the encoded signal is not transmitted.

[Adjustment method when the encoded signal period and the blanking period overlap (case 3)]
FIG. 345C shows an example where the adjustment period is less than 0 and (adjustment period + encoded signal period + blanking period) is equal to or longer than the length of one frame.

  As shown in the upper part of (d) of FIG. 345C, the adjustment period is set as the starting point at the end time Q3 of the blanking period B1, and the period corresponding to the absolute value of the adjustment period is arranged in a period that goes back.

  The second processing unit 1470 provides the adjustment period shown in the upper part of (d) of FIG. 345C, so that the period from time Q9 to time Q3, which is the adjustment period, as shown in the lower part of (d) of FIG. 345C. So that the backlight 1490 is turned on. The L. control signal is adjusted and the encoded signal is not superimposed in the ranking period B1. Adjust the L control signal.

  In this way, the second control unit 1480 can adjust the adjusted B.P. In accordance with the L control signal, the backlight 1490 is turned on during the period from time Q9 to time Q3.

  Note that the encoded signal may be superimposed on the adjustment period. In this case, the adjustment period may be lengthened by the total of the OFF period of the encoded signal. Further, when the adjustment period is longer than the blanking period, the shortage of the lighting time in the adjustment period is encoded in a predetermined period that goes back from the end time of the encoding period C1 based on the duty ratio of the encoded signal. The backlight 1490 may be turned on by not superimposing signals.

[Adjustment method when the encoded signal period and the blanking period overlap (case 4)]
FIG. 345D shows an example where the adjustment period is smaller than 0 and (adjustment period + encoded signal period + blanking period) is longer than the length of one frame.

  As shown in the upper part of (e) of FIG. 345D, the adjustment period is arranged in a period up to time Q10 starting from the end time Q3 of blanking B1 and going back the time corresponding to the absolute value of the adjustment period.

  Thereby, the backlight 1490 in the period from the time Q10 to the time Q3 overlapping with the blanking period B1 is turned on.

  Note that this adjustment period may be lengthened by the total of the OFF period of the encoded signal, and the encoded signal may be superimposed on the adjustment period.

  Similarly to the description in (e) of FIG. 344D, when the adjustment period is very long and the absolute value is larger than the blanking period B1, the remainder of the adjustment period excluding the blanking period B1 is left. For this period, the OFF period of the encoded signal in the encoded signal period may be lit.

  Here, the time Q11 needs to be determined because the original blanking period B1 is equal to the sum of the OFF periods obtained by subtracting the period between the times Q11 and Q12 from the encoded signal period C1. Specifically, the time Q11 can be calculated based on the relationship of blanking period B1 = (encoded signal period C1- (time Q12-time Q11)) × (1-Duty).

  As a result, the second processing unit 1470 allows the second control unit 1480 to continue to be lit from time Q11 to the start time Q7 of the next blanking period in addition to the blanking period B1. The L control signal can be adjusted.

[2.2.3. Effect]
As described above, according to the present embodiment, the backlight control method for improving moving image characteristics such as backlight scanning and the transmission of the visible light communication signal using the backlight are turned off by the encoded signal of the visible light communication. It is possible to achieve both by making the period uniform or performing adjustment to return to the same level as the original video signal.

  Here, for example, in the display device according to the present embodiment, the signal processing unit (second processing unit 1470) is configured to display the backlight control signal when superimposing the visible light communication signal on the backlight control signal. Of these, when there is a region of the plurality of regions in which the period of the signal indicating that the backlight is turned off and the period of the visible light communication signal to be superimposed are overlapped, A lighting adjustment period for adjusting the luminance of the overlapping region may be provided, and the on / off of the backlight control signal may be adjusted in the lighting adjustment period.

  Accordingly, in the region where the visible light communication signal period and the backlight extinguishing period overlap, an adjustment period is provided to When a visible light communication signal (encoded signal) is superimposed on the L control signal, the luminance difference in the display area can be made inconspicuous.

  In this embodiment, the reference area is described as a bright area, but may be read as an area where the opening of the display panel 1450 is set to be large.

(Example 3)
[2.3.1. Example of operation of second processing unit by second method]
In the second embodiment, the brightness of the display surface 1410 (display area) of the display panel 1450 is made uniform by providing an adjustment period in which the backlight 1490 is turned on or off. However, the present invention is not limited to this.

  In this embodiment, a method without an adjustment period will be described with reference to FIG.

  FIG. 346 is a diagram illustrating B.3 in Example 3 of the eighteenth embodiment. It is a timing chart for demonstrating the method of superimposing a visible light communication signal on L control signal. Here, (a) of FIG. The L control signal is shown. In the present embodiment, the description will be made assuming that the signal is detected only by the rising signal of the waveform.

  As shown in FIG. 346, even if the adjustment period is not provided, the luminance of the region is adjusted by changing the duty ratio of the visible light communication signal, that is, the high period of the signal by an amount corresponding to the adjustment period. Good.

  Specifically, for example, when the adjustment period in the second embodiment is a positive number, that is, when adjustment is performed to turn off the backlight 1490, as shown in FIG. The High period of the L control signal may be shortened.

  For example, when the adjustment period in the second embodiment is a negative number, that is, when adjustment is performed to turn on the backlight 1490, as shown in FIG. The High period of the L control signal may be lengthened.

It should be noted that for each display area, B.I. A case where the duty ratio of the L control signal changes is also conceivable. In this case, B.I. In order to drive the L control signal at a constant duty ratio in the plane, the adjustment period in the second embodiment calculated by redoing the calculation including the change in the duty ratio, and the high level of the visible light communication signal as in the present embodiment. A method for changing the period may be used in combination.

Further, in the above description, control of the high period of the backlight 1490 (PWM control: Pul
Although in-plane uniformity and prevention of deterioration of image quality have been described by controlling luminance using se width modulation), the present invention is not limited to this. The second control unit 180 that performs backlight control may bring the luminance in the visible light communication region closer to the luminance in the region other than the visible light communication region by controlling the current supplied to the backlight 1490 in each region. Furthermore, the luminance in the visible light communication region may be made closer to the luminance in the region other than the visible light communication region by a combination of PWM control of the backlight 1490 and current control.

[2.3.2. Effect]
As described above, according to the present embodiment, the backlight control method for improving moving image characteristics such as backlight scanning and the transmission of the visible light communication signal using the backlight are turned off by the encoded signal of the visible light communication. It is possible to achieve both by making the period uniform or performing adjustment to return to the same level as the original video signal.

  In this embodiment, the signal detection is performed only by the rising signal, but the present invention is not limited to this. B. Maintain the position of the falling part and change the rising position. In the case of the L control signal, the signal may be detected by the falling signal. In the present embodiment, B.I. The encoded signal is superimposed with reference to the rising edge of the L control signal. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

[3. effect]
As described above, according to the present embodiment, a display device that can output a visible light communication signal without greatly degrading the image quality of a display image and reduce reception errors of the output visible light communication signal is realized. can do.

(Embodiment 19)
In the eighteenth embodiment, B.I. The operation of the display device 1400 when the encoded signal period is shorter than the ON period of the L control signal has been described. In the present embodiment, B.I. An operation of the display device 1400 when the encoded signal period is longer than the ON period of the L control signal will be described.

[1. Operation of display device]
Hereinafter, the operation of the second processing unit 1470 will be mainly described.

  FIG. 347 is a flowchart for explaining the operation of the second processing unit in the nineteenth embodiment.

  First, in step S1301, the second processing unit 1470 re-encodes the visible light communication signal. More specifically, the second processing unit 1470 encodes the visible light communication signal, and then generates (re-encodes) an encoded signal to which a header or the like is added. Second processing unit 1470 calculates the transmission time of the encoded signal based on the carrier frequency of the encoded signal.

  Next, in step S1302, the second processing unit 1470 determines that the encoded signal length is B.B. It is determined whether or not it is longer than the ON period (lighting time) of the L control signal.

  More specifically, the second processing unit 1470 calculates the B.B. Based on the duty ratio of the L control signal, the time during which the backlight 1490 is turned on (lighting time) is compared with the transmission time length of the encoded signal (encoded signal length). If the second processing unit 1470 determines that the transmission time length of the encoded signal is shorter (No in S1302), the process proceeds to step S1306 and determines that the transmission time length of the encoded signal is longer (Yes in S1302), the process proceeds to step S1303.

  Next, in step S1303, the second processing unit 1470 determines whether to perform visible light communication. If it is determined that the visible light communication is to be performed (Yes in S1303), the second processing unit 1470 proceeds to step S1304. If it is determined that the visible light communication is not to be performed (No in S1303), the second processing unit 1470 proceeds to step S1309.

  Next, in step S1304, the second processing unit 1470 re-encodes the visible light communication signal. Specifically, the second processing unit 1470 is configured such that when the header portion is encoded with a signal arrangement that cannot be normal data, the signal duty ratio of the header portion is substantially OFF. Recreate (re-encode). Thereafter, the second processing unit 1470 outputs the last signal of the header portion (the signal indicating the ON state existing at the end) to the B.B. A process for advancing the transmission start time of the encoded signal is performed so as to match the rising timing of the L control signal. Details will be described later.

  In step S1305, the second processing unit 1470 determines that the encoded signal length is B.B. It is determined whether or not it is longer than the ON period (lighting time) of the L control signal.

  More specifically, the second processing unit 1470 has the B.B. The lighting time of the backlight 1490 according to the duty ratio of the L control signal is compared with the transmission time length of the encoded signal. If the second processing unit 1470 determines that the transmission time length of the encoded signal is shorter (No in S1305), the second processing unit 1470 determines that the transmission time length of the encoded signal is longer. In the case (Yes in S1305), the process proceeds to step S1307.

  Here, in step S1306, the second processing unit 1470 determines that the B.I. The encoded signal is superimposed on the portion other than the blanking period of the L control signal (that is, the ON period of the BL control signal) and output to the second control unit 1480, and the process is terminated.

  On the other hand, in step S1307, the second processing unit 1470 determines whether to divide the encoded signal. More specifically, first, second processing unit 1470 compares the transmission time length of the re-encoded encoded signal with the lighting time of backlight 1490. Then, if the transmission time length of the encoded signal is longer, the second processing unit 1470 determines to divide the encoded signal (Yes in S1307), proceeds to step S1308, and transmits the transmission time length of the encoded signal. If is shorter, it is determined that the encoded signal is not divided (No in S1307), and the process proceeds to step S1309.

  Next, in step S1308, the second processing unit 1470 divides the encoded signal so that the length is within the data length that is within the backlight lighting period. Then, the second processing unit 1470 adjusts the encoded signal so as to be superimposed on a part other than the blanking period of the backlight control signal (ON period of the BL control signal), and ends the process.

  In step S1309, second processing unit 1470 does not transmit the encoded signal to second control unit 1480. That is, the transmission of the visible light communication signal is canceled.

[2. Details of operation]
Next, details (specific example) of the operation of the display device 1400 according to Embodiment 19 will be described with reference to FIGS. 348A to 348D and 349.

[2.1. Specific Example 1]
FIGS. 348A to 348D are the same as those in FIG. It is a figure for demonstrating the specific method of superimposing an encoding signal on L control signal.

  In the present embodiment, second processing section 1470 encodes the visible light communication signal using an encoding method such as 4PPM or I-4PPM. This is because, when encoding is performed using 4PPM, I-4PPM, or the like, it can be relatively eased that the luminance greatly changes depending on the signal, and the luminance can be prevented from becoming unstable. Note that the visible light communication signal may be encoded using an encoding method such as the Manchester method.

  For example, as shown in FIG. 348A, the encoded signal includes a header 1310 and a data portion 1311 storing a code word and the like. It is assumed that an impossible signal arrangement is used for the header 1310 as a data signal. Here, in the case of encoding using I-4PPM, the ratio of the high period is determined to be 75% in principle. Further, it is common to put the ON state in the header in a form that continues for 3 slots (minimum unit of encoded signal) or more. In many cases, the end of the header is set to the OFF state as the header delimiter.

  In FIG. The case where the encoded signal period is shorter than the ON period of the L control signal is shown. In other words, as shown in FIG. If it is shorter than the period excluding the blanking period in one frame of the L control signal (that is, the ON period of the BL control signal), the B.L. The encoded signal can be superimposed without problems during the ON period of the L control signal.

  In contrast, B.I. If the encoded signal period is longer than the ON period of the L control signal, the entire encoded signal including the header is set to B.B. Since it cannot be included in the ON period of the L control signal, the encoded signal is divided and included as described in step S1307 above.

  In FIG. 348C, the entire encoded signal including the header is shown in FIG. Since the length of one frame of the L control signal is exceeded, An example in which the encoded signal is divided and superimposed in the ON period of the L control signal is shown. Specifically, the data portion 1311 of the encoded signal is divided into a data portion 1311-1 and a data portion 1311-2, including a header 1310 and a header 92, respectively. It is superimposed on the ON period of the L control signal. The header 92 is a data portion 1311 obtained by dividing the data portion 1311-2, and includes a determination signal indicating that the data is a continuation of the data portion 1311-1.

  B. When the encoded signal period is longer than the ON period of the L control signal, as shown in FIG. The data portion 1311 is stored in the B.B. It may be superimposed on the ON period of the L control signal.

[2.2. Specific Example 2]
Next, a mode different from FIG. 348D will be described. That is, B.I. When the encoded signal period is longer than the ON period of the L control signal, only the header portion of the encoded signal is changed to B.B. A specific example in the case of superimposing on the blanking period of the L control signal will be described.

  FIG. 349 is a diagram illustrating the process of B.E. It is a figure for demonstrating another specific method of superimposing an encoding signal on L control signal.

  FIG. 349 (a) shows an encoded signal encoded by I-4PRM.

  As shown in FIG. 349 (b), only the header part of FIG. 349 (a) may be re-encoded by changing the encoding method from I-4PPM to 4PPM. In this case, as shown in FIG. 349 (b), the header that has been in the ON state and has finally transitioned to the OFF state is changed to a header that has been in the OFF state and has finally transitioned to the ON state.

  Then, as shown in FIG. 349 (c), the encoded signal shown in FIG. It is superimposed on the L control signal. In the example shown in (c) of FIG. 349, an encoded signal including a header 1330 that is an OFF state signal, a header 1321 that is an ON state signal, and data 1322 is B.P. Superposed on the L control signal.

  More specifically, the second processing unit 1470 encodes the visible light communication signal to generate an encoded signal, superimposes the encoded signal on the backlight control signal as the visible light communication signal, and outputs the encoded signal. When superimposing on the backlight control signal, the signal period indicating that the backlight is turned off in the backlight control signal overlaps with the period of the encoded signal to be superimposed, and a plurality of areas When there is a region, the header portion of the encoded signal is superimposed on the backlight control signal during the signal period indicating that the backlight 1490 is turned off, and the portion other than the header portion of the encoded signal is turned off. Is superimposed on the backlight control signal in a period other than the signal period indicating.

  As a result, B.I. Even when the encoded signal period is longer than the ON period of the L control signal, The data portion of the encoded signal can be superimposed in the ON period of the L control signal.

  That is, for example, as shown in FIG. By superimposing on the blanking period of the L control signal, the encoding time length can be saved (shortened).

  When the adjustment period described in the eighteenth embodiment is provided, for example, the header 1310 of the encoded signal in FIG. It is necessary to subtract from the adjustment period the period during which lighting is performed in the blanking period superimposed on the blanking period of the L control signal.

  However, for example, as shown in FIG. 349 (c), when determining the phase by matching the last time of the header 1330 of the encoded signal (the time when the header signal is finally turned on) with the end time of the blanking period, Since it is not lit during the blanking period, there is no need to subtract from the adjustment period.

[3. Effect]
As described above, according to the present embodiment, a display device that can output a visible light communication signal without greatly degrading the image quality of a display image and reduce reception errors of the output visible light communication signal is realized. can do.

  In the present embodiment, an example using a header of an encoded signal encoded using a general 4PPM encoding method has been described, but the present invention is not limited to this. For example, when the average duty ratio of the header of the encoded signal is high, a header obtained by reversing the ON signal and the OFF signal may be raised during the blanking period. In this case, as described above, it is desirable to adjust the decrease period of the blanking period during the adjustment period.

  B. When the entire encoded signal including the header can be superimposed in the ON period of the L control signal (lighting period of the backlight 1490), the encoding may be performed so that the header has a high duty ratio.

  Even when the header is superimposed on the blanking period, it may not be able to be entered depending on the length of the blanking period. In that case, several types of headers may be prepared and used according to the length of the blanking period.

(Embodiment 20)
In the present embodiment, B.I. A method of superimposing an encoded signal after dividing a plurality of areas of the display area into several groups so that the encoded signals of the entire encoded signal period can be superimposed in the ON period of the L control signal will be described.

[1. Operation of second processing]
Hereinafter, a method for determining the timing for superimposing the encoded signal around the brightest area based on the brightness of the area will be described as an example.

  FIG. 350 is a flowchart for explaining the operation of the second process in the twentieth embodiment.

  First, in step S1311, the second processing unit 1470 encodes a visible light communication signal. More specifically, the second processing unit 1470 encodes the visible light communication signal and then generates an encoded signal to which a header or the like is added. Second processing unit 1470 calculates the transmission time of the encoded signal based on the carrier frequency of the encoded signal.

  Next, in step S1312, the second processing unit 1470 divides the display area into a plurality of areas.

  Next, in step S1313, the second processing unit 1470 detects a bright display area. More specifically, the second processing unit 1470 detects the brightness of each divided area, and selects the brightest area for display based on this. Here, the bright display means B.I. It means not the place where the duty ratio of the L control signal is large but the brightest signal level indicating the light emission energy of the image. The detection of bright places will be described in detail later.

  Next, in step S1314, the second processing unit 1470 aligns the phase of the encoded signal with a bright display area. More specifically, the second processing unit 1470 performs B.D. B. of all the areas or a part of selected areas (a plurality of selected areas) according to the timing of the L control signal. An encoded signal having the same phase is superimposed on the L control signal.

  However, as in the other embodiments, B.I. The encoded signal is not superimposed in the blanking period of the L control signal. This is because each B.I. This is equivalent to the operation of calculating the AND of the L control signal and the encoded signal. In addition, you may perform operation | movement of step S1301-S1309 of FIG. 347 as needed.

  Next, in step S1315, the second processing unit 1470 determines whether the encoded signal and the blanking period overlap. More specifically, the second processing unit 1470 performs, for each region, the encoded signal period, B.I. It is determined whether there is an overlapping portion between the blanking period of the L control signal and the encoded signal period and B.B. When the blanking period of the L control signal does not overlap (Yes in S1315), the process proceeds to step S1316, and the second processing unit 1470 converts the encoded signal to B.B. Superimpose it on the L control signal and finish the process. On the other hand, if there is an overlapping portion (No in S1315), the process proceeds to S1317.

  On the other hand, in step S1317, the second processing unit 1470 determines whether to perform a visible light communication signal. When the visible light communication is not performed (No in S1317), the process proceeds to step S1318. When performing a visible light communication signal (Yes in S1317), the process proceeds to step S1320, and the second processing unit 1470 adjusts the duty ratio so as not to transmit the encoded signal, and ends the process.

  Next, in step S1318, the second processing unit 1470 changes the phase of the encoded signal, and changes the phase of the encoded signal to B.B. It is superimposed on the L control signal.

  Next, in step S1319, the second processing unit 1470 determines whether the blanking period overlaps the bright area. If they do not overlap (No in S1319), the process proceeds to step S1320. If they overlap (Yes in S1319), the process proceeds to S1321.

  Next, in step S1321, the second processing unit 1470 determines whether the processing has been completed for each region. If the process has not ended (No in S1321), the process returns to S1315. If the processing has been completed (Yes in S1321), the process proceeds to step S1322.

  Next, in step S1322, the second processing unit 1470 determines whether there is an area where the encoded signal is not superimposed. If there is a region that is not superimposed (No in S1322), the process proceeds to step S1313. If there is no overlapping area (Yes in S1322), the process ends.

[2. Details of operation]
Next, details (specific example) of the display device 1400 of Embodiment 20 will be described with reference to FIGS. 351 and 352.

  FIG. 351 is a timing chart showing an example of area group division in the twentieth embodiment, and FIG. 352 is a timing chart showing another example of area group division in the twentieth embodiment. In FIG. 351 and FIG. 352, a shaded portion (hatched portion) indicates a period in which the encoded signal is superimposed (encoded signal period).

  For example, as shown in FIG. 351, the plurality of display areas are divided into three groups. Specifically, region A, region B, and region C are divided into group G1, region F, region G, and region H are divided into group G2, and region D and region E are divided into group G3. Then, as shown in FIG. 351, in each group, encoded signals are superimposed at the same timing within the same period. For example, in group G1, the brightest region C is superimposed on the basis, and in group G2, the brightest region E is superimposed on the group.

  Note that as shown in FIG. 352, a plurality of display areas may be divided into two groups. That is, the region A, the region B, the region C, and the region D may be divided into the group G1, and the region E, the region F, the region G, and the region H may be divided into the group G2. In each group, the encoded signal is superimposed at the same timing within the same period.

[3. Effect]
As described above, according to the display device of the present embodiment, the signal processing unit (second processing unit 1470) includes the backlight of each of a plurality of groups including a plurality of neighboring regions among the plurality of regions. The visible light communication signal is superimposed on the control signal, and the visible light communication signals superimposed on each of the plurality of groups are in phase with each other, and the backlight control signal of each of the plurality of groups All of the corresponding visible light communication signals are superimposed in the period for controlling the light emission of the backlight (1490).

  As a result, the display device is Since the encoded signal of the entire encoded signal period can be superimposed on the ON period of the L control signal, reception errors of the output visible light communication signal can be reduced. In other words, B. Since the visible light communication signal can be superimposed without being lost during the ON period of the L control signal, reception errors of the output visible light communication signal can be reduced.

  The signal processing unit (second processing unit 1470) is superimposed on each of the plurality of groups with reference to the backlight control signal in a predetermined region among the plurality of regions included in the plurality of groups. The phase of the visible light communication signal may be matched.

  Thereby, this display apparatus can output a visible light communication signal without missing more for each of a plurality of selected groups.

  Here, the predetermined area is the brightest area among the plurality of areas.

  Thus, the display device 1400 can make the difference in luminance in the display area less noticeable.

  Further, the phase of the visible light communication signal superimposed on one of the plurality of groups is different from the phase of the visible light communication signal superimposed on the other of the plurality of groups.

  As a result, the display device 1400 can output the visible light communication signal without further dropping for each of the plurality of selected groups.

  As described above, there may be a case where the group division of the area cannot be performed. That is, there may be a region where encoded signals having the same phase do not enter even if the region is divided into groups. The operation in this case will be described below.

  FIG. 353 is a timing chart illustrating another example of group division of regions in the twentieth embodiment. In FIG. 353, shaded portions (hatched portions) indicate periods during which encoded signals are superimposed (encoded signal periods).

  For example, the example shown in FIG. 353 is a special example of FIG. 351 and FIG. As shown in FIG. 353, at the stage where the region is divided into groups, it is only necessary that the encoded signal is not transmitted in a place where encoded signals having the same phase do not enter.

  Specifically, the region A, the region B, the region C, and the region D are divided into other regions and the encoded signals having the same phase are superimposed on the regions A, B, C, and D. Here, in region D, the encoded signal is not superimposed in a period in which the encoded signal overlaps the blanking period. Furthermore, in the example shown in FIG. 353, the encoded signal is not superimposed on the area after area D (area E to area H).

  If there is a region where encoded signals with the same phase do not enter even if the region is divided into groups, a reference region is simply determined and the encoded signal is superimposed only on the periphery of the region (neighboring region) It may be allowed. At this time, the range in which the encoded signal is superimposed may be based on the above-described flowchart or may be limited to a predetermined range.

  In addition, the adjustment period described above may be provided so that a luminance difference does not occur in a region where the encoded signal is superimposed, a region where the encoded signal is not superimposed, and a region where the encoded signal is superimposed.

  In the present embodiment, B.I. Although the rise of the L control signal is superimposed as a reference, other B. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

  In all of the plurality of display areas, it is very difficult to find a period that is not a blanking period, and even if there is such a period, it is very short. In this disclosure, the encoded signal is represented by B.I. Even when superposed on the L control signal, priority is given to the blanking period as much as possible, so that lighting during the blanking period is suppressed to avoid degradation of image quality.

  However, even if the blanking period and the encoded signal period do not overlap in a certain area, the blanking period and the encoded signal period often overlap in another area.

  For this reason, the present embodiment discloses a method for avoiding overlapping of the blanking period and the encoded signal period in as many areas as possible among the plurality of display areas. That is, in the present embodiment, a plurality of regions are divided into several groups, and the encoded signal is superimposed at a constant phase within each group. Thereby, the overlap of the blanking period and the encoded signal in the corresponding group can be reduced.

  In the present embodiment, an example in which a plurality of regions are divided into two groups or three groups has been described. However, the present invention is not limited to this.

  Further, as a method of grouping a plurality of regions, the regions may be divided into several groups in advance, and how the phase is shifted may be set in advance.

  In the present embodiment, a plurality of areas are grouped so that encoded signal lengths (all encoded signal periods) can be superimposed on the basis of bright areas. However, the present invention is not limited to this. Since there may be a case where the number of groups divided according to this criterion is very large, the number of groups may be limited. The area divided into groups does not necessarily need to be able to overlap the entire encoded signal period.

  Also, the encoded signals superimposed on the areas in each group may be the same or different. If two or more signals are mixed in the encoded signal obtained on the receiver side, the probability of erroneous recognition or error occurrence increases. Therefore, if two or more signals can be received by the same receiver at the same time, different encoded signals can be received by the same receiver at the same time. It means a case where it can be done or a combination thereof. As a result, the probability of erroneous recognition or error occurrence can be reduced.

  In addition, the group to be divided with some reference is not limited to the above example, and the second processing unit 1470 may be divided into groups based on the result of signal processing performed by the relationship between the video signal and the encoded signal. .

  In addition, in a backlight using LEDs, the light source is a very small object that is close to a point light source. Therefore, in order to emit surface light like an LCD, a light guide plate, a diffusion plate, or the like is used. It is spreading. Therefore, when controlling the LEDs in each region, the adjacent regions are designed to overlap, and there is more than a certain amount of leakage light.

  Therefore, in a backlight using LEDs or the like, even when divided into several groups, another signal enters as noise due to leakage light from at least the adjacent area. It is necessary to avoid the overlap of the digitized signals in time. Therefore, for example, the encoded signal may not be transmitted in the corresponding frame at the corresponding location, or the temporally continuous or overlapping encoded signals may be transmitted in a remote area.

  When the encoded signal is not transmitted in the corresponding frame at the corresponding location, it may be determined and separated from which area the encoded signal is output for each frame, or a specific location (linked to the video signal). May be preferentially transmitted.

  Even when transmission periods of encoded signals having different phases in different regions overlap, it is only necessary that the regions are not continuous or have a certain interval. This is because the signal can be received when the signal is received by limiting the area. Note that the interval between regions having different phases needs to be determined based on the extent to which the leaked light from the backlight extends, and is a numerical value based on the performance of the display device used.

  Further, the above method may be applied to each area divided into a plurality of blocks.

(Embodiment 21)
When an encoded signal is received, if a light intensity sensor with a very fast response speed such as a photodiode is used, the phase difference between the image and the encoded signal is not a big problem.

  On the other hand, when an encoded signal is captured and acquired using an image sensor such as a camera or digital still camera attached to a smartphone or mobile phone, the exposure timing is turned on due to a slight phase difference. It may not be acquired as an effective signal because it takes a very short time difference or simultaneously with the timing of the start or / and end of a series of encoded signal periods, or an OFF edge portion. In other words, the imaging cycle of the image sensor is generally 30 FPS. For example, when the video signal of 60 FPS and the encoded signal are synchronized, if the timing does not match at the time of imaging by the image sensor, it will last forever. It is also conceivable that the timing of the imaging cycle of the image sensor does not match the timing of the encoded signal.

  Therefore, in this embodiment, in order to avoid this, a method of shifting the phase of the encoded signal will be described.

[1. Operation of display device]
Hereinafter, the operation of the second processing unit 1470 will be mainly described.

  FIG. 354 is a flowchart for describing operation of the second processing unit in the twenty-first embodiment.

  First, in step S1331, the second processing unit 1470 shifts the signal synchronization. More specifically, the second processing unit 1470 shifts the synchronization of the encoded signal when the synchronization between the display panel 1450 and the backlight 1490 is not fixed. This is effective in increasing the imaging probability of the smartphone 1350.

  Next, in step S1332, the second processing unit 1470 determines that the B.D. is determined based on the duty ratio based on the video signal output from the first processing unit 1430. The AND of the L control signal and the encoded signal is calculated.

  Next, in step S1333, the second processing unit 1470 adjusts the duty ratio based on the video signal and / or the visible light communication signal.

  More specifically, as described in the eighteenth embodiment, the second processing unit 1470 checks whether or not the encoded signal period and the blanking period overlap, and provides an adjustment period by dividing the case. . The second processing unit 1470 performs B.B of the corresponding frame for the adjustment period. The duty ratio of the L control signal is based on the original video signal. When the duty ratio is different from the duty ratio of the L control signal, the duty ratio is adjusted using a period during which transmission of the encoded signal is suspended. Here, for example, the second processing unit 1470 adjusts the duty ratio by setting a period (OFF period of the BL control signal) for turning off the backlight 1490 in a period different from the blanking period. . Then, the second processing unit 1470 provides the B.B. The L control signal is output to the second control unit 1480.

  If the phase relationship between the encoded signal and the video signal returns to the original state in a certain period, these may be corrected to a predetermined phase difference.

  Further, if the phase of the encoded signal and the phase of the video signal change with time at a frequency different from the frequency of the video signal, that is, if either one is not a substantially integer multiple of the other, There is no need to control the phase alignment. This is because, even if the phases of both are not matched, the relationship between the phases of both of them will return to its original state after a certain period of time, and there are always times when it is difficult to receive signals and when they can be easily received. Because it will exist.

  FIG. 355A and FIG. It is a figure for demonstrating the relationship of the phase of L control signal and a visible light communication signal.

  For example, in FIG. The encoded signal based on the visible light communication signal and the B.L. It shows that the L control signal X has the same phase at a constant period. The hatched portion in the figure indicates the period during which the encoded signal is actually transmitted. Although the display is performed to output for a short period with a longer cycle than the L control signal, the signal length relationship may be longer as described above. Also, the actual encoded signal transmission period and B.B. Although the length of the L control signal is not necessarily longer, the encoded transmission period is B.B. Desirably shorter than the L control signal. Here, B.I. While the L control signal X is repeated 12 times, the encoded signal is repeated 7 times. When the L control signal is 60 fps, the phases of both are the same at intervals of 0.2 seconds. On the other hand, in FIG. There is no relationship between the L control signal X and the encoded signal. The first half of the L control signal, f2 is the second half, f3 is approximately the middle position, the start of the encoded signal transmission period for each frame, The relationship of the start phase of the L control signal has changed. However, although the both have the least common multiple and the phase relationship does not return to the original, the phase is shifted sequentially, so that the trouble caused by the imaging timing can be solved somewhere. B. Even at the time starting from f2 and f5 when the encoded signal is transmitted in the period corresponding to the break of the L control signal, the encoded signal is interrupted in the region X, but the encoded signal is always transmitted in some other region. There is no problem because it can be sent. The relationship between the video and the communication information may be stored in a buffer or the like, and the previous memory may be extracted and encoded as a communication signal for use. Further, when it takes a very long time for the phase relationship between the two to return to the original state, for example, several seconds or more, the phase relationship may be forcibly returned to the original state. For example, after the encoded signal is completed at f8 in FIG. The phases of the L control signal and the encoded signal may or may not be matched again. In addition, a cycle for adjusting the timing may be performed every second, for example, or may not be performed at all.

[2. Details of operation]
Next, details (specific example) of the operation of display device 1400 of Embodiment 20 will be described using FIG. 356A, FIG. 356B, and FIG.

  356A, 356B, and 356C are timing charts for explaining the operation of the second processing unit in the twenty-first embodiment. A shaded portion (hatched portion) indicates a region where an encoded signal exists. FIG. 356A is a diagram of B.B before the encoded signal is superimposed. 356B shows a timing chart of the L control signal, and FIG. The timing chart of L control signal is shown. FIG. 356C shows that the phase of the backlight control signal and the phase of the visible light communication signal are set as a result by setting the delay time from the rising or falling timing of the backlight control signal based on the encoded signal. An example in which the relationship is changed with time is shown.

  For example, as shown in FIG. The synchronization with the L control signal is shifted. Thereby, the timing which can receive an encoding signal by the receiver side, such as the smart phone 1350, can be generated reliably. Here, the adjustment period described above may be calculated and provided for each phase difference in each frame.

  As shown in FIG. 356C, for example, the time difference β1 between the rising edge U2 of the backlight control signal and the start V2 of the visible light communication signal may be set in advance as a delay time and overlapped with the region A as a reference. The time difference β2 between the rising edge U3 of the next frame and the start V3 of the visible light communication signal may be the same as or different from β1. In the example shown in FIG. 356C, β represents a positive numerical value (time) as a delay, but may be preceded by a negative numerical value (time).

  Furthermore, frames in which β is 0 may be mixed. The reference region may be anywhere, and may be selected according to the above-described criteria. The reference time is the rising edge of the backlight control signal, but another reference characteristic of the signal waveform, such as a falling edge, may be used. In addition to the characteristic part of the backlight control signal in a predetermined area, the reference time may be based on the synchronization signal of the video signal itself, or a signal delayed for a certain time from the video synchronization signal. It may be created and the signal may be used as a reference.

  In the present embodiment, since the video signal and the encoded signal do not have a one-to-one correspondence, some encoded data and image data are buffered in advance in a memory (not shown) in the display device 1400 and the like. Processing should be done.

  It should be noted that the period of the video signal (1 frame length) and the period of superimposing the encoded signal should preferably have the least common multiple within 1 second, and more preferably within 0.5 seconds. In addition, when these two periods are synchronized, tracking is performed from the timing at which these two periods are synchronized to the least common multiple period or every integral multiple of the time. The shift (phase difference) may be corrected.

  In addition, as described above, when the period or / and frequency of the video signal and the period or / and frequency of the encoded signal are in a relationship that temporally changes the phase relationship, each period is within one second. Even if it does not have the least common multiple, it is not necessary to control the relationship between the two phases, particularly when the rate of change is fast, for example, when a change exceeding the same phase relationship is repeated within one second. The change ratio is preferably the relationship as in the above example, but is not necessarily limited thereto.

[3. Effect]
As described above, in the display device according to the present embodiment, the signal processing unit uses the backlight control signal of each of the plurality of regions as a reference with respect to the backlight control signal of each of the plurality of regions. The signal processing unit (second processing unit 1470) that changes the delay time for encoding the visible light communication signal with respect to the backlight control signal in each of the plurality of regions A delay time for encoding the visible light communication signal (encoded signal) is temporally changed based on the backlight control signal in any one of the areas.

  With this configuration, it is possible to reliably generate a timing at which an encoded signal can be received on the receiver side such as the smartphone 1350.

  The signal processing unit (second processing unit 1470) superimposes a visible light communication signal (encoded signal) on the plurality of backlight control signals at a period different from the period of the backlight control signal, In each of the plurality of regions, the relationship between the phase of the backlight control signal and the phase of the visible light communication signal may change with the frame.

  Here, the period in which the visible light communication signal different from the period of the backlight control signal is superimposed may change over time.

  In addition, the phase of the visible light communication signal superimposed on the plurality of backlight control signals may be the same in all the regions where the visible light communication signal is superimposed.

  Further, the phase shift period of the visible light communication signal superimposed on each of the plurality of regions and one frame period of the backlight control signal may have a least common multiple within 1 sec.

  Thereby, the timing at which the encoded signal can be received on the receiver side such as the smartphone 1350 can be reliably generated in a relatively short time.

  In addition, the signal processing unit (second processing unit 1470) includes a phase shift period of the visible light communication signal (encoded signal) superimposed on each of the plurality of regions and one frame of the backlight control signal. The start point of the phase shift period of the visible light communication signal (encoded signal) superimposed on each of the plurality of regions is set to one frame period of the backlight control signal every time of the least common multiple or integer multiple of the period. It may be corrected.

  Accordingly, by correcting the phase shift, it is possible to prevent the timing at which the encoded signal can be received on the receiver side such as the smartphone 1350 from extending for a long time.

  Here, the time indicated by the least common multiple of the two types of periods is usually a numerical value (time) that can be received within at least this time in the positional relationship and environment where the communication signal can be received. It must be within the time that the person who thinks can hold the receiver and wait still. Even in normal NFC, etc., it is recommended to hold it for 1 second as a standard of waiting time, and it is desirable that it is equal or less. Furthermore, the least common multiple is assumed to be within 0.5 seconds as a time that does not cause a psychological burden.

(Embodiment 22)
In the eighteenth to twenty-first embodiments, the case has been described in which each area is sequentially controlled at a scanning speed driven at a normal speed to display an image signal. However, each area is scanned at a scanning speed of double speed driving higher than the normal speed. The image signal may be displayed by sequentially controlling.

  In the present embodiment, a case where a 2 × speed image signal is displayed by sequentially controlling each area at a scanning speed of 4 × speed driving will be described as an example. Hereinafter, the blanking period will be described based on the double speed driving time.

[1. Operation of display device]
Hereinafter, the operation of the second processing unit 1470 will be mainly described.

  357A and 357B are timing charts for explaining the operation of the second processing unit in the twenty-second embodiment. A shaded portion (hatched portion) indicates an area where an encoded signal exists. FIG. 357A is a diagram of B.B before the encoded signal is superimposed. FIG. 357B shows a timing chart of the L control signal, and FIG. The timing chart of L control signal is shown.

  For example, as shown in FIG. L control signals AB. There is no period during which the L control signal H is lit simultaneously. That is, it indicates that the encoded signal cannot be simultaneously superimposed on all the display areas.

  Therefore, in the present embodiment, for example, as shown in FIG. 357B, the scanning period between the blanking period regions is set to half of the normal period. Among the plurality of regions (all regions may be sufficient), B. A region H that is a region where the start of the blanking period of the L control signal is late is selected.

  The second processing unit 1470 terminates the blanking period in the region H and starts the lighting of the backlight 1490 (the time when the BL control signal H is turned ON). The encoded signal is superimposed on.

  In the example illustrated in FIG. 357B, the second processing unit 1470 performs B.B. B. The blanking period with the L control signal H ends. The encoded signal is superimposed at the timing when the L control signal H is turned ON.

  As a result, the second processing unit 1470 can set a period in which the encoded signal is superimposed in any region of the display region, up to a half frame.

[2. Effect]
As described above, in the display device according to the present embodiment, the display control unit (first control unit 1440) is configured to display the display panel according to a high-speed scanning speed that is higher than the scanning speed indicated by the video signal. The display panel (1450) is controlled to display an image on the display surface.

  Accordingly, the display device can take a longer period during which the encoded signal can be output.

  Note that the encoded signal length (encoded signal period) is long. When there is an area that overlaps with the blanking period and cannot be superimposed only in the ON period (period other than the blanking period) of the L control signal, the encoded signal is not superimposed in the blanking period in the corresponding area.

  B. An adjustment period for turning on the backlight 1490 in the blanking period may be provided for the extinguishing time by the encoded signal superimposed on the ON period of the L control signal. At that time, the adjustment period may be created by the method described in the above embodiment, or the header of the encoded signal may be superimposed on the blanking period. Further, a plurality of areas of the display area may be decomposed into several groups and the encoded signal may be superimposed.

  The same operation may be performed in an area other than the corresponding area (another area), or no signal may be output. In this case, the light-off time adjustment period may be provided by the method described in the above embodiment so that the duty ratio based on the visible light communication signal or / and the video signal is the same on the entire screen. Further, as in the case of the twentieth embodiment, the brightest area may be selected and the encoded signal may be superimposed at a timing according to the area. In the present embodiment, B.I. Although the rise of the L control signal is superimposed as a reference, other B. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

  In the present embodiment, the example in which the scanning speed at the double speed is changed to the scanning speed at the quadruple speed has been described, but the present invention is not limited to this. It is also possible to increase only the scanning speed while leaving the number of frames as it is.

  Further, in the present embodiment, it is assumed that the signal is transmitted in such a form in advance. However, the second processing unit determines the present embodiment based on the relationship between the video signal and the encoded signal. The method of transmitting the signal may be taken. In this case, since transmission of a signal from the second processing unit 1470 to the first processing unit 1430 in FIG. 340 enters, the arrow connecting the two blocks turns in both directions.

(Embodiment 23)
In the eighteenth to twenty-eighth embodiments, the description has been made assuming that the backlight scanning is the control method for providing the periods during which the turn-off control is performed at different timings in each of the plurality of regions. Local dimming may be used.

  In this embodiment, an operation at the time of local dimming adaptation will be described.

Here, in local dimming, the display area (screen) is divided into several areas, the transmittance of the liquid crystal in that area is made higher than usual, and the backlight brightness is lowered accordingly (duty is reduced). This is a backlight control method for reducing power. In the pixel with the highest luminance in the corresponding area, when there is a margin for increasing the transmittance (when the highest luminance has a relatively low value), the power can be reduced by the above control. Further, by reducing the backlight duty, the lighting period is reduced, and as a result, the contrast is improved.

[1. Backlight control by local dimming]
Next, B.B. The L control signal will be described.

  FIG. 358 is a timing chart showing backlight control during local dimming adaptation in the twenty-third embodiment.

  When performing backlight control using local dimming, for example, as shown in FIG. 358, the blanking period start timing interval T is uniform between adjacent regions, but the length of the blanking period is Not uniform.

  For this reason, the display device 1400 of the present embodiment has a B.D. After storing the blanking period of the L control signal, the processing (operation) described below may be performed.

[2. Operation of display device]
Hereinafter, the operation of the second processing unit 1470 will be mainly described. In the present embodiment, signal control in the case where the OFF periods per frame of the display areas are aligned is shown.

[2.1. Example of operation of second processing unit]
FIG. 359 is a flowchart for describing an example of operation of the second processing unit in the twenty-third embodiment.

  First, in step S1341, the second processing unit 1470 calculates an adjustment period. Specifically, the turn-off time in the encoded signal is N1, and B.B. When the turn-off time of the L control signal is N2, it can be expressed as a relational expression of an adjustment period N = N2-N1. Accordingly, the second processing unit 1470 can calculate (calculate) the adjustment period.

  Next, in step S1342, the second processing unit 1470 determines whether the sum (N + C) of the adjustment period N and the encoded signal period C is equal to or shorter than one frame period.

  If the second processing unit 1470 determines that (N + C) is equal to or shorter than one frame period (Yes in S1342), the second processing unit 1470 proceeds to S1343. On the other hand, if the second processing unit 1470 determines that (N + C) is greater than one frame period (No in S1342), the second processing unit 1470 proceeds to S1346, stops outputting the encoded signal, and ends the process.

  Next, in step S1343, the second processing unit 1470 determines whether the adjustment period N is 0 or more.

  If N is equal to or greater than 0 (Yes in S1343), the second processing unit 1470 proceeds to S1344, and provides an extinguishing period retroactively from the start of the next encoded signal. Also, during this period, no encoded signal is output, and the process is terminated.

  On the other hand, if N is smaller than 0 (No in S1343), the second processing unit 1470 proceeds to S1345. Start from the end of the blanking period of the L control signal. A lighting period (ON period) corresponding to the adjustment period is provided in the blanking period of the L control signal. Also, the encoded signal is not output during this adjustment period.

  FIG. 360 is a timing chart for explaining an example of the operation of the second processing unit in the twenty-third embodiment. Here, the bold line represents B.I. The ON period and the OFF period of the L control signal are shown, and the area A will be described below as a reference area. B. in each figure. A region controlled by the L control signal X (X is A to H) is referred to as a region X.

  For example, as illustrated in FIG. 360, the second processing unit 1470 superimposes the encoded signal having the same phase on each region from the start timing of the frame in the region A, which is the reference region, and provides an adjustment period. Note that the adjustment period may be provided according to the second method of the eighteenth embodiment, but since the second method has been described above, description thereof is omitted here.

  In this embodiment, B.B. When the L control signal is in the OFF period (blanking period), the encoded signal is not superimposed. When the L control signal is in the ON period, the encoded signal is basically superimposed. The adjustment period may be converted in consideration of the duty ratio of the encoded signal. In this case, the encoded signal is superimposed and output as long as it is an adjustment period and is a period for outputting the encoded signal. It is good.

[2.2. Example of operation of second processing unit]
Even during local dimming, priority may be given to having a blanking period sequentially as in normal backlight scan control. Processing in this case will be described below.

  FIG. 361 is a flowchart for describing an example of operation of the second processing unit in the twenty-third embodiment.

  First, in step S2101, the second processing unit 1470 calculates an adjustment period. Specifically, when the blanking period of a predetermined area is N1, the extinguishing time in the encoded signal is N2, and the blanking period of the corresponding area is N3, the relational expression N = N1-N2-N3 can be expressed. Accordingly, the second processing unit 1470 can calculate (calculate) the adjustment period.

  Next, in step S2102, the second processing unit 1470 determines whether or not the sum (N + C + N3) of the adjustment period N, the encoded signal period C, and the blanking period N3 of the corresponding region is equal to or less than one frame period. The determination result is stored.

  Next, in step S2103, the second processing unit 1470 determines whether or not the adjustment period N is 0 or more, and stores the determination result.

  After taking the steps as described above, the second processing unit 1470 provides an adjustment period based on the determination results of N1 to N3, S2102, and S2103 stored for each region, and provides an adjustment period and the like. The visible light communication signal is simultaneously displayed with the video display.

  The adjustment period may be created by a combination with the second method of the eighteenth embodiment and the methods described in the nineteenth to twenty-second embodiments.

  FIG. 362 is a timing chart for explaining an example of operation of the second processing unit in the twenty-third embodiment. In FIG. 362, an adjustment period is provided by the second method described in the eighteenth embodiment. The bold line represents B.I. The ON period and the OFF period of the L control signal are shown, and the area A will be described below as a reference area.

  For example, as illustrated in FIG. 362, the second processing unit 1470 outputs encoded signals having the same phase in the period from time P to time Q after a predetermined period has elapsed from the start timing of the frame in the area A that is the reference area. An adjustment period is provided while overlapping the area. The adjustment period may be provided in accordance with the second method of the eighteenth embodiment, but since the second method has been described above, description thereof is omitted here.

  In this embodiment, B.B. When the L control signal is in the OFF period (blanking period), the encoded signal is not superimposed. When the L control signal is in the ON period, the encoded signal is basically superimposed. Therefore, for example, in the region A, a certain period from time P is B.B. Since it is a blanking period in which the L control signal A is OFF, the encoded signal is not superimposed. An adjustment period is provided after the encoded signal period C.

  The adjustment period may be converted in consideration of the duty ratio of the encoded signal. In this case, the encoded signal is superimposed and output as long as it is an adjustment period and is a period for outputting the encoded signal. It is good.

[2.3. Example of operation of second processing unit]
FIG. 363 is a timing chart for explaining an example of operation of the second processing unit in the twenty-third embodiment.

  When backlight control is performed by applying local dimming, B. The blanking period of the L control signal varies from frame to frame and from region to region. For this reason, a provisional blanking period (described as a specified blanking period) is determined for the convenience of calculation. Then, the adjustment period can be calculated from the specified blanking period, the encoded signal period, the phase difference between them, and the original blanking period in accordance with the second method of the nineteenth embodiment. Hereinafter, an example of this case will be described with reference to FIG. A thick line in FIG. 363 indicates a waveform indicating the original blanking period.

  The specified blanking period is determined based on the average length of the blanking period in the screen and the shortest period. Here, it is assumed that the specified blanking period is an extinguishing period in which the encoded signal is not superimposed. The encoded signal period is a period in which the encoded signal is superimposed.

  The adjustment period may be provided using the second method of the eighteenth embodiment. If the adjustment period is positive, the backlight 1490 is turned off during this period. The L control signal may be adjusted. If the adjustment period is negative, the backlight 1490 is turned on. The L control signal may be adjusted. When the adjustment period is provided retroactive to the blanking period, the backlight 1490 is turned on even during the blanking period. The L control signal may be adjusted. If the adjustment period is negative, the B. When the encoded signal is superimposed on the L control signal, the adjustment period may be corrected based on the duty ratio.

[3. Effect]
As described above, in the display device according to the present embodiment, the backlight control unit (second control unit 1480) is in accordance with the backlight control signal output by the signal processing unit (second processing unit 1470). The light emission is controlled in accordance with the amount of light emitted from the backlight based on each video signal in each of the plurality of areas, and a period for controlling the turn-off at different timings in each of the plurality of areas is provided. The duty of the backlight based on the video signal and the visible light communication signal is also changed.

  In the present embodiment, B.I. Although the rise of the L control signal is superimposed as a reference, other B. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

  As described above, in this embodiment, the case where local dimming is applied has been described. However, local dimming may divide an area two-dimensionally, and a video signal may be simultaneously scanned and written in a certain direction. Although combinations of regions having the same phase and different blanking periods may occur, the present invention can be applied with the contents described in the present embodiment.

  As described above, the embodiments have been described as examples of the technology in the present disclosure. For this purpose, the accompanying drawings and detailed description are provided.

  Accordingly, among the components described in the accompanying drawings and the detailed description, not only the components essential for solving the problem, but also the components not essential for solving the problem in order to illustrate the above technique. May also be included. Therefore, it should not be immediately recognized that these non-essential components are essential as those non-essential components are described in the accompanying drawings and detailed description.

  Moreover, since the above-mentioned embodiment is for demonstrating the technique in this indication, the technique in this indication is not limited to this, A various change, replacement in a claim or its equivalent range, Additions and omissions can be made.

  For example, in the above-described embodiment, B.I. The case of superimposing on the basis of the rise of the L control signal is described as an example, but the present invention is not limited to this. Other B. Falling etc. The characteristic timing of the L control signal may be used as a reference, or the synchronization signal of the video signal itself may be used as a reference. Alternatively, a signal delayed for a certain time from the video synchronization signal may be created and used as a reference.

  The present disclosure can be applied to a display device that can output a visible light communication signal without greatly degrading the image quality of a display image and reduce reception errors of the output visible light communication signal. Specifically, since the display device according to the present disclosure can safely and actively acquire information other than images, not only devices such as televisions, PCs, and tablets at home, but also signage on the go The information terminal and the information display device can also be applied to various uses such as transfer of image-accompanying information and information transmission in various scenes in the sense that necessary information can be obtained safely because of its activeness.

  In addition, for example, the display devices in Embodiments 18 to 23 are display devices that can output a visible light communication signal, the display panel having a display surface for displaying video, and the display panel based on the video signal. A display control unit for controlling the display panel to display an image on a display surface; a backlight having a light emitting surface for illuminating the display surface of the display panel from the back; and the visible light communication signal based on the video signal. A signal processing unit that superimposes the backlight control signal generated in the above, and a light emitting surface of the backlight is divided into a plurality of regions, and in each of the plurality of regions according to the backlight control signal output by the signal processing unit A backlight control unit that performs light emission control and provides a period for performing light-off control at different timings in each of the plurality of regions. The signal processing unit, when superimposing the visible light communication signal on the backlight control signal, for the signal indicating that the backlight is turned off in the backlight control signal, the visible light communication signal May not be superimposed.

  Further, for example, the signal processing unit superimposes the visible light communication signal on the backlight control signal of each of the plurality of regions, and the visible light communication signal superimposed on each of the plurality of regions is They may be in phase with each other. Here, for example, in the display devices in Embodiments 18 to 23, the signal processing unit is superimposed on each of the plurality of regions with reference to the backlight control signal in a predetermined region among the plurality of regions. The phase of the visible light communication signal may be matched.

  Further, for example, the predetermined area may be a brightest area among the plurality of areas, and the predetermined area is an area corresponding to an end portion of the display surface among the plurality of areas. It is good.

  Further, for example, the signal processing unit superimposes the visible light communication signal on the backlight control signal of each of a plurality of groups including a plurality of neighboring regions among the plurality of regions, and The visible light communication signals superimposed on each group are in phase with each other, and the backlight control signal of each of the plurality of groups has a corresponding period of the visible light communication signal in a period for controlling the light emission of the backlight. All may be superimposed.

  Here, for example, the signal processing unit includes the visible light communication signal superimposed on each of the plurality of groups based on the backlight control signal of a predetermined region among the plurality of regions included in the plurality of groups. The phases may be matched. Alternatively, the predetermined area may be the brightest area among the plurality of areas.

  For example, the phase of the visible light communication signal superimposed on one of the plurality of groups may be different from the phase of the visible light communication signal superimposed on the other of the plurality of groups.

  Further, for example, when the visible light communication signal is superimposed on the backlight control signal, the signal processing unit is superimposed on a period of a signal indicating that the backlight is turned off in the backlight control signal. When the visible light communication signal period overlaps and there is an area of the plurality of areas, the overlapping area is provided with a lighting adjustment period for adjusting the luminance of the overlapping area, In the lighting adjustment period, on / off of the backlight control signal may be adjusted.

  In addition, for example, the signal processing unit generates the encoded signal by encoding the visible light communication signal, superimposes the encoded signal on the backlight control signal as the visible light communication signal, and the encoded signal Is superimposed on the backlight control signal, a region of the backlight control signal indicating that the backlight is turned off and a period of the encoded signal to be superimposed overlap each other, If there is a region of the encoded signal, a header portion of the encoded signal is superimposed on the backlight control signal during a signal period indicating that the backlight is turned off, and the header portion of the encoded signal is The other part may be superimposed on the backlight control signal in a period other than the period of the signal indicating that the backlight is turned off.

  Further, for example, the signal processing unit superimposes a visible light communication signal on the plurality of backlight control signals at a period different from the period of the backlight control signal, and performs the backlight control in each of the plurality of regions. The relationship between the phase of the signal and the phase of the visible light communication signal may change with the frame. Here, the period in which the visible light communication signal different from the period of the backlight control signal is superimposed may change over time.

  In addition, for example, the signal processing unit is configured to encode a visible light communication signal with respect to the backlight control signal of each of the plurality of regions based on the backlight control signal of any one of the plurality of regions. The time may be changed with time.

  Further, for example, the phase of the visible light communication signal superimposed on the plurality of backlight control signals may be the same phase in all regions where the visible light communication signal is superimposed.

  Further, for example, the phase shift period of the visible light communication signal superimposed on each of the plurality of regions and one frame period of the backlight control signal may have a least common multiple within 1 sec.

  In addition, for example, the signal processing unit may be a least common multiple or an integer multiple of a phase shift period of the visible light communication signal superimposed on each of the plurality of regions and one frame period of the backlight control signal. Furthermore, the start point of the phase shift period of the visible light communication signal superimposed on each of the plurality of regions may be corrected to one frame period of the backlight control signal.

  For example, the display control unit may control the display panel to display an image on the display surface of the display panel according to a high-speed scanning speed that is higher than the scanning speed indicated by the video signal.

  Further, the backlight control unit controls light emission in each of the plurality of areas according to a backlight control signal output from the signal processing unit, and the backlight based on each video signal in each of the plurality of areas. It is also possible to provide a period for controlling turning off at different timings according to the amount of emitted light, and to change the duty of the backlight based on the video signal and the visible light communication signal in each of the plurality of regions.

  Moreover, the control method of the display apparatus in Embodiments 18-23 is a control method of the display apparatus which can output a visible light communication signal, Comprising: The said display apparatus has a display panel which has a display surface which displays an image | video, A backlight having a light emitting surface for illuminating the display surface of the display panel from the back, and the control method controls the display panel to display an image on the display surface of the display panel based on a video signal. A display control step, a signal processing step of superimposing the visible light communication signal on a backlight control signal generated based on the video signal, a light emitting surface of the backlight being divided into a plurality of regions, and the signal processing step In accordance with the backlight control signal output at, light emission control is performed in each of the plurality of areas, and each of the plurality of areas is controlled. A backlight control step for providing a period for controlling turning off at different timings, and in the signal processing step, when the visible light communication signal is superimposed on the backlight control signal, the backlight control signal includes the backlight control signal. The visible light communication signal is not superimposed on the signal indicating that the backlight is turned off.

  In addition, according to the eighteenth to twenty-third embodiments, a display device that can output a visible light communication signal without greatly degrading the image quality of a display image and reduce reception errors of the output visible light communication signal. It is applicable to. Specifically, since the display devices according to the eighteenth to twenty-third embodiments can safely and actively acquire information other than images, the devices such as televisions, PCs, and tablets at home as well as on the go It can be used for various purposes such as transferring image supplementary information and sending information in every scene because it can be used to signify information, information terminals, and information display devices. Is possible.

(Embodiment 24)
The present disclosure relates to a display device and a display method capable of outputting a visible light communication signal.

  Japanese Unexamined Patent Application Publication Nos. 2007-43706 and 2009-212768 related to visible light communication technology using a backlight of a display disclose a display device that superimposes and displays communication information by visible light in a video signal. ing.

  The present disclosure provides a display device that outputs a visible light communication signal that can be restored by a receiving device.

  A display device according to the present disclosure is a display device capable of outputting a visible light communication signal composed of a plurality of signal units by a carousel method, which encodes a display panel that displays a video signal, a signal unit, and a plurality of blocks A visible light communication processing unit that generates a plurality of transmission frames using a plurality of blocks and generates a backlight control signal, and a backlight that emits light from the back of the display panel based on the backlight control signal. Prepare. The plurality of transmission frames for one signal unit generated by the visible light communication processing unit differ in the order of the plurality of blocks of at least two transmission frames.

  The display device of the present disclosure can output a visible light communication signal that can be restored by the receiving device.

  Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

  The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

  Hereinafter, the twenty-fourth embodiment will be described with reference to FIGS.

[1-1. Configuration of visible light communication system]
FIG. 364 is a schematic diagram of the visible light communication system according to the twenty-fourth embodiment. In FIG. 364, the visible light communication system 1500S includes a display device 1500 and a reception device 1520.

  The display device 1500 is a display, for example, and displays an image on the display surface 1510. Further, a visible light communication signal is inserted or superimposed on the video displayed on the display surface 1510 as information related to the displayed video.

  The receiving device 1520 receives the visible light communication signal output by being displayed on the display surface 1510 of the display device 1500 by capturing an image displayed on the display surface 1510. For example, the receiving device 1520 is configured as a smartphone in which a sequential exposure type image sensor is incorporated. Accordingly, the user of the reception device 1520 can receive information related to the video displayed on the display device 1500.

  In this embodiment, a display is taken as an example of the display device 1500, but is not limited thereto. The display device 1500 may be a projection display device such as a projector.

Further, although a smartphone is given as an example of the receiving device 1520, any electronic device that can receive a visible light communication signal may be used. For example, an electronic device is the “JEITA-CP1222 Visible Light ID Sy” (JEITA: Japan Electronics and Information Technology Association) defined by the Japan Electronics Information Industry Association (JEITA).
(stem) ". Furthermore, the electronic device may be a general communication terminal.

  “Visible light communication signal can be received” means that a visible light communication signal can be received and information can be obtained by decoding the received visible light communication signal.

The communication method of the visible light communication signal is, for example, “JEITA-CP-1223 visible light beacon system (Visible Light Beacon system) defined by JEITA.
System) ”or IEEE (The Institute)
of Electrical and Electronics Engineers,
Inc. ) Standardized by WPAN (Wireless Personal Area)
A communication method or the like conforming to IEEE-P802.15, which is a standard of Network), may be used.

  In other words, the receiving device 1520 may be any electronic device that can communicate with these communication methods and can receive visible light communication signals.

[1-2. Configuration of display device]
FIG. 365 is a block diagram of the display device according to the twenty-fourth embodiment. In FIG. 365, a display device 1500 includes a video signal input unit 1501, a video signal processing unit 1502, a display control unit 1503, a display panel 1504, a visible light communication signal input unit 1505, and a visible light communication signal processing unit 1506. And a backlight control unit 1507 and a backlight 1508.

  The video signal input unit 1501 relates to an image displayed on the display panel 1504 through an antenna cable, a composite cable, an HDMI (registered trademark) (High-Definition Multimedia Interface :) cable, a PJLink cable, a LAN (Local Area Network) cable, and the like. Video signal is input. The video signal input unit 1501 outputs the input video signal to the video signal processing unit 1502.

  Note that a video signal stored in a recording medium may be used as the video signal.

  The video signal processing unit 1502 performs general image processing such as decoding processing on the input video signal. The video signal processing unit 1502 transmits the video signal subjected to the image processing to the display control unit 1503 and the backlight control unit 1507. The video signal includes information about the brightness of the video.

  The display control unit 1503 controls the display panel 1504 to display an image on the display surface 1510 of the display panel 1504 based on the input video signal. More specifically, the display control unit 1503 performs liquid crystal opening control of the display panel 1504 based on the video signal input from the video signal processing unit 1502.

  The display panel 1504 is a liquid crystal panel, for example, and has a display surface 1510 for displaying an image.

  The visible light communication signal input unit 1505 receives a visible light communication signal via a visible light communication signal dedicated cable, a LAN cable, or the like.

  The visible light communication signal may be a visible light communication signal stored in a recording medium. Furthermore, the visible light communication signal may be superimposed on the video signal.

  The visible light communication signal input unit 1505 outputs the input visible light communication signal to the visible light communication signal processing unit 1506.

  The visible light communication signal processing unit 1506 encodes the input visible light communication signal by a predetermined encoding method, and further performs a process of determining the transmission order of the visible light communication signal. The visible light communication signal processing unit 1506 converts the encoded visible light communication signal into a backlight control signal. The visible light communication signal processing unit 1506 outputs the generated backlight control signal to the backlight control unit 1507.

  The backlight control unit 1507 divides the light emitting surface of the backlight 1508 into a plurality of regions, performs light emission control in each of the plurality of regions, and performs control for providing a light-off period at different timings in each of the plurality of regions on the light emitting surface. Do.

  The backlight control unit 1507 controls the luminance and timing of the backlight 1508 based on information about the brightness of the video included in the input video signal. Further, the backlight control unit 1507 controls light emission of the backlight 1508 based on the input backlight control signal.

  The backlight 1508 is installed on the back surface of the display panel 1504 and has a light emitting surface that illuminates the display surface 1510 of the display panel 1504 from the back surface. The backlight 1508 emits light from the back surface of the display panel 1504. The viewer can visually recognize the video displayed on the display panel 1504.

  In this embodiment mode, the entire display surface 1510 is a visible light communication region.

  FIG. 366 is a diagram for describing an example of generating a visible light communication signal. As shown in FIG. 366, the visible light communication signal input to the visible light communication signal input unit 1505 includes a plurality of signal units having a predetermined length. The visible light communication signal processing unit 1506 divides the signal unit into a predetermined number of data. In FIG. 366, one signal unit is composed of four data having the same data length. That is, one signal unit is divided into data 1, data 2, data 3, and data 4. The division of one signal unit is based on the carrier frequency of the visible light communication signal output from the display device 1500, the data length of the signal unit of the visible light communication signal, and further based on the period during which the backlight 1508 does not emit light. Just decide.

  Note that in FIG. 366, the data length of data that divides one signal unit is the same, but the data length of data that divides one signal unit may be different from each other, and one signal unit Among the data to be divided, the data length of one data may be different from the data length of the remaining data.

  Next, the visible light communication signal processing unit 1506 encodes the divided data, adds a header part to each data, determines a transmission order, and generates a block. Specifically, block 1, block 2, block 3, and block 4 are generated from data 1, data 2, data 3, and data 4. The visible light communication signal processing unit 1506 transmits the generated blocks to the backlight control unit 1507 in the order of block 1, block 2, block 3, and block 4 as a backlight control signal.

  The header of the block is composed of “preamble”, “address”, and “parity”. The preamble is a pattern indicating the beginning of a block, and includes an identifier indicating that the data is a visible light communication signal. For example, a signal deviating from a coding rule such as quaternary pulse position modulation (4PPM) or i-4PPM (Inverted 4PPM) is used. Parity is used to detect data errors. The address indicates the transmission order of the blocks in the signal unit.

  Four blocks generated from one signal unit are called transmission frames.

[1-3. Configuration of receiving apparatus]
FIG. 367 is a block diagram of the receiving apparatus according to the twenty-fourth embodiment. In FIG. 367, the reception device 1520 includes an imaging unit 1521, a captured image generation unit 1522, and a captured image processing unit 1523.

  The imaging unit 1521 captures an image displayed in the visible light communication area of the display device 1500. The imaging unit 1521 is, for example, a sequential exposure type image sensor. When the image sensor starts imaging, the image sensor sequentially performs exposure and sends exposure data to the captured image generation unit 1522.

  The captured image generation unit 1522 temporarily stores the exposure data sent from the imaging unit 1521 in a built-in memory. A captured image is generated based on the exposure data stored in the memory.

  The captured image processing unit 1523 restores the visible light communication signal from the captured image generated by the captured image generation unit 1522.

[1-4. Visible light communication signal output and reception]
Next, a basic operation of receiving a transmission frame output from the visible light communication area of the display device 1500 by the reception device 1520 will be described.

[1-4-1. Captured image for backlight on / off]
FIG. 368 is a diagram illustrating a captured image of the reception device 1520 with respect to turning on / off the backlight 1508 of the display device 1500.

  The imaging unit 1521 is a sequential exposure type image sensor, and exposes while scanning temporally for each line. In the present embodiment, in order to simplify the description, it is assumed that the exposure elements of the image sensor are 8 lines. It is assumed that the exposure line is configured in a vertically long band shape of the receiving device 1520.

  As shown in FIG. 368, the backlight 1508 of the display device 1500 is turned on and off as time passes. The image sensor sequentially exposes from the 1st line to the 8th line, and when the 8th line is sequentially exposed, the captured image generation unit 1522 of the receiving device 1520 generates a captured image based on the exposure data of 8 lines. Generate. Here, a sequential exposure period of the image sensor is defined as an imaging period, and a captured image generated based on exposure data sequentially exposed by the image sensor during the imaging period is defined as a reception frame L. When the exposure of the image sensor is performed up to the eighth line, the first exposure returns to the first line, and the next exposure starts from the first line. Next, the generated captured image is set as a reception frame L + 1. There is a blanking period such as the time for storing exposure data in the memory from the end of exposure until the start of the next first line until the 8th line, and this time is not exposed. . The reception frame L is the first line, the second line, the fifth line, the sixth line, and the eighth line of the exposure of the image sensor of the reception device 1520 when the backlight 1508 of the display device 1500 is lit. bright. The third and fourth lines of exposure of the image sensor of the receiving device 1520 are when the backlight 1508 of the display device 1500 is turned off, and each line is dark. Based on the received frame L, the visible light communication signal is restored.

  The reception frame L + 1 is the first line, the second line, the third line, the seventh line, and the eighth line of exposure of the image sensor of the reception device 1520 when the backlight 1508 of the display device 1500 is lit. bright. The fourth, fifth and sixth lines of exposure of the image sensor of the receiving device 1520 are when the backlight 1508 of the display device 1500 is turned off, and each line is dark. The visible light communication signal is restored based on the reception frame L + 1.

[1-4-2. Captured image for transmission frame]
FIG. 369 is a schematic diagram illustrating a captured image of the reception device 1520 with respect to the transmission frame of the display device 1500.

  As described with reference to FIG. 366, the visible light communication signal includes a plurality of signal units, and one signal unit is divided into four data, encoded, and divided into four blocks.

  In the visible light communication region, which is the display surface 1510 of the display device 1500, a period in which it is not possible to determine whether the backlight 1508 is turned on or off may occur depending on the content of the video signal. There is a possibility that the receiving device 1520 cannot receive the transmission frame output from the display device 1500 during this period.

  Therefore, a transmission frame output from the backlight 1508 of the display device 1500 uses a carousel method in which a transmission frame generated from one signal unit is repeatedly output a plurality of times. In FIG. 369, the display device 1500 outputs a transmission frame twice in succession using a visible light communication signal as one signal unit.

  As illustrated in FIG. 369, the transmission frame is output by turning on / off the backlight 1508 of the display device 1500 as time elapses. In the exposure of the image sensor of the receiving device 1520, the exposure is sequentially performed from the first line to the eighth line. When the exposure of the image sensor is performed up to the eighth line, the captured image generation unit 1522 of the reception device 1520 generates a captured image based on the exposure data of the eight lines. A received frame L that is a captured image is received by the block 1 at the first and second lines of exposure of the image sensor of the receiving device 1520, the block 2 is received at the third and fourth lines, the fifth line, Block 3 is received on the sixth line, and block 4 is received on the seventh and eighth lines. The reception frame L corresponds to the first transmission frame of one signal unit output from the display device 1500.

  In FIG. 369, a received frame L + 1, which is a captured image, receives block 1 at the first and second lines of exposure of the image sensor of the receiving device 1520, and receives block 2 at the third and fourth lines. Then, block 3 is received on the 5th and 6th lines, and block 4 is received on the 7th and 8th lines. The reception frame L + 1 corresponds to the second transmission frame of one signal unit output from the display device 1500.

In this way, by continuously outputting a transmission frame generated from one signal unit in the carousel method, even if a reception failure occurs with respect to transmission of the first transmission frame, Blocks that could not be received in the first transmission frame can be received. One signal unit can be restored by receiving all blocks, that is, four blocks, for two transmission frames.

  When the transmission frame is continuously output in the carousel method, the display device 1500 indicates that the current signal unit has been switched to the next signal unit before outputting the transmission frame of the next signal unit. May be output.

  This reset signal may be included in the preamble or data of the block of the transmission frame.

[1-5. Issues in the output and reception of visible light communication signals]
Next, problems in outputting and receiving visible light communication signals will be described. FIG. 370 is a diagram for describing the relationship between the frequency of the transmission clock of the display device 1500 and the frame rate of the imaging unit 1521 of the reception device 1520.

  The driving frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment is 120 Hz.

  Some types of liquid crystal panels operate at a drive frequency of 60 Hz, and some operate at a drive frequency of 240 Hz.

  In addition, the frame rate of the image sensor of the imaging unit 1521 of the reception device 1520 in this embodiment operates at 30 fps (frame per second).

  At this time, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or a fraction of an integer. Further, in order to control the luminance control, moving image resolution, and the like in the backlight control unit 1507 of the display device 1500, the timing of turning on and off the backlight 1508 of the display device 1500 may be synchronized with the driving frequency of the liquid crystal panel. That is, as shown in FIG. 370, the transmission frame of the display device 1500 is output in synchronization with the drive frequency of the liquid crystal panel. FIG. 370 illustrates a case where a transmission frame generated from one signal unit output from the display device 1500 in this situation is output three times by the carousel method.

  For the first transmission frame output from the display device 1500, the image sensor is exposed during an imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L. Only the blocks 2 and 3 in which all data is included in the received frame L can be restored as visible light communication signals.

  For the second transmission frame output from the display device 1500, the image sensor is exposed during the imaging period of one frame rate. The receiving device 1520 generates a received frame L + 1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L + 1. Only block 2 and block 3 in which all data is included in the reception frame L + 1 can be restored as visible light communication signals.

  For the third transmission frame output from the display device 1500, the image sensor is exposed during an imaging period of one frame rate. The receiving device 1520 generates a received frame L + 2 that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L + 2. Only block 2 and block 3 in which all data is included in the reception frame L + 2 can be restored as visible light communication signals.

  Thus, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integer multiple or a fraction of an integer, and a transmission frame for one signal unit output from the display device 1500 is transmitted to the liquid crystal panel. In the case of outputting in synchronization with the driving frequency, even if the same transmission frame is output three times by the carousel method, among the block 1, block 2, block 3, and block 4, it is possible to restore the visible light communication signal as block 2 , Only block 3. Blocks 1 and 4 cannot be restored as visible light communication signals.

[1-6. Transmission frame generation method]
In order to solve the above problem, a carousel system is used for one signal unit in order to restore all four blocks included in one signal unit output from the display device 1500 as visible light communication signals in the receiving device 1520. Instead of using the same transmission frame for each transmission frame output multiple times, a different transmission frame is generated and output each time. That is, a transmission frame that is output multiple times for one signal unit by the carousel method is generated so that the transmission order of blocks of the transmission frame for one signal unit is not the same every time.

  FIG. 371 is a diagram for explaining a first generation example of a transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 371 shows a case where one signal unit output from the display device 1500 is output three times by the carousel method, as in FIG. 370. The difference from FIG. 370 is that the transmission order of the blocks of the three transmission frames output from the display device 1500 is not the same, but is different every time.

  The order of blocks of the first transmission frame output from the display device 1500 is block 1, block 2, block 3, and block 4. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during an imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L. Only the blocks 2 and 3 in which all data is included in the received frame L can be restored as visible light communication signals.

  The order of blocks of the second transmission frame output from the display device 1500 is block 2, block 3, block 4, and block 1. The image sensor is exposed during the imaging period of one frame rate for the second signal unit output from the display device 1500. The receiving device 1520 generates a received frame L + 1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L + 1. Only blocks 3 and 4 in which all data is included in the reception frame L + 1 can be restored as visible light communication signals.

  The block order of the third transmission frame output from the display device 1500 is block 3, block 4, block 1, and block 2. For the third transmission frame output from the display device 1500, the image sensor is exposed during an imaging period of one frame rate. The receiving device 1520 generates a received frame L + 2 that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L + 2. Only block 4 and block 1 in which all data is included in the reception frame L + 2 can be restored as visible light communication signals.

  When the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or an integer, and the transmission frame is output from the display device 1500 in synchronization with the driving frequency of the liquid crystal panel. When a transmission frame for one signal unit is output three times in the carousel system by changing the block transmission order each time, all of block 1, block 2, block 3, and block 4 of one signal unit are used as visible light communication signals. Can be restored.

  In the generation example of FIG. 371, since the second and third blocks of the transmission frame output from the display device 1500 are blocks that can be restored as visible light communication signals, all the blocks are the second and third in three outputs. The transmission order of the block of signal units is changed so that it is output first.

  In the generation example of FIG. 371, the transmission frame that is output a plurality of times for one signal unit by the carousel method is changed so that the transmission order of blocks of the transmission frame for one signal unit is not the same every time. Not limited to this. A transmission frame that is output a plurality of times for one signal unit by the carousel method may be changed such that the transmission order of blocks of two adjacent transmission frames for one signal unit is different.

  Furthermore, the generation example of the transmission frame output from the display device 1500 is not limited to this.

  FIG. 372A is a diagram for explaining a second generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment.

  FIG. 372A shows the transmission order of the blocks of the transmission frame in ascending order, that is, the order of block 1, block 2, block 3, and block 4, and descending order, that is, the order of block 4, block 3, block 2, and block 1. repeat.

  When the reception frame generated by the reception device 1520 is composed of the first half part or the second half part of the transmission frame, the transmission frame as in the second generation example is output a plurality of times by the carousel method, thereby blocking one signal unit. All of 1, 1, 2, 3, and 4 can be restored as visible light communication signals.

  FIG. 372B is a diagram for explaining a third generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372B omits one of the four blocks of the signal unit and changes the transmission order for each transmission frame. The order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 2, block 3, and block 2, omitting block 4. The order of the blocks of the second transmission frame output from the display device 1500 is block 3, block 4, block 1, and block 3, omitting block 2. The block order of the third transmission frame output from the display device 1500 is block 4, block 1, block 2, and block 4 with the block 3 omitted. By changing to such a transmission order, all blocks can be transmitted the same number of times.

FIG. 372C is a diagram for explaining a fourth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. In FIG. 372C, the blocks of the signal unit are arranged in the order of block 1, block 2, block 3, and block 4, and one block is added therein. The order of the blocks of the first transmission frame output from the display device 1500 is Block 1, Block 1, Block 2, and Block 3, with Block 1 added. The order of the blocks of the second transmission frame output from the display device 1500 starts with the block 4 that was not included in the first time, added the block 2, and in the order of the block 4, the block 1, the block 2, and the block 2. is there. The order of the blocks of the third transmission frame output from the display device 1500 is the order of block 3, block 4, block 1, and block 2, starting from block 3 that was not included in the second time.

  As described above, while the transmission frame as in the fourth generation example is output a plurality of times by the carousel method, all of the block 1, block 2, block 3, and block 4 of one signal unit are used as visible light communication signals. Can be restored.

  FIG. 372D is a diagram for explaining a fifth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. FIG. 372D randomly changes the order of the blocks of the signal unit. The order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 3, block 2, and block 4. The order of blocks of the second transmission frame output from the display device 1500 is block 3, block 1, block 2, and block 4. The order of the blocks of the third transmission frame output from the display device 1500 is block 2, block 3, block 1, and block 4. While the transmission frame block order for one signal unit is randomly changed and output multiple times by the carousel method, all of block 1, block 2, block 3 and block 4 of one signal unit are visible light communication. It can be restored as a signal.

  FIG. 372E is a diagram for explaining a sixth generation example of the transmission frame for one signal unit according to the twenty-fourth embodiment. In FIG. 372E, the same block is continued twice in one transmission frame. The order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 1, block 2, and block 2. The order of blocks of the second transmission frame output from the display device 1500 is block 3, block 3, block 4, and block 4. The block order of the third transmission frame output from the display device 1500 is block 1, block 1, block 2, and block 2.

[1-7. Operation of visible light communication signal processor]
Next, the operation of the visible light communication signal processing unit 1506 of the display device 1500 will be described. FIG. 373 is a flowchart for explaining the operation of the visible light communication signal processing unit 1506 of the display device 1500.

  (Step S1501) The visible light communication signal processing unit 1506 determines whether a visible light communication signal is input from the visible light communication signal input unit 1505. If it is determined that the input of the visible light communication signal is “present” (Yes), the process proceeds to step S1502. When it is determined that the input of the visible light communication signal is “none” (No), the process of step S1501 is repeated.

  (Step S1502) The input visible light communication signal is composed of a plurality of signal units. The visible light communication signal processing unit 1506 reads one signal unit.

  (Step S1503) The visible light communication signal processing unit 1506 divides each read signal unit into a predetermined number of data, encodes each data, adds a header to each data, and generates a block. .

  (Step S1504) Based on the generated block, the visible light communication signal processing unit 1506 determines the transmission order of blocks to be included in each transmission frame of a plurality of transmission frames transmitted by the carousel method.

  (Step S1505) The visible light communication signal processing unit 1506 generates a plurality of transmission frames and outputs them to the backlight control unit 1507.

  (Step S1506) The visible light communication signal processing unit 1506 determines the presence or absence of the remaining signal units. If it is determined that the remaining signal units are “present” (Yes), the process returns to step S1501. If it is determined that the remaining signal units are “none” (No), the process is terminated.

[1-8. Effect]
As described above, the display device according to the present embodiment is a display device that can output a visible light communication signal including a plurality of signal units by the carousel method, and includes a display panel that displays a video signal, and a signal unit. Is divided into a plurality of blocks, a plurality of transmission frames are generated using the plurality of blocks, and a visible light communication processing unit which generates a backlight control signal and a display panel from the back based on the backlight control signal A backlight that emits light. The plurality of transmission frames for one signal unit generated by the visible light communication processing unit differ in the order of the plurality of blocks of at least two transmission frames.

  Thus, the display device 1500 can restore the visible light communication signal by outputting a plurality of transmission frames having different block transmission orders to one signal unit.

  In the display device according to the present embodiment, the plurality of transmission frames for one signal unit generated by the visible light communication processing unit include the same block in at least two adjacent transmission frames.

  Accordingly, the display device 1500 can restore the visible light communication signal by including the same block in at least two transmission frames adjacent to one signal unit.

  In the display device in this embodiment, the plurality of transmission frames for one signal unit generated by the visible light communication processing unit includes a plurality of the same blocks in at least one transmission frame, and the plurality of transmission frames Contains all the blocks.

  Accordingly, the display device 1500 can restore the visible light communication signal by including a plurality of the same blocks in one transmission frame and including all the blocks in the plurality of transmission frames.

  In the display device in this embodiment, the visible light communication signal processing unit inserts a reset signal between two adjacent signal units.

  Accordingly, the display device 1500 can indicate that the current signal unit has been switched to the next signal unit.

  In the display device 1500 of this embodiment, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is a relationship that is an integral multiple or a fraction of an integer, and the transmission frame from the display device 1500 is the driving frequency of the liquid crystal panel. This is particularly effective when output is synchronized with the output.

  In the present embodiment, the number of transmissions of a transmission frame output from the display device 1500 by the carousel method has been described as three, but the present invention is not limited to this. The number of transmissions of the transmission frame output by the carousel method may be any number as long as it is a plurality of times.

(Embodiment 25)
Hereinafter, Embodiment 25 will be described with reference to FIGS. 374 to 376.

[2-1. Configuration of visible light communication system]
The visible light communication system in the present embodiment has the same configuration as that of the visible light communication system 1500S described in the twenty-fourth embodiment. Different points in the visible light communication system according to the present embodiment will be mainly described.

[2-2. Relationship between image brightness and visible light communication signal output]
The display panel 1504 of the display device 1500 in this embodiment is a liquid crystal panel. When an image is displayed, the liquid crystal panel is visually recognized as an image by opening and closing a liquid crystal shutter on the display surface 1510 or controlling gradation and the backlight 1508.

  Therefore, even if the backlight 1508 is set to be very bright, if the video signal is dark, a dark region is formed in the visible light communication region. In the dark area of the video signal, the light of the backlight 1508 is shielded by the liquid crystal shutter of the display panel 1504. When a visible light communication signal is output in a dark region, the visible light communication signal may not be restored from a captured image captured by the imaging unit 1521 of the receiving device 1520.

  Therefore, in the present embodiment, when the ratio of the high-luminance area that is an area having a predetermined brightness or more to the visible light communication area that is the entire display surface 1510 of the display device 1500 is small, one signal unit The visible light communication signal can be restored by outputting the number of transmissions of the block included in the plurality of times. On the contrary, when the ratio of the high luminance area is large with respect to the visible light communication area, the number of transmissions of the blocks included in one signal unit is reduced or one signal is compared with the case where the ratio of the high luminance area is small. The block included in the unit is transmitted once.

[2-3. Operation of visible light communication signal processor]
The difference between the twenty-fifth embodiment and the twenty-fourth embodiment is mainly the operation of the visible light communication signal processing unit 1506. Next, the operation of the visible light communication signal processing unit 1506 will be described. FIG. 374 is a flowchart for explaining the operation of the visible light communication signal processing unit 1506 of the display device 1500 according to the twenty-fifth embodiment.

  The operations from step S1501 to step S1503 are the same as those in the twenty-fourth embodiment.

  (Step S1511) The visible light communication signal processing unit 1506 detects the high luminance area of the visible light communication area from the video signal input from the video signal processing unit 1502. The visible light communication signal processing unit determines the number of transmissions of each block of the transmission unit based on the ratio of the high luminance region in the visible light communication region. A method for determining the number of transmissions will be described later.

  (Step S1512) The visible light communication signal processing unit 1506 determines the block transmission order based on the number of transmissions of each block of the signal unit. A method for determining the block transmission order will be described later.

  The operations in steps S1505 and S1506 are the same as those in the twenty-fourth embodiment.

[2-4. How to determine the number of block transmissions]
Next, a method for determining the number of block transmissions will be described. FIG. 375 is a diagram for explaining an example of a method for determining the number of transmissions of an arbitrary block of a transmission frame for one signal unit.

  In FIG. 375, the horizontal axis represents the ratio of the high-luminance region of the visible light communication region, and the vertical axis represents the number of transmissions of an arbitrary block in the signal unit.

FIG. 375 assumes that the visible light communication signal can be reconstructed by the receiving device 1520 when the high-luminance region of the visible light communication region is about 80% or more and the number of times of transmission of any block in the signal unit is one. Then, it is assumed that the visible light communication signal can be restored by the receiving device 1520 by increasing the number of transmissions of an arbitrary block in the signal unit as the ratio of the high luminance region in the visible light communication region decreases. Specifically, if the high-luminance area is 90% (point A) in the visible light communication area, the number of transmissions of an arbitrary block in the signal unit is one, and the high-luminance area is in the visible light communication area. If 50% (point B), the number of transmissions of an arbitrary block in the signal unit is 3, and if the high-luminance area is 10% (point C) in the visible light communication area, an arbitrary block in the signal unit Is sent 6 times. In FIG. 375, the number of transmissions of an arbitrary block in the signal unit is such that the ratio of the high luminance region in the visible light communication region is 80% to about 15%, and the number of transmissions of the arbitrary block in the signal unit is one time. Increasing by one.

  Note that the ratio of the number of transmissions is not limited to this, and may be changed as appropriate.

[2-5. How to determine block transmission order]
Next, a method for determining the transmission order of blocks for one signal unit will be described. FIG. 376 is a diagram for explaining a transmission frame generation example for one signal unit according to the twenty-fifth embodiment. The driving frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment mode is 120 Hz, and the frame rate of the image sensor of the imaging unit 1521 of the reception device 1520 operates at 30 fps. Further, the transmission frame of the display device 1500 is output in synchronization with the driving frequency of the liquid crystal panel. FIG. 376 shows a case where one signal unit of a visible light communication signal output from the display device 1500 is output three times by the carousel method. One signal unit is composed of six data having the same data length, and is encoded to generate six blocks.

  In FIG. 376, the number of transmissions of blocks included in three transmission frames for one signal unit is determined according to the ratio of the high-luminance area of the visible light communication area.

  In the first transmission frame output from the display device 1500, the ratio of the high-luminance region is 80%, so the number of transmissions of any block of the signal unit is one. Therefore, the block order of the first transmission frame output from the display device 1500 is block 1, block 2, block 3, block 4, block 5, and block 6. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during an imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L. Block 2, block 3, block 4, and block 5 in which all data is included in the received frame L can be restored as visible light communication signals.

  Next, in the second transmission frame output from the display device 1500, the ratio of the high luminance region is 50%. Therefore, the number of transmissions of an arbitrary block of the signal unit is three. Accordingly, the block order of the second transmission frame output from the display device 1500 repeats the block 1 and the block 2 three times in order. For the second transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The receiving device 1520 generates a received frame L + 1 that is a captured image based on the exposure data. Of the received frame L + 1, a block in a region that is not a high luminance region cannot be restored. The reception device 1520 restores the visible light communication signal from the reception frame L + 1. Blocks 1 and 2 in which all data is included in the reception frame L + 1 can be restored as visible light communication signals.

  Next, in the third transmission frame output from the display device 1500, the ratio of the high luminance region is 10%, and therefore the number of transmissions of any block of the signal unit is six. The order of the blocks of the third transmission frame output from the display device 1500 repeats the block 6 six times in succession. For the third transmission frame output from the display device 1500, the reception device 1520 performs exposure of the image sensor during an imaging period of one frame rate. The receiving device 1520 generates a received frame L + 2 that is a captured image based on the exposure data. Of the received frame L + 2, a block in a region that is not a high luminance region cannot be restored. The receiving device 1520 restores the visible light communication signal from the received frame L + 2. The block 6 in which all data is included in the reception frame L + 2 can be restored as a visible light communication signal.

  When the transmission frame for one signal unit is output three times in the carousel method by determining the transmission order of the blocks based on the ratio of the high luminance region, the block 1, block 2, block 3, block 4, All of the blocks 5 and 6 can be restored as visible light communication signals.

[2-6. Effect]
As described above, in the display device according to the present embodiment, the visible light communication processing unit detects an area having a luminance higher than a predetermined level of the display panel, and includes the same block included in the transmission frame according to the size of the area And a plurality of transmission frames for the signal unit are generated.

  Accordingly, the display device 1500 restores the visible light communication signal by outputting a plurality of transmission frames by changing the number of block transmissions for one signal unit in accordance with the ratio of the high luminance region. it can.

  In this embodiment, three transmission frames are output by the carousel method to one signal unit output from the display device 1500, but the present invention is not limited to this. For example, a transmission frame may be used in which a transmission frame is output three or more times by the carousel method, and the combination of repeating block 1 and block 2 three times is changed as the block order of the second transmission frame.

  In the display device 1500 of this embodiment, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is a relationship that is an integral multiple or a fraction of an integer, and the transmission frame from the display device 1500 is the driving frequency of the liquid crystal panel. This is particularly effective when output is synchronized with the output.

(Embodiment 26)
The twenty-sixth embodiment will be described below with reference to FIGS. 377 to 380.

[3-1. Configuration of visible light communication system]
The visible light communication system in the present embodiment has the same configuration as that of the visible light communication system 1500S described in the twenty-fourth embodiment. Different points in the visible light communication system according to the present embodiment will be mainly described.

[3-2. Relationship between transmission of visible light communication signal and distance from display device]
The case where the distance between the display device 1500 and the receiving device 1520 is relatively short will be compared with the case where the distance is long. When the distance between the display device 1500 and the reception device 1520 is relatively short, there are more blocks included in the captured image captured by the reception device 1520 than when the distance between the display device 1500 and the reception device 1520 is relatively long. Become.

  This is because when the distance between the display device 1500 and the reception device 1520 is relatively short, the captured image that can be generated by the imaging unit 1521 of the reception device 1520 is relatively large, and the distance between the display device 1500 and the reception device 1520 is relative. This is because a captured image that can be generated by the imaging unit 1521 of the reception device 1520 is relatively small.

  Therefore, display device 1500 in the present embodiment changes the number of transmissions of an arbitrary block in the transmission frame of one signal unit in accordance with the distance from reception device 1520.

[3-3. Operation of visible light communication signal processor]
The difference of the twenty-sixth embodiment from the twenty-fourth embodiment is mainly the operation of the visible light communication signal processing unit 1506. The operation of the visible light communication signal processing unit 1506 will be described. FIG. 377 is a flowchart for explaining the operation of the visible light communication signal processing unit 1506 of the display device 1500 according to the twenty-sixth embodiment.

  The operations from step S1501 to step S1503 are the same as those in the twenty-fourth embodiment.

  (Step 1401) The visible light communication signal processing unit 1506 determines the number of transmissions of each block of the transmission unit according to the distance from the receiving device 1520. A method for determining the number of transmissions will be described later.

  (Step 1402) The visible light communication signal processing unit 1506 determines the block transmission order based on the number of block transmissions of the signal unit. A method for determining the transmission order will be described later.

  The operations in steps S1505 and S1506 are the same as those in the twenty-fourth embodiment.

[3-4. How to determine the number of block transmissions]
Next, a method for determining the number of block transmissions will be described. FIG. 378 is a diagram illustrating an example of a method for determining the number of transmissions of an arbitrary block of a transmission frame for one signal unit.

  In FIG. 378, the horizontal axis represents the distance between the display device 1500 and the receiving device 1520, and the vertical axis represents the number of transmissions of an arbitrary block in the signal unit. When the distance is short, the number of transmissions of each block of the signal unit is reduced. In FIG. 378, if the distance is 3 m or less, the number of transmissions of each block of the signal unit is set to one.

  When the distance is long, the number of transmissions of each block of the signal unit is increased. In FIG. 378, the transmission count of each block of the signal unit is increased once at a rate of 2 m from a distance of 3 m or more.

  In addition, you may change this ratio suitably.

[3-5. How to determine block transmission order]
Next, a method for determining the transmission order of blocks for one signal unit will be described. FIG. 379 is a diagram for explaining a transmission frame generation example for one signal unit output from the display apparatus 1500 according to the twenty-sixth embodiment. FIG. 379 shows a case where the distance is 3 m. The driving frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment is 120 Hz, and the frame rate of the image sensor of the imaging unit 1521 of the reception device 1520 operates at 30 fps. Further, the transmission frame of the display device 1500 is output in synchronization with the driving frequency of the liquid crystal panel. FIG. 379 shows a case where one signal unit of a visible light communication signal output from the display device 1500 is transmitted four times by the carousel method. One signal unit is composed of four data having the same data length and is encoded to generate four blocks.

  In FIG. 378, when the distance is 3 m, the number of transmissions of an arbitrary block of one transmission frame of the signal unit is two. Therefore, as shown in FIG. 379, an arbitrary block is transmitted twice in one transmission frame.

  The order of the blocks of the first transmission frame output from the display device 1500 is block 1, block 1, block 2, and block 2 so that block 1 and block 2 are output twice. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during an imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L. Blocks 1 and 2 in which all data is included in the reception frame L can be restored as visible light communication signals.

  The order of the blocks of the second transmission frame output from the display device 1500 is block 3, block 3, block 4, and block 4 so that block 3 and block 4 are output twice. For the second transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The receiving device 1520 generates a received frame L + 1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L + 1. Blocks 3 and 4 in which all data is included in the reception frame L + 1 can be restored as visible light communication signals.

  The order of the blocks of the third transmission frame output from the display device 1500 is block 1, block 1, block 2, and block 2 so that block 1 and block 2 are output twice. For the third transmission frame output from the display device 1500, the reception device 1520 performs exposure of the image sensor during an imaging period of one frame rate. The receiving device 1520 generates a received frame L + 2 that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L + 2. Blocks 1 and 2 in which all data is included in the reception frame L + 2 can be restored as visible light communication signals.

  The order of the blocks of the fourth transmission frame output from the display device 1500 is block 3, block 3, block 4, and block 4 such that block 3 and block 4 are output twice. For the fourth transmission frame output from the display device 1500, the reception device 1520 performs exposure of the image sensor during an imaging period of one frame rate. The reception device 1520 generates a reception frame L + 3 that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L + 3. Blocks 3 and 4 in which all data is included in the reception frame L + 3 can be restored as visible light communication signals.

  As described above, each received frame can receive one block in which an arbitrary block included in the transmission frame is output twice. That is, two different blocks can be received from each received frame.

  FIG. 380 is a diagram illustrating another example of generating a transmission frame for one signal unit output from the display apparatus according to the twenty-sixth embodiment. FIG. 380 shows a case where the distance is 8 m. The driving frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment is 120 Hz, and the frame rate of the image sensor of the imaging unit 1521 of the reception device 1520 operates at 30 fps. Further, the transmission frame of the display device 1500 is output in synchronization with the driving frequency of the liquid crystal panel. FIG. 380 shows a case where one signal unit of a visible light communication signal output from the display device 1500 is transmitted four times by the carousel method. One signal unit is composed of four data having the same data length and is encoded to generate four blocks.

  In FIG. 378, when the distance is 8 m, the number of transmissions of an arbitrary block of one transmission frame of the signal unit is four. Therefore, as shown in FIG. 380, an arbitrary block is transmitted four times in one transmission frame.

  As for the block order of the first transmission frame output from the display device 1500, the block 1 is output four times. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during an imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L. The block 1 in which all data is included in the reception frame L can be restored as a visible light communication signal.

  The second transmission frame block output from the display device 1500 outputs block 2 four times. For the second transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during the imaging period of one frame rate. The receiving device 1520 generates a received frame L + 1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L + 1. Block 2 in which all data is included in reception frame L + 1 can be restored as a visible light communication signal.

  The block order of the third transmission frame output from the display device 1500 outputs block 3 four times. For the third transmission frame output from the display device 1500, the reception device 1520 performs exposure of the image sensor during an imaging period of one frame rate. The receiving device 1520 generates a received frame L + 2 that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L + 2. The block 3 in which all data is included in the reception frame L + 2 can be restored as a visible light communication signal.

  The block 4 of the fourth transmission frame output from the display device 1500 outputs block 4 four times. For the fourth transmission frame output from the display device 1500, the reception device 1520 performs exposure of the image sensor during an imaging period of one frame rate. The reception device 1520 generates a reception frame L + 3 that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L + 3. Block 2 in which all data is included in reception frame L + 3 can be restored as a visible light communication signal.

  As described above, each received frame can receive one block from which any block included in the transmitted frame is output four times. That is, one block can be received from each received frame.

[3-6. Effect]
As described above, in the present embodiment, the visible light communication processing unit determines the number of the same blocks included in the transmission frame according to the distance between the display device and the receiving device that can receive the output visible light communication signal. Determining and generating a plurality of transmission frames for the signal unit.

  Accordingly, the display device 1500 can restore the visible light communication signal by changing the number of block transmissions and outputting a plurality of transmission frames according to the distance between the display device 1500 and the reception device 1520. .

  In the display device 1500 of this embodiment, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is a relationship that is an integral multiple or a fraction of an integer, and the transmission frame from the display device 1500 is the driving frequency of the liquid crystal panel. This is particularly effective when output is synchronized with the output.

  Note that the distance between the display device 1500 and the reception device 1520 can be set in advance by the display device 1500, and is preferably changed as appropriate depending on the application and the installation status of the display device 1500.

  The receiving apparatus 1520 may request the display apparatus 1500 to set the distance designation through wireless communication such as Wi-Fi (Wireless Fidelity), Bluetooth (registered trademark), or LTE (Long Yield Evolution).

  Further, the distance may be estimated using a sensor or a camera on either the display device 1500 or the reception device 1520.

  Moreover, in this Embodiment, the produced | generated transmission frame is an example and is not limited to this.

  Further, in the present embodiment, when two blocks are output a plurality of times in a transmission frame, the same number is used, but the same number is not necessarily required.

(Embodiment 27)
Hereinafter, Embodiment 27 will be described with reference to FIGS. 381 to 383.

[4-1. Configuration of visible light communication system]
The visible light communication system in the present embodiment has the same configuration as that of the visible light communication system 1500S described in the twenty-fourth embodiment. Different points in the visible light communication system according to the present embodiment will be mainly described.

[4-2. Insert blank]
FIG. 381 is a diagram for explaining a generation example of a transmission frame for one signal unit according to the twenty-seventh embodiment. The driving frequency of the liquid crystal panel which is the display panel 1504 of the display device 1500 in this embodiment mode is 120 Hz, and the frame rate of the image sensor of the imaging unit 1521 of the reception device 1520 operates at 30 fps. Further, the transmission frame of the display device 1500 is output in synchronization with the driving frequency of the liquid crystal panel. One signal unit of the visible light communication signal output from the display device 1500 is output four times by the carousel method. One signal unit is composed of four data having the same data length and is encoded to generate four blocks.

  In the present embodiment, a blank having the same size as the block is inserted into the transmission frame so that the same block does not become the same position.

  In FIG. 381, the first transmission frame output from the display device 1500 is in the order of block 1, block 2, block 3, block 4, and blank. For the first transmission frame output from the display device 1500, the reception device 1520 exposes the image sensor during an imaging period of one frame rate. The reception device 1520 generates a reception frame L that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L. Only the blocks 2 and 3 in which all data is included in the received frame L can be restored as visible light communication signals.

  The second transmission frame output from the display device 1500 is in the order of block 1, block 2, block 3, block 4, and blank. The image sensor is exposed during the imaging period of one frame rate for the second signal unit output from the display device 1500. The receiving device 1520 generates a received frame L + 1 that is a captured image based on the exposure data. The reception device 1520 restores the visible light communication signal from the reception frame L + 1. Only block 1 and block 2 in which all data is included in the reception frame L + 1 can be restored as visible light communication signals.

  The third transmission frame output from the display device 1500 is in the order of block 1, block 2, block 3, block 4, and blank. For the third transmission frame output from the display device 1500, the image sensor is exposed during an imaging period of one frame rate. The receiving device 1520 generates a received frame L + 2 that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L + 2. Only the block 1 in which all data is included in the reception frame L + 2 can be restored as a visible light communication signal.

  The fourth transmission frame output from the display device 1500 is in the order of block 1, block 2, block 3, block 4, and blank. For the fourth transmission frame output from the display device 1500, the image sensor is exposed during the imaging period of one frame rate. The reception device 1520 generates a reception frame L + 3 that is a captured image based on the exposure data. The receiving device 1520 restores the visible light communication signal from the received frame L + 3. Only the block 4 in which all data is included in the reception frame L + 3 can be restored as a visible light communication signal.

  The blank signal pattern to be inserted may be anything as long as it is a pattern different from the data included in the signal unit.

  As described above, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is an integral multiple or a fraction of an integer, and the transmission frame is output from the display device 1500 in synchronization with the driving frequency of the liquid crystal panel. In this case, by inserting a blank into the transmission frame for one signal unit, the timing of turning on and off the backlight 1508 of the display device 1500 is prevented from synchronizing with the drive frequency of the liquid crystal panel, and the same transmission frame is used four times. , All of block 1, block 2, block 3, and block 4 of one signal unit can be restored as visible light communication signals.

  Further, by making the size of the blank to be inserted the same as the block size, it is possible to prevent fluctuations in the luminance of the video signal, and it is also effective as a luminance adjustment period.

  In addition, although it demonstrated that the size of the blank to insert is made the same as the size of a block, it is not restricted to this. The size of the blank to be inserted may be determined so that the timing of turning on and off the backlight 1508 of the display device 1500 is not synchronized with the driving frequency of the liquid crystal panel.

  Note that the size of the blank to be inserted may not always be the same size.

  Furthermore, the generation example of the transmission frame in which the blank is inserted is not limited to this.

  FIG. 382A is a diagram for explaining a second generation example of the transmission frame for one signal unit according to the twenty-seventh embodiment.

  In FIG. 382A, a blank is inserted at the end of the transmission frame, and the transmission order of blocks of the transmission frame is changed every time as described in the twenty-fourth embodiment. That is, the block order of the first transmission frame output from the display device 1500 is block 1, block 2, block 3, block 3, and blank. The second transmission frame output from the display device 1500 is in the order of block 4, block 3, block 2, block 1, and blank. The third transmission frame output from the display device 1500 is in the order of block 2, block 3, block 4, block 1, and blank.

  FIG. 382B is a diagram for explaining a third generation example of the transmission frame for one signal unit according to the twenty-seventh embodiment.

  FIG. 382B inserts a blank after each block of the transmission frame. That is, the transmission frames output from the display device 1500 are in the order of block 1, blank, block 2, blank, block 3, blank, block 4, and blank. The size of the blank to be inserted is block length × α (0 <α ≦ 1 decimal), and α is determined so that the timing of turning on / off the backlight 1508 of the display device 1500 is not synchronized with the driving frequency of the liquid crystal panel. To do.

  FIG. 382C is a diagram for explaining a fourth generation example of the transmission frame for one signal unit according to the twenty-seventh embodiment.

  FIG. 382C inserts a blank after any block of the transmission frame. That is, the transmission frames output from the display device 1500 are in the order of block 1, blank, block 2, blank, block 3, and block 4.

[4-3. Operation of visible light communication signal processor]
The difference between the twenty-seventh embodiment and the twenty-fourth embodiment is mainly the operation of the visible light communication signal processing unit 1506. Next, the operation of the visible light communication signal processing unit 1506 will be described. FIG. 383 is a flowchart for explaining the operation of the visible light communication signal processing unit 1506 of the display device 1500 according to the twenty-seventh embodiment.

  The operations from step S1501 to step S1502 are the same as those in the twenty-fourth embodiment.

  (Step S1531) The visible light communication signal processing unit 1506 determines the position where the blank of the transmission unit is inserted.

  (Step S1532) The visible light communication signal processing unit 1506 determines the size of the blank.

  The operations in steps S1503 to S1506 are the same as those in the twenty-fourth embodiment.

[4-4. Effect]
As described above, in the display device according to the present embodiment, the visible light communication processing unit inserts a blank into at least one transmission frame among a plurality of transmission frames for one signal unit.

  Thus, by inserting a blank in the transmission frame for one signal unit, the timing of turning on and off the backlight 1508 of the display device 1500 is avoided from synchronizing with the driving frequency of the liquid crystal panel, and the receiving device 1520 can perform visible light communication. The signal can be restored.

  In the display device 1500 of this embodiment, the relationship between the driving frequency of the liquid crystal panel and the frame rate of the image sensor is a relationship that is an integral multiple or a fraction of an integer, and the transmission frame from the display device 1500 is the driving frequency of the liquid crystal panel. This is particularly effective when output is synchronized with the output.

(Other embodiments)
As described above, Embodiments 24 to 27 have been described as examples of the technology of the present disclosure. The technology of the present disclosure is not limited to this, and can also be applied to embodiments in which changes, replacements, additions, omissions, and the like are made. Moreover, it is also possible to combine each component demonstrated in the said Embodiment 24-27 and it can also be set as a new embodiment.

  In addition, although the display apparatus of this indication showed the example of generation of a transmission frame in case a transmission frame is outputted synchronizing with the drive frequency of a liquid crystal panel, it is not limited to this.

  For example, even when a transmission frame is output without being synchronized with the driving frequency of the liquid crystal panel from the display device, this embodiment is effective when the carrier frequency at which the transmission frame is output is an integral multiple of the frequency of the image sensor. It is.

  Moreover, although the case where the display panel of the display device is a liquid crystal panel has been described, the present invention is not limited to this.

  For example, even if the display device is a signboard whose image film is illuminated from the back by illumination such as an LED, the carrier frequency of the transmission frame output from the display device is an integral multiple of the frequency of the image sensor of the reception device. In this case, the present embodiment is effective.

  The display device according to the present disclosure is applicable to a display device that can output a visible light communication signal, for example, a device such as a television, a personal computer, or a tablet terminal in a home, a signage terminal, an information terminal, or an information display device at a go Is possible.

(Summary)
A display device according to a first aspect of the present disclosure is a display device capable of outputting a visible light communication signal including a plurality of signal units by a carousel method, the display panel displaying a video signal, and the signal unit. A visible light communication processing unit that generates a plurality of transmission frames using the plurality of blocks and generates a backlight control signal, and the display panel based on the backlight control signal. A plurality of transmission frames for one signal unit generated by the visible light communication processing unit, the order of the plurality of blocks of at least two of the transmission frames being different from each other. .

  In the display device according to the second aspect of the present disclosure, the plurality of transmission frames for one signal unit generated by the visible light communication processing unit includes the same block in at least two adjacent transmission frames. The display device according to the first aspect.

  In the display device according to the third aspect of the present disclosure, the plurality of transmission frames for one signal unit generated by the visible light communication processing unit includes a plurality of identical blocks in at least one transmission frame. A display device according to the first aspect, including all of the plurality of blocks in the plurality of transmission frames.

  In the display device according to the fourth aspect of the present disclosure, the visible light communication processing unit detects an area having a luminance of a predetermined level or more of the display panel, and includes the area in the transmission frame according to the size of the area. The display device according to a third aspect, wherein the number of identical blocks is determined, and a plurality of the transmission frames for the signal unit are generated.

  In the display device according to the fifth aspect of the present disclosure, the visible light communication processing unit transmits the transmission frame to the transmission frame according to a distance between the display device and a receiving device that can receive the output visible light communication signal. In the display device according to the third aspect, the number of the same blocks to be included is determined, and a plurality of the transmission frames for the signal unit are generated.

  The display device according to a sixth aspect of the present disclosure is the display device according to the first aspect, in which the visible light communication signal processing unit inserts a reset signal between two adjacent signal units.

  In the display device according to a seventh aspect of the present disclosure, in the first aspect, the visible light communication processing unit inserts a blank in at least one of the transmission frames among the plurality of transmission frames for one signal unit. It is a display apparatus concerning.

  A display method according to an eighth aspect of the present disclosure is a display method capable of outputting a visible light communication signal composed of a plurality of signal units by a carousel method, wherein the signal unit is encoded and divided into a plurality of blocks. A first step of generating a plurality of transmission frames to be output in a carousel method using the plurality of blocks, and outputting the transmission frame as a backlight control signal; and a step of controlling the backlight based on the backlight control signal. In the display method, the plurality of transmission frames for one signal unit generated in step 2 and the first step are different in order of the plurality of blocks in at least two of the transmission frames.

(Embodiment 28)
FIG. 384 is a diagram for describing switching control of visible light communication (VLC) when the transmission device is a moving image display device such as a television.

  Specifically, FIG. 384 (a) is a diagram showing a moving image composed of a plurality of pictures, and FIG. 384 (b) is a backlight of the moving image display device when visible light communication is OFF. FIG. 384 (c) is a diagram showing ON / OFF control of the backlight of the moving image display apparatus when visible light communication is ON.

  As shown in FIG. 384 (a), when a moving image 1600 composed of a plurality of pictures P1601, P1602, P1603, P1604, P1605, P1606,... Is reproduced, a plurality of pictures P1601, P1602, P1603,. P1604, P1605, P1606,... Are displayed on the moving image display device at times t1601, t1603, t1605, t1607, t1609, t1611,. Note that the time t1 is a display start time of the moving image 1600, and may be an absolute time or a time specified by the user. Further, times t1603, t1605, t1607, t1609, t1601,... Are times at predetermined time intervals Δt1600 from time t1. That is, times t1603, t1605, t1607, t1609, t1611,... Are times determined at a constant cycle (predetermined time interval Δt1600).

  When reproducing such a moving image 1600, particularly in a liquid crystal display, there is a liquid crystal display that controls to insert a black picture between adjacent pictures in order to reduce the reproduction of the moving image 1600. In the case of such a moving image display apparatus, as shown in FIG. 384 (b), times t1601, t1603, t1605, t1607 at which a plurality of pictures P1601, P1602, P1603, P1604, P1605, P1606,. , T1609, t1611,... At times t1602, t1604, t1606, t1608, t1610, t1612,..., Control for turning off the backlight of the moving image display device in order to insert a black picture Is going. That is, at times t1601, t1603, t1605, t1607, t1609, t1611,... At which a plurality of pictures P1601, P1602, P1603, P1604, P1605, P1606,. At t1602, t1604, t1606, t1608, t1610, t1612,..., the backlight is turned off.

  However, if the backlight is turned off during visible light communication, the communication will be interrupted while the backlight is off. For this reason, as shown in FIG. 384 (c), when visible light communication is performed (that is, when the VLC is ON), the backlight is kept ON even while the moving image 1600 is being reproduced. Take control. As described above, in the transmission device in this case, when visible light communication is performed, the backlight is kept on as shown in FIG. 384 (c), and when visible light communication is not performed, Control to repeat ON / OFF of the backlight is switched as in 384 (b). Thereby, when visible light communication is performed, it can suppress that communication is interrupted, and when visible light communication is not performed, it can reduce that the moving image 1600 is blurred and reproduced | regenerated.

(Embodiment 29)
In this embodiment, a protocol transmission method for visible light communication will be described.

  FIG. 385 and FIG. 386 are diagrams illustrating a procedure in the case where logical data (for example, ID) used in the application layer is transmitted by visible light communication.

  First, the logical data error correction code assigning unit 1701 assigns a logical data correction code 1712 that is an error correction code to the logical data 1711 used in the application layer.

  Next, the logical data dividing unit 1702 generates a plurality of divided logical data 1713 by dividing the logical data 1711 and the logical data correction code 1712 into data sizes that can be transmitted. Further, the logical data dividing unit 1702 gives a division type 1714 and an address 1715 to each divided logical data 1713.

  The data modulation unit 1703 generates physical data 1716 to be transmitted by converting the data generated by the logical data dividing unit 1702 into a data string that can be transmitted.

  The logical data error correction code assigning unit 1701 uses encoding such as CRC or Reed-Solomon code according to the size of the logical data or the state of the transmission path. Further, when the logical data correction code 1712 is added to the head of the logical data 1711, or when it is added to the end of the logical data 1711, it may be inserted at a specific position of the logical data 1711.

  The logical data dividing unit 1702 can determine the limit distance and the receiving speed that can be received by visible light communication by changing the size after the division. Further, the logical data division unit 1702 can improve the resistance to burst errors in addition to the error resistance by the logical data correction code 1712 and the physical data correction code 1717 by changing the division method, and can also be concealed during data decoding. Can be improved.

  Note that the data modulation unit 1703 is required to maintain the device characteristics of the visible light communication transmission unit regardless of the modulation type such as PPM modulation or Manchester modulation (for example, in the case of a display, it is necessary to keep the brightness as much as possible. Brightness control or modulation rate control can be realized by changing the number of quantizations or sample values corresponding to one bit of logical data depending on whether it is necessary to coexist with moving images or still images. For example, the data modulation unit 1703 uses two values such as “1” of physical data when light is emitted, “0” of physical data when light is not emitted, and brightness at the time of light emission is 100. % Is set to “2”, the brightness at the time of light emission is set to 50%, “1” is set to 0%, and the brightness at the time of light emission is set to “0”. Brightness control can be realized. Further, the data modulation unit 1703 sets the physical data “0” when the light is emitted, sets the physical data “0” when the light is not emitted, and changes the logical data “01” to the physical data “0100”. The average brightness in the physical data transmission size can be controlled by switching the modulation to “11001111”.

  Next, the physical data error correction code giving unit 1704 gives a physical data correction code 1717 that is an error correction code to the physical data 1716 generated by the data modulation unit 1703.

  Next, the physical data header insertion unit 1705 adds a header 1718 for indicating the start position of the physical data 1716 to the physical data 1716. The obtained data is transmitted as visible light communication data by the visible light communication transmitter.

  Note that the physical data error correction code assigning unit 1704 uses encoding such as CRC or Reed-Solomon code according to the size of the physical data 1716 or the state of the transmission path. In addition, when the physical data correction code 1717 is added to the physical data 1716, it may be inserted at a specific position of the physical data 1716 when it is added to the end.

  The physical data header insertion unit 1705 inserts, as a header, preamble data that allows the visible light communication receiving unit to identify the head of the physical data of the visible light communication data. The preamble data to be inserted is a data string that does not appear in the combined data of the physical data 1716 and the physical data correction code 1717 to be transmitted. The physical data header insertion unit 1705 can control the flickering condition and necessary brightness of the visible light communication transmission unit by changing the size of the preamble data and the preamble data string. Furthermore, the preamble data can also be used for identification of the device type at the visible light communication receiver. For example, flicker can be reduced by setting the preamble data so that the difference between the brightness during transmission of the data including the physical data 1716 and the physical data correction code 1717 and the brightness during transmission of the preamble data is minimized. Moreover, the brightness of the preamble data can be adjusted to be small by reducing the light emission period in the preamble.

  In addition, a general interleaving method can be used for the division by the logical data division unit 1702. FIG. 387 is a diagram for explaining the dividing process by the logical data dividing unit 1702.

  FIG. 387 is a diagram illustrating an example of division when the logical data is “010011000111010” and the number of divisions is three. For example, as shown in FIG. 387 (a), the logical data dividing unit 1702 generates a plurality of divided logical data 1713 by dividing the logical data 1711 and the logical data correction code 1712 into 5 bits from the beginning. Alternatively, as shown in FIG. 387 (b), the logical data dividing unit 1702 assigns the logical data 1711 and the logical data correction code 1712 to each of the divided logical data 1713 by allocating 1 bit from the head to each divided logical data 1713. Logical data 1713 is generated.

  Also, as shown in FIG. 388, the logical data dividing unit 1702 defines the number of SKIPs necessary for dividing logical data, and the logical data 1711 and the logical data correction code 1712 are divided into logical units divided by BITs corresponding to the number of SKIPs from the top. A plurality of divided logical data 1713 may be generated by allocating to the data 1713.

  In this case, the logical data dividing unit 1702 can arbitrarily specify the number of SKIPs, thereby providing confidentiality so that the visible light communication receiving unit that does not know the set number of SKIPs cannot restore the logical data. It becomes. The logical data dividing unit 1702 may perform division using a hash value output by applying a hash function based on an arbitrary value, or an arbitrary value that uniquely specifies a division designation BIT by an arbitrary value. An arithmetic expression may be used.

  Furthermore, the logical data dividing unit 1702 can secure confidentiality so that it can be received only at a specific time by using time as an arbitrary value. Further, the logical data dividing unit 1702 can be developed into a service that can receive only a specific channel by using the TV channel number as an arbitrary value. In addition, the logical data dividing unit 1702 can use the data only at the place by using the numerical value related to the place as an arbitrary value.

  In addition, this invention may include the following aspects.

  The transmitter includes a visible light transmission unit and a human sensor unit. The human sensor unit detects that there is a person and starts transmission. Transmission is performed in the direction detected by the human sensor unit when a person is present. Thereby, power consumption can be suppressed.

  The receiver receives the ID of the transmitter, adds address information or current position information, and transmits it to the server. The server sends a code to the receiver for optimal settings for the received address or location. The receiver displays the code received from the server on the screen and indicates to the user to set in the transmitter. Thereby, for example, the setting of the rice cooker and the washing machine can be set to the optimum setting for the water quality in the residential area.

  The receiver changes the exposure time setting for each imaging frame, receives a visible light signal in a frame with a short exposure time, and receives other signals and markers, for example, a two-dimensional barcode, in a frame with a long exposure time. Or perform object recognition or image recognition. Thereby, visible light reception and reception of other signals and markers can be performed simultaneously.

  The receiver performs imaging by changing the exposure time little by little for each frame. Thereby, even if the modulation frequency of the transmission signal is unknown, the image of any frame is captured with an appropriate exposure time, and the signal can be demodulated. Further, the received signal can be demodulated more efficiently by imaging the same signal with a plurality of exposure times.

  When the receiver receives an ID in a predetermined range, the receiver does not make an inquiry to the server, and passes the received ID to another processing unit as it is. Thereby, a quick response is obtained. Moreover, even in a state where connection to the server is not possible, processing can be performed. In addition, the operation can be confirmed before setting the content in the server.

  The transmitter expresses a transmission signal by amplitude modulation. At this time, in a plurality of symbols representing different signals, the duration of either the low luminance state or the high luminance state is made equal. Thereby, a signal can be expressed even with a low control clock.

  The transmitter registers the transmission ID and content in the server at the time of activation. Thereby, desired content can be transmitted from the server to the receiver.

  A part of the ID can be freely set by the transmitter. Thereby, the code indicating the state of the transmitter can be included in the ID. The receiver and the server may change the display content depending on this part, or may ignore it.

(Multi-value amplitude pulse signal)
389, 390, and 391 are diagrams illustrating an example of a transmission signal in this embodiment.

  By giving meaning to the amplitude of the pulse, more information can be expressed per unit time. For example, when the amplitude is classified into three stages, as shown in FIG. 389, three values can be expressed by the transmission time of two slots while keeping the average luminance at 50%. However, if (c) in FIG. 389 is transmitted continuously, there is no change in luminance, so the presence of a signal is difficult to understand. Also, ternary values are a little difficult to handle in digital processing.

  Therefore, by using the four types of symbols (a) to (d) in FIG. 390, four values can be expressed with an average transmission time of 3 slots while maintaining the average luminance at 50%. Although the transmission time differs depending on the symbol, it is possible to recognize the end point of the symbol by setting the last state of the symbol to a low luminance state. The same effect can be obtained even when the high luminance state and the low luminance state are switched. 390 (e) is not suitable because it cannot be distinguished from transmitting (a) twice. 390 (f) and (g) in FIG. 390 are somewhat difficult to recognize because the intermediate luminance is continuous, but can be used.

  Consider using the patterns of (a) and (b) of FIG. 391 as headers. Since these patterns have strong specific frequency components in frequency analysis, signal detection can be performed by frequency analysis by using these patterns as headers.

  As shown in (c) of FIG. 391, a transmission packet is configured using the patterns (a) and (b). Data can be delimited by using a pattern with a specific length as a header of the entire packet and using a pattern with a different length as a separator. Moreover, signal detection can be facilitated by including this pattern in the middle. Thus, even when one packet is longer than the image capturing time of one frame image, the receiver can connect and decode the data. This also makes it possible to make the packet length variable by adjusting the number of separators. The length of the entire packet may be expressed by the length of the packet header pattern. Moreover, the receiver can synthesize the partially received data by using the separator as a packet header and the length of the separator as the data address.

  The transmitter repeatedly transmits the packet configured as described above. The contents of packets 1 to 4 in (c) of FIG. 391 may all be the same, or may be combined as different data on the receiving side.

Embodiment 30
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  FIG. 392A is a diagram for describing a transmitter in this embodiment.

  The transmitter in this embodiment is configured as a backlight of a liquid crystal display, for example, and includes a blue LED 2303 and a phosphor 2310 including a green fluorescent component 2304 and a red fluorescent component 2305.

  The blue LED 2303 emits blue (B) light. The phosphor 2310 emits yellow (Y) light when receiving blue light emitted from the blue LED 2303 as excitation light. That is, the phosphor 2310 emits yellow light. Specifically, since the phosphor 2130 includes a green fluorescent component 2304 and a red fluorescent component 2305, yellow light is emitted by the emission of these fluorescent components. Of these two fluorescent components, the green fluorescent component 2304 emits green light when receiving blue light emitted from the blue LED 2303 as excitation light. That is, the green fluorescent component 2304 emits green (G) light. Of the two fluorescent components described above, the red fluorescent component 2305 emits red light when receiving blue light emitted from the blue LED 2303 as excitation light. That is, the red fluorescent component 2305 emits red (R) light. Thereby, since each light of RGB or Y (RG) B is emitted, a transmitter outputs white light as a backlight.

  This transmitter transmits a visible light signal of white light by changing the luminance of the blue LED 2303 in the same manner as in the above embodiments. At this time, a visible light signal having a predetermined carrier frequency is output as the luminance of white light changes.

  Here, the barcode reader irradiates the barcode with red laser light, and reads the barcode based on the luminance change of the red laser light reflected from the barcode. The barcode reading frequency of the red laser light may coincide with or approximate the carrier frequency of the visible light signal output from a general transmitter currently in practical use. Therefore, in such a case, when the barcode reader attempts to read a barcode illuminated with white light, which is a visible light signal from the general transmitter, the luminance of the red light contained in the white light Depending on the change, the reading may fail. In other words, a barcode reading error occurs due to interference between the carrier frequency of the visible light signal (particularly red light) and the barcode reading frequency.

  Therefore, a fluorescent material having a longer afterglow duration than that of the green fluorescent component 2304 is used for the red fluorescent component 2305 in the present embodiment. That is, the red fluorescent component 2305 in the present embodiment changes in luminance at a frequency sufficiently lower than the luminance change frequency of the blue LED 2303 and the green fluorescent component 2304. In other words, the red fluorescent component 2305 smoothes the frequency of the red luminance change included in the visible light signal.

  FIG. 392B is a diagram illustrating changes in luminance of RGB.

  The blue light from the blue LED 2303 is included in the visible light signal and output, as shown in (a) of FIG. 392B. As shown in (b) of FIG. 392B, the green fluorescent component 2304 emits green light when receiving blue light from the blue LED 2303. The duration of afterglow in the green fluorescent component 2304 is short. Therefore, when the blue LED 2303 changes in luminance, the green fluorescent component 2304 emits green light whose luminance changes at substantially the same frequency as the luminance change frequency of the blue LED 2303 (that is, the visible light signal carrier frequency). .

  As shown in (c) of FIG. 392B, the red fluorescent component 2305 emits red light when receiving blue light from the blue LED 2303. The duration of the afterglow in the red fluorescent component 2305 is long. Therefore, when the blue LED 2303 changes in luminance, the red fluorescent component 2305 emits red light whose luminance changes at a frequency lower than the frequency of luminance change of the blue LED 2303 (that is, the carrier frequency of the visible light signal). .

  FIG. 393 is a diagram illustrating the afterglow characteristics of the green fluorescent component 2304 and the red fluorescent component 2305 in the present embodiment.

  For example, when the blue LED 2303 is lit without a change in luminance, the green fluorescent component 2304 emits green light having an intensity I = I0 without changing the luminance (that is, light having a luminance change frequency f = 0). . Further, even if the blue LED 2303 changes in luminance at a low frequency, the green fluorescent component 2304 emits green light having an intensity I = I0 that changes in luminance at substantially the same frequency f as the low frequency. However, when the luminance of the blue LED 2303 changes at a high frequency, the intensity I of the green light emitted from the green fluorescent component 2304 that changes in luminance at substantially the same frequency f as the high frequency is influenced by the afterglow in the green fluorescent component 2304. Therefore, the intensity becomes smaller than the intensity I0. As a result, the intensity I of green light emitted from the green fluorescent component 2304 is maintained at I = I0 when the frequency f of the luminance change of the light is less than the threshold value fb, as shown by the dotted line in FIG. However, when the frequency f exceeds the threshold value fb, it gradually decreases.

  Further, the afterglow duration of the red fluorescent component 2305 in this embodiment is longer than the duration of the afterglow of the green fluorescent component 2304. Therefore, the intensity I of the red light emitted from the red fluorescent component 2305 is, as shown by the solid line in FIG. 393, until the frequency f of the luminance change of the light is less than the threshold fa lower than the threshold fb. Although it is kept at I0, it becomes gradually smaller when the frequency f becomes higher than the threshold value fb. In other words, the red light emitted from the red fluorescent component 2305 does not exist in the high frequency region of the frequency band of the green light emitted from the green fluorescent component 2304 and exists only in the low frequency region.

  More specifically, for the red fluorescent component 2305 in the present embodiment, a fluorescent material is used in which the intensity I of red light emitted at the same frequency f as the carrier frequency f1 of the visible light signal is I = I1. It is done. The carrier frequency f1 is a carrier frequency of luminance change by the blue LED 2303 provided in the transmitter. The intensity I1 is 1/3 of the intensity I0 or -10 dB of the intensity I0. For example, the carrier frequency f1 is 10 kHz or 5 to 100 kHz.

  That is, the transmitter according to the present embodiment is a transmitter that transmits a visible light signal, and receives a blue LED that emits blue light whose luminance changes as light included in the visible light signal, and the blue light. A green fluorescent component that emits green light as light included in the visible light signal, and a red fluorescent component that emits red light as light included in the visible light signal by receiving the blue light. The duration of afterglow in the red fluorescent component is longer than the duration of afterglow in the green fluorescent component. The green fluorescent component and the red fluorescent component may be included in a single phosphor that receives yellow light and emits yellow light as light included in the visible light signal. Alternatively, the green fluorescent component may be included in a green phosphor, and the red fluorescent component may be included in a red phosphor that is separate from the green phosphor.

  Thereby, since the duration of the afterglow in the red fluorescent component is long, the luminance of red light can be changed at a frequency lower than the frequency in the luminance change of blue and green light. Therefore, even if the frequency of the luminance change of blue and green light included in the visible light signal of white light is the same as or close to the barcode reading frequency of the red laser light, it is included in the visible light signal of white light. The frequency of the red light can be significantly different from the barcode reading frequency. As a result, the occurrence of barcode reading errors can be suppressed.

  Here, the red fluorescent component may emit red light whose luminance changes at a frequency lower than the frequency of luminance change of light emitted from the blue LED.

  The red fluorescent component may include a red fluorescent material that emits red light by receiving blue light and a low-pass filter that transmits only light in a predetermined frequency band. For example, the low-pass filter transmits only light in a low frequency band out of blue light emitted from the blue LED and applies the light to the red fluorescent material. The red fluorescent material may have the same afterglow characteristics as the green fluorescent component. Alternatively, the low-pass filter transmits only light in a low frequency band out of red light emitted from the red fluorescent material when blue light emitted from the blue LED hits the red fluorescent material. Let Even when such a low-pass filter is used, the occurrence of barcode reading errors can be suppressed as described above.

  The red fluorescent component may be made of a fluorescent material having a predetermined afterglow characteristic. For example, predetermined afterglow characteristics are: (a) the intensity of the red light when the frequency f of the luminance change of the red light emitted from the red fluorescent component is 0 is I0, and (b) When the carrier frequency in the luminance change of the light emitted from the blue LED is f1, when the frequency f of the red light is f = f1, the intensity of the red light is 1/3 or less of the I0. Or -10 dB or less.

  As a result, the frequency of the red light included in the visible light signal can be surely greatly varied from the barcode reading frequency. As a result, the occurrence of barcode reading errors can be reliably suppressed.

  Further, the carrier frequency f1 may be approximately 10 kHz.

  As a result, since the carrier frequency used for transmitting visible light signals, which is currently in practical use, is 9.6 kHz, it is effective to generate bar code reading errors in this practical transmission of visible light signals. Can be suppressed.

  The carrier frequency f1 may be approximately 5 to 100 kHz.

  With the advance of image sensors (imaging devices) of receivers that receive visible light signals, it is assumed that carrier frequencies such as 20 kHz, 40 kHz, 80 kHz, or 100 kHz will be used in future visible light communication. Therefore, by setting the above carrier frequency f1 to approximately 5 to 100 kHz, it is possible to effectively suppress the occurrence of barcode reading errors in future visible light communication.

  In the present embodiment, regardless of whether the green fluorescent component and the red fluorescent component are contained in a single phosphor, or each of these two fluorescent components is contained in a separate phosphor, The above effects can be achieved. That is, even when a single phosphor is used, the afterglow characteristics, that is, the frequency characteristics, of the red light and the green light emitted from the phosphor are different. Therefore, the above-mentioned effects can also be achieved by using a single phosphor that is inferior in afterglow characteristics or frequency characteristics in red light and superior in afterglow characteristics or frequency characteristics in green light. Note that poor afterglow characteristics or frequency characteristics means that the duration of afterglow is long or the intensity of light in a high frequency band is weak, and that the afterglow characteristics or frequency characteristics are superior means that afterglow characteristics The duration of the light is short, or the light intensity in the high frequency band is high.

  Here, in the example shown in FIGS. 392A to 393, the occurrence of barcode reading errors is suppressed by smoothing the frequency of the red luminance change included in the visible light signal. By making it high, the occurrence of the reading error may be suppressed.

  FIG. 394 is a diagram for describing a problem newly generated in order to suppress occurrence of a barcode reading error.

  As shown in FIG. 394, when the carrier frequency fc of the visible light signal is about 10 kHz, the reading frequency of the red laser light used for reading the barcode is also about 10 to 20 kHz. A barcode reading error occurs.

  Therefore, by increasing the carrier frequency fc of the visible light signal from about 10 kHz to, for example, 40 kHz, occurrence of barcode reading errors can be suppressed.

  However, if the carrier frequency fc of the visible light signal is about 40 kHz, the sampling frequency fs for the receiver to sample the visible light signal by photographing needs to be 80 kHz or more.

  That is, since the sampling frequency fs required in the receiver is high, there arises a new problem that the processing burden on the receiver increases. In order to solve this new problem, the receiver in this embodiment performs downsampling.

  FIG. 395 is a diagram for describing downsampling performed by the receiver in this embodiment.

  The transmitter 2301 in the present embodiment is configured as, for example, a liquid crystal display, digital signage, or a lighting device. The transmitter 2301 then outputs a frequency-modulated visible light signal. At this time, the transmitter 2301 switches the carrier frequency fc of the visible light signal to, for example, 40 kHz and 45 kHz.

  The receiver 2302 in this embodiment photographs the transmitter 2301 at a frame rate of 30 fps, for example. At this time, similarly to the receiver in each of the above embodiments, the receiver 2302 performs imaging with a short exposure time so that bright lines are generated in each image (specifically, each frame) obtained by imaging. The image sensor used for photographing by the receiver 2302 has, for example, 1000 exposure lines. Accordingly, in one-frame shooting, visible light signals are sampled by starting exposure at different timings for 1000 exposure lines. As a result, in 1 second, 30 fps × 1000 lines = 30000 samplings (30 ks / second) are performed. In other words, the sampling frequency fs of the visible light signal is 30 kHz.

  According to a general sampling theorem, at a sampling frequency fs = 30 kHz, only a visible light signal having a carrier frequency of 15 kHz or less can be demodulated.

  However, the receiver 2302 in this embodiment downsamples a visible light signal having a sampling frequency fs = 30 kHz and a carrier frequency fc = 40 kHz or 45 kHz. Although aliasing occurs in the frame by this downsampling, the receiver 2302 in this embodiment estimates the carrier frequency fc of the visible light signal by observing and analyzing the aliasing.

  FIG. 396 is a flowchart illustrating processing operations of the receiver 2302 in this embodiment.

  First, the receiver 2302 performs down-sampling with a sampling frequency fs = 30 kHz on a visible light signal with a carrier frequency fc = 40 kHz or 45 kHz by photographing a subject (step S2310).

  Next, the receiver 2302 observes and analyzes an alias generated in the frame obtained by the downsampling (step S2311). As a result, the receiver 2302 specifies the frequency of the alias as, for example, 5.1 kHz or 5.5 kHz.

  Then, the receiver 2302 estimates the carrier frequency fc of the visible light signal based on the identified alias frequency (step S2311). That is, the receiver 2302 restores the original frequency from the alias. Thereby, the receiver 2302 estimates the carrier frequency fc of the visible light signal as 40 kHz or 45 kHz, for example.

  As described above, the receiver 2302 in this embodiment can appropriately receive a visible light signal having a high carrier frequency by performing downsampling and restoration of a frequency based on an alias. For example, the receiver 2302 can receive a visible light signal having a carrier frequency of 30 kHz to 60 kHz even if the sampling frequency is fs = 30 kHz. Accordingly, the carrier frequency of the visible light signal can be increased from 30 kHz to 60 kHz from the frequency (about 10 kHz) currently in practical use. As a result, the carrier frequency of the visible light signal and the barcode reading frequency (10 to 20 kHz) can be greatly different, and interference between the frequencies can be suppressed. As a result, the occurrence of barcode reading errors can be suppressed.

  Such a receiving method in the present embodiment is a receiving method for acquiring information from a subject, and corresponds to a plurality of exposure lines included in the image sensor in a frame obtained by photographing the subject by an image sensor. An exposure time setting step for setting an exposure time of the image sensor so that a plurality of bright lines are generated according to a change in luminance of the subject, and exposure of each of the plurality of exposure lines included in the image sensor is sequentially performed at different times. The image sensor causes the image sensor to shoot the subject whose luminance changes at a predetermined frame rate and at the set exposure time, and for each frame obtained by the shooting. , The data specified by the plurality of bright line patterns included in the frame. Including an information acquisition step of acquiring information by demodulating the data. In the photographing step, each of the plurality of exposure lines sequentially repeats starting exposure at different times, so that the sampling frequency is lower than the carrier frequency of the visible light signal transmitted by the luminance change of the subject. The visible light signal is down-sampled, and in the information acquisition step, the frequency of the alias specified by the pattern of the plurality of bright lines included in the frame is specified and specified for each frame obtained by the imaging. The information is obtained by estimating the frequency of the visible light signal from the frequency of the alias and demodulating the estimated frequency of the visible light signal.

  In such a reception method, a visible light signal having a high carrier frequency can be appropriately received by performing downsampling and restoration of the frequency based on the alias.

  In the downsampling, a visible light signal having a carrier frequency higher than 30 kHz may be downsampled. Thereby, interference between the carrier frequency of the visible light signal and the barcode reading frequency (10 to 20 kHz) can be avoided, and barcode reading errors can be more effectively suppressed.

(Embodiment 31)
FIG. 397 is a diagram illustrating processing operations of the reception device (imaging device). Specifically, FIG. 397 is a diagram for describing an example of switching processing between the normal imaging mode and the macro imaging mode in the case of receiving visible light communication.

  Here, the reception device 1610 receives visible light emitted from a transmission device including a plurality of light sources (four light sources in FIG. 397).

  First, when the receiving device 1610 transitions to a mode for performing visible light communication, the receiving device 1610 activates the imaging unit in the normal imaging mode (S1601). Note that the receiving device 1610 displays a frame 1611 for imaging a light source on the screen when the mode is changed to a mode for performing visible light communication.

  After a predetermined time, the receiving device 1610 switches the imaging mode of the imaging unit to the macro imaging mode (S1602). Note that the timing of switching from step S1601 to step S1602 may not be after a predetermined time from step S1601, but may be when the receiving device 1610 determines that the light source is captured within the frame 1611. By switching to the macro imaging mode in this way, the user can easily place the light source in the frame 1611 because the light source can be stored in the frame 1611 with a clear image in the normal imaging mode before the image is blurred in the macro imaging mode. Can fit in.

  Next, the receiving device 1610 determines whether or not a signal from the light source has been received (S1603). If it is determined that the signal from the light source is received (Yes in S1603), the process returns to the normal imaging mode in Step S1601, and if it is determined that the signal from the light source is not received (No in S1603), the macro imaging in Step 1602 is performed. Continue mode. In the case of Yes in step S1603, processing based on the received signal (for example, processing for displaying an image indicated by the received signal) may be performed.

  According to the receiving device 1610, the user can take an image in a blurred state by switching from the normal imaging mode to the macro imaging mode by touching the display unit of the light source 1611 of the smartphone with a finger. For this reason, the image captured in the macro imaging mode includes more bright areas than the image captured in the normal imaging mode. In particular, between two adjacent light sources among a plurality of light sources, the light from the two light sources overlaps, so that the striped images are separated as shown in the left diagram of FIG. The problem that it cannot be received as a signal can be demodulated as a continuous reception signal for forming a continuous stripe as shown in the right figure. Since a long code can be received at once, the response time is shortened. As shown in FIG. 397 (b), when a photographed image is first photographed with a normal shutter and a normal focus, a beautiful normal image is obtained. However, if the light source is far away, such as text, even if the shutter speed is increased, continuous data cannot be obtained and cannot be demodulated. Next, when the speed of the shutter is increased and the focus driving unit of the lens is set to a short distance (macro), the light source is blurred and spreads, and the four light sources are connected, so that data can be received. Next, when the focus is returned and the shutter speed is returned to normal, the original beautiful image can be obtained. As shown in (c), by recording and displaying a beautiful image in the memory on the display unit, there is an effect that only the beautiful image is displayed on the display unit. The image captured in the macro imaging mode includes more areas brighter than the predetermined brightness than the image captured in the normal imaging mode. Therefore, in the macro imaging mode, the number of exposure lines that can generate bright lines for the subject can be increased.

  FIG. 398 is a diagram illustrating processing operations of the reception device (imaging device). Specifically, FIG. 398 is a diagram for describing another example of the switching process between the normal imaging mode and the macro imaging mode when receiving visible light communication.

  Here, the reception device 1620 receives visible light emitted from a transmission device including a plurality of light sources (four light sources in FIG. 398).

  First, when the receiving device 1620 transitions to a mode for performing visible light communication, the imaging device is activated in the normal imaging mode, and an image 1623 having a wider range than the image 1622 displayed on the screen of the receiving device 1620 is captured. . The image data indicating the captured image 1623 and the posture information indicating the posture of the receiving device 1620 detected by the gyro sensor, the geomagnetic sensor, and the acceleration sensor of the receiving device 1620 when the image 1623 is captured are stored in the memory. (S1611). Note that the captured image 1623 is an image having a wide range by a predetermined width in the vertical direction and the horizontal direction with reference to the image 1622 displayed on the screen of the reception device 1620. In addition, when the receiving apparatus 1620 transitions to a mode for performing visible light communication, the receiving apparatus 1620 displays a frame 1621 for imaging a light source on the screen.

  After a predetermined time, the receiving device 1620 switches the imaging mode of the imaging unit to the macro imaging mode (S1612). Note that the timing of switching from step S1611 to step S1612 is not after a predetermined time from step S1611, but when the image 1623 is captured and it is determined that the image data indicating the captured image 1623 is held in the memory. Good. At this time, the receiving device 1620 displays an image 1624 having a size corresponding to the screen size of the receiving device 1620 among the images 1623 based on the image data held in the memory.

  Note that an image 1624 displayed on the receiving device 1620 at this time is a part of the image 1623, and the posture of the receiving device 1620 indicated by the posture information acquired in step S1611 (indicated by a white broken line). Position) and the current posture of the receiving device 1620. This is an image of an area predicted to be captured by the current receiving device 1620. That is, the image 1624 is a partial image of the image 1623 and is an image of an area corresponding to the imaging target of the image 1625 actually captured in the macro imaging mode. That is, in step S1612, the posture (imaging direction) changed from the time of step S1611 is acquired, and the imaging target that is presumed to be currently imaged from the acquired current posture (imaging direction) is identified and imaged in advance. An image 1624 corresponding to the current posture (imaging direction) is specified from the image 1623, and processing for displaying the image 1624 is performed. For this reason, as shown by an image 1623 in FIG. 398, when the receiving device 1620 moves from the position indicated by the white broken line in the direction of the white arrow, the receiving device 1620 cuts out the image 1623 according to the amount of movement. An area 1624 can be determined, and an image 1624 that is the image 1623 in the determined area can be displayed.

  Accordingly, the reception device 1620 does not display the image 1625 captured in the macro imaging mode even when capturing in the macro imaging mode, and starts from the image 1623 captured in the clearer normal imaging mode. An image 1624 cut out in accordance with the current posture of the receiving device 1620 can be displayed. In the method of the present invention in which continuous visible light information is obtained from a plurality of light sources that are separated from a defocused image, the stored normal plane image is displayed on the display unit, and the user takes a picture using a smartphone. In this case, camera shake occurs, and the direction of the actual captured image and the direction of the still image displayed from the memory shifts, and a problem that the user cannot adjust the direction to the target light source is expected to occur. In this case, countermeasures are necessary because data from the light source cannot be received. However, according to the present invention, even if the camera shake is detected, the camera shake is detected by the image swing detection means or the swing gyro detection means, and the target image in the still image is shifted in a predetermined direction. The user can see the deviation from the direction. This display makes it possible for the user to point the camera at the target light source, so that multiple divided light sources can be photographed while displaying normal images, and signals are received continuously. can do. Thereby, since the normal image is displayed, the light source divided into a plurality of parts can be received. In this case, it is possible to easily adjust the posture of the reception device 1620 so that a plurality of light sources fits the frame 1621. Note that when the focal point is blurred, the light source is dispersed and the luminance is equivalently reduced. Therefore, there is an effect that the visible light data can be received more reliably by increasing the sensitivity of the ISO of the camera.

Next, the receiving device 1620 determines whether or not a signal from the light source has been received (S1613). If it is determined that the signal from the light source has been received (Yes in S1613), the process returns to the normal imaging mode in step S1611, and if it is determined that the signal from the light source has not been received (S1).
In step 613, the macro imaging mode in step 1612 is continued. In the case of Yes in step S1613, processing based on the received signal (for example, processing for displaying an image indicated by the received signal) may be performed.

  Similarly to the reception device 1610, the reception device 1620 can capture an image including a brighter region in the macro imaging mode. For this reason, in the macro imaging mode, the number of exposure lines that can generate bright lines for the subject can be increased.

  FIG. 399 is a diagram illustrating processing operations of the reception device (imaging device).

  Here, the transmission device 1630 is a display device such as a television, for example, and transmits different transmission IDs by visible light communication at a predetermined time interval Δ1630. Specifically, at times t1631, t1632, t1633, and t1634, ID1631, ID1632, ID1633, and ID1634, which are transmission IDs associated with the data corresponding to the displayed images 1631, 1632, 1633, and 1634, respectively, are transmitted. To do. That is, ID 1631 to ID 1634 are transmitted one after another at a predetermined time interval Δt 1630 from the transmission device 1630.

  The receiving device 1640 requests the data associated with each transmission ID from the server 1650 based on the transmission ID received by visible light communication, receives the data from the server, and displays an image corresponding to the data. Specifically, images 1641, 1642, 1643, and 1644 respectively corresponding to ID1631, ID1632, ID1633, and ID1634 are displayed at times t1631, t1632, t1633, and t1634, respectively.

  When receiving the ID 1631 received at time t1631, the reception device 1640 may acquire ID information indicating a transmission ID scheduled to be transmitted from the transmission device 1630 from the server 1650 at subsequent times t1632 to t1634. In this case, the reception device 1640 uses the acquired ID information, so that the data associated with ID 1632 to ID 1634 at time t1632 to t1634 can be stored in the server 1650 without receiving a transmission ID from the transmission device 1630 each time. The received data can be displayed at times t1632 to t1634.

  Further, the receiving device 1640 may request data corresponding to the ID 1631 at time t1631 without acquiring information indicating the transmission ID scheduled to be transmitted from the transmitting device 1630 from time t1632 to t1634 from the server 1650. The data associated with the transmission ID corresponding to the subsequent times t1632 to t1634 may be received from the server 1650, and the received data may be displayed at each time t1632 to t1634. That is, when the server 1650 receives a request for data associated with the ID 1631 transmitted from the receiving device 1640 at the time t1631, the server 1650 transmits the data associated with the transmission ID corresponding to the subsequent times t1632 to t1634 to the receiving device. Even if there is no request from 1640, transmission is made to receiving apparatus 1640 at times t1632 to t1634. In other words, in this case, the server 1650 holds association information in which each time t1631 to 1634 is associated with data associated with the transmission ID corresponding to each time t1631 to 1634, and based on the association information. The predetermined data associated with the predetermined time is transmitted at the predetermined time.

  As described above, if the receiving device 1640 can acquire the transmission ID 1631 by visible light communication at time t1631, data corresponding to each time t1632 to t1634 from the server 1650 can be obtained without performing visible light communication at subsequent times t1632 to t1634. Can be received. For this reason, the user does not need to keep the receiving device 1640 directed at the transmitting device 1630 in order to acquire the transmission ID by visible light communication, and can easily display the data acquired from the server 1650 on the receiving device 1640. In this case, when the receiving device 1640 obtains data corresponding to the ID from the server every time, the time delay from the server occurs and the response time becomes longer. Therefore, in order to speed up the response, data corresponding to the ID is stored in advance in the storage unit of the receiver from the server or the like, and the response time is displayed by displaying the data corresponding to the ID in the storage unit. Can be quick. In this method, if the time information for outputting the next ID is included in the transmission signal from the visible light transmitter, the receiver side can obtain the time even if the visible light signal is not continuously received. In this case, since it is possible to know the transmission time of the next ID, there is an effect that it is not necessary to keep the receiving device directed toward the light source. This system only synchronizes the time information (clock) on the transmitter side with the time information (clock) on the receiver side when receiving visible light. However, there is an effect that a screen synchronized with the transmitter can be continuously displayed.

  In the above-described example, the receiving device 1640 displays images 1641, 1642, 1643, and 1644 corresponding to the transmission IDs of ID1631, ID1632, ID1633, and ID1634, respectively, at times t1631, t1632, t1633, and t1634. Displayed respectively. Here, as illustrated in FIG. 400, the receiving device 1640 may present not only the image but also other information at each time. That is, at time t1631, the receiving device 1640 displays the image 1641 corresponding to the ID 1631 and outputs sound or sound corresponding to the ID 1631. At this time, the receiving device 1640 may further display, for example, a purchase site for the product displayed in the image. Such sound output and purchase site display are performed in the same manner at times t1632, t1633, and t1634 other than time t1631.

Next, in the case of a smartphone equipped with two right and left cameras for stereoscopic use as shown in FIG. 397 (b), an image with a normal image quality is displayed with a normal shutter speed and a normal focus for the left eye. At the same time, the right-eye camera uses a shutter that is faster than the left eye and / or is set to a focal point or macro at a short distance to obtain the stripe-like bright line of the present invention and demodulate the data. As a result, an image having normal image quality is displayed on the display unit, and the optical communication data of a plurality of light sources divided in distance can be received by the right eye camera.

(Embodiment 32)
Here, an application example of audio synchronized playback will be described below.

  FIG. 401 is a diagram illustrating an example of an application in the thirty-second embodiment.

  For example, the receiver 1800a configured as a smartphone receives a signal (visible light signal) transmitted from a transmitter 1800b configured as street digital signage, for example. That is, the receiver 1800a receives the timing of image reproduction by the transmitter 1800b. The receiver 1800a reproduces sound at the same timing as the image reproduction. In other words, the receiver 1800a performs synchronized reproduction of the sound so that the image and sound reproduced by the transmitter 1800b are synchronized. Note that the receiver 1800a may reproduce the same image as the image (reproduced image) reproduced by the transmitter 1800b or a related image related to the reproduced image together with the sound. Further, the receiver 1800a may cause a device connected to the receiver 1800a to reproduce sound and the like. Further, after receiving the visible light signal, the receiver 1800a may download content such as sound or related images associated with the visible light signal from the server. The receiver 1800a performs synchronous reproduction after the download.

  Thereby, even when the sound from the transmitter 1800b cannot be heard or when the sound from the transmitter 1800b is not reproduced because the street sound reproduction is prohibited, the user can select the sound that matches the display of the transmitter 1800b. Can hear. Further, even when there is a distance that takes time to reach the voice, it is possible to listen to the voice that matches the display.

  Here, the multilingual correspondence by the audio synchronous reproduction will be described below.

  FIG. 402 is a diagram illustrating an example of an application in the thirty-second embodiment.

  Each of the receiver 1800a and the receiver 1800c obtains and reproduces audio corresponding to a video such as a movie displayed on the transmitter 1800d from the server, in the language set in the receiver. To do. Specifically, the transmitter 1800d transmits a visible light signal indicating an ID for identifying the displayed video to the receiver. When the receiver receives the visible light signal, the receiver transmits a request signal including the ID indicated in the visible light signal and the language set in the receiver to the server. The receiver acquires the audio corresponding to the request signal from the server and reproduces it. Thereby, the user can enjoy the work displayed on the transmitter 1800d in the language set by the user.

  Here, the audio synchronization method will be described below.

  403 and 404 are diagrams illustrating an example of a transmission signal and an example of a voice synchronization method in Embodiment 32. FIG.

  Different data (for example, data shown in FIG. 403: 1 to 6 and the like) are associated with a time every fixed time (N seconds). These data may be, for example, an ID for identifying time, may be time, or may be audio data (for example, 64 Kbps data). The following description is based on the assumption that the data is an ID. Different IDs may have different additional information parts attached to the ID.

  It is desirable that the packets constituting the ID are different. Therefore, it is desirable that IDs are not continuous. Alternatively, it is desirable to use a packetizing method in which the discontinuous portion is configured as one packet when the ID is packetized. Since error correction signals tend to have different patterns even with consecutive IDs, the error correction signals may be configured to be distributed in a plurality of packets rather than being combined into one packet.

  The transmitter 1800d transmits the ID in accordance with the reproduction time of the displayed image, for example. The receiver can recognize the reproduction time (synchronization time) of the image of the transmitter 1800d by detecting the timing when the ID is changed.

  In the case of (a), since the change time points of ID: 1 and ID: 2 are received, the synchronization time can be accurately recognized.

  When the time N during which the ID is transmitted is long, there are few such opportunities, and the ID may be received as shown in (b). Even in this case, the synchronization time can be recognized by the following method.

  (B1) It is assumed that the midpoint of the reception section where the ID has changed is the ID change point. Further, the time after an integer multiple of the time N from the ID change point estimated in the past is also estimated as the ID change point, and the midpoint of the plurality of ID change points is estimated as a more accurate ID change point. With such an estimation algorithm, an accurate ID change point can be gradually estimated.

  (B2) In addition to the above, it is possible that the reception section where the ID has not changed and the time after an integer multiple of the time N are gradually included in the ID change point by estimating that the ID change point is not included. There are fewer sections of the ID, and an accurate ID change point can be estimated.

  By setting N to 0.5 seconds or less, accurate synchronization can be achieved.

  By setting N to 2 seconds or less, synchronization can be performed without causing the user to feel a delay.

  By setting N to 10 seconds or less, it is possible to synchronize while suppressing waste of ID.

  404 is a diagram illustrating an example of a transmission signal in Embodiment 32. FIG.

  In FIG. 404, it is possible to avoid wasting ID by performing synchronization using time packets. A time packet is a packet that holds the time of transmission. When it is necessary to express a long time, the time packet is divided into a time packet 1 representing a fine time and a time packet 2 representing a rough time. For example, time packet 2 indicates the hour and minute of the time, and time packet 1 indicates only the second of the time. A packet indicating the time may be divided into three or more time packets. Since the coarse time is less necessary, the receiver can recognize the synchronization time quickly and accurately by transmitting more fine time packets than coarse time packets.

  That is, in the present embodiment, the visible light signal includes the second information (hour packet 2) indicating the hour and minute of the time, and the first information (time packet 1) indicating the second of the time. The time when the visible light signal is transmitted from the transmitter 1800d is indicated. The receiver 1800a receives the second information and receives the first information more times than the number of times of receiving the second information.

  Here, the synchronization time adjustment will be described below.

  FIG. 405 is a diagram illustrating an example of process flow of the receiver 1800a in Embodiment 32.

  Since it takes a certain amount of time for the audio or video to be played back after the signal is transmitted and processed by the receiver 1800a, it is possible to accurately reproduce the audio or video in anticipation of this processing time. Synchronous playback can be performed.

  First, a processing delay time is designated for the receiver 1800a (step S1801). This may be stored in the processing program or specified by the user. When the user performs correction, it is possible to realize more accurate synchronization according to the individual receiver. This processing delay time can be synchronized more accurately by changing it depending on the receiver model, the temperature of the receiver, and the CPU usage rate.

  The receiver 1800a determines whether a time packet has been received or whether an ID associated for voice synchronization has been received (step S1802). Here, when the receiver 1800a determines that it has been received (Y in step S1802), it further determines whether there is an image waiting for processing (step S1804). If it is determined that there is an image waiting for processing (Y in step S1804), the receiver 1800a discards the image waiting for processing or delays processing of the image waiting for processing to receive from the latest acquired image. Is performed (step S1805). Thereby, it is possible to avoid an unexpected delay due to the amount of waiting for processing.

  The receiver 1800a measures the position in the image where the visible light signal (specifically, the bright line) is located (step S1806). In other words, by measuring the position in the direction perpendicular to the exposure line from the first exposure line in the image sensor, the time difference (delay time in the image) from the image acquisition start time to the signal reception time is obtained. Can be calculated.

  The receiver 1800a can accurately perform synchronized reproduction by reproducing the sound or moving image at the time obtained by adding the processing delay time and the in-image delay time to the recognized synchronization time (step S1807).

  On the other hand, if the receiver 1800a determines in step S1802 that it has not received the time packet or the audio synchronization ID, it receives a signal from the image obtained by imaging (step S1803).

  FIG. 406 is a diagram illustrating an example of a user interface of the receiver 1800a in Embodiment 32.

  As shown in FIG. 406 (a), the user can adjust the processing delay time described above by pressing one of the buttons Bt1 to Bt4 displayed on the receiver 1800a. Further, the processing delay time may be set by a swipe operation as shown in FIG. Thereby, synchronous reproduction can be performed more accurately based on the user's sense.

  Here, the earphone limited reproduction will be described below.

  FIG. 407 is a diagram illustrating an example of processing flow of the receiver 1800a in Embodiment 32.

  With the earphone-only reproduction indicated by this processing flow, it is possible to reproduce sound without causing trouble to the surroundings.

  The receiver 1800a confirms whether the setting limited to the earphone is performed (step S1811). When the setting limited to the earphone is performed, for example, the setting limited to the earphone is set in the receiver 1800a. Alternatively, settings that are limited to earphones are made in the received signal (visible light signal). Alternatively, it is recorded in the server or the receiver 1800a in association with the received signal that it is limited to the earphone.

  When the receiver 1800a confirms that the earphone is limited (Y in step S1811), the receiver 1800a determines whether the earphone is connected to the receiver 1800a (step S1813).

  When the receiver 1800a confirms that the earphone is not limited (N in Step S1811) or determines that the earphone is connected (Y in Step S1813), the receiver 1800a reproduces the sound (Step S1812). When playing back audio, the receiver 1800a adjusts the volume so that the volume is within the set range. This setting range is set similarly to the setting limited to the earphone.

  When the receiver 1800a determines that the earphone is not connected (N in step S1813), the receiver 1800a performs a notification prompting the user to connect the earphone (step S1814). This notification is performed by, for example, screen display, audio output, or vibration.

  In addition, when the setting for prohibiting forced audio reproduction is not set, the receiver 1800a prepares an interface for forced reproduction and determines whether or not the user has performed an operation of forced reproduction ( Step S1815). If it is determined that the forced playback operation has been performed (Y in step S1815), the receiver 1800a plays back the audio even when the earphone is not connected (step S1812).

  On the other hand, if it is determined that the forced regeneration operation is not performed (N in step S1815), the receiver 1800a retains the audio data received in advance and the analyzed synchronization time so that the earphone is connected. Quickly synchronize audio playback.

  FIG. 408 is a diagram illustrating another example of process flow of the receiver 1800a in Embodiment 32.

  First, the receiver 1800a receives an ID from the transmitter 1800d (step S1821). That is, the receiver 1800a receives a visible light signal indicating the ID of the transmitter 1800d or the ID of the content displayed on the transmitter 1800d.

  Next, the receiver 1800a downloads information (content) associated with the received ID from the server (step S1822). Alternatively, the receiver 1800a reads out the information from the data holding unit in the receiver 1800a. Hereinafter, this information is referred to as related information.

  Next, the receiver 1800a determines whether or not the synchronous reproduction flag included in the related information indicates ON (step S1823). If it is determined that the synchronous reproduction flag does not indicate ON (N in step S1823), the receiver 1800a outputs the content indicated by the related information (step S1824). That is, when the content is an image, the receiver 1800a displays an image, and when the content is audio, the receiver 1800a outputs audio.

  On the other hand, when receiver 1800a determines that the synchronous reproduction flag indicates ON (Y in step S1823), is the time adjustment mode included in the related information set to the transmitter reference mode? Then, it is determined whether or not the absolute time mode is set (step S1825). If it is determined that the absolute time mode is set, the receiver 1800a determines whether or not the last time adjustment has been performed within a certain time from the current time (step S1826). The time adjustment at this time is processing for obtaining time information by a predetermined method and using the time information to adjust the time of a clock provided in the receiver 1800a to the absolute time of the reference clock. The predetermined method is a method using, for example, a GPS (Global Positioning System) radio wave or an NTP (Network Time Protocol) radio wave. Note that the current time described above may be a time when the receiver 1800a, which is a terminal device, receives a visible light signal.

  If the receiver 1800a determines that the last time adjustment has been performed within a certain time (Y in step S1826), the receiver 1800a outputs the related information based on the time of the clock of the receiver 1800a, and is displayed on the transmitter 1800d. Content and related information are synchronized (step S1827). When the content indicated by the related information is, for example, a moving image, the receiver 1800a displays the moving image so as to be synchronized with the content displayed on the transmitter 1800d. When the content indicated by the related information is, for example, audio, the receiver 1800a outputs the audio so as to be synchronized with the content displayed on the transmitter 1800d. For example, when the related information indicates sound, the related information includes each frame constituting the sound, and these frames are time stamped. The receiver 1800a outputs a sound synchronized with the content of the transmitter 1800d by playing back a frame with a type stamp corresponding to the time of its own clock.

  If the receiver 1800a determines that the last time adjustment has not been performed within a certain time (N in step S1826), the receiver 1800a attempts to obtain the time information by a predetermined method, and whether or not the time information has been obtained. Is determined (step S1828). If it is determined that the time information has been obtained (Y in step S1828), the receiver 1800a updates the time of the clock of the receiver 1800a using the time information (step S1829). Then, the receiver 1800a executes the process of step S1827 described above.

  If it is determined in step S1825 that the time adjustment mode is the transmitter reference mode, or if it is determined in step S1828 that time information could not be obtained (N in step S1828), the receiver 1800a The time information is acquired from the transmitter 1800d (step S1830). That is, the receiver 1800a acquires time information that is a synchronization signal from the transmitter 1800d through visible light communication. For example, the synchronization signals are time packet 1 and time packet 2 shown in FIG. Alternatively, the receiver 1800a acquires time information from the transmitter 1800d by radio waves such as Bluetooth (registered trademark) or Wi-Fi. Then, the receiver 1800a executes the processes of steps S1829 and S1827 described above.

  In the present embodiment, as in steps S1829 and S1830, processing (time adjustment) is performed for synchronization between the clock of the terminal device that is the receiver 1800a and the reference clock by GPS radio waves or NTP radio waves. The time of the terminal device, the time of the terminal device, and the time of the transmitter according to the time indicated by the visible light signal transmitted from the transmitter 1800d. Synchronize between. Accordingly, the terminal device can reproduce the content (moving image or sound) at the timing synchronized with the transmitter-side content reproduced by the transmitter 1800d.

  FIG. 409A is a diagram for describing a specific method of synchronous playback in Embodiment 32. There are methods a to e shown in FIG.

(Method a)
In the method a, the transmitter 1800d outputs a visible light signal indicating the content ID and the content playback time by changing the luminance of the display, as in the above embodiments. The content playback time is the playback time of data that is part of the content that is being played back by the transmitter 1800d when the content ID is transmitted from the transmitter 1800d. The data is a picture or a sequence constituting the moving image if the content is a moving image, or a frame constituting the sound if the content is sound. The playback time indicates, for example, the playback time from the beginning of the content as the time. If the content is a moving image, the playback time is included in the content as a PTS (Presentation Time Stamp). That is, the content includes the reproduction time (display time) of the data for each data constituting the content.

  The receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the content ID indicated by the visible light signal to the server 1800f. The server 1800f receives the request signal, and transmits the content associated with the content ID included in the request signal to the receiver 1800a.

  When the receiver 1800a receives the content, the receiver 1800a reproduces the content from the time of (content reproduction time + elapsed time since ID reception). The elapsed time from the reception of the ID is an elapsed time from when the content ID is received by the receiver 1800a.

(Method b)
In the method b, the transmitter 1800d outputs a visible light signal indicating the content ID and the content playback time by changing the luminance of the display, as in the above embodiments. The receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the content ID indicated by the visible light signal and the content playback time to the server 1800f. The server 1800f receives the request signal, and transmits only a part of the content after the content playback time to the receiver 1800a among the content associated with the content ID included in the request signal.

  When the receiver 1800a receives the part of the content, the part 1800a reproduces the part of the content from the time point (elapsed time since the ID reception).

(Method c)
In the method c, the transmitter 1800d outputs a visible light signal indicating the transmitter ID and the content reproduction time by changing the luminance of the display, as in the above embodiments. The transmitter ID is information for identifying the transmitter.

  The receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the transmitter ID indicated by the visible light signal to the server 1800f.

  The server 1800f holds, for each transmitter ID, a reproduction schedule that is a time table of content reproduced by the transmitter with the transmitter ID. Further, the server 1800f includes a clock. When such a server 1800f receives the request signal, the content associated with the transmitter ID included in the request signal and the clock time (server time) of the server 1800f is the content being played back. Identify from the playback schedule. Then, the server 1800f transmits the content to the receiver 1800a.

  When the receiver 1800a receives the content, the receiver 1800a reproduces the content from the time of (content reproduction time + elapsed time since ID reception).

(Method d)
In the method d, the transmitter 1800d outputs a visible light signal indicating the transmitter ID and the transmitter time by changing the luminance of the display as in the above embodiments. The transmitter time is a time indicated by a clock provided in the transmitter 1800d.

  The receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the transmitter ID indicated by the visible light signal and the transmitter time to the server 1800f.

  The server 1800f holds the above reproduction schedule table. When such a server 1800f receives the request signal, the server 1800f identifies the content associated with the transmitter ID and the transmitter time included in the request signal as the content being reproduced from the reproduction schedule. Furthermore, the server 1800f specifies the content playback time from the transmitter time. That is, the server 1800f finds the playback start time of the specified content from the playback schedule, and specifies the time between the transmitter time and the playback start time as the content playback time. Then, the server 1800f transmits the content and the content playback time to the receiver 1800a.

  Upon receiving the content and the content playback time, the receiver 1800a plays the content from the time of (content playback time + elapsed time since ID reception).

  Thus, in the present embodiment, the visible light signal indicates the time when the visible light signal is transmitted from the transmitter 1800d. Therefore, the receiver 1800a, which is a terminal device, can receive content associated with the time (transmitter time) at which the visible light signal is transmitted from the transmitter 1800d. For example, if the transmitter time is 5:43, content played back at 5:43 can be received.

  In the present embodiment, server 1800f has a plurality of contents each associated with a time. However, the content associated with the time indicated by the visible light signal may not exist in the server 1800f. In such a case, the receiver 1800a as the terminal device is closest to the time indicated by the visible light signal and is associated with the time after the time indicated by the visible light signal among the plurality of contents. Content may be received. Thereby, even if the content associated with the time indicated by the visible light signal does not exist in the server 1800f, it is possible to receive appropriate content from among the plurality of contents in the server 1800f.

  In addition, the reproduction method according to the present embodiment includes a signal receiving step of receiving a visible light signal by a sensor of the receiver 1800a (terminal device) from a transmitter 1800d that transmits a visible light signal according to a change in luminance of the light source, and a receiver. 1800a transmits a request signal for requesting the content associated with the visible light signal to the server 1800f, the receiver 1800a receives the content from the server 1800f, and reproduces the content. A playback step. The visible light signal indicates a transmitter ID and a transmitter time. The transmitter ID is ID information. The transmitter time is the time indicated by the clock of the transmitter 1800d, and the time when the visible light signal is transmitted from the transmitter 1800d. In the content reception step, the receiver 1800a receives the content associated with the transmitter ID and the transmitter time indicated by the visible light signal. As a result, the receiver 1800a can reproduce appropriate content with respect to the transmitter ID and the transmitter time.

(Method e)
In the method e, the transmitter 1800d outputs a visible light signal indicating the transmitter ID by changing the luminance of the display as in the above embodiments.

  The receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the transmitter ID indicated by the visible light signal to the server 1800f.

  The server 1800f holds the above-described reproduction schedule and further includes a clock. When such a server 1800f receives the request signal, the server 1800f identifies the content associated with the transmitter ID and the server time included in the request signal from the reproduction schedule as content being reproduced. The server time is the time indicated by the clock of the server 1800f. Further, the server 1800f finds the reproduction start time of the specified content from the reproduction schedule table. Then, the server 1800f transmits the content and the content reproduction start time to the receiver 1800a.

  When receiving the content and the content reproduction start time, the receiver 1800a reproduces the content from the time of (receiver time-content reproduction start time). The receiver time is a time indicated by a clock provided in the receiver 1800a.

  As described above, the reproduction method according to the present embodiment includes a signal receiving step of receiving a visible light signal by a sensor of the receiver 1800a (terminal device) from a transmitter 1800d that transmits a visible light signal due to a luminance change of the light source; The transmitting step of transmitting a request signal for requesting the content associated with the visible light signal from the receiver 1800a to the server 1800f, and the receiver 1800a include each time and data reproduced at each time A content receiving step of receiving content from the server 1800f and a playback step of playing back data corresponding to the time of the clock provided in the receiver 1800a among the content. Therefore, the receiver 1800a can appropriately reproduce the data in the content at the correct time indicated by the content without reproducing the data at the wrong time. Also, in the transmitter 1800d, if content related to the content (transmitter-side content) is reproduced, the receiver 1800a can reproduce the content in synchronization with the transmitter-side content appropriately. .

  Even in the above methods c to e, like the method b, the server 1800f may transmit only a part of the content after the content playback time to the receiver 1800a.

  Further, in the above methods a to e, the receiver 1800a transmits a request signal to the server 1800f and receives necessary data from the server 1800f, but the data in the server 1800f is transmitted in advance without performing such transmission / reception. You may keep it.

  FIG. 409B is a block diagram illustrating a configuration of a playback device that performs synchronized playback by the method e described above.

  The playback device B10 is a receiver 1800a or a terminal device that performs synchronous playback by the method e described above, and includes a sensor B11, a request signal transmission unit B12, a content reception unit B13, a clock B14, and a playback unit B15. I have.

  The sensor B11 is an image sensor, for example, and receives the visible light signal from the transmitter 1800d that transmits the visible light signal according to the luminance change of the light source. The request signal transmission unit B12 transmits a request signal for requesting content associated with the visible light signal to the server 1800f. The content receiving unit B13 receives content including each time and data reproduced at each time from the server 1800f. The reproduction unit B15 reproduces data corresponding to the time of the clock B14 in the content.

  FIG. 409C is a flowchart illustrating the processing operation of the terminal device that performs synchronized playback by the method e described above.

  The playback device B10 is a receiver 1800a or a terminal device that performs synchronized playback by the method e described above, and executes each process of steps SB11 to SB15.

  In Step SB11, the visible light signal is received from the transmitter 1800d that transmits the visible light signal according to the luminance change of the light source. In step SB12, a request signal for requesting content associated with the visible light signal is transmitted to server 1800f. In step SB13, content including each time and data reproduced at each time is received from server 1800f. In step SB15, data corresponding to the time of the clock B14 is reproduced from the content.

  As described above, in the playback device B10 and the playback method in the present embodiment, the data in the content can be appropriately played back at the correct time indicated by the content without being played back at the wrong time.

  In the present embodiment, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the playback apparatus B10 and the like of the present embodiment is a program that causes a computer to execute each step included in the flowchart shown in FIG. 409C.

  FIG. 410 is a diagram for describing preparations for synchronized playback in the thirty-second embodiment.

  The receiver 1800a adjusts the time of the clock provided in the receiver 1800a to the time of the reference clock in order to perform synchronized reproduction. For this time adjustment, the receiver 1800a performs the following processes (1) to (5).

  (1) The receiver 1800a receives a signal. This signal may be a visible light signal transmitted by a change in luminance of the display of the transmitter 1800d, or a radio signal based on Wi-Fi or Bluetooth (registered trademark) from a wireless device. Alternatively, the receiver 1800a acquires position information indicating the position of the receiver 1800a by, for example, GPS instead of receiving such a signal. Then, the receiver 1800a recognizes that the receiver 1800a has entered a predetermined place or building based on the position information.

  (2) When the receiver 1800a receives the above signal or recognizes that it has entered a predetermined location, it receives a request signal for requesting data (related information) associated with the signal or location. It transmits to the server (visible light ID resolution server) 1800f.

  (3) The server 1800f transmits the above-described data and a time adjustment request for causing the receiver 1800a to adjust the time to the receiver 1800a.

  (4) When receiving the data and the time adjustment request, the receiver 1800a transmits the time adjustment request to the GPS time server, the NTP server, or the base station of the telecommunication carrier (carrier).

  (5) Upon receiving the time adjustment request, the server or the base station transmits time data (time information) indicating the current time (reference clock time or absolute time) to the receiver 1800a. The receiver 1800a adjusts the time by adjusting the time of the clock provided to the receiver 1800a to the current time indicated by the time data.

  As described above, in this embodiment, a GPS (Global Positioning System) radio wave or an NTP (Network Time Protocol) radio wave is used between the clock provided in the receiver 1800a (terminal device) and the reference clock. Synchronized. Therefore, the receiver 1800a can reproduce the data corresponding to the time at an appropriate time according to the reference clock.

  FIG. 411 is a diagram illustrating an example of application of the receiver 1800a in Embodiment 32.

  The receiver 1800a is configured as a smartphone as described above, and is used by being held by, for example, a holder 1810 formed of a member such as resin or glass having translucency. The holder 1810 includes a back plate portion 1810a and a locking portion 1810b provided upright on the back plate portion 1810a. The receiver 1800a is inserted between the back plate portion 1810a and the locking portion 1810b so as to be along the back plate portion 1810a.

  FIG. 412A is a front view of receiver 1800a held by holder 1810 in Embodiment 32. FIG.

  Receiver 1800a is held by holder 1810 in the inserted state as described above. At this time, the locking portion 1810b locks with the lower portion of the receiver 1800a and sandwiches the lower portion with the back plate portion 1810a. In addition, the back surface of the receiver 1800a faces the back plate portion 1810a, and the display 1801 of the receiver 1800a is exposed.

  FIG. 412B is a rear view of receiver 1800a held by holder 1810 according to Embodiment 32.

  Further, a through hole 1811 is formed in the back plate portion 1810a, and a variable filter 1812 is attached in the vicinity of the through hole 1811. When receiver 1800a is held by holder 1810, camera 1802 of receiver 1800a is exposed through back hole 1811 from back plate portion 1810a. The flashlight 1803 of the receiver 1800a faces the variable filter 1812.

  The variable filter 1812 is formed in a disk shape, for example, and has three color filters (a red filter, a yellow filter, and a green filter) each having a fan shape and the same size. The variable filter 1812 is attached to the back plate portion 1810a so as to be rotatable about the center of the variable filter 1812. The red filter is a filter having red translucency, the yellow filter is a filter having yellow translucency, and the green filter is a filter having green translucency.

  Therefore, the variable filter 1812 is rotated and, for example, the red filter is disposed at a position facing the flashlight 1803a. In this case, the light emitted from the flashlight 1803a is diffused inside the holder 1810 as red light by passing through the red filter. As a result, substantially the entire holder 1810 emits red light.

  Similarly, the variable filter 1812 is rotated, and, for example, a yellow filter is disposed at a position facing the flashlight 1803a. In this case, the light emitted from the flashlight 1803a is diffused inside the holder 1810 as yellow light by passing through the yellow filter. As a result, substantially the entire holder 1810 emits yellow light.

  Similarly, the variable filter 1812 is rotated, and, for example, the green filter is disposed at a position facing the flashlight 1803a. In this case, the light emitted from the flashlight 1803a is diffused inside the holder 1810 as green light by passing through the green filter. As a result, substantially the entire holder 1810 emits green light.

  That is, the holder 1810 lights in red, yellow, or green like a penlight.

  FIG. 413 is a diagram for describing a use case of the receiver 1800a held by the holder 1810 in the thirty-second embodiment.

  For example, a receiver with a holder that is a receiver 1800a held by a holder 1810 is used in an amusement park or the like. That is, the plurality of receivers with holders that are directed to the float moving in the amusement park blink in synchronization with the music flowing from the float. That is, the float is configured as a transmitter in each of the above embodiments, and transmits a visible light signal by a change in luminance of a light source attached to the float. For example, the float transmits a visible light signal indicating the ID of the float. And the receiver with a holder receives the visible light signal, ie, ID, by imaging | photography with the camera 1802 of the receiver 1800a similarly to said each embodiment. The receiver 1800a that has received the ID acquires a program associated with the ID from, for example, a server. This program includes instructions for turning on the flashlight 1803 of the receiver 1800a at each predetermined time. Each predetermined time is set in accordance with the music flowing from the float (so as to be synchronized). Then, the receiver 1800a blinks the flashlight 1803a according to the program.

  Thus, the holder 1810 of each receiver 1800a that has received the ID repeats lighting at the same timing according to the music flowing from the float of the ID.

  Here, each receiver 1800a blinks the flashlight 1803 in accordance with a set color filter (hereinafter referred to as a setting filter). The setting filter is a color filter that faces the flashlight 1803 of the receiver 1800a. Each receiver 1800a recognizes the current setting filter based on an operation by the user. Alternatively, each receiver 1800a recognizes the current setting filter based on the color of an image obtained by photographing with the camera 1802.

  That is, among the plurality of receivers 1800a that have received the ID, only the holders 1810 of the plurality of receivers 1800a that recognize that the setting filter is a red filter are lit simultaneously at a predetermined time. At the next time, only the holders 1810 of the plurality of receivers 1800a that recognize that the setting filter is a green filter are lit simultaneously. Further, at the next time, only the holders 1810 of the plurality of receivers 1800a that recognize that the setting filter is a yellow filter are lit simultaneously.

  As described above, the receiver 1800a held in the holder 1810 is synchronized with the float music and the receiver 1800a held in the other holder 1810 in the same manner as the synchronous playback shown in FIGS. 401 to 407 described above. Then, the flashlight 1803, that is, the holder 1810 is blinked.

  FIG. 414 is a flowchart illustrating a processing operation of the receiver 1800a held by the holder 1810 in the thirty-second embodiment.

  The receiver 1800a receives the float ID indicated by the visible light signal from the float (step S1831). Next, the receiver 1800a acquires a program associated with the ID from the server (step S1832). Next, the receiver 1800a executes the program to turn on the flashlight 1803 at each predetermined time according to the setting filter (step S1833).

  Here, the receiver 1800a may cause the display 1801 to display an image corresponding to the received ID or the acquired program.

  FIG. 415 is a diagram illustrating an example of an image displayed by the receiver 1800a in Embodiment 32.

  For example, when receiving an ID from a Santa Claus float, the receiver 1800a displays a Santa Claus image as shown in FIG. Furthermore, as shown in FIG. 415 (b), the receiver 1800a may change the background color of the Santa Claus image to the color of the setting filter simultaneously with the lighting of the flashlight 1803. For example, when the color of the setting filter is red, the holder 1810 is lit red by turning on the flashlight 1803, and at the same time, a Santa Claus image having a red background color is displayed on the display 1801. That is, the blinking of the holder 1810 and the display on the display 1801 are synchronized.

  FIG. 416 is a diagram illustrating another example of the holder according to Embodiment 32. In FIG.

The holder 1820 is configured in the same manner as the above-described holder 1810, but does not have the through hole 1811 and the variable filter 1812. Such a holder 1820 holds the receiver 1800a in a state where the display 1801 of the receiver 1800a is directed to the back plate portion 1820a. In this case, the receiver 1800a causes the display 1801 to emit light instead of the flashlight 1803. As a result, light from the display 1801 is diffused over substantially the entire holder 1820. Therefore, when the receiver 1800a causes the display 1801 to emit light with red light in accordance with the above-described program, the holder 1820 is lit red. Similarly, when the receiver 1800a causes the display 1801 to emit light with yellow light according to the above-described program, the holder 1820 lights up in yellow. Receiver 1800a
However, when the display 1801 is caused to emit light with green light in accordance with the above program, the holder 1820 is lit in green. If such a holder 1820 is used, the setting of the variable filter 1812 can be omitted.

(Embodiment 33)
(Visible light signal)
417A to 417D are diagrams illustrating an example of a visible light signal in Embodiment 33. FIG.

  As described above, the transmitter generates a 4PPM visible light signal and changes the luminance according to the visible light signal, for example, as shown in FIG. 417A. Specifically, the transmitter allocates 4 slots to one signal unit, and generates a visible light signal composed of a plurality of signal units. The signal unit indicates High (H) or Low (L) for each slot. The transmitter emits light brightly in the H slot and emits light darkly or extinguishes in the L slot. For example, one slot is a period corresponding to a time of 1/9600 seconds.

  Further, for example, as shown in FIG. 417B, the transmitter may generate a visible light signal in which the number of slots allocated to one signal unit is variable. In this case, the signal unit includes a signal indicating H in one or more consecutive slots and a signal indicating L in one slot following the H signal. Since the number of slots of H is variable, the total number of slots in the signal unit is variable. For example, as shown in FIG. 417B, the transmitter generates a visible light signal including these signal units in the order of a signal unit of 3 slots, a signal unit of 4 slots, and a signal unit of 6 slots. Also in this case, the transmitter emits light brightly in the H slot and emits light darkly or extinguishes in the L slot.

  Further, for example, as illustrated in FIG. 417C, the transmitter may allocate an arbitrary period (signal unit period) to one signal unit without allocating a plurality of slots to one signal unit. The signal unit period includes an H period and an L period following the H period. The period of H is adjusted according to the signal before modulation. The period L may be fixed and may be a period corresponding to the slot. The H period and the L period are, for example, periods of 100 μs or more. For example, as shown in FIG. 417C, the transmitter transmits a visible light signal including signal units in the order of a signal unit having a signal unit period of 210 μs, a signal unit having a signal unit period of 220 μs, and a signal unit having a signal unit period of 230 μs. Send. In this case as well, the transmitter emits light brightly during the H period and emits light darkly or extinguishes during the L period.

  Further, for example, as illustrated in FIG. 417D, the transmitter may generate a signal that alternately indicates L and H as a visible light signal. In this case, the L period and the H period in the visible light signal are adjusted according to the signals before modulation. For example, as shown in FIG. 417D, the transmitter indicates H for a period of 100 μs, then indicates L for a period of 120 μs, then indicates H for a period of 110 μs, and further indicates L for a period of 200 μs. A visible light signal is transmitted. In this case as well, the transmitter emits light brightly during the H period and emits light darkly or extinguishes during the L period.

  418 is a diagram illustrating a structure of a visible light signal in Embodiment 33. FIG.

  The visible light signal includes, for example, a signal 1, a brightness adjustment signal corresponding to the signal 1, a signal 2, and a brightness adjustment signal corresponding to the signal 2. When the transmitter generates the signal 1 and the signal 2 by modulating the signals before modulation, the transmitter generates a brightness adjustment signal for the signals and generates the above-described visible light signal.

  The brightness adjustment signal corresponding to the signal 1 is a signal that compensates for increase / decrease in brightness due to a luminance change according to the signal 1. The brightness adjustment signal corresponding to the signal 2 is a signal that compensates for increase / decrease in brightness due to a luminance change according to the signal 2. Here, the brightness B1 is expressed by the luminance change according to the signal 1 and the brightness adjustment signal of the signal 1, and the brightness change according to the signal 2 and the brightness adjustment signal of the signal 2 Brightness B2 is expressed. The transmitter in the present embodiment generates the brightness adjustment signals of signal 1 and signal 2 as part of the visible light signal so that the brightness B1 and brightness B2 are equal. Thereby, the brightness is kept constant and flicker can be suppressed.

  Further, when the transmitter 1 generates the signal 1, the transmitter 1 generates a signal 1 including data 1, a preamble (header) following the data 1, and data 1 following the preamble. Here, the preamble is a signal corresponding to data 1 arranged before and after the preamble. For example, this preamble is a signal serving as an identifier for reading data 1. Thus, since the signal 1 is composed of the two data 1 and the preamble arranged between them, even if the receiver reads the visible light signal from the middle of the preceding data 1, Data 1 (ie, signal 1) can be correctly demodulated.

(Bright line image)
FIG. 419 is a diagram illustrating an example of bright line images obtained by imaging with a receiver in Embodiment 33. In FIG.

  As described above, the receiver captures a bright line image including a visible light signal transmitted from the transmitter as a bright line pattern by capturing an image of the transmitter that changes in luminance. With such imaging, a visible light signal is received by the receiver.

  For example, as shown in FIG. 419, the receiver captures an image at time t1 using N exposure lines included in the image sensor, so that each line includes a region a and a region b where a bright line pattern appears. Get an image. Regions a and b are regions in which bright line patterns appear when the luminance of the transmitter, which is the subject, changes.

  Here, the receiver demodulates the visible light signal from the bright line pattern of the region a and the region b. However, if the receiver determines that the demodulated visible light signal alone is not sufficient, only M (M <N) consecutive exposure lines corresponding to the area a are used among the N exposure lines. The image is taken at time t2. Thereby, the receiver acquires a bright line image including only the region a out of the regions a and b. The receiver repeatedly performs such imaging at times t3 to t5. As a result, a visible light signal having a sufficient amount of data from the subject corresponding to the region a can be received at high speed. Further, the receiver captures an image at time t6 using only L (L <N) consecutive exposure lines corresponding to the region b among the N exposure lines. Thereby, the receiver acquires a bright line image including only the region b out of the regions a and b. The receiver repeatedly performs such imaging at times t7 to t9. As a result, a visible light signal having a sufficient amount of data from the subject corresponding to the region b can be received at high speed.

  Further, the receiver may acquire the bright line image including only the region a by performing the same imaging at the times t2 to t5 at the times t10 and t11. Furthermore, the receiver may acquire a bright line image including only the region b by performing imaging similar to that at times t6 to t9 at times t12 and t13.

  In the above-described example, when the receiver determines that the visible light signal is insufficient, the receiver performs continuous shooting of the bright line image including only the region a at time t2 to t5. If bright lines appear in the image obtained by the above, the above-described continuous shooting may be performed. Similarly, when the receiver determines that the visible light signal is insufficient, the receiver performs continuous shooting of the bright line image including only the region b from time t6 to t9, but the image is obtained by imaging at time t1. If bright lines appear in the image, the above-described continuous shooting may be performed. The receiver may alternately perform acquisition of a bright line image including only the region a and acquisition of a bright line image including only the region b.

  Note that the M consecutive exposure lines corresponding to the area a are exposure lines that contribute to the generation of the area a, and the L consecutive exposure lines corresponding to the area b are the generation of the area b. Is an exposure line that contributes to

  FIG. 420 is a diagram illustrating another example of bright line images obtained by imaging by the receiver in Embodiment 33.

  For example, as shown in FIG. 420, the receiver captures an image at time t1 using N exposure lines included in the image sensor, so that each line includes a region a and a region b where a bright line pattern appears. Get an image. Each of the areas a and b is an area where a bright line pattern appears when the luminance of the transmitter, which is a subject, changes as described above. In addition, each of the region a and the region b has a region overlapping with each other along the direction of the bright line or the exposure line (hereinafter referred to as an overlapping region).

  Here, when the receiver determines that the visible light signal demodulated from the bright line pattern of the area a and the area b is insufficient, P (P <N) corresponding to the overlapping area among the N exposure lines. An image is taken at time t2 using only the continuous exposure lines. As a result, the receiver acquires a bright line image including only the overlapping regions of the region a and the region b. The receiver repeatedly performs such imaging at times t3 and t4. As a result, a visible light signal having a sufficient amount of data from the subject corresponding to each of the region a and the region b can be received substantially simultaneously and at high speed.

  421 is a diagram illustrating another example of a bright line image obtained by imaging by a receiver in Embodiment 33. FIG.

  For example, as illustrated in FIG. 421, the receiver uses N exposure lines included in the image sensor to capture an image at time t <b> 1, thereby clearly displaying a portion a in which the bright line pattern appears unclearly. A bright line image including an area including the appearing portion b is acquired. Similar to the above, this region is a region where a bright line pattern appears when the luminance of the transmitter that is the subject changes.

  In such a case, when the receiver determines that the visible light signal demodulated from the bright line pattern in the above-described region is insufficient, Q (Q <N) corresponding to the portion b of the N exposure lines. The image is taken at time t2 using only the continuous exposure lines. Thereby, the receiver acquires the bright line image including only the part b in the above-described region. The receiver repeatedly performs such imaging at times t3 and t4. As a result, a visible light signal having a sufficient amount of data from the subject corresponding to the above-described region can be received at high speed.

  The receiver may further perform continuous shooting of the bright line image including only the part a after the continuous shooting of the bright line image including only the part b.

  As described above, when a plurality of regions (or portions) where the bright line pattern appears in the bright line image are included, the receiver assigns an order to each region, and according to the order, only that region is included. Performs continuous shooting of bright line images including. In this case, the order may be an order corresponding to the magnitude of the signal (area or area size) or an order corresponding to the clarity of the bright line. Further, the order may be an order corresponding to the color of light from the subject corresponding to these areas. For example, the first continuous shooting is performed on a region corresponding to red light, and the next continuous shooting is performed on a region corresponding to white light. Further, only continuous shooting of the area with respect to red light may be performed.

(HDR synthesis)
422 is a diagram for describing adaptation of the receiver in Embodiment 33 to a camera system that performs HDR combining. FIG.

  A vehicle is equipped with a camera system for preventing collisions. This camera system performs HDR (High Dynamic Range) composition using an image obtained by imaging by a camera. By this HDR synthesis, an image with a wide dynamic range of luminance can be obtained. The camera system recognizes surrounding vehicles, obstacles, or people based on this wide dynamic range image.

  For example, the camera system has a normal setting mode and a communication setting mode as setting modes. When the setting mode is the normal setting mode, for example, as shown in FIG. 422, the camera system performs four times of imaging at the same shutter speed of 1/100 seconds and at different sensitivities at times t1 to t4. . The camera system performs HDR synthesis using the four images obtained by the four imaging operations.

  On the other hand, when the setting mode is the communication setting mode, for example, as shown in FIG. 422, the camera system captures three images at the same 1/100 second shutter speed and at different sensitivities at times t5 to t7. I do. Furthermore, the camera system captures an image with a maximum sensitivity (for example, ISO = 1600) at a shutter speed of 1/10000 seconds at time t8. The camera system performs HDR synthesis using the three images obtained by the first three images out of the four images. Further, the camera system receives a visible light signal by the last imaging among the above four imaging operations, and demodulates the bright line pattern appearing in the image obtained by the imaging.

  Further, when the setting mode is the communication setting mode, the camera system may not perform HDR synthesis. For example, as shown in FIG. 422, the camera system performs imaging at a time t9 with a shutter speed of 1/100 second and with low sensitivity (for example, ISO = 200). Furthermore, the camera system performs imaging three times at a shutter speed of 1/10000 seconds and at different sensitivities at times t10 to t12. The camera system recognizes a surrounding vehicle, an obstacle, a person, or the like from an image obtained by the first one of the four images. Further, the camera system receives a visible light signal by the last three imagings among the above four imagings, and demodulates the bright line pattern appearing in the image obtained by the imaging.

  In the example shown in FIG. 422, imaging is performed with different sensitivities at times t10 to t12. However, imaging may be performed with the same sensitivity.

  In such a camera system, HDR synthesis can be performed and visible light signals can also be received.

(Security)
423 is a diagram for describing a processing operation of the visible light communication system according to Embodiment 33. FIG.

  This visible light communication system includes, for example, a transmitter disposed at a cash register, a smartphone as a receiver, and a server. Note that the communication between the smartphone and the server and the communication between the transmitter and the server are each performed via a secure communication line. Communication between the transmitter and the smartphone is performed by visible light communication. The visible light communication system according to the present embodiment ensures security by determining whether a visible light signal from a transmitter is accurately received by a smartphone.

  Specifically, the transmitter transmits, for example, a visible light signal indicating a value “100” to the smartphone by changing the luminance at time t1. When the smartphone receives the visible light signal at time t2, the smartphone transmits a radio signal indicating the value “100” to the server. The server receives the radio signal from the smartphone at time t3. At this time, the server performs a process for determining whether or not the value “100” indicated by the radio signal is the value of the visible light signal received by the smartphone from the transmitter. That is, the server transmits, for example, a radio signal indicating a value “200” to the transmitter. The transmitter that has received the radio wave signal transmits a visible light signal indicating the value “200” to the smartphone by changing the luminance at time t4. When the smartphone receives the visible light signal at time t5, the smartphone transmits a radio signal indicating the value “200” to the server. The server receives the radio signal from the smartphone at time t6. The server determines whether or not the value indicated by the received radio signal is the same as the value indicated by the radio signal transmitted at time t3. If they are the same, the server determines that the value “100” indicated by the visible light signal received at time t3 is the value of the visible light signal transmitted from the transmitter to the smartphone and received. On the other hand, if not the same, the server determines that the value “100” indicated by the visible light signal received at time t3 is suspicious as the value of the visible light signal transmitted from the transmitter to the smartphone and received.

  Thereby, the server can determine whether the smart phone has surely received the visible light signal from the transmitter. That is, it is possible to prevent the smartphone from transmitting the signal to the server by making it appear as if it has received the visible light signal even though it has not received the visible light signal from the transmitter.

  In the above example, communication using a radio wave signal is performed between the smartphone, the server, and the transmitter. However, communication using an optical signal other than a visible light signal or communication using an electrical signal may be performed. Good. The visible light signal transmitted from the transmitter to the smartphone indicates, for example, a charging value, a coupon value, a monster value, or a bingo value.

(Vehicle related)
424A is a diagram illustrating an example of vehicle-to-vehicle communication using visible light in Embodiment 33. FIG.

  For example, the head vehicle recognizes that there is an accident in the traveling direction by a sensor (such as a camera) mounted on the vehicle. When an accident is recognized in this way, the leading vehicle transmits a visible light signal by changing the brightness of the tail lamp. For example, the leading vehicle transmits a visible light signal that prompts the subsequent vehicle to decelerate. When the succeeding vehicle receives the visible light signal by imaging with a camera mounted on the vehicle, the following vehicle decelerates according to the visible light signal and further transmits a visible light signal that prompts the subsequent vehicle to decelerate. To do.

  As described above, the visible light signal for prompting deceleration is sequentially transmitted from the head to a plurality of vehicles traveling in a line, and the vehicle that has received the visible light signal decelerates. Since the transmission of the visible light signal to each vehicle is performed quickly, the plurality of vehicles can be decelerated in the same manner at substantially the same time. Therefore, it is possible to reduce traffic congestion due to accidents.

  FIG. 424B is a diagram illustrating another example of vehicle-to-vehicle communication using visible light according to Embodiment 33.

  For example, the preceding vehicle may transmit a visible light signal indicating a message (eg, “thank you”) to the subsequent vehicle by changing the brightness of the tail lamp. This message is generated, for example, by a user operation on a smartphone. And a smart phone transmits the signal which shows the message to the above-mentioned previous vehicle. As a result, the preceding vehicle can transmit a visible light signal indicating the message to the subsequent vehicle.

  425 is a diagram illustrating an example of a position determination method of a plurality of LEDs in Embodiment 33. FIG.

  For example, a vehicle headlight has a plurality of LEDs (Light Emitting Diodes). The transmitter of this vehicle transmits a visible light signal from each LED by individually changing the brightness of each of the plurality of LEDs of the headlight. Other vehicle receivers receive visible light signals from their LEDs by imaging the vehicle with the headlights.

  At this time, the receiver determines the position of each of the plurality of LEDs from the image obtained by the imaging in order to recognize which LED the received visible light signal is transmitted from. . Specifically, the receiver uses an acceleration sensor attached to the same vehicle as the receiver, and uses a plurality of gravity sensors based on the direction of gravity indicated by the acceleration sensor (for example, a downward arrow in FIG. 425). Determine the position of each LED.

  In the above-described example, an LED is used as an example of a light emitter that changes in luminance, but a light emitter other than the LED may be used.

  426 is a diagram illustrating an example of bright line images obtained by capturing an image of the vehicle in Embodiment 33. FIG.

  For example, the receiver mounted on the traveling vehicle acquires the bright line image shown in FIG. 426 by imaging the subsequent vehicle (following vehicle). The transmitter mounted on the following vehicle transmits a visible light signal to the preceding vehicle by changing the brightness of the two headlights of the vehicle. A camera for imaging the rear is attached to the rear part of the front vehicle or the side mirror. The receiver acquires a bright line image by imaging with the camera of the following vehicle as a subject, and demodulates a bright line pattern (visible light signal) included in the bright line image. Thereby, the visible light signal transmitted from the transmitter of the following vehicle is received by the receiver of the preceding vehicle.

  Here, the receiver acquires the ID of the vehicle having the headlight, the speed of the vehicle, and the type of the vehicle from each of the demodulated visible light signals transmitted from the two headlights. If the IDs of the two visible light signals are the same, the receiver determines that the two visible light signals are signals transmitted from the same vehicle. And a receiver specifies the length (distance between lights) between the two headlights which the vehicle has from the model of the vehicle. Further, the receiver measures a distance L1 between two regions where the bright line pattern appears, which is included in the bright line image. Then, the receiver calculates the distance (inter-vehicle distance) from the vehicle on which the receiver is mounted to the following vehicle by triangulation using the distance L1 and the inter-light distance. The receiver determines the risk of collision based on the inter-vehicle distance and the vehicle speed acquired from the visible light signal, and notifies the vehicle driver of a warning corresponding to the determination result. Thereby, the collision of a vehicle can be avoided.

  In the above-described example, the receiver specifies the inter-light distance from the vehicle type included in the visible light signal, but may specify the inter-light distance from information other than the vehicle type. In the above-described example, the receiver issues a warning when it is determined that there is a risk of a collision. However, the receiver may output a control signal for causing the vehicle to perform an operation to avoid the risk. Good. For example, the control signal is a signal for accelerating the vehicle or a signal for causing the vehicle to change lanes.

  Moreover, in the above-mentioned example, although a camera images a succeeding vehicle, you may image an oncoming vehicle. In addition, when the receiver determines from the image obtained by imaging by the camera that fog is in the vicinity of the receiver (that is, the vehicle equipped with the receiver), the receiver enters a mode for receiving the visible light signal as described above. May be. As a result, the receiver of the vehicle can identify the position and speed of the oncoming vehicle by receiving the visible light signal transmitted from the headlight of the oncoming vehicle, even if fog is in the vicinity.

  427 is a diagram illustrating an example of application of a receiver and a transmitter in Embodiment 33. FIG. FIG. 427 is a view of the automobile from the back.

  For example, a transmitter (car) 7006a having two tail lamps (light emitting unit or light) of a car transmits identification information (ID) of the transmitter 7006a to a receiver configured as a smartphone, for example. When the receiver receives the ID, the receiver acquires information associated with the ID from the server. For example, the information includes the ID of the car or transmitter, the distance between the light emitting parts, the size of the light emitting part, the size of the car, the shape of the car, the weight of the car, the car number, the appearance in front, or the danger. This is information indicating the presence or absence of. In addition, the receiver may acquire these pieces of information directly from the transmitter 7006a.

  FIG. 428 is a flowchart illustrating an example of process operations of the receiver and the transmitter 7006a in Embodiment 33.

  The ID of the transmitter 7006a and the information passed to the receiver that received the ID are associated with each other and stored in the server (Step 7106a). The information to be passed to the receiver includes identification of the size of the light emitting unit that becomes the transmitter 7006a, the distance between the light emitting units, the shape of the object having the transmitter 7006a as a component, the weight, the body number, etc. Information such as numbers, places that are difficult to observe from the receiver, and presence or absence of danger may be included.

  The transmitter 7006a transmits the ID (Step 7106b). The transmission content may include the URL of the server and information stored in the server.

  The receiver receives the transmitted information such as an ID (Step 7106c). The receiver acquires information associated with the received ID from the server (Step 7106d). The receiver displays the received information and the information acquired from the server (Step 7106e).

  The receiver and the light emitting unit can be triangulated from the size information of the light emitting unit and the appearance size of the imaged light emitting unit, or from the distance information between the light emitting units and the distance between the imaged light emitting units. Is calculated (Step 7106f). The receiver issues a warning of danger based on information such as the state of the place that is difficult to observe from the receiver and the presence or absence of danger (Step 7106g).

  429 is a diagram illustrating an example of application of a receiver and a transmitter in Embodiment 33. FIG.

  For example, a transmitter (car) 7007b having two tail lamps (light emitting units or lights) of a car transmits information on the transmitter 7007b to a receiver 7007a configured as a transmission / reception device for a parking lot, for example. The information of the transmitter 7007b indicates identification information (ID) of the transmitter 7007b, a car number, a car size, a car shape, or a car weight. When receiving the information, the receiver 7007a transmits information indicating whether parking is possible, billing information, or a parking position. Note that the receiver 7007a may receive the ID and acquire information other than the ID from the server.

  FIG. 430 is a flowchart illustrating an example of processing operations of the receiver 7007a and the transmitter 7007b in Embodiment 33. Note that the transmitter 7007b includes an in-vehicle transmitter and an in-vehicle receiver in order to perform not only transmission but also reception.

  The ID of the transmitter 7007b and the information passed to the receiver 7007a that received the ID are associated with each other and stored in the server (parking lot management server) (Step 7107a). Information to be passed to the receiver 7007a includes an identification number such as the shape, weight, and body number of the object having the transmitter 7007b as a component, the identification number of the user of the transmitter 7007b, and information for payment. May be included.

  The transmitter 7007b (in-vehicle transmitter) transmits the ID (Step 7107b). The contents of transmission may include the URL of the server and information stored in the server. The parking lot receiver 7007a (parking lot transmission / reception device) transmits the received information to a server (parking lot management server) that manages the parking lot (Step 7107c). The parking lot management server acquires information associated with the ID using the ID of the transmitter 7007b as a key (Step 7107d). The parking lot management server investigates the parking lot availability (Step 7107e).

  The parking lot receiver 7007a (parking lot transmission / reception device) transmits whether or not parking is possible, parking position information, or the address of a server that holds these pieces of information (Step 7107f). Or a parking lot management server transmits such information to another server. The transmitter (on-vehicle receiver) 7007b receives the information transmitted above (Step 7107g). Or an in-vehicle system acquires these information from another server.

  The parking lot management server controls the parking lot so as to facilitate parking (Step 7107h). For example, control of a multistory parking lot is performed. The transmission / reception device of the parking lot transmits the ID (Step 7107i). The in-vehicle receiver (transmitter 7007b) makes an inquiry to the parking lot management server based on the user information of the in-vehicle receiver and the received ID (Step 7107j).

  The parking lot management server charges according to the parking time or the like (7107k). The parking lot management server controls the parking lot so that the parked vehicle can be easily accessed (Step 7107m). For example, control of a multistory parking lot is performed. The in-vehicle receiver (transmitter 7007b) displays a map to the parking position and performs navigation from the current location (Step 7107n).

(On the train)
FIG. 431 is a diagram illustrating a configuration of a visible light communication system applied to the inside of a train according to Embodiment 33.

  The visible light communication system includes, for example, a plurality of lighting devices 1905 arranged in a train, a smartphone 1906 held by a user, a server 1904, and a camera 1903 arranged in the train.

  Each of the plurality of lighting devices 1905 is configured as the above-described transmitter, illuminates the light, and transmits a visible light signal by changing the luminance. This visible light signal indicates the ID of the illumination device 1905 that transmits the visible light signal.

  The smartphone 1906 is configured as the above-described receiver, and receives a visible light signal transmitted from the lighting device 1905 by imaging the lighting device 1905. For example, when a user is involved in a trouble (for example, a molester or a fight) on a train, the user causes the smartphone 1906 to receive the visible light signal. When the smartphone 1906 receives the visible light signal, the smartphone 1906 notifies the server 1904 of the ID indicated by the visible light signal.

  Upon receiving the notification of the ID, the server 1904 specifies the camera 1903 whose imaging range is the range illuminated by the lighting device 1905 identified by the ID. Then, the server 1904 causes the identified camera 1903 to image the range illuminated by the lighting device 1905.

  The camera 1903 captures an image in response to an instruction from the server 1904 and transmits an image obtained by the image capture to the server 1904.

  Thereby, the image which shows the condition of the trouble in a train can be acquired. This image can be used as evidence of trouble.

  In addition, the user may operate the smartphone 1906 to transmit an image obtained by imaging with the camera 1903 from the server 1904 to the smartphone 1906.

  In addition, the smartphone 1906 may display an imaging button on the screen, and when the imaging button is touched by the user, a signal that prompts imaging may be transmitted to the server 1904. Thereby, the user can determine the timing of imaging himself.

  FIG. 432 is a diagram illustrating a configuration of a visible light communication system applied to a facility such as an amusement park in Embodiment 33.

  The visible light communication system includes, for example, a plurality of cameras 1903 arranged in a facility and a jewelry 1907 attached to a person.

  The accessory 1907 is, for example, a headband having a ribbon to which a plurality of LEDs are attached. Moreover, this accessory 1907 is comprised as an above-mentioned transmitter, and transmits a visible light signal by changing the brightness | luminance of several LED.

  Each of the plurality of cameras 1903 is configured as the above-described receiver, and has a visible light communication mode and a normal imaging mode. In addition, each of the plurality of cameras 1903 is disposed at a different location on the path in the facility.

  Specifically, when the accessory 1907 is imaged as a subject when the camera 1903 is set to the visible light communication mode, the camera 1903 receives a visible light signal from the accessory 1907. Upon receiving the visible light signal, the camera 1903 switches the set mode from the visible light communication mode to the normal imaging mode. As a result, the camera 1903 images a person wearing the accessory 1907 as a subject.

  Therefore, when a person wearing the accessory 1907 is walking on the street in the facility, the camera 1903 near the person sequentially captures the person. As a result, it is possible to automatically acquire and save an image showing the person enjoying at the facility.

  Note that the camera 1903 may not perform imaging in the normal imaging mode immediately after receiving a visible light signal, but may perform imaging in the normal imaging mode when receiving an instruction to start imaging from a smartphone, for example. Accordingly, the user can cause the camera 1903 to image itself at the timing when the user touches the imaging start button displayed on the screen of the smartphone.

  FIG. 433 is a diagram illustrating an example of a visible light communication system including a playground device and a smartphone according to Embodiment 33.

  The play equipment 1901 is configured as the above-described transmitter including a plurality of LEDs, for example. That is, the playground equipment 1901 transmits a visible light signal by changing the luminance of the plurality of LEDs.

  The smartphone 1902 receives the visible light signal transmitted from the play equipment 1901 by imaging the play equipment 1901. And as shown to (a) of FIG. 433, when the smart phone 1902 receives the visible light signal for the first time, the moving image 1 associated with the visible light signal and the first time is received from, for example, a server or the like. Download and play. On the other hand, when the smartphone 1902 receives the visible light signal for the second time, as shown in FIG. 433 (b), the smartphone 1902 displays the moving image 2 associated with the visible light signal and the second time from, for example, a server or the like. Download and play.

  That is, even if the smartphone 1902 receives the same visible light signal, the smartphone 1902 switches the reproduced video according to the number of times the visible light signal is received. The number of times the visible light signal is received may be counted by the smartphone 1902 or may be counted by the server. Or even if the smartphone 1902 receives the same visible light signal a plurality of times, the smartphone 1902 does not continuously reproduce the same moving image. Alternatively, the smartphone 1902 reduces the appearance probability of a video that has already been played among a plurality of videos associated with the same visible light signal, and preferentially downloads and plays a video with a high appearance probability. May be.

  In addition, the smartphone 1902 may receive a visible light signal transmitted from a touch panel provided in a guidance office of a facility having a plurality of stores, and may display an image corresponding to the visible light signal. For example, when the touch panel displays an initial screen showing the outline of the facility, the touch panel transmits a visible light signal indicating the outline of the facility by a change in luminance. Therefore, when the smartphone receives the visible light signal by capturing an image of the touch panel displaying the initial screen, the smartphone can display an image showing an outline of the facility on its display. Here, when the touch panel is operated by the user, the touch panel displays, for example, a store image indicating information on a specific store. At this time, the touch panel transmits a visible light signal indicating the information of the specific store. Therefore, the smart phone can display the store image which shows the information of a specific store, if the visible light signal is received by imaging the touch panel which displays the store image. Thus, the smartphone can display an image synchronized with the touch panel.

(Summary of the above embodiment)
The reproduction method according to an aspect of the present invention includes a signal receiving step of receiving the visible light signal by a sensor of a terminal device from a transmitter that transmits a visible light signal according to a luminance change of a light source, and the visible light signal from the terminal device. A transmission step of transmitting a request signal for requesting the content associated with the optical signal to the server, and the terminal device transmits content including each time and data reproduced at each time from the server. A content receiving step of receiving, and a playback step of playing back data corresponding to a time of a clock provided in the terminal device among the content.

  As a result, as shown in FIG. 409C, content including each time and data reproduced at each time is received by the terminal device, and data corresponding to the clock time of the terminal device is reproduced. Therefore, the terminal device can appropriately reproduce the data in the content at the correct time indicated by the content without reproducing the data at the wrong time. Specifically, as in the method e of FIG. 409A, the receiver that is the terminal device reproduces the content from the time of (receiver time−content reproduction start time). The data corresponding to the clock time of the terminal device described above is data at the time of (receiver time-content reproduction start time) in the content. Also, in the transmitter, if content related to the content (transmitter-side content) is being played back, the terminal device can play back the content appropriately synchronized with the transmitter-side content. The content is sound or image.

  Further, the clock provided in the terminal device and the reference clock may be synchronized by a GPS (Global Positioning System) radio wave or an NTP (Network Time Protocol) radio wave.

  As a result, as shown in FIGS. 408 and 410, since the clock of the terminal device (receiver) is synchronized with the reference clock, it corresponds to the appropriate time according to the reference clock. Can be played back.

  The visible light signal may indicate a time at which the visible light signal is transmitted from the transmitter.

  Thereby, as shown in the method d of FIG. 409A, the terminal device (receiver) can receive the content associated with the time (transmitter time) at which the visible light signal is transmitted from the transmitter. For example, if the transmitter time is 5:43, content played back at 5:43 can be received.

  Further, in the reproduction method, the time at which the process for synchronizing the clock of the terminal device and the reference clock is performed by the GPS radio wave or the NTP radio wave is determined by the terminal device as the visible signal. If it is before a predetermined time from the time when the optical signal is received, synchronization is performed between the clock of the terminal device and the clock of the transmitter according to the time indicated by the visible light signal transmitted from the transmitter. It may be taken.

  For example, if a predetermined time elapses after the processing for synchronizing the clock of the terminal device and the reference clock is performed, the synchronization may not be properly maintained. In such a case, there is a possibility that the terminal device cannot reproduce the content at a time synchronized with the transmitter-side content reproduced by the transmitter. Therefore, in the playback method according to one aspect of the present invention, as shown in steps S1829 and S1830 in FIG. 408, when a predetermined time has elapsed, the clock of the terminal device (receiver) and the clock of the transmitter are set. Synchronized. Therefore, the terminal device can reproduce the content at a time synchronized with the transmitter-side content reproduced by the transmitter.

  Further, the server has a plurality of contents each associated with a time, and in the contents receiving step, when the contents associated with the time indicated by the visible light signal does not exist in the server, Among the plurality of contents, content that is closest to the time indicated by the visible light signal and that is associated with a time after the time indicated by the visible light signal may be received.

  As a result, as shown in the method d in FIG. 409A, even when the content associated with the time indicated by the visible light signal does not exist in the server, the appropriate content is received from the plurality of contents in the server. be able to.

  In addition, a signal receiving step of receiving the visible light signal by a sensor of a terminal device from a transmitter that transmits a visible light signal due to a luminance change of the light source, and a content associated with the visible light signal from the terminal device. A transmission step of transmitting a request signal for requesting to the server, a content reception step in which the terminal device receives the content from the server, and a reproduction step of reproducing the content, wherein the visible light signal is: ID information and a time at which the visible light signal is transmitted from the transmitter are indicated. In the content receiving step, the ID information indicated by the visible light signal and the content associated with the time are received. Good.

  As a result, as in the method d of FIG. 409A, the visible light signal is associated with the time (transmitter time) at which the visible light signal is transmitted from the transmitter among the plurality of contents associated with the ID information (transmitter ID). The received content is received and played back. Therefore, it is possible to reproduce content appropriate for the transmitter ID and transmitter time.

  The visible light signal includes second information indicating the hour and minute of the time and first information indicating the second of the time, so that the visible light signal is transmitted from the transmitter. In the signal receiving step, the second information may be received and the first information may be received more times than the number of times the second information is received.

  Thereby, for example, when notifying the terminal device of the time at which each packet included in the visible light signal is transmitted in seconds, the packet indicating the current time expressed using all of the hour, minute, and second Can be saved in every second. That is, as shown in FIG. 404, if the hour and minute of the time when the packet is transmitted is not updated from the time and minute indicated in the previously transmitted packet, a packet indicating only the second (time packet 1 Only the first information is required to be transmitted. Therefore, by reducing the second information that is the packet indicating the hour and the minute (time packet 2) than the first information that is the packet indicating the second (time packet 1) transmitted by the transmitter, Transmission of packets containing redundant contents can be suppressed.

  The sensor of the terminal device is an image sensor, and in the signal receiving step, the shutter speed of the image sensor is set to a first speed and a second speed higher than the first speed. While continuously switching, the image sensor performs continuous shooting, and (a) when the subject imaged by the image sensor is a barcode, by shooting when the shutter speed is the first speed, A barcode identifier is obtained by acquiring an image in which a barcode is reflected and decoding the barcode reflected in the image. (B) When the subject imaged by the image sensor is the light source A plurality of exposure lines included in the image sensor by photographing when the shutter speed is the second speed; In the reproducing method, the visible light signal is obtained as a visible light identifier by acquiring a bright line image that is an image including a bright line corresponding to the acquired bright line image, and decoding a plurality of bright line patterns included in the acquired bright line image. Furthermore, an image obtained by photographing when the shutter speed is the first speed may be displayed.

  As a result, as shown in FIG. 309, an identifier corresponding to the barcode or the light source can be appropriately acquired from the barcode or the visible light signal, and an image on which the barcode or the light source as the subject is projected is displayed. Can be displayed.

  In addition, in the acquisition of the visible light identifier, a first packet including a data part and an address part is acquired from the plurality of bright line patterns, and at least one already acquired before the first packet is obtained. It is determined whether or not there are a predetermined number or more of second packets, which are packets including the same address part as the address part of the first packet, and the second packet is the predetermined number If it is determined that there are more than the predetermined number, the pixel value of a part of the bright line image corresponding to each data portion of the second packet more than the predetermined number and the data portion of the first packet The visible light identifier is calculated by calculating a composite pixel value by combining pixel values of a part of the region of the corresponding bright line image and decoding a data portion including the composite pixel value. You may obtain at least a portion.

  As a result, as shown in FIG. 281, even if the data part is slightly different in each of a plurality of packets including the same address part, an appropriate data part can be obtained by matching the pixel values of the data part of those packets. It can be decoded and at least a portion of the visible light identifier can be obtained correctly.

  The first packet further includes a first error correction code for the data portion and a second error correction code for the address portion. In the signal receiving step, a second error correction code is sent from the transmitter. The data part transmitted by the luminance change according to the first frequency higher than the second frequency, receiving the address part and the second error correction code transmitted by the luminance change according to the frequency of The first error correction code may be received.

  As a result, as shown in FIG. 279, it is possible to suppress erroneous reception of the address part and to quickly acquire a data part with a large amount of data.

  In addition, in the acquisition of the visible light identifier, a first packet including a data part and an address part is acquired from the plurality of bright line patterns, and at least one already acquired before the first packet is obtained. It is determined whether or not there is at least one second packet that is a packet including the same address part as the address part of the first packet, and if there is the at least one second packet. If it is determined, it is determined whether or not the data portions of the at least one second packet and the first packet are all equal, and when it is determined that the data portions are not all equal. In each of the at least one second packet, among the portions included in the data portion of the second packet, the first packet It is determined whether there are more than a predetermined number of parts different from each part included in the data portion, and the number of different parts of the at least one second packet is greater than or equal to the predetermined number. When there is a second packet determined to exist, the at least one second packet is discarded, and the number of different portions of the at least one second packet is greater than or equal to the predetermined number. If there is no determined second packet, a plurality of packets having the largest number of packets having the same data part are identified from among the first packet and the at least one second packet, By decoding the data part included in each of the packets as a data part corresponding to the address part included in the first packet, the number of visible light identifiers is reduced. And you may also get the part.

  As a result, as shown in FIG. 280, when a plurality of packets having the same address part are received, even if the data parts of those packets are different, the appropriate data part can be decoded and visible. At least a part of the optical identifier can be acquired correctly. That is, a plurality of packets having the same address part transmitted from the same transmitter basically have the same data part. However, when the terminal device switches the transmitter that is the transmission source of the packet, the terminal device may receive a plurality of packets having different data portions even though they have the same address portion. In such a case, in the reproduction method according to one aspect of the present invention, the already received packet (second packet) is discarded as in step S10106 of FIG. Data portion of the packet) can be decoded as a correct data portion corresponding to the address portion. Furthermore, even when there is no transmitter switching as described above, the data portions of a plurality of packets having the same address portion may be slightly different depending on the transmission / reception state of the visible light signal. In such a case, in the reproduction method according to one aspect of the present invention, an appropriate data portion can be decoded by so-called majority decision as in step S10107 of FIG.

  Further, in the acquisition of the visible light identifier, a plurality of packets each including a data portion and an address portion are acquired from the plurality of bright line patterns, and all of the plurality of acquired packets included in the data portion are acquired. It is determined whether or not there is a 0-termination packet that is a packet whose bit indicates 0, and if it is determined that the 0-termination packet exists, the address part of the 0-termination packet is included in the plurality of packets. When it is determined whether or not all N related packets (N is an integer of 1 or more) that are packets including the associated address part exist, and when it is determined that all the N related packets exist The visible light identifier may be acquired by arranging and decoding the data portions of the N related packets. For example, the address part associated with the address part of the zero-termination packet is an address part that indicates an address that is smaller than or equal to the address indicated in the address part of the zero-termination packet.

  Specifically, as shown in FIG. 282, it is determined whether or not all packets having an address equal to or smaller than the address of the zero-termination packet are prepared as related packets. Each data portion of the packet is decoded. As a result, the terminal device does not know in advance how many related packets are necessary to obtain the visible light identifier, and further does not know the addresses of those related packets in advance. Also, when the zero-termination packet is acquired, it can be easily known. As a result, the terminal device can obtain an appropriate visible light identifier by arranging and decoding the data portions of the N related packets.

(Embodiment 34)
Hereinafter, the variable length / variable division number compatible protocol will be described.

  434 to 438 are diagrams illustrating an example of a transmission signal in this embodiment.

  The transmission packet includes a preamble, a TYPE, a payload, and a check unit. Packets may be transmitted continuously or intermittently. By providing a period during which no packet is transmitted, the state of the liquid crystal can be changed when the backlight is turned off, and the dynamic resolution of the liquid crystal display can be improved. Interference can be avoided by making the packet transmission interval random.

  A pattern that does not appear in 4PPM is used for the preamble. The reception process can be simplified by using a short basic pattern. As shown in (b) and (c) of FIG. 436, when the basic pattern is arranged behind, a packet can be received even when a short section is received. As shown in (c) of FIG. 436, the preamble can be accurately received by setting the beginning and end of the preamble to 1 (high brightness state).

  By expressing the number of data divisions according to the type of preamble, the number of data divisions can be made variable without using extra transmission slots.

  By changing the payload length according to the value of TYPE, transmission data can be made variable. In TYPE, the payload length may be expressed, or the data length before division may be expressed. By expressing the packet address by the value of TYPE, the receiver can arrange the received packets correctly. Since the required address length varies depending on the number of divisions, the length of TYPE may be changed depending on the number of divisions as shown in FIG. Further, the payload length (data length) represented by the value of TYPE may be changed depending on the type of preamble or the number of divisions.

  By changing the length of the check unit according to the payload length, efficient error correction (detection) can be performed. By setting the shortest length of the check part to 2 bits, it can be efficiently converted to 4PPM. Also, error correction (detection) can be performed efficiently by changing the type of error correction (detection) code depending on the payload length. The length of the check unit or the type of error correction (detection) code may be changed depending on the type of preamble or the value of TYPE.

  There are combinations that have the same data length with different combinations of payload and number of divisions. In such a case, even if the data value is the same, more values can be expressed by giving different meanings to each combination.

  Hereinafter, the high-speed transmission / luminance modulation protocol will be described.

  439 and 440 are diagrams each illustrating an example of a transmission signal in this embodiment.

  The transmission packet includes a preamble part, a body part, and a brightness adjustment part. The body includes an address part, a data part, and an error correction (detection) code part. By allowing intermittent transmission, the same effect as described above can be obtained.

  As shown in FIG. 440, by combining the luminance change of the preamble portion of two types of average luminance, the body portion of three types of average luminance, and the luminance adjustment unit capable of continuously changing the average luminance, The average luminance can be continuously changed as a whole. The continuous luminance change of the luminance adjusting unit can be realized by changing the maximum luminance or changing the ratio of the time between the bright period and the dark period. Since there are two types of expression methods around the luminance of 50%, they may be used separately at the boundary of 50%, and more values can be expressed by letting the two types of expression methods mean different data. it can.

  At the time of reception, assuming that four combinations of a preamble part and a body part are performed, maximum likelihood decoding is performed, and a pattern with the highest likelihood is used as a reception signal, so that even if the dimming state is unknown Can be received.

  Since the preamble portion has only two types of luminance changes and is a signal detection start portion, it is possible to efficiently receive a signal by setting a luminance reference in this portion.

  Hereinafter, a 4PPM convolutional decoding pattern will be described.

  441 to 444 are diagrams illustrating an example of a reception algorithm in the present embodiment.

  When the convolution length is 3, if the preamble of FIG. 440 is used, 1 and 0 are three consecutive, and therefore the preamble can be detected as it is without performing convolution decoding.

  At each luminance of 4 PPM, the convolution state transitions as shown in FIGS. By performing maximum likelihood estimation in each pattern, a signal can be received while performing convolutional decoding. The start states “000” and “111” in FIG. 444 do not exist in the end state, but the end state of the preamble is one of these states, so this pattern is used in the maximum likelihood decoding of the first four slots of the body part. By using this, maximum likelihood decoding can be accurately performed.

Here, as the sensitivity is set higher or the exposure time is set shorter, the noise associated with the imaging increases. Therefore, the noise is reduced by increasing the number of pixels used for obtaining the average luminance in the exposure line. can do. When the sensitivity is increased by N times, the amount of noise can be suppressed equally by setting the average number of pixels to the square of N. Conversely, when the sensitivity is set low and the exposure time is set long, the calculation load can be reduced by reducing the number of pixels to be averaged.

  If the value obtained by dividing the likelihood of the maximum likelihood path at the time of maximum likelihood decoding by the path length is lower than a predetermined value, the decoding result is unreliable and the decoded signal is discarded. Thereby, reception errors can be reduced.

  When the likelihood at the time of maximum likelihood decoding is low, or when error detection is performed, it may be possible to receive correctly by performing decoding again in a state where the number of pixels to be averaged is increased and noise is reduced. . In the same case, if the pixel value is high, it may be received correctly by reducing the sensitivity or shortening the exposure time. If the pixel value is low, the sensitivity may be increased. It may be possible to receive correctly by increasing the value or increasing the exposure time.

When the sampling rate of the receiver (= the reciprocal of the time difference between exposure timings between adjacent exposure lines) is f hertz, the transmission frequency is Nf / 2 + k hertz. However, N is an integer and k <f / 2. At this time, when the received signal is subjected to frequency analysis, high frequency components are not observed, and k hertz is obtained as an alias. Therefore, for example, when representing four values, k = {0,
By assigning f / 4, f / 2, f × 3/4}, the receiver can identify each signal. At this time, since the frequency is higher than that of simply using k hertz as a transmission signal, effects such as suppressing flickering can be obtained. For example, a barcode reader reading error is induced at a frequency of several kilohertz to several tens of kilohertz, but this problem can be avoided by using a frequency higher than that.

  Since the barcode reader reads the reflected light of the red light beam, the adverse effect on the reading of the barcode reader can be removed by removing the red component from the visible light communication signal. In order to remove the red component from the visible light communication signal, there are methods such as using a filter or a phosphor that emits red light having a phase opposite to that of the signal or that increases the afterglow of a component having a long wavelength. The receiver can receive the signal correctly by receiving the signal from the luminance of the green or blue component light.

  Depending on the last action performed by the user on the transmitter, the transmitter changes the signal to be transmitted. As a result, it is possible to transmit information on a phenomenon that occurs as a result of the operation, or to instruct a signal that the user wants to transmit.

  The transmitter transmits a signal for a predetermined time after the user performs the last operation. Thereby, power consumption can be suppressed and the timer of the microcomputer can be assigned to another function.

  The transmitter sends a signal only when the microcomputer timer is not used for another function. Or a transmitter performs the function using the timer of the microcomputer including signal transmission alternately. As a result, the transmitter can be configured with a small number of timers.

  The receiver performs different operations only for a predetermined time after receiving predetermined data by visible light communication. For example, products and services can be discounted or campaign items used in games can be downloaded within 1 hour after receiving an ID from a billboard. Thereby, an offline-to-online service or an offline-to-online-to-offline service can be realized.

  The receiver receives the setting information from the transmitter and stores it in the storage in the receiver in association with the identification information of the transmitter. The receiver receives the identification information from the same or another transmitter, and sets the associated setting information in the transmitter. The setting information may be, for example, the language setting of the receiver other than the information received from the transmitter. As a result, the transmitter settings can be quickly restored. In addition, settings suitable for the user can be performed on the transmitter without input by the user.

  While the reception process is continuing, the receiver performs normal imaging and displays it on the preview screen. As a result, a smoother preview can be displayed without degrading the reception performance.

  The receiver captures a predetermined number of visible light images continuously or intermittently, stores the images in the memory until the processing of the captured image of the previous frame is completed, and sequentially performs reception processing. Thereby, even when the time when the receiver is directed to the transmitter is short, the reception can be completed.

(Embodiment 35)
(Frame configuration of Single frame transmission)
445 and 446 are diagrams illustrating an example of a transmission signal in this embodiment.

  The transmission frame includes a preamble (PRE), a frame length (FLEN), an ID type (IDTYPE), a content (ID / DATA), and a check code (CRC), and may include a content type (CONTENTTYPE). The number of bits in each area is an example.

  By specifying the ID / DATA length with FLEN, it is possible to transmit variable-length content.

  The CRC is a check code for correcting or detecting an error in a portion other than the PRE. By changing the CRC length in accordance with the length of the inspection area, the inspection capability can be maintained above a certain level. Moreover, the inspection capability per CRC length can be improved by using different inspection codes depending on the length of the inspection region.

  As shown in FIG. 446 (e), when CONTENTTYPE is a predetermined bit, ID / DATA indicates ID, and when it is not the predetermined bit, ID / DATA indicates data. When CONTENTTYPE is not the predetermined bit, the IDTYPE area is also an ID / DATA area, so that a larger data capacity can be transmitted.

  As shown in (f) and (g) of FIG. 446, by changing the IDTYPE length according to the ID / DATA length, it is possible to define an appropriate amount of ID types according to the ID / DATA length. For example, when the ID / DATA length is long with many types of IDs, many IDs can be defined by increasing the IDTYPE length. Alternatively, many ID systems can be defined by increasing the IDTYPE length for a short and easy-to-use ID / DATA length. Alternatively, by setting the IDTYPE length to 0 in a specific ID / DATA length, the length of the entire frame can be shortened, and quick transmission / reception and long-distance communication can be performed.

(Frame configuration of multiple frame transmission)
FIG. 447 is a diagram illustrating an example of a transmission signal in this embodiment.

  The transmission frame includes a preamble (PRE), an address (ADDR), and a part of the divided data (DATAPART), and may include a division number (PARTNUM) and an address flag (ADDRFRAG). The number of bits in each area is an example.

  A long-distance communication can be performed by dividing the content into a plurality of parts and transmitting the content.

  By dividing the size into equal parts, the maximum frame length can be reduced, and stable communication can be performed.

  If equal division is not possible, data of just the right size can be transmitted by making some of the divided portions smaller than other divided portions.

  More sizes of information can be transmitted by making the size of division different and giving meaning to combinations of division sizes. For example, even if the data is the same value of 32 bits, when 8 bits are transmitted 4 times, when 16 bits are transmitted 2 times, when 15 bits are transmitted once and 17 is transmitted once Then, it is possible to express a larger amount of information by treating it as different information.

  By indicating the number of divisions with PARTNUM, the receiver can immediately know the number of divisions and can accurately display the progress of reception.

  When ADDRFRAG is 0, it is not the last address, but when it is 1, the area indicating the number of divisions is unnecessary, and transmission can be performed in a shorter time.

  The CRC is a check code for correcting or detecting an error in a portion other than the PRE, as described above. By this inspection, interference can be detected when transmission frames from a plurality of transmission sources are received. By setting the CRC length to an integer multiple of the DATAPART length, it is possible to detect interference most efficiently.

  A check code for inspecting a part other than the PRE of each frame may be added to the end of the divided frame (the frame indicated by (a), (b), or (c) in FIG. 447).

  The IDTYPE shown by (d) in FIG. 447 may be a fixed length such as 4 bits or 5 bits as in (a) to (d) of FIG. 445, or as shown in (f) and (g) of FIG. In addition, the IDTYPE length may be changed according to the ID / DATA length. Thereby, the same effect as described above can be obtained.

(Designation of ID / DATA length)
448 and 449 are diagrams illustrating an example of a transmission signal in this embodiment.

  In the case of (a) to (d) in FIG. 445, by setting the tables (a) and (b) shown in FIG. 448 and the tables (a) and (b) shown in FIG. Sometimes ucode can be expressed.

(CRC length and generator polynomial)
FIG. 450 is a diagram illustrating an example of a transmission signal in this embodiment.

  By setting the CRC length in this way, the inspection capability can be maintained regardless of the length of the inspection target.

  The generator polynomial is an example, and another generator polynomial may be used. A check code other than CRC may be used. As a result, the inspection capability can be improved.

(Designation of DATAPART length by the type of preamble and designation of the last address)
FIG. 451 is a diagram illustrating an example of a transmission signal in this embodiment.

  By indicating the DATAPART length as the type of preamble, an area indicating the DATAPART length is not necessary, and information can be transmitted in a shorter transmission time. Further, by indicating whether or not it is the last address, an area indicating the number of divisions is not necessary, and information can be transmitted in a shorter transmission time. In the case of (b) in FIG. 451, since the DATAPART length is not known in the case of the last address, it is estimated that it is the same as the DATAPART length of the frame that is not the last address received immediately before or after the frame reception. By performing the receiving process, it is possible to receive normally.

  The address length may be different depending on the type of preamble. Thereby, the combination of the length of transmission information can be increased, or it can transmit in a short time.

  In the case of (c) in FIG. 451, the number of divisions is defined by the preamble, and an area indicating the DATAPART length is added.

(Specify address)
FIG. 452 is a diagram illustrating an example of a transmission signal in this embodiment.

  By indicating the address of the frame with the value of ADDR, the receiver can reconstruct correctly transmitted information.

  By indicating the number of divisions by the value of PARTNUM, the receiver can know the number of divisions when the first frame is received, and can accurately display the progress of reception.

(Prevention of interference due to differences in the number of divisions)
FIGS. 453 and 454 are a diagram and a flowchart illustrating an example of a transmission / reception system in this embodiment.

  When transmission information is equally divided and transmitted separately, the signals from transmitter A and transmitter B in FIG. 453 have different preambles. Therefore, even when these signals are received simultaneously, the receiver confuses the transmission source. Transmission information can be reconstructed without any problem.

  Since the transmitters A and B include the division number setting unit, the user can set the division numbers of transmitters installed nearby to be different, and can prevent interference.

  The receiver registers the number of divisions of the received signal with the server, so that the server can know the number of divisions set by the transmitter, and the other receivers obtain the information from the server, The progress of reception can be accurately displayed.

  The receiver obtains from the server or the storage unit of the receiver whether or not the signal from the nearby or corresponding transmitter is an equal-length division. When the acquired information is equal length division, a signal is restored only from a frame having the same DATAPART length. If this is not the case, or if a situation in which all addresses are not complete in a frame with the same DATAPART length continues for a predetermined time or longer, the signal is restored by combining frames with different DATAPART lengths.

(Prevention of interference due to differences in the number of divisions)
FIG. 455 is a flowchart showing the operation of the server in this embodiment.

  The server receives from the receiver the ID received by the receiver and the divided configuration (what combination of DATAPART lengths the signal was received in). When the ID is an object to be expanded by a divided configuration, a numerical value of the divided configuration pattern is used as an auxiliary ID, and information associated with the extended ID obtained by combining the ID and the auxiliary ID as a key is received by the receiver. To pass.

  If it is not the target of expansion by the divided configuration, it is confirmed whether the divided configuration associated with the ID exists in the storage unit, and whether it is the same as the received divided configuration. If they are different, send a reconfirmation command to the receiver. Thereby, it is possible to prevent erroneous information from being presented due to a reception error of the receiver.

  If the same divided configuration is received with the same ID within a predetermined time after transmitting the reconfirmation command, it is determined that the divided configuration has been changed, and the divided configuration associated with the ID is updated. As a result, as described in the explanation of FIG. 453, it is possible to cope with a case where the division configuration is changed.

  When the division configuration is not stored, when the received division configuration matches the stored division configuration, or when the division configuration is updated, the associated information is passed to the receiver using the ID as a key, The divided configuration is associated with the ID and stored in the storage unit.

(Display of reception progress)
FIGS. 456 to 461 are a flowchart and a diagram illustrating an example of operation of a receiver in this embodiment.

  The receiver acquires the type and ratio of the number of divisions of the transmitter that is supported by the receiver or the transmitter in the vicinity of the receiver from the storage area of the server or the receiver. Further, when a part of the divided data has already been received, the type and ratio of the number of divisions of the transmitter that transmits the information that matches the part of the divided data is acquired.

  The receiver receives the divided frame.

  If the last address has already been received, if the obtained number of divisions is only one, or if the number of divisions supported by the receiving application being executed is only one, the number of divisions is Since it is known, the progress status is displayed based on the number of divisions.

  Otherwise, if the available processing resources are low or the energy saving mode is set, the receiver calculates and displays the progress status in the simple mode. On the other hand, when there are many available processing resources or when the energy saving mode is not set, the progress status is calculated and displayed in the maximum likelihood estimation mode.

  FIG. 457 is a flowchart showing a method of calculating the progress status in the simple mode.

  First, the receiver acquires the standard division number Ns from the server. Alternatively, the receiver reads the standard division number Ns from its own data holding unit. Note that the standard number of divisions is (a) the mode value or expected value of the number of transmitters to be transmitted with the number of divisions, (b) the number of divisions determined for each packet length, and (c) the number of divisions determined for each application. Or (d) the number of divisions determined for each identifiable range where the receiver is located.

  Next, the receiver determines whether or not a packet indicating the final address has been received. If it is determined that the packet is received, the address of the last packet is set to N. On the other hand, if it is determined that it has not been received, the number obtained by adding a number of 1 or 2 to the received maximum address Amax is set to Ne. Here, the receiver determines whether Ne> Ns. If it is determined that Ne> Ns, the receiver sets N = Ne. On the other hand, if it is determined that Ne> Ns is not satisfied, the receiver sets N = Ns.

  Then, the receiver calculates the ratio of the number of received packets among the packets necessary for reception of the entire signal, assuming that the number of divisions of the signal being received is N.

  In such a simple mode, the progress situation can be calculated by a simpler calculation than the maximum likelihood estimation mode, which is advantageous in terms of processing time or energy consumption.

  FIG. 458 is a flowchart illustrating a method for calculating the progress in the maximum likelihood estimation mode.

  First, the receiver acquires the prior distribution of the number of divisions from the server. Alternatively, the receiver reads the prior distribution from its own data holding unit. The prior distribution is defined as (a) the distribution of the number of transmitters to be transmitted in the division number, (b) defined for each packet length, (c) defined for each application, or (D) The location where the receiver is located, and is determined for each identifiable range.

  Next, the receiver receives the packet x, and calculates the probability P (x | y) of receiving the packet x when the division number is y. Then, when the receiver receives the packet x, the receiver obtains a probability P (y | x) that the number of transmission signal divisions is y as P (x | y) × P (y) ÷ A (note that A Is a normalized multiplier). Further, the receiver sets P (y) = P (y | x).

  Here, the receiver determines whether the division number estimation mode is the maximum likelihood mode or the likelihood average mode. In the case of the maximum likelihood mode, the receiver calculates the ratio of the number of received packets, with y being the maximum P (y) as the division number. On the other hand, in the likelihood average mode, the receiver calculates the ratio of the number of received packets with the sum of y × P (y) as the number of divisions.

  In such maximum likelihood estimation mode, a more accurate degree of progress can be calculated than in the simple mode.

  In addition, when the division number estimation mode is the maximum likelihood mode, the likelihood of what the last address is from the addresses received so far is calculated, and the maximum likelihood is estimated as the division number. Displays the progress of reception. This display method can display the progress status closest to the actual progress status.

  FIG. 459 is a flowchart showing a display method in which the progress status does not decrease.

  First, the receiver calculates the ratio of the number of received packets among the packets necessary for receiving the entire signal. Then, the receiver determines whether or not the calculated ratio is smaller than the ratio being displayed. If it is determined that the ratio is smaller than the display ratio, the receiver further determines whether the display ratio is a calculation result before a predetermined time or more. If it is determined that the calculation result is more than a predetermined time, the receiver displays the calculated ratio. On the other hand, if it is determined that the calculation result is not more than a predetermined time ago, the receiver continues to display the displayed ratio.

  When the receiver determines that the calculated ratio is equal to or higher than the ratio being displayed, Ne sets the number obtained by adding a number of 1 or 2 to the received maximum address Amax. The receiver then displays the calculated percentage.

  When the final packet is received, it is unnatural that the calculation result of the progress status becomes smaller than before, that is, the displayed progress status (degree of progress) decreases. However, the above-described display method can suppress such unnatural display.

  FIG. 460 is a flowchart illustrating a progress display method when there are a plurality of packet lengths.

  First, the receiver calculates the ratio P of the number of received packets for each packet length. Here, the receiver determines whether the display mode is the maximum mode, the full display mode, or the latest mode. If it is determined that the mode is the maximum mode, the receiver displays the maximum ratio among the ratios P of the plurality of packet lengths. When it is determined that the display mode is the full display mode, the receiver displays all the ratios P. If it is determined that the mode is the latest mode, the receiver displays the packet length ratio P of the last received packet.

  In FIG. 461, (a) is the progress status calculated as the simple mode, (b) is the progress status calculated as the maximum likelihood mode, and (c) is the minimum of the obtained number of divisions. Is the progress of the case. Since the progress status increases in the order of (a), (b), and (c), all the progress statuses can be displayed at the same time by displaying (a), (b), and (c) in this manner. .

(Split transmission)
FIG. 462 is a diagram illustrating an example of a transmission signal in this embodiment.

  The check codes used in this frame configuration include the two CRCs shown in FIGS. By increasing the CRC length as the bit length of the portion to be checked increases, it is possible to prevent a decrease in inspection capability.

  Table (a) in FIG. 462 shows a case where the CRC length is a multiple of 4, and Table (b) in FIG. 462 shows a case where the CRC length is a multiple of 2. Since 4PPM encodes every 2 bits, if it is a multiple of 2, it can be allocated without waste. When the inspection target bit length is N, the CRC length is defined not to be less than 2 log (N). Note that log is a logarithm with 2 as the base. Thereby, it can set so that it may not fall below fixed inspection capability. The numerical values in the table are examples, and other CRC lengths may be assigned or CRC lengths other than multiples of 2 and 4 may be used.

  The generator polynomial in the table is an example, and another generator polynomial may be used. Also, another check code may be used instead of the CRC.

  By dividing the check code, even if the frame cannot be received to the end, a certain amount of check can be performed and the error rate can be reduced.

(Light emission control by common switch and pixel switch)
In the transmission method according to the present embodiment, for example, each LED included in the LED display for video display is changed in luminance according to the switching of the common switch and the pixel switch, so that a visible light signal (also a visible light communication signal) is obtained. Send).

  The LED display is configured as a large display installed outdoors, for example. The LED display includes a plurality of LEDs arranged in a matrix, and displays an image by blinking these LEDs in accordance with a video signal. Such an LED display includes a plurality of common lines (COM lines) and a plurality of pixel lines (SEG lines). Each common line is composed of a plurality of LEDs arranged in a line in the horizontal direction, and each pixel line is composed of a plurality of LEDs arranged in a line in the vertical direction. Each of the plurality of common lines is connected to a common switch corresponding to the common line. The common switch is, for example, a transistor. Each of the plurality of pixel lines is connected to a pixel switch corresponding to the pixel line. A plurality of pixel switches corresponding to a plurality of pixel lines are provided in, for example, an LED driver circuit (constant current circuit). The LED driver circuit is configured as a pixel switch control unit that switches a plurality of pixel switches.

  More specifically, one of the anode and the cathode of each LED included in the common line is connected to a terminal such as a collector of a transistor corresponding to the common line. The other of the anode and the cathode of each LED included in the pixel line is connected to a terminal (pixel switch) corresponding to the pixel line in the LED driver circuit.

  When such an LED display displays an image, a common switch control unit that controls a plurality of common switches turns on the common switches in a time-sharing manner. For example, the common switch control unit turns on only the first common switch among the plurality of common switches during the first period, and the second common among the plurality of common switches during the next second period. Turn on only the switch. Then, the LED driver circuit turns on each pixel switch according to the video signal during a period when any one of the common switches is turned on. As a result, the LEDs corresponding to the common switch and the pixel switch are lit only during a period in which the common switch is on and the pixel switch is on. The luminance of the pixels in the video is expressed by this lighting period. That is, the luminance of the image pixels is PWM-controlled.

  In the transmission method in the present embodiment, a visible light signal is transmitted using such an LED display, a common switch and a pixel switch, and a common switch control unit and a pixel switch control unit. In addition, the transmission device (also referred to as a transmitter) in the present embodiment that transmits a visible light signal by such a transmission method includes the common switch control unit and the pixel switch control unit.

  FIG. 463 is a diagram illustrating an example of a transmission signal in this embodiment.

  The transmitter transmits each symbol included in the visible light signal according to a predetermined symbol period. For example, when transmitting the symbol “00” by 4PPM, the transmitter switches the common switch according to the symbol (the luminance change pattern of “00”) in a symbol period of 4 slots. Then, the transmitter switches the pixel switch according to the average luminance indicated by the video signal or the like.

  More specifically, when the average luminance in the symbol period is 75% ((a) in FIG. 463), the transmitter turns off the common switch during the first slot, and the second to fourth slots. The common switch is turned on during the period up to the slot. Further, the transmitter turns off the pixel switch during the period of the first slot, and turns on the pixel switch during the period from the second slot to the fourth slot. As a result, the LEDs corresponding to the common switch and the pixel switch are lit only during a period in which the common switch is on and the pixel switch is on. In other words, the luminance of the LED changes by lighting at the luminance of LO (Low), HI (High), HI, and HI in each of the four slots. As a result, the symbol “00” is transmitted.

  Further, when the average luminance in the symbol period is 25% ((e) in FIG. 463), the transmitter turns off the common switch during the first slot and during the period from the second slot to the fourth slot. Turn on the common switch. Further, the transmitter turns off the pixel switch during the first slot, the third slot, and the fourth slot, and turns on the pixel switch during the second slot. As a result, the LEDs corresponding to the common switch and the pixel switch are lit only during a period in which the common switch is on and the pixel switch is on. In other words, the luminance of the LED changes by being lit like LO (Low), HI (High), LO, and LO in each of the four slots. As a result, the symbol “00” is transmitted. In addition, since the transmitter in this Embodiment transmits the visible light signal close | similar to the above-mentioned V4PPM (variable 4PPM), even when transmitting the same symbol, it can make an average luminance variable. That is, when transmitting the same symbol (for example, “00”) with mutually different average luminances, the transmitter, as shown in FIGS. ) Is constant regardless of the average brightness. Thereby, the receiver can receive a visible light signal without being conscious of luminance.

  The common switch is switched by the above-described common switch control unit, and the pixel switch is switched by the above-described pixel switch control unit.

  As described above, the transmission method in the present embodiment is a transmission method for transmitting a visible light signal by a change in luminance, and includes a determination step, a common switch control step, and a first pixel switch control step. In the determining step, the luminance change pattern is determined by modulating the visible light signal. In the common switch control step, a common switch for commonly lighting a plurality of light sources (LEDs) included in a light source group (common line) provided in the display and representing each pixel in the video is displayed with the brightness. Switching according to the change pattern. In the first pixel switch control step, the common switch is turned on by turning on the first pixel switch for turning on the first light source among the plurality of light sources included in the light source group, and The visible light signal is transmitted by turning on the first light source only during the period when the first pixel switch is on.

  Thereby, a visible light signal can be appropriately transmitted from a display including a plurality of LEDs as light sources. Therefore, the communication between the apparatuses of the aspect containing apparatuses other than illumination is enabled. When the display is a display for displaying an image by controlling the common switch and the first pixel switch, a visible light signal may be transmitted using the common switch and the first pixel switch. it can. Therefore, a visible light signal can be easily transmitted without making a significant change to the configuration for displaying an image on a display.

  Further, by making the control timing of the pixel switch coincide with the transmission symbol (for one 4PPM) and controlling as shown in FIG. 463, a visible light signal can be transmitted from the LED display without flickering. Image signals (that is, video signals) usually change at 1/30 second or 1/60 second cycles, but by changing the image signal according to the symbol transmission cycle (symbol cycle), this can be achieved without changing the circuit. can do.

  Thus, in the determination step of the transmission method in the present embodiment, the luminance change pattern is determined for each symbol period. In the first pixel switch control step, the pixel switch is switched in synchronization with the symbol period. Thereby, even if a symbol period is 1/2400 second, a visible light signal can be appropriately transmitted according to the symbol period.

  When the signal (symbol) is “10” and the average luminance is around 50%, the luminance change pattern is close to 0101, and the luminance rises at two locations. However, in that case, the receiver can correctly receive the signal by giving priority to the subsequent rising point. In other words, the subsequent rising point is the timing at which the luminance rising inherent to the symbol “10” is obtained.

  As the average luminance is higher, a signal closer to a signal modulated by 4 PPM can be output. Therefore, when the luminance of the entire screen or the portion where the power supply line is common is low, the HI section can be lengthened and the error can be reduced by reducing the current and decreasing the instantaneous luminance value. . In this case, the maximum brightness of the screen is reduced, but when high brightness is not necessary in the first place, such as indoor use, or when priority is given to visible light communication, enabling the switch to enable this enables communication. The balance between quality and image quality can be set optimally.

  In the first pixel switch control step of the transmission method according to the present embodiment, when an image is displayed on the display (LED display), the pixel value of the pixel in the image corresponding to the first light source is expressed. The first pixel switch is switched so that the lighting period is supplemented only during the period during which the first light source is turned off in order to transmit the visible light signal. That is, in the transmission method in the present embodiment, a visible light signal is transmitted when an image is displayed on the LED display. Therefore, the LED may be turned off for transmission of a visible light signal in a period in which the LED is to be turned on in order to express a pixel value (specifically, a luminance value) indicated by the video signal. In such a case, in the transmission method according to the present embodiment, the first pixel switch is switched so as to compensate for the lighting period only during the period when the LED is turned off.

  For example, when displaying an image indicated by a video signal without transmitting a visible light signal, the common switch is turned on during one symbol period, and the pixel switch is an average value that is the pixel value indicated by the video signal. It is turned on only for the period according to the brightness. When the average luminance is 75%, the common switch is turned on in the first to fourth slots of the symbol period. Further, the pixel switch is turned on in the first to third slots of the symbol period. Thereby, during the symbol period, the LEDs are lit in the first slot to the third slot, so that the above-described pixel values can be expressed. However, in order to transmit the symbol “01”, the second slot is turned off. Therefore, in the transmission method according to the present embodiment, the pixel switch is switched so that only the second slot in which the LED is turned off compensates for the lighting period of the LED, that is, the LED is turned on in the fourth slot. .

  In the transmission method in this embodiment, the lighting period is compensated by changing the pixel value of the pixel in the video. For example, in the above case, the pixel value with an average luminance of 75% is changed to a pixel value with an average luminance of 100%. When the average luminance is 100%, the LED tries to turn on in the first slot to the fourth slot, but the first slot is turned off for transmission of the symbol “01”. Therefore, even when a visible light signal is transmitted, the LED can be lit with the original pixel value (average luminance 75%).

  Thereby, it is possible to suppress the image from being corrupted by the transmission of the visible light signal.

(Light emission control shifted for each pixel)
FIG. 464 is a diagram illustrating an example of a transmission signal in this embodiment.

  When transmitting the same symbol (for example, “10”) from the pixel A and the pixels in the vicinity of the pixel A (for example, the pixel B and the pixel C) as shown in FIG. The light emission timing of the pixels is shifted. However, the transmitter causes the pixels to emit light without shifting the rise timing of the luminance specific to the symbol between the pixels. Note that each of the pixels A to C corresponds to a light source (specifically, an LED). In addition, when the symbol is “10”, the luminance rise timing specific to the symbol is the boundary timing between the third slot and the fourth slot. Such timing is hereinafter referred to as symbol specific timing. The receiver can receive a symbol corresponding to the timing by specifying the symbol specific timing.

  By shifting the light emission timing in this way, the waveform indicating the average luminance transition between pixels has a gradual rise or fall except for the rise at the symbol specific timing, as shown in FIG. That is, the rise at the symbol specific timing is steeper than the rise at other timings. Therefore, the receiver can identify an appropriate symbol specific timing by giving priority to receiving the steepest rise among a plurality of rises, and as a result, reception errors can be suppressed.

  In other words, when the symbol “10” is transmitted from a predetermined pixel and the luminance of the predetermined pixel is an intermediate value of 25% to 75%, the transmitter opens the pixel switch corresponding to the predetermined pixel. Set the section short or long. Further, the transmitter reversely adjusts the open period of the pixel switch corresponding to the pixel in the vicinity of the predetermined pixel. Thus, errors can also be suppressed by setting the open interval of each pixel switch so that the overall luminance including the predetermined pixel and the neighboring pixels does not change. The open section is a section where the pixel switch is on.

  Thus, the transmission method in the present embodiment further includes the second pixel switch control step. In this second pixel switch control step, by turning on the second pixel switch for turning on the second light source around the first light source included in the light source group (common line) described above. The visible light signal is transmitted by turning on the second light source only during the period when the common switch is on and the second pixel switch is on. Note that the second light source is, for example, a light source adjacent to the first light source.

  In the first and second pixel switch control steps, when the same symbol included in the visible light signal is simultaneously transmitted from each of the first and second light sources, the first and second pixel switches are controlled. Among a plurality of timings each of which is turned on or off to transmit the same symbol, the timing at which a rise in luminance specific to the same symbol is obtained is made the same in each of the first and second pixel switches, Other timings are made different for each of the first and second pixel switches, and the overall average luminance of the first and second light sources in a period in which the same symbol is transmitted is set to a predetermined luminance. Match.

  As a result, as shown in FIG. 464, the rise of the brightness can be made sharp only at the timing when the rise of the brightness specific to the symbol is obtained in the spatially averaged brightness as in the transition between the average brightness of pixels. The occurrence of errors can be suppressed. That is, the reception error of the visible light signal by the receiver can be suppressed.

  In addition, when the symbol “10” is transmitted from a predetermined pixel and the luminance of the predetermined pixel is an intermediate value of 25% to 75%, the transmitter transmits the symbol “10” to the predetermined pixel in the first period. The open section of the corresponding pixel switch is set to be short or long. Further, the transmitter reversely adjusts the open period of the pixel switch in a second period (for example, a frame) before or after the first period. As described above, the error can be suppressed by setting the open period of the pixel switch so that the entire time average luminance including the first period and the second period in the predetermined pixel does not change.

  That is, in the first pixel switch control step in the transmission method according to the present embodiment, for example, it is included in the visible light signal in the first period and the second period following the first period. Send the same symbol. At this time, in each of the first and second periods, a rise in luminance specific to the same symbol among a plurality of timings at which the first pixel switch is turned on or off to transmit the same symbol. The obtained timing is made the same, and other timings are made different. Then, the average luminance of the first light source in the entire first and second periods is matched with a predetermined luminance. Each of the first period and the second period may be a period for displaying a frame and a period for displaying the next frame. Further, the first period and the second period may each be a symbol period. That is, the first period and the second period may be a period for transmitting one symbol and a period for transmitting the next symbol, respectively.

  As a result, like the average luminance transition between pixels shown in FIG. 464, in the temporally averaged luminance, the rise can be made sharp only at the timing when the luminance rise inherent to the symbol is obtained. The occurrence of reception errors can be suppressed. That is, the reception error of the visible light signal by the receiver can be suppressed.

(Light emission control when the pixel switch can be driven at double speed)
FIG. 465 is a diagram illustrating an example of a transmission signal in this embodiment.

  When the pixel switch can be opened and closed at half the transmission symbol period, that is, when the pixel switch can be driven at double speed, the same light emission pattern as V4PPM (see FIG. 325) can be obtained as shown in FIG.

  In other words, when the symbol period (symbol transmission period) is composed of 4 slots, a pixel switch control unit such as an LED driver circuit that controls the pixel switch can control the pixel switch every two slots. That is, the pixel switch control unit can turn on the pixel switch for an arbitrary time in a period of two slots from the first time point of the symbol period. Further, the pixel switch control unit can turn on the pixel switch for an arbitrary time in a period of two slots from the first time point of the third slot of the symbol period.

  That is, in the transmission method according to the present embodiment, the pixel value may be changed at a period that is ½ of the above-described symbol period.

  In this case, the fineness per opening / closing of the pixel switch may be reduced (accuracy is reduced). Therefore, by performing this only when the transmission priority switch is valid, the balance between image quality and transmission quality can be set optimally.

(Light emission control block by pixel value adjustment)
FIG. 466 is a block diagram illustrating an example of a transmitter in this embodiment.

  FIG. 466 (a) is a block diagram showing a configuration of a device that does not transmit a visible light signal and only displays a video, that is, a display device that displays a video on the LED display described above. As shown in FIG. 466 (a), the display device includes an image / video input unit 1911, an N-times speed increasing unit 1912, a common switch control unit 1913, and a pixel switch control unit 1914.

  The image / video input unit 1911 outputs a video signal indicating an image or video at a frame rate of 60 Hz, for example, to the N-times speed increasing unit 1912.

  The N-times speed increasing unit 1912 increases the frame rate of the video signal input from the image / video input unit 1911 by N (N> 1) times and outputs the video signal. For example, the N-times speed increasing unit 1912 increases the frame rate to 10 times (N = 10), that is, to a frame rate of 600 Hz.

  The common switch control unit 1913 switches the common switch based on the 600 Hz frame rate video. Similarly, the pixel switch control unit 1914 switches pixel switches based on the 600 Hz frame rate video. Thus, flickering due to opening and closing of a switch such as a common switch or a pixel switch can be avoided by increasing the frame rate by the N-times speed increasing unit 1912. Even when the LED display is imaged by the imaging device with a high-speed shutter, it is possible to cause the imaging device to capture an image without missing pixels or flicker.

  FIG. 466 (b) is a block diagram showing a configuration of a display device that transmits not only video but also the above-described visible light signal, that is, a transmitter (transmitting device). This transmitter includes an image / video input unit 1911, a common switch control unit 1913, a pixel switch control unit 1914, a signal input unit 1915, and a pixel value adjustment unit 1916. The signal input unit 1915 outputs a visible light signal including a plurality of symbols to the pixel value adjustment unit 1916 at a symbol rate (frequency) of 2400 symbols / second.

  The pixel value adjustment unit 1916 duplicates the image input from the image / video input unit 1911 in accordance with the symbol rate of the visible light signal, and adjusts the pixel value according to the above-described method. Accordingly, the common switch control unit 1913 and the pixel switch control unit 1914 in the subsequent stage from the pixel value adjustment unit 1916 can output a visible light signal without changing the luminance of the image or video.

  For example, in the example shown in FIG. 466, when the symbol rate of the visible light signal is 2400 symbols / second, the pixel value adjustment unit 1916 is included in the video signal so that the frame rate of the video signal 60 Hz is 4800 Hz. Duplicate the image. For example, the value of the symbol included in the visible light signal is “00”, and the pixel value (luminance value) of the pixel included in the first image before duplication is 50%. In this case, the pixel value adjustment unit 1916 adjusts the pixel value to 100% for the first image after duplication and to 50% for the second image. As a result, the luminance is 50% due to AND of the common switch and the pixel switch as in the luminance change in the case of the symbol “00” shown in FIG. As a result, a visible light signal can be transmitted while maintaining the same luminance as the original image. Note that the AND of the common switch and the pixel switch means that the light source (that is, the LED) corresponding to the common switch and the pixel switch is lit only when the common switch is on and the pixel switch is on. .

  In the transmission method in this embodiment, the video display and the visible light signal transmission may not be performed at the same time, but may be performed separately for the signal transmission period and the video display time.

  That is, in the above-described first pixel switch control step in the present embodiment, the first pixel switch is turned on during the signal transmission period in which the common switch is switched according to the luminance change pattern. The transmission method in the present embodiment further turns on the common switch during a video display period different from the signal transmission period, and turns on the first pixel switch according to the video to be displayed in the video display period. Accordingly, a video display step of displaying the pixels in the video by turning on the first light source only during a period in which the common switch is on and the first pixel switch is on may be included. .

  As a result, since the display of the video and the transmission of the visible light signal are performed in different periods, the display and transmission can be easily performed.

(Power change timing)
When the power supply line is changed, a signal off period occurs. However, even if the last part of 4PPM is not emitting light, it does not affect reception. By changing the power supply line according to the transmission period of the 4PPM symbol, The power line can be changed without affecting the reception quality.

  Also, changing the power supply line during the 4PPM LO period can change the power supply line without affecting the reception quality. In this case, transmission can be performed while keeping the maximum luminance high.

(Drive timing)
In this embodiment, the LED display may be driven at the timings shown in FIGS. 467 to 469.

  FIGS. 467 to 469 are timing charts when the LED display is driven by the optical ID modulation signal of the present invention.

  For example, as shown in FIG. 468, when the common switch (COM1) is turned off (period t1) in order to transmit a visible light signal (light ID), the LED is turned on with the luminance indicated by the video signal. Therefore, the LED is lit after the period t1. Thereby, the video can be appropriately displayed without breaking the video indicated by the video signal while appropriately transmitting the visible light signal.

(Summary)
FIG. 470A is a flowchart illustrating a transmission method according to one embodiment of the present invention.

  A transmission method according to one embodiment of the present invention is a transmission method for transmitting a visible light signal by a change in luminance, and includes steps SC11 to SC13.

  In step SC11, the luminance change pattern is determined by modulating the visible light signal as in the above-described embodiments.

  In Step SC12, a common switch for commonly lighting a plurality of light sources each representing a pixel in the video included in the light source group provided in the display is switched according to the luminance change pattern.

  In step S13, the common switch is turned on by turning on the first pixel switch (that is, the pixel switch) for turning on the first light source among the plurality of light sources included in the light source group, and The visible light signal is transmitted by turning on the first light source only during a period in which the first pixel switch is on.

  FIG. 470B is a block diagram illustrating a functional configuration of the transmission device according to one embodiment of the present invention.

  A transmission device C10 according to an aspect of the present invention is a transmission device (or transmitter) that transmits a visible light signal by a luminance change, and includes a determination unit C11, a common switch control unit C12, a pixel switch control unit C13, Is provided. The determination unit C11 determines the luminance change pattern by modulating the visible light signal, as in the above-described embodiments. In addition, this determination part C11 is provided in the signal input part 1915 shown in FIG. 466, for example.

  The common switch control unit C12 switches the common switch according to the luminance change pattern. This common switch is a switch for commonly lighting a plurality of light sources included in a light source group provided in the display, each representing a pixel in an image.

  The pixel switch control unit C13 turns on the pixel switch for turning on the light source to be controlled among the plurality of light sources included in the light source group, so that the common switch is on and the pixel switch is on. The visible light signal is transmitted by turning on the light source to be controlled only during a certain period. The light source to be controlled is the first light source described above.

  Thereby, a visible light signal can be appropriately transmitted from a display including a plurality of LEDs as light sources. Therefore, the communication between the apparatuses of the aspect containing apparatuses other than illumination is enabled. Further, when the display is a display for displaying an image by controlling the common switch and the pixel switch, the visible light signal can be transmitted using the common switch and the pixel switch. Therefore, a visible light signal can be easily transmitted without making a significant change to the configuration for displaying an image on a display (that is, a display device).

(Frame configuration of Single frame transmission)
FIG. 471 is a diagram illustrating an example of a transmission signal in this embodiment.

  As shown in FIG. 471 (a), the transmission frame includes a preamble (PRE), an ID length (IDLEN), an ID type (IDTYPE), content (ID / DATA), and a check code (CRC). The number of bits in each area is an example.

  By using the preamble as shown in FIG. 471 (b), the receiver can distinguish from other parts encoded by 4PPM, I-4PPM or V4PPM, and can find a signal delimiter. .

  As shown in FIG. 471 (c), variable length content can be transmitted by specifying the length of ID / DATA by IDLEN.

  The CRC is a check code for correcting or detecting an error in a portion other than the PRE. By changing the CRC length in accordance with the length of the inspection area, the inspection capability can be maintained above a certain level. Moreover, the inspection capability per CRC length can be improved by using different inspection codes depending on the length of the inspection region.

(Frame configuration of multiple frame transmission)
472 and 473 are diagrams illustrating an example of a transmission signal in this embodiment.

  A partition type (PTYPE) and a check code (CRC) are added to the transmission data (BODY) to become Joined data. Joined data is divided into several DATA PARTs, and transmitted with a preamble (PRE) and an address (ADDR) added.

  PTYPE (or partition mode (PMODE)) indicates the division method or meaning of BODY. By using 2 bits as shown in FIG. 472 (a), it is possible to encode with 4PPM. As shown in FIG. 472 (b), the transmission time can be shortened by setting 1 bit.

  CRC is an inspection code for inspecting PTYPE and BODY. As shown in FIG. 450, by changing the CRC code length according to the length of the portion to be inspected, the inspection capability can be maintained at a certain level or more.

  By defining the preamble as shown in FIG. 451, the transmission time can be shortened while ensuring variations of the division pattern.

  By determining the address as shown in FIG. 452, the receiver can restore the data regardless of the order in which the frames are received.

  FIG. 473 shows a combination of possible Joined data length and the number of frames. The underlined combination is a combination used when a PTYPE described later is a single frame compatible.

(Configuration of BODY field)
FIG. 474 is a diagram illustrating an example of a transmission signal in this embodiment.

  By setting BODY to a field configuration as shown in the figure, it is possible to transmit the same ID as in single frame transmission.

  In the case of the same IDTYPE and the same ID, it is possible to flexibly signal when the continuous transmission / reception time is short by expressing the same meaning regardless of the combination of single frame transmission or multiframe transmission and packet transmission. Can be sent.

  The ID length is specified by IDLEN, and PADDING is transmitted for the remaining part. This part may be all 0 or 1, may transmit data for extending the ID, or may be a check code. PADDING may be left-justified.

  In (b), (c), or (d) of FIG. 474, the transmission time can be made shorter than in (a) of FIG. At this time, the length of the ID is assumed to be the maximum of the lengths that can be taken as the ID.

  In the case of (b) or (c) in FIG. 474, the number of bits of IDTYPE is an odd number, but when combined with the 1-bit PTYPE shown in FIG. Can do.

  In FIG. 474 (c), a longer ID can be transmitted.

  In FIG. 474 (d), more IDTYPEs can be expressed.

(PTYPE)
FIG. 475 is a diagram illustrating an example of a transmission signal in this embodiment.

  When PTYPE is a predetermined bit, it indicates that BODY is in a single frame compatible mode. Thereby, the same ID as in the case of single frame transmission can be transmitted.

  For example, when PTYPE = 00, the ID or ID type corresponding to the PTYPE can be handled in the same manner as the ID or ID type transmitted by single frame transmission, and the management of the ID or ID type can be simplified. it can.

  When PTYPE is a predetermined bit, BODY indicates a data stream mode. At this time, all combinations of the number of transmission frames and the DATAPART length can be used, and different combinations of data can have different meanings. Depending on the PTYPE bits, the different combinations may have the same meaning or different meanings. Thereby, a transmission method can be selected flexibly.

  For example, when PTYPE = 01, an ID having a size not defined for single frame transmission can be transmitted. Further, even if the ID corresponding to the PTYPE is the same as the ID for single frame transmission, the ID corresponding to the PTYPE can be handled as an ID different from the ID for the single frame transmission. As a result, the number of IDs that can be expressed can be increased.

(Field structure in Single frame compatible mode)
FIG. 476 is a diagram illustrating an example of a transmission signal in this embodiment.

  When (a) in FIG. 474 is used, in the single frame compatible mode, it is most efficient to transmit in the combination of tables shown in FIG.

  When (b), (c), or (d) in FIG. 474 is used, when the ID is 32 bits, the combination of the number of frames 13 and the DATAPART length 4 bits is efficient. When the ID is 64 bits, the combination of 11 frames and DATAPART length of 8 bits is efficient.

  By transmitting only by the combination of tables, it becomes possible to determine different combinations as reception errors, and the reception error rate can be lowered.

  The present invention can be used, for example, in a transmission device that transmits a visible light signal from a display or the like, and in particular, can be used in a transmission device that transmits a visible light signal, for example, from a large LED display or the like.

C10 Transmitter (transmitter)
C11 determination unit C12 common switch control unit C13 pixel switch control unit

Claims (10)

A transmission method for transmitting a visible light signal by luminance change,
A determining step of determining a luminance change pattern by modulating the visible light signal;
A common switch control step of switching a common switch for commonly lighting a plurality of light sources for representing pixels in each image included in the light source group provided in the display, according to the luminance change pattern;
By turning on a first pixel switch for turning on a first light source among a plurality of light sources included in the light source group, the common switch is turned on, and the first pixel switch is turned on. A first pixel switch control step of transmitting the visible light signal by turning on the first light source only during a period of being on. In the determination step,
Determining the luminance change pattern for each symbol period;
In the first pixel switch control step,
The transmission method according to claim 1, wherein the first pixel switch is switched in synchronization with the symbol period. In the first pixel switch control step,
When displaying an image on the display, the first light source for transmitting the visible light signal in a lighting period corresponding to the first light source for expressing a pixel value of a pixel in the image. The transmission method according to claim 2, wherein the first pixel switch is switched so as to compensate for the lighting period only during a period in which is turned off. The transmission method according to claim 3, wherein the lighting period is supplemented by changing a pixel value of a pixel in the video. The transmission method according to claim 4, wherein the pixel value is changed at a period that is ½ of the symbol period. The transmission method further includes:
By turning on a second pixel switch for turning on a second light source included in the light source group and surrounding the first light source, the common switch is turned on, and the second light source is turned on. A second pixel switch control step of transmitting the visible light signal by turning on the second light source only during a period in which the pixel switch is on,
In the first and second pixel switch control steps,
When simultaneously transmitting the same symbol included in the visible light signal from each of the first and second light sources,
Of the plurality of timings at which each of the first and second pixel switches is turned on or off to transmit the same symbol, a timing at which a rise in luminance specific to the same symbol is obtained is determined by the first And the same for each of the second pixel switches, and different timings for each of the first and second pixel switches,
4. The transmission method according to claim 3, wherein an overall average luminance of the first and second light sources in a period in which the same symbol is transmitted is matched with a predetermined luminance. In the first pixel switch control step,
When transmitting the same symbol included in the visible light signal in a first period and a second period following the first period,
In each of the first and second periods, a rise in luminance specific to the same symbol is obtained among a plurality of timings at which the first pixel switch is turned on or off to transmit the same symbol. The same timing, different timing,
The transmission method according to claim 3, wherein an average luminance of the first light source in the entire first period and the second period is made to coincide with a predetermined luminance. In the first pixel switch control step,
During the signal transmission period in which the common switch is switching according to the luminance change pattern, the first pixel switch is turned on,
The transmission method further includes:
During the video display period different from the signal transmission period, the common switch is turned on, and the common switch is turned on by turning on the first pixel switch according to the video to be displayed in the video display period, The transmission method according to claim 1, further comprising: a video display step of displaying pixels in the video by turning on the first light source only during a period in which the first pixel switch is on. A transmission device that transmits a visible light signal according to a luminance change,
A determining unit that determines a luminance change pattern by modulating a visible light signal;
A common switch control unit for switching a common switch for commonly lighting a plurality of light sources for representing pixels in each image included in a light source group included in the display, according to the luminance change pattern;
By turning on a pixel switch for turning on a light source to be controlled among a plurality of light sources included in the light source group, only when the common switch is on and the pixel switch is on. And a pixel switch controller that transmits the visible light signal by turning on the light source to be controlled. A program for transmitting a visible light signal according to a luminance change,
A determining step of determining a luminance change pattern by modulating the visible light signal;
A common switch control step of switching a common switch for commonly lighting a plurality of light sources for representing pixels in each image included in the light source group provided in the display, according to the luminance change pattern;
By turning on a pixel switch for turning on a light source to be controlled among a plurality of light sources included in the light source group, only when the common switch is on and the pixel switch is on. A program for causing a computer to execute a pixel switch control step of transmitting the visible light signal by turning on the light source to be controlled.