JP2018032884A - Reception program and receiver - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a reception program that enables communication between various devices.SOLUTION: A reception program for receiving information from a light emitter causes a computer to execute: an exposure time setting step SA21 of setting an exposure time of an image sensor using automatic exposure; a bright line image acquisition step SA22 of acquiring a bright line image which is an image including a plurality of bright lines corresponding to a plurality of exposure lines included in the image sensor, by capturing a light emitter changing in luminance by the image sensor with the set exposure time; and an information acquisition step SA23 of acquiring information by decoding a pattern of the plurality of bright lines included in the acquired bright line image. In the exposure time setting step SA21, the sensitivity of the image sensor is set to the maximum value within a predetermined range for the image sensor, and an exposure time according to the sensitivity at the maximum value is set by the automatic exposure.SELECTED DRAWING: Figure 261A

Description

  The present invention relates to a program for performing communication between a mobile terminal such as a smartphone, a tablet, and a mobile phone and home appliances such as an air conditioner, a lighting device, and a rice cooker.

  In recent home networks, in addition to cooperation of AV home appliances by IP (Internet Protocol) connection in Ethernet (registered trademark) and wireless LAN (Local Area Network), management of power consumption corresponding to environmental problems, and from outside the home Introduction of a home appliance linkage function in which various home appliances are connected to a network by a home energy management system (HEMS) having a function of turning on / off the power of the home appliance. However, there are home appliances that do not have sufficient computing power to have a communication function, and home appliances that are difficult to install a communication function in terms of cost.

  In order to solve such a problem, in Patent Literature 1, in an optical space transmission device that transmits information to free space using light, limited transmission is performed by performing communication using a plurality of monochromatic light sources of illumination light. A technology for efficiently realizing communication between devices is described in the apparatus.

JP 2002-290335 A

  However, the conventional method is limited to a case where a device to be applied has a three-color light source such as illumination. The present invention solves such problems and provides an information processing program, a reception program, or the like that enables communication between various devices including a device with a small computing power.

  A receiving program according to an aspect of the present invention is a receiving program for receiving information from a light emitter, an exposure time setting step for setting an exposure time of an image sensor using automatic exposure, and the light emission that changes in luminance. A bright line image acquisition step of acquiring a bright line image that is an image including a bright line corresponding to each of a plurality of exposure lines included in the image sensor by causing the image sensor to capture an image of the body at the set exposure time. An information acquisition step of acquiring information by decoding the plurality of bright line patterns included in the acquired bright line image, and in the exposure time setting step, the sensitivity of the image sensor is Set to the maximum value in a predetermined range for the image sensor, and respond to the sensitivity of the maximum value. The exposure time is set by the automatic exposure has.

  Note that these comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, and computer program. And any combination of recording media.

  According to the present invention, it is possible to realize an information processing program and a reception program that enable communication between various devices including a device having a small computing power.

FIG. 1 is a diagram illustrating an example of an observation method of luminance of a light emitting unit in the first embodiment. FIG. 2 is a diagram illustrating an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 3 is a diagram illustrating an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 4 is a diagram illustrating an example of a method of observing the luminance of the light emitting unit in the first embodiment. FIG. 5A is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5B is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5C is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5D is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5E is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5F is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5G is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 5H is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1. FIG. 6A is a flowchart of the information communication method in Embodiment 1. FIG. 6B is a block diagram of the information communication apparatus according to Embodiment 1. FIG. 7 is a diagram illustrating an example of each mode of the receiver in the second embodiment. FIG. 8 is a diagram illustrating an example of a photographing operation of the receiver in the second embodiment. FIG. 9 is a diagram illustrating another example of the photographing operation of the receiver in the second embodiment. 10A is a diagram illustrating another example of imaging operation of a receiver in Embodiment 2. FIG. FIG. 10B is a diagram illustrating another example of imaging operation of a receiver in Embodiment 2. FIG. 10C is a diagram illustrating another example of imaging operation of a receiver in Embodiment 2. FIG. 11A is a diagram illustrating an example of a camera arrangement of a receiver in Embodiment 2. FIG. 11B is a diagram illustrating another example of camera arrangement of a receiver in Embodiment 2. FIG. 12 is a diagram illustrating an example of display operation of the receiver in Embodiment 2. FIG. 13 is a diagram illustrating an example of display operation of the receiver in Embodiment 2. FIG. 14 is a diagram illustrating an example of operation of a receiver in Embodiment 2. FIG. 15 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 16 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 17 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 18 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 19 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 20 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 21 is a diagram illustrating an example of operations of the receiver, the transmitter, and the server in the second embodiment. FIG. 22 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 23 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 24 is a diagram illustrating an example of initial setting of a receiver in the second embodiment. FIG. 25 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 26 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 27 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 28 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 29 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 30 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 31A is a diagram illustrating a pen used for operation of a receiver in Embodiment 2. FIG. 31B is a diagram illustrating operation of a receiver using a pen in Embodiment 2. FIG. 32 is a diagram illustrating an example of appearance of a receiver in Embodiment 2. FIG. 33 is a diagram illustrating another example of appearance of a receiver in Embodiment 2. FIG. 34 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 35A is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 35B is a diagram illustrating an application example using the receiver in Embodiment 2. FIG. 36A is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 36B is a diagram illustrating an application example using the receiver in Embodiment 2. FIG. 37A is a diagram illustrating an example of operation of a transmitter in Embodiment 2. FIG. 37B is a diagram illustrating another example of operation of a transmitter in Embodiment 2. FIG. 38 is a diagram illustrating another example of operation of a transmitter in Embodiment 2. FIG. 39 is a diagram illustrating another example of operation of a transmitter in Embodiment 2. FIG. 40 is a diagram illustrating an example of a communication form between the plurality of transmitters and the receiver in the second embodiment. FIG. 41 is a diagram illustrating an example of operations of a plurality of transmitters in the second embodiment. FIG. 42 is a diagram illustrating another example of a communication form between a plurality of transmitters and a receiver in the second embodiment. FIG. 43 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 44 is a diagram illustrating an example of application of a receiver in Embodiment 2. FIG. 45 is a diagram illustrating an example of application of a receiver in Embodiment 2. FIG. 46 is a diagram illustrating an example of application of a receiver in Embodiment 2. FIG. 47 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 48 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 49 is a diagram illustrating an example of application of the reception method in Embodiment 2. FIG. 50 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 51 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 52 is a diagram illustrating an example of application of a transmitter in Embodiment 2. FIG. 53 is a diagram illustrating another example of operation of a receiver in Embodiment 2. FIG. 54 is a flowchart illustrating an example of operation of the receiver in Embodiment 3. FIG. 55 is a flowchart illustrating another example of operation of a receiver in Embodiment 3. FIG. 56A is a block diagram illustrating an example of a transmitter in Embodiment 3. FIG. 56B is a block diagram illustrating another example of the transmitter in Embodiment 3. FIG. 57 is a diagram illustrating a configuration example of a system including a plurality of transmitters in the third embodiment. FIG. 58 is a block diagram illustrating another example of the transmitter in the third embodiment. FIG. 59A is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 59B is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 59C is a diagram illustrating an example of a transmitter in Embodiment 3. 60A is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. FIG. 60B is a diagram illustrating an example of a transmitter in Embodiment 3. FIG. 61 is a diagram illustrating an example of processing operations of a receiver, a transmitter, and a server in Embodiment 3. FIG. 62 is a diagram illustrating an example of processing operations of a receiver, a transmitter, and a server in Embodiment 3. FIG. 63 is a diagram illustrating an example of processing operations of the receiver, the transmitter, and the server in Embodiment 3. FIG. 64A is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment. FIG. 64B is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment. FIG. 65 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 66 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 67 is a diagram illustrating an example of operation of a transmitter, a receiver, and a server in Embodiment 3. FIG. 68 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 69 is a diagram illustrating an example of appearance of a receiver in Embodiment 3. FIG. 70 is a diagram illustrating an example of operation of a transmitter, a receiver, and a server in Embodiment 3. 71 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. FIG. 72 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 73 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 74 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG. 75A is a diagram illustrating an example of a structure of information transmitted by a transmitter in Embodiment 3. FIG. 75B is a diagram illustrating another example of a structure of information transmitted by a transmitter in Embodiment 3. FIG. 76 is a diagram illustrating an example of a 4-level PPM modulation scheme by a transmitter in Embodiment 3. 77 is a diagram illustrating an example of a PPM modulation scheme by a transmitter in Embodiment 3. FIG. FIG. 78 is a diagram illustrating an example of a PPM modulation scheme in a transmitter in Embodiment 3. FIG. 79A is a diagram showing an example of a luminance change pattern corresponding to the header (preamble portion) in the third embodiment. FIG. 79B is a diagram showing an example of a luminance change pattern in Embodiment 3. FIG. 80A is a diagram showing an example of a luminance change pattern in Embodiment 3. FIG. 80B is a diagram showing an example of a luminance change pattern in Embodiment 3. FIG. 81 is a diagram illustrating an example of operation of a receiver in a situation in front of a store in Embodiment 4. FIG. 82 is a diagram illustrating another example of operation of a receiver in a situation in front of a store in Embodiment 4. FIG. 83 is a diagram illustrating an example of next operation of the receiver in the situation in front of the store in the fourth embodiment. FIG. 84 is a diagram illustrating an example of next operation of the receiver in the situation in front of the store in the fourth embodiment. FIG. 85 is a diagram illustrating an example of next operation of the receiver in the situation in front of the store in the fourth embodiment. FIG. 86 is a diagram illustrating an example of operation of the display device in the in-store situation according to Embodiment 4. FIG. 87 is a diagram illustrating an example of next operation of the display device in the in-store situation in the fourth embodiment. FIG. 88 is a diagram illustrating an example of next operation of the display device in the in-store situation in the fourth embodiment. FIG. 89 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 90 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 91 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 92 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 93 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 94 is a diagram illustrating an example of next operation of the receiver in the in-store situation in the fourth embodiment. FIG. 95 is a diagram illustrating an example of operation of a receiver in a store search situation in the fourth embodiment. FIG. 96 is a diagram illustrating an example of next operation of the receiver in the store search situation in the fourth embodiment. FIG. 97 is a diagram illustrating an example of next operation of the receiver in the store search situation in the fourth embodiment. FIG. 98 is a diagram illustrating an example of operation of a receiver in a movie advertisement situation in Embodiment 4. FIG. 99 is a diagram illustrating an example of next operation of a receiver in a movie advertisement situation in the fourth embodiment. FIG. 100 is a diagram illustrating an example of next operation of a receiver in a movie advertisement situation in Embodiment 4. FIG. 101 is a diagram illustrating an example of next operation of a receiver in a movie advertisement situation in Embodiment 4. FIG. 102 is a diagram illustrating an example of operation of a receiver in a museum situation in Embodiment 4. FIG. 103 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 104 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 105 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 106 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 107 is a diagram illustrating an example of next operation of a receiver in a museum situation in Embodiment 4. FIG. 108 is a diagram illustrating an example of operation of a receiver in a bus stop situation in Embodiment 4. FIG. 109 is a diagram illustrating an example of next operation of a receiver in a bus stop situation in the fourth embodiment. FIG. 110 is a diagram for describing imaging in the fourth embodiment. FIG. 111 is a diagram for explaining transmission and imaging in the fourth embodiment. FIG. 112 is a diagram for explaining transmission in the fourth embodiment. FIG. 113 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. 114 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. FIG. 115 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. 116 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 117 is a diagram illustrating an example of operation of a receiver in Embodiment 5. FIG. 118 is a diagram illustrating an example of operation of a receiver in Embodiment 5. FIG. FIG. 119 is a diagram illustrating an example of operation of a system including a transmitter, a receiver, and a server in Embodiment 5. 120 is a block diagram illustrating a configuration of a transmitter in Embodiment 5. In FIG. FIG. 121 is a block diagram illustrating a configuration of a receiver in Embodiment 5. In FIG. 122 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. FIG. 123 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. 124 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 125 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. FIG. 126 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. 127 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. 128 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG. FIG. 129 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 130 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. 131 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. In FIG. 132 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 133 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. In FIG. 134 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. FIG. 135 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. 136 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. In FIG. FIG. 137 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 138 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 139 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 140 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 141 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 142 is a diagram illustrating an encoding method according to the fifth embodiment. FIG. 143 is a diagram illustrating an encoding method capable of receiving light even when an image is captured from an oblique direction according to the fifth embodiment. FIG. 144 is a diagram illustrating an encoding scheme in which the amount of information differs according to the distance in the fifth embodiment. FIG. 145 is a diagram illustrating an encoding method in which the amount of information differs depending on the distance in the fifth embodiment. FIG. 146 is a diagram illustrating a coding scheme obtained by dividing data according to the fifth embodiment. FIG. 147 is a diagram illustrating the effect of inserting a reverse phase image in the fifth embodiment. FIG. 148 is a diagram showing the effect of inserting a reverse phase image in the fifth embodiment. FIG. 149 is a diagram illustrating super-resolution processing according to the fifth embodiment. FIG. 150 is a diagram illustrating a display indicating that it supports visible light communication in the fifth embodiment. FIG. 151 is a diagram illustrating information acquisition using a visible light communication signal in Embodiment 5. FIG. 152 is a diagram showing a data format in the fifth embodiment. FIG. 153 is a diagram illustrating reception by estimating a three-dimensional shape according to Embodiment 5. FIG. 154 is a diagram illustrating reception by estimating a three-dimensional shape according to Embodiment 5. FIG. 155 is a diagram illustrating stereoscopic projection in the fifth embodiment. FIG. 156 is a diagram illustrating stereoscopic projection in the fifth embodiment. FIG. 157 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 158 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG. 159 is a diagram illustrating an example of a transmission signal according to the sixth embodiment. FIG. 160 is a diagram illustrating an example of a transmission signal according to the sixth embodiment. 161A is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 161B is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 161C is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. FIG. 162A is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 162B is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 163A is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 163B is a diagram illustrating an example of an image (bright line image) captured by a receiver in Embodiment 6. FIG. 163C is a diagram illustrating an example of an image (bright line image) captured by the receiver in Embodiment 6. 164 is a diagram illustrating an example of a captured image (bright line image) of a receiver in Embodiment 6. FIG. FIG. 165 is a diagram illustrating an example of a transmission signal in Embodiment 6. FIG. 166 is a diagram illustrating an example of operation of a receiver in Embodiment 6. FIG. 167 is a diagram illustrating an example of instructions to the user displayed on the receiver screen in the sixth embodiment. FIG. 168 is a diagram illustrating an example of instructions to the user displayed on the receiver screen in the sixth embodiment. FIG. 169 is a diagram illustrating an example of a signal transmission method in Embodiment 6. 170 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. FIG. 171 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. 172 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG. 173 is a diagram for describing a use case in the sixth embodiment. FIG. 174 is a diagram illustrating an information table transmitted to the server by the smartphone according to Embodiment 6. FIG. 175 is a block diagram of the server in the sixth embodiment. FIG. 176 is a flowchart showing overall processing of the system in the sixth embodiment. FIG. 177 is a diagram illustrating an information table transmitted from the server according to the sixth embodiment to a smartphone. FIG. 178 is a diagram illustrating a screen flow displayed on the wearable device from when the user receives information from the server in front of the store until when the user actually purchases the product. FIG. 179 is a diagram for explaining another use case in the sixth embodiment. FIG. 180 is a diagram illustrating a service providing system using the reception method described in each embodiment. FIG. 181 is a flowchart illustrating a service provision flow. FIG. 182 is a flowchart showing service provision in another example. FIG. 183 is a flowchart illustrating service provision in another example. FIG. 184A is a diagram for describing a modulation scheme easily received in the eighth embodiment. FIG. 184B is a diagram for explaining a modulation scheme easy to receive in the eighth embodiment. FIG. 185 is a diagram for explaining a modulation scheme easy to receive in the eighth embodiment. FIG. 186 is a diagram for describing the combination of bright line communication and image recognition in the eighth embodiment. FIG. 187A is a diagram for describing a method of using an imaging element suitable for visible light signal reception in Embodiment 8. FIG. 187B is a diagram for describing a method of using an imaging element suitable for visible light signal reception in Embodiment 8. FIG. 187C is a diagram for describing a usage method of an imaging element suitable for visible light signal reception in Embodiment 8. FIG. 187D is a diagram for describing a method of using an imaging element suitable for receiving visible light signals according to Embodiment 8. FIG. 187E is a flowchart for describing a method of using an imaging element suitable for visible light signal reception in Embodiment 8. FIG. 188 is a diagram illustrating captured image sizes suitable for visible light signal reception in the eighth embodiment. 189A is a diagram illustrating captured image sizes suitable for visible light signal reception in Embodiment 8. FIG. FIG. 189B is a flowchart illustrating an operation for switching to a captured image size suitable for reception of a visible light signal in Embodiment 8. FIG. 189C is a flowchart illustrating an operation for switching to a captured image size suitable for reception of a visible light signal in the eighth embodiment. FIG. 190 is a diagram for describing reception of a visible light signal using zoom according to the eighth embodiment. FIG. 191 is a diagram for describing a compression method of an image data size suitable for visible light signal reception in the eighth embodiment. FIG. 192 is a diagram for explaining a modulation scheme with high reception error detection accuracy in the eighth embodiment. FIG. 193 is a diagram for describing a change in operation of the receiver due to a difference in situation in the eighth embodiment. FIG. 194 is a diagram for describing notification of visible light communication to a human according to the eighth embodiment. FIG. 195 is a diagram for explaining expansion of the reception range by the light diffusing plate in the eighth embodiment. FIG. 196 is a diagram for describing a synchronization method of signal transmission from a plurality of projectors according to the eighth embodiment. FIG. 197 is a diagram for describing a synchronization method of signal transmission from a plurality of displays in the eighth embodiment. FIG. 198 is a diagram for describing reception of a visible light signal by the illuminance sensor and the image sensor in the eighth embodiment. FIG. 199 is a diagram for describing a trigger to start reception in the eighth embodiment. FIG. 200 is a diagram for describing a reception start gesture in the eighth embodiment. FIG. 201 is a diagram for describing an application example to the car navigation system in the eighth embodiment. FIG. 202 is a diagram for describing an example of application to car navigation in the eighth embodiment. FIG. 203 is a diagram for describing an application example to content protection in the eighth embodiment. 204A is a diagram for describing an application example as an electronic lock in Embodiment 8. FIG. FIG. 204B is a flowchart of an information communication method in Embodiment 8. FIG. 204C is a block diagram of an information communication device in Embodiment 8. FIG. 205 is a diagram for explaining an application example as store visit information transmission in the eighth embodiment. FIG. 206 is a diagram for describing an application example of order control according to a place in the eighth embodiment. FIG. 207 is a diagram for describing an example of application to route guidance in the eighth embodiment. FIG. 208 is a diagram for describing an example of application to location communication in the eighth embodiment. FIG. 209 is a diagram for describing an application example to use log accumulation and analysis in the eighth embodiment. FIG. 210 is a diagram for describing an example of application to screen sharing in the eighth embodiment. FIG. 211 is a diagram for describing an example of application to screen sharing in the eighth embodiment. FIG. 212 is a diagram for describing an application example of position estimation using a wireless access point in the eighth embodiment. FIG. 213 is a diagram illustrating a configuration in which position estimation is performed by visible light communication and wireless communication in the eighth embodiment. FIG. 214 is a diagram illustrating an example of application of the information communication method in Embodiment 8. FIG. 215 is a flowchart illustrating an application example of the information communication method in the eighth embodiment. FIG. 216 is a flowchart illustrating an application example of the information communication method in the eighth embodiment. FIG. 217 is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9. FIG. 218 is a diagram illustrating an example of application of the transmitter in Embodiment 9. FIG. 219 is a flowchart of the information communication method in the ninth embodiment. FIG. 220 is a block diagram of an information communication device according to the ninth embodiment. FIG. 221A is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9. FIG. 221B is a flowchart illustrating operation of the receiver in Embodiment 9. FIG. 222 is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9. FIG. 223 is a diagram illustrating an example of application of the transmitter in Embodiment 9. FIG. 224A is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9. FIG. 224B is a flowchart illustrating operation of the receiver in Embodiment 9. 225 is a diagram illustrating operation of a receiver in Embodiment 9. FIG. 226 is a diagram illustrating an example of application of a transmitter in Embodiment 9. FIG. FIG. 227 is a diagram illustrating an example of application of the receiver in Embodiment 9. FIG. 228A is a flowchart illustrating an example of operation of a transmitter in Embodiment 9. FIG. 228B is a flowchart illustrating an example of operation of a transmitter in Embodiment 9. FIG. 229 is a flowchart illustrating an example of operation of a transmitter in Embodiment 9. FIG. 230 is a flowchart illustrating an example of operation of the imaging device according to the ninth embodiment. FIG. 231 is a flowchart illustrating an example of operation of the imaging device according to the ninth embodiment. FIG. 232 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9. FIG. 233 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9. 234 is a diagram illustrating an example of signals transmitted by a transmitter in Embodiment 9. FIG. FIG. 235 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9. FIG. 236 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. FIG. 237 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. 238 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. In FIG. 239 is a diagram illustrating an example of operation of a transmitter in Embodiment 9. FIG. 240 is a diagram illustrating an example of operation of a transmitter in Embodiment 9. FIG. FIG. 241 is a diagram illustrating an example of a structure and operation of a transmitter in Embodiment 9. 242 is a diagram illustrating an example of a structure and operation of a transmitter in Embodiment 9. FIG. FIG. 243 is a diagram illustrating a timepiece including the optical sensor according to the tenth embodiment. FIG. 244 is a diagram illustrating an example of a receiver in Embodiment 10. FIG. 245 is a diagram illustrating an example of a receiver in Embodiment 10. FIG. 246A is a flowchart of an information communication method according to an aspect of the present invention. FIG. 246B is a block diagram of a mobile terminal according to one embodiment of the present invention. FIG. 247 is a diagram illustrating an example of a reception system in Embodiment 10. FIG. 248 is a diagram illustrating an example of a reception system in Embodiment 10. FIG. 249A is a diagram illustrating an example of a modulation scheme in Embodiment 10. FIG. 249B is a diagram illustrating an example of a modulation scheme in Embodiment 10. FIG. 249C is a diagram illustrating an example of a modulation scheme in Embodiment 10. FIG. 249D is a diagram illustrating an example of mixing signal separation in Embodiment 10. FIG. 249E is a diagram illustrating an example of mixing signal separation in Embodiment 10. FIG. 249F is a flowchart illustrating processing of the information processing program in the tenth embodiment. FIG. 249G is a block diagram of an information processing device in Embodiment 10. FIG. 250A is a diagram illustrating an example of a visible light communication system in Embodiment 10. FIG. 250B is a diagram for describing a use case in Embodiment 10. FIG. 250C is a diagram illustrating an example of a signal transmission / reception system in Embodiment 10. FIG. 251 is a flowchart illustrating a reception method in which interference is eliminated in the tenth embodiment. FIG. 252 is a flowchart illustrating a transmitter direction estimation method according to the tenth embodiment. FIG. 253 is a flowchart illustrating a reception start method in Embodiment 10. FIG. 254 is a flowchart illustrating an ID generation method using information of another medium according to the tenth embodiment. FIG. 255 is a flowchart illustrating a reception method selection method based on frequency separation in the tenth embodiment. FIG. 256 is a flowchart showing a signal reception method when the exposure time is long in the tenth embodiment. FIG. 257 is a diagram illustrating an example of a transmitter dimming (adjusting brightness) method in Embodiment 10. 258 is a diagram illustrating an example of a method of configuring a dimming function of a transmitter in Embodiment 10. [FIG. FIG. 259A is a flowchart illustrating an example of operation of a receiver in Embodiment 11. FIG. 259B is a flowchart illustrating an example of operation of a receiver in Embodiment 11. FIG. 259C is a flowchart illustrating an example of operation of a receiver in Embodiment 11. FIG. 259D is a flowchart illustrating an example of operation of a receiver in Embodiment 11. FIG. 260 is a diagram for explaining the EX zoom. FIG. 261A is a flowchart illustrating processing of a reception program in the tenth embodiment. FIG. 261B is a block diagram of a receiving apparatus in Embodiment 10. FIG. 262 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 263 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 264 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 265 is a diagram illustrating an example of a screen display method of a receiver in Embodiment 12. 266 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. FIG. 267 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 268 is a flowchart illustrating an example of a signal reception method in Embodiment 12. FIG. 269 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG. 270A is a flowchart showing processing of a reception program in the twelfth embodiment. FIG. 270B is a block diagram of a receiving apparatus in Embodiment 12. FIG. 271 is a diagram illustrating an example of display on the receiver when a visible light signal is received. FIG. 272 is a diagram illustrating an example of display on the receiver when a visible light signal is received. FIG. 273 is a diagram illustrating an example of a display of the acquired data image. FIG. 274 is a diagram illustrating an example of an operation for saving or discarding acquired data. FIG. 275 is a diagram illustrating a display example when browsing acquired data. FIG. 276 is a diagram illustrating an example of a transmitter in Embodiment 12. FIG. 277 is a diagram illustrating an example of a reception method in Embodiment 12. FIG. 278 is a diagram illustrating an example of a header pattern in the thirteenth embodiment. FIG. 279 is a diagram for describing an example of the structure of a communication protocol packet in the thirteenth embodiment. FIG. 280 is a flowchart illustrating an example of a reception method in Embodiment 13. FIG. 281 is a flowchart illustrating an example of a reception method in Embodiment 13. FIG. 282 is a flowchart illustrating an example of a reception method in Embodiment 13. FIG. 283 is a diagram for illustrating a reception method in which the receiver in Embodiment 13 uses an exposure time longer than the period of the modulation frequency (modulation period). FIG. 284 is a diagram for illustrating a reception method in which the receiver in Embodiment 13 uses an exposure time longer than the period of the modulation frequency (modulation period). FIG. 285 is a diagram illustrating an efficient division number with respect to the size of transmission data in the thirteenth embodiment. FIG. 286A is a diagram illustrating an example of a setting method in Embodiment 13. FIG. 286B is a diagram illustrating another example of the setting method according to the thirteenth embodiment. FIG. 287A is a flowchart illustrating processing of the information processing program in the thirteenth embodiment. FIG. 287B is a block diagram of an information processing device in Embodiment 13. FIG. 288 is a diagram for describing an example of application of the transmission and reception system in Embodiment 13. FIG. 289 is a flowchart illustrating processing operations of the transmission and reception system in the thirteenth embodiment. FIG. 290 is a diagram for describing an example of application of the transmission and reception system in Embodiment 13. FIG. 291 is a flowchart showing processing operations of the transmission / reception system in the thirteenth embodiment. FIG. 292 is a diagram for describing an example of application of the transmission and reception system in Embodiment 13. FIG. 293 is a flowchart illustrating processing operations of the transmission and reception system in the thirteenth embodiment. FIG. 294 is a diagram for describing an example of application of a transmitter in Embodiment 13. FIG. 295 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. FIG. 296 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. FIG. 297 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. FIG. 298 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. 299 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. FIG. 300 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. 301 is a diagram for describing an application example of the transmission and reception system in Embodiment 14. 302 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. FIG. 303 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. 304 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. 305 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG. FIG. 306 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. FIG. 307 is a diagram for describing an example of application of the transmission and reception system in Embodiment 14. 308 is a diagram for describing an example of application of a transmission and reception system in Embodiment 14. FIG.

  A receiving program according to an aspect of the present invention is a receiving program for receiving information from a light emitter, an exposure time setting step for setting an exposure time of an image sensor using automatic exposure, and the light emission that changes in luminance. A bright line image acquisition step of acquiring a bright line image that is an image including a bright line corresponding to each of a plurality of exposure lines included in the image sensor by causing the image sensor to capture an image of the body at the set exposure time. An information acquisition step of acquiring information by decoding the plurality of bright line patterns included in the acquired bright line image, and in the exposure time setting step, the sensitivity of the image sensor is Set to the maximum value in a predetermined range for the image sensor, and respond to the sensitivity of the maximum value. The exposure time is set by the automatic exposure has.

  As a result, as shown in FIGS. 259A to 261B, even if the exposure time of the image sensor cannot be set directly, an appropriate bright line can be obtained using the automatic exposure function installed in a general camera. An exposure time that is short enough to acquire an image can be set. That is, in the automatic exposure, the exposure is adjusted based on the brightness of the image acquired by imaging by the image sensor. Therefore, if the sensitivity of the image sensor is set to a large value, the image becomes bright. Therefore, in order to suppress exposure, the exposure time of the image sensor is set short. If the sensitivity of the image sensor is set to the maximum value, the exposure time can be set shorter and an appropriate bright line image can be acquired. That is, information from the light emitter can be appropriately received. As a result, communication between various devices can be enabled. The sensitivity is, for example, ISO sensitivity.

  In the exposure time setting step, a value indicating a level of exposure correction of the image sensor is set to a minimum value in a range preset for the image sensor, and the sensitivity of the maximum value and the minimum value are set. The exposure time corresponding to the value level exposure correction may be set by the automatic exposure.

  As a result, since the value indicating the level of exposure correction is set to the minimum value, the exposure time can be set shorter by automatic exposure processing that attempts to suppress exposure, and a more appropriate bright line image is acquired. be able to. The unit of the value indicating the exposure correction level is, for example, EV.

  In the exposure time setting step, a brighter part than the other part is identified in the first image acquired by imaging the subject including the light emitter by the image sensor, and one of the subjects corresponding to the bright part is identified. The exposure time is set by using a second image obtained by enlarging a part by optical zoom and using the second image acquired by imaging the part of the enlarged subject by the image sensor, In the bright line image acquisition step, the bright line image may be acquired by causing the image sensor to capture a part of the enlarged subject with the set exposure time.

  Thereby, a part of the subject corresponding to the bright part of the first image is enlarged by the optical zoom, that is, the bright light emitter is enlarged by the optical zoom. Can also be brightened overall. Then, since the bright second image is used for the input of automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can.

  Further, in the exposure time setting step, the central portion of the first image acquired by imaging the subject including the light emitter by the image sensor is more than the average brightness of a plurality of positions in the first image. When it is determined whether the central portion is bright, a part of the subject corresponding to the central portion is enlarged by optical zoom, and the enlarged image of the subject by the image sensor is determined. The exposure time is set by using a second image acquired by a part of imaging for the input of the automatic exposure. In the bright line image acquisition step, a part of the enlarged subject is set. The bright line image may be acquired by causing the image sensor to take an image with the exposure time.

  Accordingly, a part of the subject corresponding to the bright central portion of the first image is enlarged by the optical zoom, that is, a bright light-emitting body is enlarged by the optical zoom, so that the second image is changed to the first image. Can be brighter overall. Then, since the bright second image is used for the input of automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can. Further, in the optical zoom, if the enlargement center position cannot be arbitrarily set, the angle of view or the central portion of the image is enlarged. Therefore, even if the center position cannot be arbitrarily set, if the central portion of the first image is bright, the second image can be brightened as a whole by using the optical zoom. it can. Further, even when the central portion of the first image is dark, if the enlargement by the optical zoom is performed, the second image becomes dark and the exposure time becomes long. Therefore, as described above, it is possible to prevent the exposure time from becoming longer by enlarging with the optical zoom only when it is determined that the central portion is bright.

  Further, in the exposure time setting step, among K (K is an integer of 3 or more) image pickup elements included in the image sensor, N (N is K) that are uniformly distributed in the image sensor. In a first image acquired by imaging an object including the light emitter using only an imaging element of less than 2 (an integer greater than or equal to 2), a portion brighter than other portions is specified, and K images included in the image sensor are identified. Among the elements, the exposure time is set by using a second image acquired by imaging with only the dense N imaging elements corresponding to the bright part for the input of the automatic exposure, and the bright line image In the acquisition step, the bright line image is acquired by causing only the N dense image pickup elements included in the image sensor to capture images with the set exposure time. It may be.

  Thereby, even if the bright part in the first image is not at the center by so-called EX zoom, the second image can be brightened as a whole, and the exposure time can be set shorter.

  In the exposure time setting step, a photometric position in an image acquired by the image sensor capturing an image of the subject including the light emitter is set, and the exposure according to the brightness of the set photometric position. The time may be set by the automatic exposure.

  As a result, if the bright part in the image acquired by imaging is set as the photometry position, the exposure time can be set shorter by the automatic exposure process to suppress exposure, and an appropriate bright line image is acquired. can do.

  Further, the reception program further sets an imaging mode setting for switching the imaging mode of the image sensor from a color imaging mode for acquiring a color image by imaging to a monochrome imaging mode for acquiring a monochrome image by imaging. The step may be executed by the computer, and in the exposure time setting step, the exposure time may be set by using an image acquired in the monochrome imaging mode as an input for the automatic exposure.

  Accordingly, since an image acquired in the monochrome imaging mode is used for automatic exposure input, an appropriate exposure time can be set without being influenced by color information. Further, when the exposure time is set in the monochrome imaging mode, the bright line image is acquired by imaging according to the mode. Therefore, when information is transmitted from the light emitter only by a luminance change, the information can be appropriately acquired.

  In the exposure time setting step, the exposure time of the image sensor is updated by using the acquired image for input of the automatic exposure every time an image is acquired by imaging the light emitter by the image sensor. Then, the exposure time may be set by ending the update of the exposure time by the automatic exposure when the fluctuation range of the exposure time updated as needed falls below a predetermined range.

  Thereby, when the fluctuation of the exposure time is stabilized, that is, when the brightness of the image acquired by imaging falls within the target brightness by the automatic exposure, the exposure time at that time is the bright line image. It is used for imaging for acquisition. Therefore, an appropriate bright line image can be acquired.

  Note that these comprehensive or specific aspects may be realized by a recording medium such as an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM. You may implement | achieve in arbitrary combinations of a circuit, a computer program, or a recording medium.

  Hereinafter, embodiments will be specifically described with reference to the drawings.

  It should be noted that each of the embodiments described below shows a comprehensive or specific example. The numerical values, shapes, materials, constituent elements, arrangement positions and connecting forms of the constituent elements, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Embodiment 1)
The first embodiment will be described below.

(Observation of luminance of light emitting part)
We propose an imaging method that starts and ends the exposure at different times for each image sensor, rather than exposing all the image sensors at the same timing when capturing one image. FIG. 1 shows an example in which imaging devices arranged in one row are exposed simultaneously, and imaging is performed by shifting the exposure start time in the order of closer rows. Here, the exposure line of the image sensor that is exposed simultaneously is referred to as an exposure line, and the pixel line on the image corresponding to the image sensor is referred to as a bright line.

  When this image capturing method is used to capture an image of a blinking light source on the entire surface of the image sensor, bright lines (light and dark lines of pixel values) along the exposure line appear on the captured image as shown in FIG. . By recognizing the bright line pattern, it is possible to estimate the light source luminance change at a speed exceeding the imaging frame rate. Thereby, by transmitting a signal as a change in light source luminance, communication at a speed higher than the imaging frame rate can be performed. When a signal is expressed by the light source taking two types of luminance values, the lower luminance value is called low (LO), and the higher luminance value is called high (HI). Low may be in a state where the light source is not shining, or may be shining weaker than high.

  By this method, information is transmitted at a speed exceeding the imaging frame rate.

  When there are 20 exposure lines in which the exposure time does not overlap in one captured image and the imaging frame rate is 30 fps, it is possible to recognize a luminance change with a period of 1.67 milliseconds. When there are 1000 exposure lines whose exposure times do not overlap, it is possible to recognize a luminance change with a period of 1 / 30,000 second (about 33 microseconds). The exposure time is set shorter than 10 milliseconds, for example.

  FIG. 2 shows a case where the exposure of the next exposure line is started after the exposure of one exposure line is completed.

  In this case, when the number of frames per second (frame rate) is f and the number of exposure lines constituting one image is 1, if information is transmitted depending on whether or not each exposure line receives a certain amount of light, the maximum Can transmit information at a rate of fl bits per second.

  Note that when exposure is performed with a time difference for each pixel, not for each line, communication at higher speed is possible.

  At this time, when the number of pixels per exposure line is m pixels and information is transmitted depending on whether each pixel receives light above a certain level, the transmission speed is a maximum of flm bits per second.

  As shown in FIG. 3, if the exposure state of each exposure line by the light emission of the light emitting unit can be recognized at a plurality of levels, the light emission time of the light emitting unit is controlled by a unit time shorter than the exposure time of each exposure line. More information can be transmitted.

  If the exposure state can be recognized in the Elv stage, information can be transmitted at a maximum rate of flElv bits per second.

  Moreover, the basic period of transmission can be recognized by causing the light emitting unit to emit light at a timing slightly shifted from the exposure timing of each exposure line.

  FIG. 4 shows a case where the exposure of the next exposure line is started before the exposure of one exposure line is completed. That is, the exposure times of adjacent exposure lines are partially overlapped in time. With such a configuration, (1) it is possible to increase the number of samples within a predetermined time as compared with the case where the exposure of the next exposure line is started after waiting for the end of the exposure time of one exposure line. By increasing the number of samples in a predetermined time, it becomes possible to more appropriately detect the optical signal generated by the optical transmitter that is the subject. That is, it is possible to reduce the error rate when detecting an optical signal. Further, (2) the exposure time of each exposure line can be made longer than when the exposure time of the next exposure line is started after waiting for the end of the exposure time of one exposure line. However, a brighter image can be acquired. That is, the S / N ratio can be improved. In all exposure lines, it is not necessary that the exposure time of adjacent exposure lines has a partial overlap in time, and a configuration in which some exposure lines have no partial overlap. It is also possible. By configuring a part of the exposure lines so as not to partially overlap in time, it is possible to suppress the generation of intermediate colors due to the overlap of exposure times on the imaging screen, and to detect bright lines more appropriately. .

  In this case, the exposure time is calculated from the brightness of each exposure line, and the light emission state of the light emitting unit is recognized.

  When the brightness of each exposure line is determined by a binary value indicating whether the luminance is equal to or higher than a threshold value, in order to recognize the state where no light is emitted, the state where the light emitting unit does not emit light is indicated for each line. It must last longer than the exposure time.

  FIG. 5A shows the influence of the difference in exposure time when the exposure start times of the exposure lines are equal. 7500a is the case where the exposure end time of the previous exposure line is equal to the exposure start time of the next exposure line, and 7500b is the case where the exposure time is longer than that. As in the case of 7500b, the exposure time of adjacent exposure lines is partially overlapped in time, so that the exposure time can be increased. That is, the light incident on the image sensor increases and a bright image can be obtained. In addition, since the imaging sensitivity for capturing images with the same brightness can be suppressed to a low level, an image with less noise can be obtained, so that communication errors are suppressed.

  FIG. 5B shows the influence of the difference in the exposure start time of each exposure line when the exposure times are equal. 7501a is the case where the exposure end time of the previous exposure line is equal to the exposure start time of the next exposure line, and 7501b is the case where the exposure of the next exposure line is started earlier than the end of exposure of the previous exposure line. By adopting a configuration in which the exposure times of adjacent exposure lines partially overlap in time as in 7501b, the number of lines that can be exposed per time can be increased. Thereby, the resolution becomes higher and a large amount of information can be obtained. Since the sample interval (= difference in exposure start time) becomes dense, the change in the light source luminance can be estimated more accurately, the error rate can be reduced, and the change in the light source luminance in a shorter time is recognized. be able to. By making the exposure time overlap, it is possible to recognize blinking of the light source that is shorter than the exposure time by using the difference in exposure amount between adjacent exposure lines.

  As described with reference to FIGS. 5A and 5B, in the configuration in which each exposure line is sequentially exposed so that the exposure times of adjacent exposure lines partially overlap in time, the exposure time is set to be longer than that in the normal shooting mode. By using the bright line pattern generated by setting it short for signal transmission, the communication speed can be dramatically improved. Here, it is possible to generate an appropriate bright line pattern by setting the exposure time during visible light communication to 1/480 seconds or less. Here, when the frame frequency = f, the exposure time needs to be set as exposure time <1/8 × f. Blanking that occurs during shooting is at most half the size of one frame. That is, since the blanking time is less than half of the shooting time, the actual shooting time is 1 / 2f at the shortest time. Furthermore, since it is necessary to receive quaternary information within a time of 1 / 2f, at least the exposure time needs to be shorter than 1 / (2f × 4). Since the normal frame rate is 60 frames / second or less, it is possible to generate an appropriate bright line pattern in the image data and perform high-speed signal transmission by setting the exposure time to 1/480 seconds or less. Become.

FIG. 5C shows an advantage when the exposure times are short when the exposure times of the exposure lines do not overlap. When the exposure time is long, even if the light source has a binary luminance change as in 7502a, the captured image has an intermediate color portion as in 7502e, and it becomes difficult to recognize the luminance change of the light source. There is. However, as the 7502D, after completion exposure of one exposure line, by a configuration in which the free time (predetermined waiting time) t _D2 not predetermined exposure start exposure of the next exposure line, the luminance variation of the light source Can be easily recognized. That is, a more appropriate bright line pattern such as 7502f can be detected. As in 7502D, be provided with a free time without predetermined exposure becomes an exposure time t _E can be realized to be smaller than the time difference t _D of the exposure start time of each exposure line. When the normal shooting mode has a configuration in which the exposure times of adjacent exposure lines partially overlap in time, the exposure time is set shorter than the normal shooting mode until a predetermined idle time occurs. This can be realized. Further, even when the normal photographing mode is the case where the exposure end time of the previous exposure line and the exposure start time of the next exposure line are equal, by setting the exposure time short until a predetermined non-exposure time occurs, Can be realized. Further, as 7502G, also by increasing the distance t _D of the exposure start time of each exposure line, after the exposure of one exposure line, following exposure line exposure start until a predetermined exposure was not free time (predetermined Waiting time) t _D2 can be provided. In this configuration, since the exposure time can be extended, a bright image can be taken, and noise is reduced, so that error tolerance is high. On the other hand, in this configuration, since the number of exposure lines that can be exposed within a certain time is reduced, there is a disadvantage that the number of samples is reduced as in 7502h. For example, when the imaging target is bright, the former configuration is used, and when the imaging target is dark, the latter configuration can be used to reduce the estimation error of the light source luminance change.

  In all exposure lines, it is not necessary that the exposure time of adjacent exposure lines has a partial overlap in time, and a configuration in which some exposure lines have no partial overlap. It is also possible. Further, in all exposure lines, it is not necessary to provide a configuration in which an idle time (predetermined waiting time) in which a predetermined exposure is not performed is provided after the exposure of one exposure line is completed until the exposure of the next exposure line is started. It is also possible to have a configuration in which the lines partially overlap in time. With such a configuration, it is possible to take advantage of the advantages of each configuration. Further, the same readout method is used in the normal shooting mode in which shooting is performed at a normal frame rate (30 fps, 60 fps) and in the visible light communication mode in which shooting is performed with an exposure time of 1/480 second or less in which visible light communication is performed. Alternatively, a signal may be read by a circuit. By reading out signals with the same reading method or circuit, it is not necessary to use different circuits for the normal imaging mode and the visible light communication mode, and the circuit scale can be reduced.

FIG. 5D shows the relationship between the minimum change time t _S of the light source luminance, the exposure time t _E , the time difference t _D of the exposure start time of each exposure line, and the captured image. When t _E + t _D <t _S , since one or more exposure lines are always imaged in a state where the light source does not change from the start to the end of exposure, an image with clear brightness as in 7503d is obtained. It is easy to recognize the luminance change of the light source. When 2t _E > t _S , a bright line having a pattern different from the luminance change of the light source may be obtained, and it becomes difficult to recognize the luminance change of the light source from the captured image.

Figure 5E, the transition and time t _T of the light source luminance, which shows the relationship between the time difference t _D of the exposure start time of each exposure line. as t _D is larger than the t _T, exposure lines to be neutral is reduced, it is easy to estimate the light source luminance. When t _D > t _T , the exposure line of the intermediate color is continuously 2 or less, which is desirable. t _T, the light source is less than 1 microsecond in the case of LED, light source for an approximately 5 microseconds in the case of organic EL, a t _D by 5 or more microseconds, to facilitate estimation of the light source luminance be able to.

Figure 5F shows a high frequency noise _{t HT} of light source luminance, the relationship between the exposure time _{t E.} As t _E is larger than t _HT , the captured image is less affected by high frequency noise, and light source luminance is easily estimated. When t _E is an integral multiple of t _HT , the influence of high frequency noise is eliminated, and the light source luminance is most easily estimated. For estimation of the light source luminance, it is desirable that t _E > t _HT . The main cause of high frequency noise derived from the switching power supply circuit, since many of the t _HT in the switching power supply for the lamp is less than 20 microseconds, by the t _E and 20 micro-seconds or more, the estimation of the light source luminance It can be done easily.

Figure 5G is the case t _HT is 20 microseconds, which is a graph showing the relationship between the size of the exposure time t _E and the high frequency noise. When t _HT is considered that there is variation by the light source, from the graph, t _E is the value becomes equal to the value when the amount of noise takes a maximum, 15 microseconds or more, or, 35 microseconds or more, or, It can be confirmed that the efficiency is good when it is set to 54 microseconds or more, or 74 microseconds or more. From the viewpoint of reducing high-frequency noise, it is desirable that t _E be large. However, as described above, there is a property that light source luminance can be easily estimated in that the smaller the t _E , the more difficult the intermediate color portion is generated. Therefore, the period of variation of the light source luminance is 15 to 35 _{t E} when microseconds 15 microseconds or more, the period of variation of the light source luminance thirty-five to fifty-four _{t E} when microseconds 35 microseconds or more, the light source luminance t _E is 54 microseconds or more when the period of the change is 54 to 74 microseconds, t _E when the period of the change in light source luminance is 74 microseconds or more may be set as 74 microseconds or more.

Figure 5H shows the relationship between the exposure time t _E and the recognition success rate. Since the exposure time t _E has a relative meaning with respect to the time when the luminance of the light source is constant, the value (relative exposure time) obtained by dividing the period t _S where the luminance of the light source changes by the exposure time t _E is taken as the horizontal axis. Yes. From the graph, it can be seen that if the recognition success rate is desired to be almost 100%, the relative exposure time should be 1.2 or less. For example, when the transmission signal is 1 kHz, the exposure time may be about 0.83 milliseconds or less. Similarly, when it is desired to set the recognition success rate to 95% or more, the relative exposure time may be set to 1.25 or less, and when the recognition success rate is set to 80% or more, the relative exposure time may be set to 1.4 or less. Recognize. Also, the recognition success rate drops sharply when the relative exposure time is around 1.5, and becomes almost 0% at 1.6, so it can be seen that the relative exposure time should not be set to exceed 1.5. . It can also be seen that after the recognition rate becomes 0 at 7507c, it rises again at 7507d, 7507e, and 7507f. Therefore, when it is desired to take a bright image by extending the exposure time, use an exposure time in which the relative exposure time is 1.9 to 2.2, 2.4 to 2.6, and 2.8 to 3.0. Just do it. For example, these exposure times may be used as the intermediate mode in FIG.

  FIG. 6A is a flowchart of the information communication method in the present embodiment.

  The information communication method in the present embodiment is an information communication method for acquiring information from a subject, and includes steps SK91 to SK93.

  That is, in this information communication method, a plurality of bright lines corresponding to a plurality of exposure lines included in the image sensor are generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. A first exposure time setting step SK91 for setting a first exposure time of the image sensor; and the image sensor shoots the subject whose luminance changes with the set first exposure time, First image acquisition step SK92 for acquiring a bright line image including a plurality of bright lines, and information acquisition for acquiring information by demodulating data specified by the patterns of the plurality of bright lines included in the acquired bright line image In the first image acquisition step SK92, each of the plurality of exposure lines is included. Starts exposure at successively different times, and the exposure of the adjacent exposure line after a predetermined idle time from the end, the exposure is started adjacent to the exposure line.

  FIG. 6B is a block diagram of the information communication apparatus in the present embodiment.

  The information communication device K90 in the present embodiment is an information communication device that acquires information from a subject, and includes constituent elements K91 to K93.

  That is, the information communication apparatus K90 causes a plurality of bright lines corresponding to a plurality of exposure lines included in the image sensor to be generated in response to a change in luminance of the subject in an image obtained by photographing the subject by an image sensor. And an exposure time setting unit K91 for setting an exposure time of the image sensor, and the image sensor for acquiring a bright line image including the plurality of bright lines by photographing the subject whose luminance changes with the set exposure time. And an information acquisition unit K93 that acquires information by demodulating data specified by the patterns of the plurality of bright lines included in the acquired bright line image, and the plurality of exposure lines. Each of the exposure starts sequentially at different times and is adjacent to the exposure line. Exposure after a predetermined idle time from the end, to initiate the exposure of.

  In the information communication method and the information communication apparatus K90 shown in FIGS. 6A and 6B as described above, for example, as shown in FIG. 5C, each of the plurality of exposure lines is exposed to the adjacent exposure line adjacent to the exposure line. Since exposure is started after a lapse of a predetermined idle time after the end, it is possible to easily recognize a change in luminance of the subject. As a result, information can be appropriately acquired from the subject.

  In the above embodiment, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. For example, the program causes the computer to execute the information communication method shown by the flowchart of FIG. 6A.

(Embodiment 2)
In the present embodiment, each application example using a receiver such as a smartphone that is the information communication device K90 in the first embodiment and a transmitter that transmits information as a blinking pattern of a light source such as an LED or an organic EL. explain.

  FIG. 7 is a diagram illustrating an example of each mode of the receiver in this embodiment.

  In the normal shooting mode, the receiver 8000 acquires a normal shot image by shooting at a shutter speed of 1/100 seconds, for example, and displays the normal shot image on the display. In this case, the normal photographed image clearly shows a subject such as a streetlight or signage configured as a store signboard and its surroundings.

  Further, in the visible light communication mode, the receiver 8000 acquires a visible light communication image by shooting at a shutter speed of 1/10000 seconds, for example. For example, in the case where the streetlight or signage described above transmits a signal by luminance change as the light source shown in the first embodiment, that is, a transmitter, a location where the signal is transmitted in this visible light communication image One or a plurality of bright lines (hereinafter referred to as a bright line pattern) is projected, and nothing is projected except for the portions. That is, in this visible light communication image, only the bright line pattern is displayed, and the portion where the luminance of the subject is not changed and the periphery of the subject are not projected.

  Further, in the intermediate mode, the receiver 8000 acquires an intermediate image by shooting at a shutter speed of 1/3000 seconds, for example. In this intermediate image, a bright line pattern is displayed, and the above-described portion of the subject where the luminance is not changed and the periphery of the subject are also shown. Therefore, the receiver 8000 displays the intermediate image on the display, so that the user can know where or from where the signal is transmitted. Note that the bright line pattern, the subject, and the surrounding area displayed by the intermediate image are not clearer than the bright line pattern of the visible light communication image and the subject and the surrounding area of the normal captured image, respectively, but have a sharpness recognized by the user. .

  In the following description, shooting in the normal shooting mode or the normal shooting mode is referred to as normal shooting, and shooting in the visible light communication mode or the visible light communication mode is referred to as visible light shooting (visible light communication). Further, instead of normal shooting and visible light shooting, shooting in an intermediate mode may be used, and an intermediate image may be used instead of a composite image described later.

  FIG. 8 is a diagram illustrating an example of a photographing operation of the receiver in this embodiment.

  The receiver 8000 switches the shooting mode to normal shooting, visible light communication, normal shooting, and so on. Then, the receiver 8000 generates a composite image in which the bright line pattern, the subject, and the surrounding area are clearly displayed by combining the normal captured image and the visible light communication image, and displays the composite image on the display. . This composite image is an image generated by superimposing the bright line pattern of the visible light communication image on the portion where the signal in the normal captured image is transmitted. Further, the bright line pattern, the subject, and the surroundings displayed by the composite image are clear and have a sharpness sufficiently recognized by the user. By displaying such a composite image, the user can more clearly know from where or from where the signal is transmitted.

  FIG. 9 is a diagram illustrating another example of the photographing operation of the receiver in this embodiment.

  The receiver 8000 includes a camera Ca1 and a camera Ca2. In such a receiver 8000, the camera Ca1 performs normal photographing, and the camera Ca2 performs visible light photographing. Thereby, the camera Ca1 acquires the normal captured image as described above, and the camera Ca2 acquires the visible light communication image as described above. Then, the receiver 8000 generates the above-described combined image by combining the normal captured image and the visible light communication image, and displays the combined image on the display.

  FIG. 10A is a diagram illustrating another example of imaging operation of a receiver in this embodiment.

  In the receiver 8000 having two cameras, the camera Ca1 switches the shooting mode to normal shooting, visible light communication, normal shooting, and so on. On the other hand, the camera Ca2 continuously performs normal shooting. When the normal shooting is performed simultaneously with the cameras Ca1 and Ca2, the receiver 8000 receives from the normal shooting images acquired by these cameras using stereo vision (the principle of triangulation). The distance from the machine 8000 to the subject (hereinafter referred to as subject distance) is estimated. By using the subject distance estimated in this way, the receiver 8000 can superimpose the bright line pattern of the visible light communication image on an appropriate position of the normal captured image. That is, an appropriate composite image can be generated.

  FIG. 10B is a diagram illustrating another example of imaging operation of a receiver in this embodiment.

  The receiver 8000 includes, for example, three cameras (camera Ca1, camera Ca2, and camera Ca3). In such a receiver 8000, two cameras (camera Ca2 and camera Ca3) continue to perform normal imaging, and the remaining one camera (camera Ca1) continues to perform visible light communication. Accordingly, the subject distance can be estimated based on the normal captured images obtained by the two cameras that are normally capturing at any timing.

  FIG. 10C is a diagram illustrating another example of imaging operation of a receiver in this embodiment.

  The receiver 8000 includes, for example, three cameras (camera Ca1, camera Ca2, and camera Ca3). In such a receiver 8000, each camera switches the shooting mode to normal shooting, visible light communication, normal shooting, and so on. Here, in one period, any two of these cameras perform normal shooting, and the remaining one camera performs visible light communication so that the shooting mode of each camera is switched for each period. It is done. In other words, the combination of cameras that are normally shooting changes periodically. Thereby, in any period, the subject distance can be estimated based on the normal captured images obtained by the two cameras that are normally capturing.

  FIG. 11A is a diagram illustrating an example of a camera arrangement of a receiver in this embodiment.

  When the receiver 8000 includes two cameras Ca1 and Ca2, as shown in FIG. 11A, the two cameras Ca1 and Ca2 are arranged at positions separated from each other. As a result, the subject distance can be estimated with high accuracy. That is, the longer the distance between the two cameras, the more accurately the subject distance can be estimated.

  FIG. 11B is a diagram illustrating another example of the camera arrangement of the receiver in this embodiment.

  When receiver 8000 includes three cameras Ca1, camera Ca2, and camera Ca3, as shown in FIG. 11B, two cameras Ca1 and Ca2 for normal photographing are arranged at positions separated from each other. Moreover, the camera Ca3 for visible light communication is arrange | positioned, for example between camera Ca1 and camera Ca2. As a result, the subject distance can be estimated with high accuracy. That is, the subject distance can be accurately estimated by using the two most distant cameras for normal shooting.

  FIG. 12 is a diagram illustrating an example of display operation of the receiver in this embodiment.

  As described above, the receiver 8000 switches the photographing mode to visible light communication, normal photographing, visible light communication, and so on. Here, the receiver 8000 activates an application program when performing visible light communication for the first time. Then, the receiver 8000 estimates its own position based on the signal received by visible light communication. Next, when performing normal shooting, the receiver 8000 displays AR (Augmented Reality) information on the normal shot image acquired by the normal shooting. This AR information is acquired based on the position estimated as described above. Further, the receiver 8000 estimates the movement and direction change of the receiver 8000 based on the detection result of the 9-axis sensor and the motion detection of the normal captured image, and matches the estimated movement and direction change. To move the display position of the AR information. Thereby, the AR information can be made to follow the subject image of the normal captured image.

  In addition, when the imaging mode is switched from normal imaging to visible light communication, the receiver 8000 superimposes AR information on the latest normal captured image acquired at the time of the normal imaging immediately before the visible light communication. The receiver 8000 displays a normal captured image on which the AR information is superimposed. Similarly to the normal shooting, the receiver 8000 estimates the movement and direction change of the receiver 8000 on the basis of the detection result by the 9-axis sensor, and AR in accordance with the estimated movement and direction change. Move information and normal captured images. Thereby, AR information can be made to follow the subject image of the normal captured image in accordance with the movement of the receiver 8000 or the like in the case of visible light communication as in the case of normal imaging. Further, the normal image can be enlarged and reduced in accordance with the movement of the receiver 8000 or the like.

  FIG. 13 is a diagram illustrating an example of display operation of the receiver in this embodiment.

  For example, the receiver 8000 may display the composite image on which the bright line pattern is projected, as shown in FIG. Further, as shown in FIG. 13B, the receiver 8000 normally captures a signal explicit object that is an image having a predetermined color for notifying that a signal is transmitted instead of the bright line pattern. A composite image may be generated by superimposing on the image, and the composite image may be displayed.

  In addition, as shown in FIG. 13C, the receiver 8000 normally has a location where a signal is transmitted indicated by a dotted frame and an identifier (for example, ID: 101, ID: 102, etc.). The captured image may be displayed as a composite image. Further, as shown in FIG. 13D, the receiver 8000 recognizes a signal that is an image having a predetermined color for notifying that a specific type of signal is transmitted instead of the bright line pattern. A composite image may be generated by superimposing an object on a normal captured image, and the composite image may be displayed. In this case, the color of the signal identification object differs depending on the type of signal output from the transmitter. For example, when the signal output from the transmitter is position information, a red signal identification object is superimposed, and when the signal output from the transmitter is a coupon, the green signal identification object is Superimposed.

  FIG. 14 is a diagram illustrating an example of operation of a receiver in this embodiment.

  For example, when receiving a signal through visible light communication, the receiver 8000 may display a normal captured image and output a sound for notifying the user that the transmitter has been found. In this case, the receiver 8000 varies the type of output sound, the number of outputs, or the output time depending on the number of transmitters found, the type of received signal, or the type of information specified by the signal. It may be allowed.

  FIG. 15 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, when the user touches the bright line pattern displayed in the composite image, the receiver 8000 generates an information notification image based on a signal transmitted from the subject corresponding to the touched bright line pattern, and the information notification Display an image. This information notification image indicates, for example, a store coupon or a place. The bright line pattern may be a signal explicit object, a signal identification object, a dotted line frame, or the like shown in FIG. The same applies to the bright line patterns described below.

  FIG. 16 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, when the user touches the bright line pattern displayed in the composite image, the receiver 8000 generates an information notification image based on a signal transmitted from the subject corresponding to the touched bright line pattern, and the information notification Display an image. For example, the information notification image indicates the current location of the receiver 8000 by a map or the like.

  FIG. 17 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, the receiver 8000 receives signals from two street lamps that are subjects configured as transmitters. Then, similarly to the above, the receiver 8000 estimates its current location based on these signals. The receiver 8000 displays a normal captured image and displays an information notification image (an image indicating latitude and longitude) indicating the estimation result superimposed on the normal captured image. Note that the receiver 8000 may display the auxiliary information notification image superimposed on the normal captured image. This auxiliary information notification image recommends to the user an operation for calibrating, for example, a 9-axis sensor (in particular, a geomagnetic sensor), that is, an operation for canceling drift. By performing such an operation, the current location is estimated with high accuracy.

  Further, when the displayed information notification image is touched by the user, the receiver 8000 may display a map indicating the estimated position instead of the normal captured image.

  FIG. 18 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, when the user swipes the receiver 8000 displaying the composite image, the receiver 8000 performs normal shooting having a dotted frame and an identifier similar to the normal shot image illustrated in FIG. An image is displayed and a list of information is displayed so as to follow the swipe operation. In this list, information specified by a signal transmitted from a location (transmitter) indicated by each identifier is shown. The swipe may be, for example, an operation of moving a finger from outside the right side of the display in the receiver 8000. The swipe may be an operation of moving a finger from the upper side, the lower side, or the left side of the display.

  Further, when information included in the list is tapped by the user, the receiver 8000 may display an information notification image (for example, an image showing a coupon) that shows the information in more detail.

  FIG. 19 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, when the user performs a swipe on the receiver 8000 on which the composite image is displayed, the receiver 8000 displays an information notification image superimposed on the composite image so as to follow the swipe operation. This information notification image shows the subject distance with an arrow in an easy-to-understand manner for the user. The swipe may be, for example, an operation of moving a finger from outside the lower side of the display in the receiver 8000. The swipe may be an operation of moving a finger from the left side of the display, from the upper side, or from the right side.

  FIG. 20 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, the receiver 8000 images a transmitter, which is a signage indicating a plurality of stores, as a subject, and displays a normal captured image acquired by the imaging. Here, when the user taps the signage image of one store included in the subject displayed in the normal captured image, the receiver 8000 generates an information notification image based on a signal transmitted from the signage of the store Then, the information notification image 8001 is displayed. This information notification image 8001 is an image showing, for example, a vacant seat situation in a store.

  FIG. 21 is a diagram illustrating an example of operations of the receiver, the transmitter, and the server in this embodiment.

  First, a transmitter 8012 configured as a television transmits a signal to a receiver 8011 by a luminance change. This signal includes, for example, information for prompting the user to purchase content related to the program being viewed. When the receiver 8011 receives the signal through visible light communication, the receiver 8011 displays an information notification image that prompts the user to purchase content based on the signal. When the user performs an operation for purchasing the content, the receiver 8011 receives information included in a SIM (Subscriber Identity Module) card inserted into the receiver 8011, user ID, terminal ID, credit card information, billing At least one of the information, the password, and the transmitter ID is transmitted to the server 8013. The server 8013 manages a user ID and payment information in association with each user. Then, the server 8013 identifies the user ID based on the information transmitted from the receiver 8011, and confirms the payment information associated with the user ID. By this confirmation, the server 8013 determines whether or not to allow the user to purchase content. If the server 8013 determines to permit, the server 8013 transmits permission information to the receiver 8011. When receiving the permission information, the receiver 8011 transmits the permission information to the transmitter 8012. The transmitter 8012 that has received the permission information acquires and reproduces the content via a network, for example.

  Further, the transmitter 8012 may transmit information including the ID of the transmitter 8012 to the receiver 8011 by changing the luminance. In this case, the receiver 8011 transmits the information to the server 8013. When the server 8013 obtains the information, the server 8013 can determine that, for example, a television program is being viewed by the transmitter 8012, and can perform a viewing rate survey of the television program.

  Further, the receiver 8011 includes the content operated by the user (voting or the like) in the above information and transmits the content to the server 8013, so that the server 8013 can reflect the content in the television program. That is, a viewer participation type program can be realized. Further, when the receiver 8011 accepts writing by the user, the contents of the writing are included in the above-described information and transmitted to the server 8013 so that the server 8013 can write the writing to the TV program or a bulletin board on the network. Etc. can be reflected.

  Further, when the transmitter 8012 transmits the above-described information, the server 8013 can charge for viewing a television program by pay broadcasting or an on-demand program. Further, the server 8013 displays an advertisement on the receiver 8011, displays detailed information of a TV program displayed on the transmitter 8012, and displays a URL of a site indicating the detailed information. Can do. Further, the server 8013 obtains the number of times the advertisement is displayed by the receiver 8011 or the amount of the product purchased by the advertisement, and thereby charges the advertiser according to the number or amount. be able to. Such billing can be made even if the user who saw the advertisement does not purchase the product immediately. Further, when the server 8013 acquires information indicating the manufacturer of the transmitter 8012 from the transmitter 8012 via the receiver 8011, the server 8013 provides a service (for example, a reward for sales of the above-described product to the manufacturer indicated by the information). Payment).

  FIG. 22 is a diagram illustrating another example of operation of a receiver in this embodiment.

  For example, the user points the camera of the receiver 8021 toward a plurality of transmitters 8020a-8020d configured as illumination. At this time, the receiver 8021 is moved so that each of the transmitters 8020a to 8020d is sequentially photographed as a subject. The receiver 8021 receives signals from the transmitters 8020a to 8020d by performing visible light communication while being moved. These signals include information indicating the position of the transmitter. The receiver 8021 is based on the position indicated by the received signals from the transmitters 8020a to 8020d, the detection result of the 9-axis sensor provided in the receiver 8021, and the movement of the image obtained by photographing. The position of the receiver 8021 is estimated using the principle of triangulation. In this case, since the receiver 8021 is moved, the drift of the 9-axis sensor (particularly, the geomagnetic sensor) is eliminated, so that a more accurate position can be estimated.

  FIG. 23 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 is configured as, for example, a head mounted display provided with a camera. The receiver 8030 starts photographing in the visible light communication mode, that is, visible light communication when the start button is pressed. When a signal is received through visible light communication, the receiver 8030 notifies the user of information corresponding to the received signal. This notification is performed, for example, by outputting sound from a speaker provided in the receiver 8030 or by displaying an image. In addition, when the start button is pressed, the visible light communication is received by the receiver 8030 when an input of a voice instructing the start is received by the receiver 8030 or a signal instructing the start is received by wireless communication. It may be started when it is done. In addition, visible light communication is started when the change width of the value obtained by the 9-axis sensor provided in the receiver 8030 exceeds a predetermined range or when a bright line pattern appears even in a normal photographed image. May be.

  FIG. 24 is a diagram illustrating an example of initial setting of the receiver in this embodiment.

  The receiver 8030 displays an alignment image 8031 at the time of initial setting. This alignment image 8031 is for performing alignment between the position indicated by the user in the image obtained by photographing with the camera of the receiver 8030 and the image displayed by the receiver 8030. When the user aligns the fingertip with the position of the circle indicated by the alignment image 8031, the receiver 8030 performs alignment by associating the position of the fingertip with the position of the circle. That is, the position indicated by the user is calibrated.

  FIG. 25 is a diagram illustrating another example of operation of a receiver in this embodiment.

  When the receiver 8030 identifies a location where a signal is transmitted by visible light communication, the receiver 8030 displays a composite image 8034 on which a bright line pattern is projected. The user performs an operation such as a tap or a double tap on the bright line pattern. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 26 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of moving the fingertip so as to surround the bright line pattern in the composite image 8034. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 27 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of placing the fingertip on the bright line pattern in the composite image 8034 for a predetermined time or more. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 28 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation of moving the fingertip toward the bright line pattern of the composite image 8034 by swiping. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 29 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 as described above. Here, the user performs an operation to keep the line of sight toward the bright line pattern of the composite image 8034 for a predetermined time or more. Alternatively, the user performs an operation of blinking a predetermined number of times with the line of sight directed toward the bright line pattern. Upon receiving this operation, the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 30 is a diagram illustrating another example of operation of a receiver in this embodiment.

  Similarly to the above, the receiver 8030 displays the composite image 8034 and displays an arrow associated with each bright line pattern in the composite image 8034. These arrows point in different directions for each bright line pattern. Here, the user performs an operation of moving the head along any of the arrows. When receiving this operation based on the detection result by the 9-axis sensor, the receiver 8030 specifies the bright line pattern associated with the arrow corresponding to the operation, that is, the arrow pointing in the direction in which the head is moved. The receiver 8030 displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.

  FIG. 31A is a diagram illustrating a pen used for operation of a receiver in this embodiment.

  The pen 8033 includes a transmitter 8033a that transmits a signal according to a luminance change, a button 8033b, and a button 8033c. When the button 8033b is pressed, the transmitter 8033a transmits a predetermined first signal, and when the button 8033c is pressed, the transmitter 8033a has a predetermined first signal different from the first signal. 2 signal is transmitted.

  FIG. 31B is a diagram illustrating operation of a receiver using a pen in this embodiment.

  The pen 8033 is used in place of the above-described user's fingertip, and is used like a stylus pen. Further, by properly using the button 8033b and the button 8033c, the pen 8033 can be used as a normal pen or used like an eraser.

  FIG. 32 is a diagram illustrating an example of appearance of a receiver in this embodiment.

  The receiver 8030 includes a first touch sensor 8030a and a second touch sensor 8030b. These touch sensors are attached to the frame of the receiver 8030. For example, when the user moves the fingertip on the first touch sensor 8030a, the receiver 8030 moves the pointer in accordance with the movement of the fingertip on the image displayed to the user. Further, when the user touches the second touch sensor 8030b, the receiver 8030 performs a process of selecting an object to which the pointer is placed on an image displayed to the user.

  FIG. 33 is a diagram illustrating another example of the appearance of the receiver in this embodiment.

  The receiver 8030 includes a touch sensor 8030c. The touch sensor 8030c is attached to the frame of the receiver 8030. For example, when the user touches and moves the touch sensor 8030c with the fingertip, the receiver 8030 moves the pointer in accordance with the movement of the fingertip on the image displayed to the user. When the user presses the touch sensor 8030c, the receiver 8030 performs a process of selecting an object to which the pointer is placed on an image displayed to the user. That is, the touch sensor 8030c is configured as a so-called clickable touch sensor.

  FIG. 34 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays the composite image 8034 in the same manner as described above, and displays the pointer 8035 in the composite image 8034. When the receiver 8030 includes the first touch sensor 8030a and the second touch sensor 8030b, the user moves the pointer by moving the first touch sensor 8030a while placing the fingertip on the first touch sensor 8030a. Point the object at. Then, the user touches the second touch sensor 8030b to cause the receiver 8030 to select the bright line pattern. When the receiver 8030 selects a bright line pattern, the receiver 8030 displays an information notification image 8032 based on a signal transmitted from the portion of the bright line pattern.

  In the case where the receiver 8030 includes the touch sensor 8030c, the user moves the pointer by placing the fingertip on the touch sensor 8030c and moves the pointer to the object having the bright line pattern. Then, the user causes the receiver 8030 to select the bright line pattern by pressing the touch sensor 8030c. When the receiver 8030 selects a bright line pattern, the receiver 8030 displays an information notification image 8032 based on a signal transmitted from the portion of the bright line pattern.

  FIG. 35A is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays a gesture confirmation image 8036 based on a signal obtained by performing visible light communication. This gesture confirmation image 8036 prompts the user for a predetermined gesture in order to provide a service to the user, for example.

  FIG. 35B is a diagram illustrating an application example using the receiver in this embodiment.

  A user 8038 wears the receiver 8030 and is at a store, for example. Here, the receiver 8030 displays the above-described gesture confirmation image 8036 on the user 8038. The user 8038 performs a predetermined gesture according to the gesture confirmation image 8036. Here, a clerk 8039 in the store is wearing a receiver 8037. The receiver 8037 is configured as a head mounted display provided with a camera, and may have the same configuration as the receiver 8030. The receiver 8037 also displays a gesture confirmation image 8036 based on a signal obtained by performing visible light communication. The salesclerk 8039 determines whether or not the predetermined gesture indicated by the displayed gesture confirmation image 8036 matches the gesture performed by the user 8038. When the clerk 8039 determines that they match, the clerk 8039 provides the user 8038 with a service associated with the gesture confirmation image 8036.

  FIG. 36A is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8030 displays a gesture confirmation image 8040 based on a signal obtained by performing visible light communication. This gesture confirmation image 8040 prompts the user for a predetermined gesture in order to permit wireless communication, for example.

  FIG. 36B is a diagram illustrating an application example using the receiver in this embodiment.

  A user 8038 is wearing a receiver 8030. Here, the receiver 8030 displays the above-described gesture confirmation image 8040 on the user 8038. The user 8038 performs a predetermined gesture according to the gesture confirmation image 8040. Here, a person 8041 around the user 8038 wears the receiver 8037. The receiver 8037 is configured as a head mounted display provided with a camera, and may have the same configuration as the receiver 8030. The receiver 8037 acquires authentication information such as a password included in the gesture by photographing a predetermined gesture performed by the user 8038. When the receiver 8037 determines that the authentication information matches the predetermined information, the receiver 8037 establishes a wireless connection with the receiver 8030. After the establishment, the receiver 8030 and the receiver 8037 can perform wireless communication with each other.

  FIG. 37A is a diagram illustrating an example of operation of a transmitter in this embodiment.

  The transmitter transmits the signal 1 and the signal 2 alternately at a predetermined cycle, for example. Transmission of the signal 1 and transmission of the signal 2 are performed by luminance changes such as blinking of visible light. Further, the luminance change pattern for transmitting the signal 1 and the luminance change pattern for transmitting the signal 2 are different from each other.

  FIG. 37B is a diagram illustrating another example of operation of a transmitter in this embodiment.

  The transmitter may transmit the signal 1 and the signal 2 intermittently with a buffer time without continuously transmitting the signal 1 and the signal 2 as described above. Here, the transmitter does not change the luminance during the buffer time. Alternatively, the transmitter may transmit a signal indicating that it is a buffer time by a luminance change or a luminance change different from the luminance change for transmitting each of the signal 1 and the signal 2 during the buffer time. . As a result, the receiver can appropriately receive the signal 1 and the signal 2 without interference.

  FIG. 38 is a diagram illustrating another example of operation of a transmitter in this embodiment.

  The transmitter repeatedly transmits a signal sequence of a unit composed of a preamble, block 1, block 2, block 3, and a check signal according to a luminance change. Here, the block 1 has a preamble, an address 1, data 1 and a check signal. Block 2 and block 3 are configured in the same manner as block 1. Further, specific information can be obtained by using data included in each of block 1, block 2 and block 3.

  That is, in the signal sequence as described above, one piece of data or information is stored in three blocks. Therefore, even if a receiver that requires a blanking period for imaging cannot receive all the data of block 1, block 2 and block 3 from one signal string, the remaining data from other signal strings Can be received. As a result, even a receiver that requires a blanking period can appropriately acquire specific information from at least one signal sequence.

  In the signal sequence as described above, a preamble and a check signal are arranged for a set of three blocks. Therefore, a receiver that can receive light without requiring a blanking period, for example, a receiver equipped with an illuminance sensor or the like, uses a preamble and a check signal that are arranged for the set, and thereby uses one signal sequence. It can be received at once and specific information can be acquired in a short time.

  FIG. 39 is a diagram illustrating another example of operation of a transmitter in this embodiment.

  As described above, when the transmitter repeatedly transmits the signal sequence of the structural unit including the block 1, the block 2, and the block 3, for each signal sequence, the transmitter changes the arrangement of the blocks included in the signal sequence. Also good. For example, each block is arranged in the order of block 1, block 2, and block 3 in the first signal sequence, and each block is arranged in the order of block 3, block 1, and block 2 in the next signal sequence. Thereby, it can be avoided that only the same block is acquired by a receiver that requires a periodic blanking period.

  FIG. 40 is a diagram illustrating an example of a communication form between a plurality of transmitters and a receiver in this embodiment.

  The receiver 8050 may receive a signal (visible light) transmitted from the transmitters 8051a and 8051b configured as illumination and reflected by the reflecting surface. Thereby, signals from many transmitters can be received together. In this case, the transmitter 8051a and the transmitter 8051b transmit signals of different frequencies or protocols. Thereby, the receiver 8050 can receive the signals from those transmitters without interference.

  FIG. 41 is a diagram illustrating an example of operation of a plurality of transmitters in this embodiment.

  One of the transmitter 8051a and the transmitter 8051b may monitor the transmission status of the signal from the other and transmit a signal so as to prevent interference with the other signal. For example, one transmitter receives a signal transmitted from the other transmitter and transmits a signal of a protocol different from that signal. Alternatively, one transmitter detects a period during which no signal is transmitted from the other transmitter, and transmits a signal during that period.

  FIG. 42 is a diagram illustrating another example of a communication form between a plurality of transmitters and a receiver in this embodiment.

  The transmitter 8051a and the transmitter 8051b may transmit signals having the same frequency or protocol. In this case, the receiver 8050 specifies the intensity of the signal transmitted from those transmitters, that is, the edge intensity of the bright line included in the image obtained by imaging. This intensity becomes weaker as the distance between the receiver 8050 and the transmitter is longer. When the distance between the receiver 8050 and each of the transmitter 8051a and the transmitter 8051b is different, such a difference in distance can be used. That is, the receiver 8050 can appropriately separate and receive the signals transmitted from the transmitter 8051a and the transmitter 8051b according to the specified strength.

  FIG. 43 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver 8050 receives a signal transmitted from the transmitter 8051a and reflected by the reflecting surface. At this time, the receiver 8050 may estimate the position of the transmitter 8051a based on the luminance intensity distribution (luminance difference at a plurality of positions) in an image obtained by photographing.

  FIG. 44 is a diagram illustrating an example of application of a receiver in this embodiment.

  For example, the receiver 7510a configured as a smartphone captures an image of the light source 7510b with a back camera (out camera) 7510c, receives a signal transmitted from the light source 7510b, and acquires the position and orientation of the light source 7510b from the received signal. The receiver 7510a estimates the position and orientation of the receiver 7510a itself from how the light source 7510b is captured in the captured image and the sensor values of the 9-axis sensor provided in the receiver 7510a. The receiver 7510a captures the user 7510e with a front camera (face camera, in-camera) 7510f, and estimates the position and orientation of the head of the 7510e and the line-of-sight direction (eyeball position and orientation) by image processing. . The receiver 7510a transmits the estimation result to the server. The receiver 7510a changes the behavior (display content and playback sound) according to the viewing direction of the user 7510e. The imaging by the back camera 7510c and the imaging by the front camera 7510f may be performed simultaneously or alternately.

  FIG. 45 is a diagram illustrating an example of application of a receiver in this embodiment.

  For example, similarly to the above, the receivers 7511d and 7511i configured as smartphones receive signals from the light sources 7511b and 7511g, estimate their positions and orientations, and estimate the gaze directions of the users 7511e and 7511j. In addition, the receivers 7511d and 7511i acquire information on the surrounding objects 7511a to 7511c and 7511f to 7511h from the server based on the received data. The receivers 7511d and 7511i change the display on the display as if the object on the other side is seen through the receivers 7511d and 7511i when viewed from the user. The receivers 7511d and 7511i display an AR (augmented reality) object such as 7511k in accordance with the content shown on the display. When the line of sight of the user 7511j exceeds the imaging range of the imaging of the camera, the receiver 7511i displays that it is out of the range, such as 7511l. Alternatively, the AR object and other information are displayed in an area outside the range. Alternatively, the images obtained when the out-of-range region is captured in the past are connected and displayed.

  FIG. 46 is a diagram illustrating an example of application of a receiver in this embodiment.

  For example, similarly to the above, the receiver 7512c configured as a smartphone receives a signal from the light source 7512a, estimates its own position and orientation, and estimates the visual line direction of the user 7512d. The receiver 7512c performs processing related to the object 7512b that is in the line-of-sight direction of the user 7512d. For example, information related to the object 7512b is displayed on the screen. When the line-of-sight direction of the user 7512h moves from the object 7512f to the receiver 7512g, the receiver 7512g determines that the user 7512h is interested in the object 7512f, and continues processing related to the object 7512f. For example, information on the object 7512f is kept displayed on the screen.

  FIG. 47 is a diagram illustrating an example of application of a transmitter in this embodiment.

  For example, the transmitter 7513a configured as an illumination has a high luminance, and when the luminance is high (high) or low (low) as a transmission signal, it exceeds the upper limit brightness when imaged by the receiver. , 7513b, no bright line appears. Therefore, as shown in 7513c, by providing a portion 7513d that diffuses or weakens light such as a diffuser plate or a prism to reduce the luminance, the receiver can image bright lines like 7513e. .

  FIG. 48 is a diagram illustrating an example of application of a transmitter in this embodiment.

  For example, in the transmitter 7514a configured as illumination, since the light source is not uniform, the picked-up image causes unevenness in brightness as in the case of 7514b and induces a reception error. Therefore, as shown by 7514c, a reception error can be suppressed by providing a portion 7514d for diffusing light such as a diffuser plate or a prism so that the luminance is uniform.

  FIG. 49 is a diagram illustrating an application example of the reception method in this embodiment.

  In the transmitters 7515a and 7515b, the luminance of the central portion is high, and bright lines do not appear in the captured image of the receiver, but bright lines appear in the peripheral portions. The receiver cannot receive a signal from the portion 7515d because the bright line is interrupted, but can receive a signal from the portion 7515c. The receiver can receive signals from more bright lines than the part 7515c by reading the bright lines in the path 7515e.

  FIG. 50 is a diagram illustrating an example of application of a transmitter in this embodiment.

  For example, transmitters 7516a, 7516b, 7516c, and 7516d configured as illumination have high luminance like 7513a, and bright lines are less likely to occur when imaged by a receiver. Therefore, the diffuser plate / prism 7516e, the reflector plate 7516f, the reflector plate / half mirror 7516g, the reflector plate 7516h, and the diffuser plate / prism 7516j are provided to diffuse the light and widen the portion where the bright line is generated. Can do. With these transmitters, the picked-up image is picked up in the form of bright lines around 7515a. Since the receiver estimates the distance between the receiver and the transmitter using the size of the transmitter on the captured image, the part where the light is diffused is set as the size of the light source, and the server is associated with the transmission ID. By storing in the receiver, the receiver can accurately estimate the distance to the transmitter.

  FIG. 51 is a diagram illustrating an example of application of a transmitter in this embodiment.

  For example, the transmitter 7517a configured as illumination has high luminance like the 7513a, and a bright line is hardly generated when imaged by the receiver. Therefore, by providing the reflecting plate 7517b, it is possible to diffuse light and widen a portion where a bright line is generated.

  FIG. 52 is a diagram illustrating an example of application of a transmitter in this embodiment.

  The transmitter 7518a reflects light from the light source at 7518c, so that the receiver can capture bright lines in a wide range. The transmitter 7518d directs the light source toward the diffuser plate or the prism 7518e, so that the receiver can image the bright lines in a wide range.

  FIG. 53 is a diagram illustrating another example of operation of a receiver in this embodiment.

  The receiver displays the bright line pattern by the composite image or the intermediate image as described above. At this time, the receiver may not be able to receive a signal from the transmitter corresponding to the bright line pattern. Here, when the bright line pattern is selected by the user performing an operation (for example, tapping) on the bright line pattern, the receiver performs an optical zoom to enlarge the composite line image or the intermediate image in which the bright line pattern is enlarged. Display an image. By performing such optical zoom, the receiver can appropriately receive a signal from the transmitter corresponding to the bright line pattern. That is, even if an image obtained by imaging is too small to acquire a signal, the signal can be appropriately received by performing optical zoom. Even when an image having a size capable of acquiring a signal is displayed, fast reception can be performed by performing optical zoom.

(Summary of this embodiment)
The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, and an bright line corresponding to an exposure line included in the image sensor is included in an image obtained by photographing the subject with an image sensor. A first exposure time setting step for setting an exposure time of the image sensor so as to occur in accordance with a change in luminance of the subject; and the image sensor photographs the subject whose luminance changes with the set exposure time. The bright line image acquisition step of acquiring a bright line image that is an image including the bright line, and the spatial position of the portion where the bright line appears can be identified based on the bright line image, and the subject and the subject An image display step for displaying a display image in which the surroundings of the subject are projected, and the image included in the acquired bright line image Including an information acquisition step of acquiring transmission information by demodulating the data identified by the pattern of lines.

  For example, a composite image or an intermediate image as shown in FIGS. 7 to 9 and FIG. 13 is displayed as a display image. In the display image in which the subject and the surroundings of the subject are projected, the spatial position of the part where the bright line appears is identified by a bright line pattern, a signal explicit object, a signal identification object, a dotted line frame, or the like. Therefore, the user can easily find a subject that is transmitting a signal due to a change in luminance by viewing such a display image.

  The information communication method further includes a second exposure time setting step for setting an exposure time longer than the exposure time, and the image sensor photographs the subject and the surroundings of the subject with the long exposure time. A normal image acquisition step of acquiring a normal photographed image, a part where the bright line appears in the normal photographed image is identified based on the bright line image, and a signal object which is an image indicating the part is designated as the normal image A composite step of generating a composite image by superimposing the captured image, and the composite image may be displayed as the display image in the image display step.

  For example, the signal object is a bright line pattern, a signal explicit object, a signal identification object, a dotted line frame, or the like, and a composite image is displayed as a display image as shown in FIGS. Thus, the user can more easily find the subject that is transmitting the signal due to the luminance change.

  In the first exposure time setting step, an exposure time is set to 1/3000 sec. In the bright line image acquisition step, the bright line image in which the periphery of the subject is projected is acquired, and in the image display step. The bright line image may be displayed as the display image.

  For example, as shown in FIG. 7, the bright line image is acquired and displayed as an intermediate image. Therefore, it is not necessary to perform processing such as acquiring and synthesizing the normal captured image and the visible light communication image, and the processing can be simplified.

  The image sensor includes a first image sensor and a second image sensor. In the normal image acquisition step, the first image sensor captures the normal captured image, and the bright line is acquired. In the image acquisition step, the bright line image may be acquired by capturing the second image sensor simultaneously with the capturing of the first image sensor.

  For example, as shown in FIG. 9, a normal photographed image and a visible light communication image that is a bright line image are acquired by each camera. Therefore, compared with the case where a normal captured image and a visible light communication image are acquired with one camera, those images can be acquired earlier, and the processing can be speeded up.

  Further, in the information communication method, when the part where the bright line appears in the display image is designated by a user operation, the information communication method further includes the transmission information acquired from the pattern of the bright line of the designated part. An information presentation step of presenting presentation information based on the information may be included. For example, the operation by the user is shown in association with a tap, swipe, an operation in which a fingertip is continuously applied to the part for a predetermined time, an operation in which a line of sight is directed to the part for a predetermined time or more, An operation of moving a part of the user's body to an arrow, an operation of applying a pen tip that changes in luminance to the part, or a touch sensor is touched, and a pointer displayed on the display image is applied to the part It is an operation.

  For example, as shown in FIGS. 15 to 20 and FIGS. 25 to 34, the presentation information is displayed as an information notification image. Thereby, desired information can be presented to the user.

  Further, the image sensor may be provided in a head mounted display, and in the image display step, a projector mounted on the head mounted display may display the display image.

  Thereby, for example, as shown in FIGS. 23 to 30, information can be easily presented to the user.

  In addition, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. As described above, the first exposure time setting step of setting the exposure time of the image sensor, and the image including the bright line by the image sensor photographing the subject whose luminance changes with the set exposure time. A bright line image acquiring step for acquiring the bright line image, and an information acquiring step for acquiring information by demodulating data specified by the pattern of the bright line included in the acquired bright line image. In the obtaining step, a plurality of the subjects are photographed during the period in which the image sensor is moved. By acquiring the bright line image including a plurality of parts where the bright line appears, in the information acquisition step, for each part, by demodulating data specified by the pattern of the bright line of the part, The position of each of the subjects is acquired, and the information communication method further includes a position for estimating the position of the image sensor based on the acquired positions of the plurality of subjects and the movement state of the image sensor. An estimation step may be included.

  Thereby, for example, as shown in FIG. 22, the position of the receiver including the image sensor can be accurately estimated based on a luminance change caused by a subject such as a plurality of illuminations.

  In addition, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. As described above, the first exposure time setting step of setting the exposure time of the image sensor, and the image including the bright line by the image sensor photographing the subject whose luminance changes with the set exposure time. A bright line image acquisition step of acquiring a bright line image, an information acquisition step of acquiring information by demodulating data specified by a pattern of the bright line included in the acquired bright line image, and the acquired information An information presentation step for presenting the image sensor, wherein in the information presentation step, a user of the image sensor Against THE, it may present an image that prompts the gesture that has been predetermined as the information.

  As a result, for example, as shown in FIGS. 35A to 36B, depending on whether or not the user performs the gesture as prompted, the user can be authenticated and the convenience can be improved.

  In addition, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. As described above, the exposure time setting step for setting the exposure time of the image sensor, and the image sensor captures the bright line image including the bright line by photographing the subject whose luminance changes at the set exposure time. And an information acquisition step of acquiring information by demodulating data specified by the bright line pattern included in the acquired bright line image. In the image acquisition step, the image is reflected on the reflection surface. The bright line image is acquired by photographing a plurality of the subjects, and the information acquisition step is performed. Then, in accordance with the intensity of the bright line included in the bright line image, the bright line is separated into bright lines corresponding to each of the plurality of subjects, and each subject is specified by a bright line pattern corresponding to the subject. Information may be acquired by demodulating data.

  Thereby, for example, as shown in FIG. 42, even when the subject such as a plurality of illuminations changes in luminance, appropriate information can be acquired from each of the subjects.

  In addition, in the information communication method for acquiring information from a subject, a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject. As described above, the exposure time setting step for setting the exposure time of the image sensor, and the image sensor captures the bright line image including the bright line by photographing the subject whose luminance changes at the set exposure time. And an information acquisition step of acquiring information by demodulating data specified by the bright line pattern included in the acquired bright line image. In the image acquisition step, the image is reflected on the reflection surface. The bright line image is acquired by photographing the subject, and the information communication method further includes: , Based on the luminance distribution of the bright line image may include position estimation step for estimating the position of the object.

  Thereby, for example, as shown in FIG. 43, an appropriate subject position can be estimated based on the luminance distribution.

  An information communication method for transmitting a signal according to a luminance change, wherein a first determination step of determining a first pattern of luminance change by modulating a first signal to be transmitted, A second determination step of determining a second pattern of luminance change by modulating the signal of 2; and a luminance change according to the first pattern determined by the light emitter; A transmission step of transmitting the first and second signals by alternately performing a luminance change according to the second pattern.

  Thereby, for example, as shown in FIG. 37A, the first signal and the second signal can be transmitted without delay.

  In the transmission step, when the luminance change is switched between the luminance change according to the first pattern and the luminance change according to the second pattern, the luminance change may be performed after a buffer time.

  Thereby, for example, as shown in FIG. 37B, interference between the first signal and the second signal can be suppressed.

  An information communication method for transmitting a signal according to a luminance change, wherein a determination step of determining a luminance change pattern by modulating a signal to be transmitted, and the light emitter changes in luminance according to the determined pattern And transmitting the signal to be transmitted, the signal comprising a plurality of large blocks, each of the plurality of large blocks including first data and a preamble for the first data. And a check signal for the first data, wherein the first data is composed of a plurality of small blocks, and the small blocks include second data, a preamble for the second data, and the first data. And a check signal for the second data may be included.

  Thus, for example, as shown in FIG. 38, data can be appropriately acquired by a receiver that requires a blanking period or a receiver that does not require a blanking period.

  An information communication method for transmitting a signal by luminance change, wherein a plurality of transmitters each modulate a signal to be transmitted to determine a luminance change pattern, and for each transmitter, And a transmitting step in which a light emitter provided in the transmitter changes the luminance according to the determined pattern and transmits the signal to be transmitted. In the transmitting step, signals having different frequencies or protocols are transmitted. May be.

  Thereby, for example, as shown in FIG. 40, interference of signals from a plurality of transmitters can be suppressed.

  An information communication method for transmitting a signal by luminance change, wherein a plurality of transmitters each modulate a signal to be transmitted to determine a luminance change pattern, and for each transmitter, A transmitter in which a light emitter provided in the transmitter transmits a signal to be transmitted by changing in luminance according to the determined pattern, wherein in the transmitting step, one of the plurality of transmitters One transmitter may receive a signal transmitted from the other transmitter and transmit another signal in a manner that does not interfere with the received signal.

  Thereby, for example, as shown in FIG. 41, interference of signals from a plurality of transmitters can be suppressed.

(Embodiment 3)
In this embodiment, each application example using a receiver such as a smartphone in Embodiment 1 or 2 and a transmitter that transmits information as a blinking pattern such as an LED or an organic EL will be described.

  FIG. 54 is a flowchart illustrating an example of operation of the receiver in Embodiment 3.

  First, the receiver receives a signal with an illuminance sensor (8101). Next, the receiver acquires information such as position information from the server based on the received signal (8102). Next, the receiver activates an image sensor that can capture the light receiving direction of the illuminance sensor (8103). Next, the receiver receives part or all of the signal by the image sensor, and checks whether part or all of the signal is the same as the signal received by the illuminance sensor (8104). Next, the receiver estimates the position of the receiver from the position of the transmitter in the captured image (captured image), the information from the 9-axis sensor provided in the receiver, and the position information of the transmitter. (8105). As described above, the receiver activates the illuminance sensor with low power consumption, and activates the image sensor when a signal is received by the illuminance sensor. Then, the receiver performs position estimation using imaging by the image sensor. As a result, the position of the receiver can be accurately estimated while suppressing power consumption.

  FIG. 55 is a flowchart illustrating another example of operation of a receiver in Embodiment 3.

  The receiver recognizes a periodic change in luminance from the sensor value of the illuminance sensor (8111). Next, the receiver activates an image sensor that can capture the light receiving direction of the illuminance sensor, and receives a signal (8112). That is, similarly to the above, the receiver activates an illuminance sensor with low power consumption, and activates the image sensor when a periodic change in luminance is received by the illuminance sensor. And a receiver receives an exact signal by imaging with the image sensor. Thereby, it is possible to receive an accurate signal while suppressing power consumption.

  FIG. 56A is a block diagram illustrating an example of a transmitter in Embodiment 3.

  The transmitter 8115 includes a power supply unit 8115a, a signal control unit 8115b, a light emitting unit 8115c, and a light emitting unit 8115d. The power supply unit 8115a supplies power to the signal control unit 8115b. The signal control unit 8115b distributes the power supplied from the power supply unit 8115a to the light emitting unit 8115c and the light emitting unit 8115d, and controls the luminance change of the light emitting unit 8115c and the light emitting unit 8115d.

  FIG. 56B is a block diagram illustrating another example of the transmitter in Embodiment 3.

  The transmitter 8116 includes a power supply unit 8116a, a signal control unit 8116b, a light emitting unit 8116c, and a light emitting unit 8116d. The power supply unit 8116a supplies power to the light emitting unit 8116c and the light emitting unit 8116d. Here, the signal control unit 8116b controls changes in luminance of the light emitting unit 8116c and the light emitting unit 8116d by controlling the power supplied from the power supply unit 8116a. In this manner, the power control unit 8116a that supplies power to each of the light emitting unit 8116c and the light emitting unit 8116d is controlled by the signal control unit 8116b, so that the power use efficiency can be increased.

  FIG. 57 is a diagram illustrating a configuration example of a system including a plurality of transmitters in the third embodiment.

  This system includes a central control unit 8118, a transmitter 8117, and a transmitter 8120. The central control unit 8118 controls transmission of signals due to luminance changes of the transmitter 8117 and the transmitter 8120, respectively. For example, the central control unit 8118 causes the transmitter 8117 and the transmitter 8120 to transmit the same signal at the same timing, or causes only one of the transmitters to transmit a unique signal to the transmitter.

  The transmitter 8120 includes two transmission units 8121 and 8122, a signal change unit 8123, a signal storage unit 8124, a synchronization signal input unit 8125, a synchronization control unit 8126, and a light receiving unit 8127.

  Each of the two transmission units 8121 and 8122 has a configuration similar to that of the transmitter 8115 illustrated in FIG. 56A, and transmits a signal by changing the luminance. Specifically, the transmission unit 8121 includes a power supply unit 8121a, a signal control unit 8121b, a light emitting unit 8121c, and a light emitting unit 8121d. The transmission unit 8122 includes a power supply unit 8122a, a signal control unit 8122b, a light emitting unit 8122c, and a light emitting unit 8122d.

  The signal changing unit 8123 modulates the transmission target signal into a signal indicating a luminance change pattern. The signal storage unit 8124 stores a signal indicating the luminance change pattern. The signal control unit 8121b of the transmission unit 121 reads the signal stored in the signal storage unit 8124, and changes the luminance of the light emitting unit 8121c and the light emitting unit 8121d according to the signal.

  The synchronization signal input unit 8125 acquires a synchronization signal in accordance with control by the central control unit 8118. When the synchronization signal is acquired, the synchronization control unit 8126 synchronizes the luminance change between the transmission unit 8121 and the transmission unit 8122. That is, the synchronization control unit 8126 synchronizes the luminance change between the transmission unit 8121 and the transmission unit 8122 by controlling the signal control unit 8121b and the signal control unit 8122b. Here, the light receiving unit 8127 detects light emission from the transmission unit 8121 and the transmission unit 8122. The synchronization control unit 8126 performs feedback control to the signal control unit 8121b and the signal control unit 8122b in accordance with the light detected by the light receiving unit 8127.

  FIG. 58 is a block diagram illustrating another example of the transmitter in the third embodiment.

  The transmitter 8130 includes a transmission unit 8131 that transmits a signal according to a luminance change, and a non-transmission unit 8132 that emits light without transmitting the signal.

  The transmission unit 8131 has a configuration similar to that of the transmitter 8115 illustrated in FIG. 56A and includes a power supply unit 8131a, a signal control unit 8131b, and light emitting units 8131c to 8131f. The non-transmission unit 8132 includes a power supply unit 8132a and light emitting units 8132c to 8132f, and does not include a signal control unit. In other words, when there are multiple units including a power supply and synchronization control of luminance change is not possible between these units, only one unit is provided with a signal control unit as shown in FIG. Change the brightness of only one unit.

  Here, in such a transmitter 8130, the light emitting units 8131c to 8131f of the transmission unit 8131 are continuously arranged in a line. That is, any of the light emitting units 8132c to 8132f of the non-transmission unit 8132 is not mixed with the set of the light emitting units 8131c to 8131f. As a result, the size of the illuminant that changes in luminance increases, so that the receiver can easily receive a signal transmitted by the luminance change.

  FIG. 59A is a diagram illustrating an example of a transmitter in Embodiment 3.

  The transmitter 8134 is configured as a signage, for example, and includes three light emitting units (light emitting regions) 8134a to 8134c. In addition, the light from these light emission parts 8134a-8134c does not interfere with each other. Here, when only one of the light emitting units 8134a to 8134c can change the luminance and transmit the signal, the light emitting unit 8134b at the center is connected as shown in FIG. It is desirable to change the brightness. When two of the light emitting units 8134a to 8134c can change in luminance, as shown in FIG. 59A (b), the light emitting unit 8134b at the center and the light emitting unit 8134a at the end or the light emitting unit It is desirable to change the brightness of the portion 8134c. By changing the luminance of the light emitting unit at such a position, the receiver can appropriately receive a signal transmitted by the luminance change.

  FIG. 59B is a diagram illustrating an example of a transmitter in Embodiment 3.

  The transmitter 8135 is configured as a signage, for example, and includes three light emitting units 8135a to 8135c. Note that light from the light emitting units adjacent to each other among these light emitting units 8135a to 8135c interferes with each other. Here, when only one of the light emitting units 8135a to 8135c can change the luminance to transmit a signal, as shown in (a) of FIG. 59B, the light emission arranged at the end. It is desirable to change the luminance of the portion 8135a or the light emitting portion 8135c. Thereby, it can suppress that the luminance change for transmitting a signal interferes with the light from another light emission part. When two of the light emitting units 8135a to 8135c can be changed in luminance, as shown in FIG. 59B (b), the light emitting unit 8135b at the center and the light emitting unit 8135a at the end or the light emitting unit It is desirable to change the brightness of the portion 8135c. By changing the luminance of the light emitting unit at such a position, the area of the luminance change increases, so that the receiver can appropriately receive a signal transmitted by the luminance change.

  FIG. 59C is a diagram illustrating an example of a transmitter in Embodiment 3.

  If the transmitter 8134 can change the luminance of two of the three light emitting units 8134a to 8134c, the transmitter 8134 can change the luminance of the light emitting units 8134a and 8134c at both ends as shown in FIG. 50C. Good. In this case, in the imaging by the receiver, the imaging range in which the part where the luminance changes is reflected can be expanded.

  60A is a diagram illustrating an example of a transmitter in Embodiment 3. FIG.

  The transmitter 8137 is configured as a signage, for example, and transmits a signal when the character portion “A Shop” and the light emitting unit 8137a change in luminance. The light emitting unit 8137a is formed in, for example, a rectangular shape that is long in the horizontal direction, and changes in luminance uniformly. When the light emitting unit 8137a uniformly changes in luminance, the receiver can appropriately receive a signal transmitted by the luminance change.

  FIG. 60B is a diagram illustrating an example of a transmitter in Embodiment 3.

  The transmitter 8138 is configured as a signage, for example, and transmits a signal when the character portion “A Shop” and the light emitting unit 8138a change in luminance. The light emitting unit 8138a is formed in a frame shape along the edge of the signage, for example, and the luminance changes uniformly. That is, the light emitting portion 8138a is formed so that the length of the continuous projected portion is maximized when the light emitting portion is projected onto an arbitrary straight line. When the light emitting unit 8138a changes in luminance uniformly, the receiver can more appropriately receive a signal transmitted by the luminance change.

  FIG. 61 is a diagram illustrating an example of processing operations of a receiver, a transmitter, and a server in Embodiment 3.

  For example, the receiver 8142 configured as a smartphone acquires position information indicating its own position, and transmits the position information to the server 8141. Note that the receiver 8142 acquires position information when, for example, GPS is used or other signals are received. The server 8141 transmits the ID list associated with the position indicated by the position information to the receiver 8142. The ID list includes, for each ID such as “abcd”, the ID and information associated with the ID.

  The receiver 8142 receives a signal from a transmitter 8143 configured as, for example, a lighting device. At this time, the receiver 8142 may be able to receive only a part of the ID (for example, “b”) as the above signal. In this case, the receiver 8142 searches the ID list for an ID including a part of the ID. If the unique ID is not found, the receiver 8142 further receives a signal from the transmitter 8143 that includes another portion of that ID. Thereby, the receiver 8142 obtains a larger part (for example, “bc”) of the ID. Then, the receiver 8142 searches the ID list again for an ID including a part of the ID (for example, “bc”). By performing such a search, the receiver 8142 can specify all of the IDs even if only a part of the IDs can be acquired. When receiving a signal from the transmitter 8143, the receiver 8142 receives not only a part of the ID but also a check part such as CRC (Cyclic Redundancy Check).

  FIG. 62 is a diagram illustrating an example of processing operations of a receiver, a transmitter, and a server in Embodiment 3.

  For example, the receiver 8152 configured as a smartphone acquires position information indicating its own position. Note that the receiver 8152 acquires position information when, for example, GPS is used or other signals are received. Furthermore, the receiver 8152 receives a signal from a transmitter 8153 configured as, for example, a lighting device. At this time, the signal includes only a part of the ID (for example, “b”). Here, the receiver 8152 transmits the position information and a part of the ID to the server 8151.

  The server 8151 searches for an ID including a part of the ID from the ID list associated with the position indicated by the position information. If the unique ID is not found, the server 8151 notifies the receiver 8152 that the identification of the ID has failed.

  Next, the receiver 8152 receives a signal including another part of the ID from the transmitter 8153. Thereby, the receiver 8152 obtains a larger part (for example, “be”) of the ID. Then, the receiver 8152 transmits a part of the ID (for example, “be”) and the position information to the server 8151.

  The server 8151 searches for an ID including a part of the ID from the ID list associated with the position indicated by the position information. When the unique ID is found, the server 8151 notifies the receiver 8152 that the ID (for example, “abef”) has been specified, and transmits information associated with the ID to the receiver 8152.

  FIG. 63 is a diagram illustrating an example of processing operations of the receiver, the transmitter, and the server in Embodiment 3.

  The receiver 8152 may transmit not only a part of the ID but also all of the ID to the server 8151 together with the position information. At this time, if the ID of the complete state (for example, “wxyz”) is not included in the ID list, the server 8151 notifies the receiver 8152 of an error.

  FIG. 64A is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment.

  The transmitter 8155a and the transmitter 8155b transmit signals by changing the luminance. Here, the transmitter 8155a transmits a synchronization signal to the transmitter 8155b, whereby the luminance changes in synchronization with the transmitter 8155b. In addition, the transmitter 8155a and the transmitter 8155b each acquire a signal from the transmission source, and change the luminance according to the signal. Here, the time required for signal transmission from the transmission source to the transmitter 8155a (first delay time) may be different from the time required for signal transmission from the transmission source to the transmitter 8155b (second delay time). is there. Therefore, the round trip time of signals between the transmitters 8155a and 8155b and the transmission source is measured, and ½ of the round trip time is specified as the first or second delay time. The transmitter 8155a performs a luminance change synchronized with the transmitter 8155b by transmitting a synchronization signal such that the difference between the first and second delay times is canceled.

  FIG. 64B is an explanatory diagram for explaining synchronization of a plurality of transmitters in the third embodiment.

  The light receiving sensor 8156 detects light from the transmitter 8155a and the transmitter 8155b, and outputs the result as a detection signal to the transmitter 8155a and the transmitter 8155b. When the transmitter 8155a and the transmitter 8155b receive the detection signal from the light receiving sensor 8156, the transmitter 8155a and the transmitter 8155b perform luminance changes synchronized with each other based on the detection signal and adjust the intensity of the signal.

  FIG. 65 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  For example, the transmitter 8165 configured as a television acquires an image and an ID (ID 1000) associated with the image from the control unit 8166. The transmitter 8165 displays the image and transmits the ID (ID 1000) to the receiver 8167 by changing the luminance. The receiver 8167 receives the ID (ID 1000) by imaging and displays information associated with the ID (ID 1000). Here, the control unit 8166 changes the image output to the transmitter 8165 to another image. At this time, the control unit 8166 also changes the ID output to the transmitter 8165. That is, the control unit 8166 outputs the other ID (ID 1001) associated with the other image to the transmitter 8165 together with the other image. Thus, the transmitter 8165 displays another image and transmits another ID (ID 1001) to the receiver 8167 by changing the luminance. The receiver 8167 receives the other ID (ID 1001) by imaging and displays information associated with the other ID (ID 1001).

  FIG. 66 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  The transmitter 8170 is configured as a signage, for example, and switches and displays images. Then, when displaying the image, the transmitter 8170 transmits ID time information indicating the ID corresponding to the displayed image and the time when the image is displayed to the receiver 8171 by changing the luminance. For example, the transmitter 8170 displays an image showing a round graphic at time t1, and also displays an ID (ID: 1000) corresponding to the image and a time (TIME: t1) when the image is displayed. ID time information shown is transmitted.

  Here, the transmitter 8170 transmits not only the ID time information corresponding to the currently displayed image but also at least one ID time information corresponding to the image displayed in the past. For example, the transmitter 8170 displays an image showing a square graphic at time t2, and also displays an ID (ID: 1001) corresponding to the image and a time (TIME: t2) at which the image is displayed. ID time information shown is transmitted. Further, at this time, the transmitter 8170 transmits ID time information indicating the ID (ID: 1000) corresponding to the image showing the round graphic and the time (TIME: t1) when the image was displayed. Similarly, at time t3, the transmitter 8170 displays an image showing a triangular figure, ID corresponding to the image (ID: 1002), and time (TIME: t3) when the image is displayed. ID time information indicating is transmitted. Further, at this time, the transmitter 8170 transmits ID time information indicating the ID (ID: 1001) corresponding to the image indicating the square graphic and the time (TIME: t2) when the image was displayed. That is, the transmitter 8170 transmits a plurality of ID time information at the same timing.

  For example, in order to obtain information related to an image showing a square graphic, the user holds the image sensor of the receiver 8171 over the transmitter 8170 at time t2 when the image showing the square graphic is displayed. Imaging with 8171 is started.

  Here, even if the receiver 8171 starts imaging at time t2, the receiver 8171 cannot acquire ID time information corresponding to the image while the image indicating the square graphic is displayed on the transmitter 8170. There is a case. Even in such a case, as described above, since the ID time information corresponding to the image displayed in the past is also transmitted from the transmitter 8170, the receiver 8171 has a triangular shape at time t3. Not only ID time information (ID: 1002, TIME: t3) corresponding to an image showing a graphic but also ID time information (ID: 1001, TIME: t2) corresponding to an image showing a square graphic can be acquired. The receiver 8171 selects ID time information (ID: 1001, TIME: t2) indicating the time (t2) held over the transmitter 8170 from the ID time information, and the ID time information indicates the ID time information. The indicated ID (ID: 1001) is specified. Accordingly, the receiver 8171 can obtain information on an image showing a square graphic from a server or the like based on the specified ID (ID: 1001) at time t3.

  Note that the above-described time is not limited to an absolute time, and is a time between the time when the receiver 8171 is held over the transmitter 8170 and the time when the receiver 8171 acquires the ID time information (so-called relative time). It may be. The transmitter 8170 has transmitted ID time information corresponding to an image displayed in the past, together with ID time information corresponding to the currently displayed image, but corresponds to an image scheduled to be displayed in the future. ID time information may be transmitted. Further, the transmitter 8170 may increase the number of past or future ID time information to be transmitted when reception by the receiver 8171 is difficult.

  Further, when the transmitter 8170 is configured as a television instead of signage, the transmitter 8170 may transmit information indicating a channel corresponding to the displayed image instead of the ID time information. That is, when an image of a broadcast television program is displayed on the transmitter 8170 in real time, the display time of the image displayed on the transmitter 8170 can be uniquely specified for each channel. Therefore, the receiver 8171 specifies the time when the receiver 8171 is held over the transmitter 8170, that is, the time when the receiver 8171 starts imaging, based on the image obtained by imaging and the channel. Can do. The receiver 8171 can obtain information on an image obtained by imaging from, for example, a server based on the channel and the time. The transmitter 8170 may transmit information indicating the display time of the displayed image instead of the ID time information. In this case, the receiver 8171 searches for the TV program including the image obtained by imaging from all the TV programs broadcast at that time, and sets the channel of the TV program and its display time. Based on this, information related to the image can be obtained from a server or the like.

  FIG. 67 is a diagram illustrating an example of operation of a transmitter, a receiver, and a server in Embodiment 3.

  As shown in FIG. 67A, the receiver 8176 captures an image of the transmitter 8175 to acquire an image including a bright line, and specifies (acquires) the ID of the transmitter 8175 from the image. Furthermore, the receiver 8176 transmits the ID to the server 8177, and acquires information associated with the ID from the server 8177.

  On the other hand, as illustrated in FIG. 67B, the receiver 8176 may acquire an image including a bright line by capturing an image of the transmitter 8175 and transmit the image to the server 8177 as captured data. The receiver 8176 may perform preprocessing for an image including a bright line so that the amount of information of the image is reduced, and transmit the preprocessed image to the server 8177 as imaging data. This preprocessing is, for example, image binarization processing. When the server 8177 acquires the imaging data, the server 8177 identifies (acquires) the ID of the transmitter 8175 from the image indicated by the imaging data. Furthermore, the server 8177 transmits information associated with the ID to the receiver 8176.

  FIG. 68 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  When the user is at position A, the receiver 8183 identifies the position of the receiver 8183 by acquiring a signal transmitted from the transmitter 8181 that changes in luminance. As a result, the receiver 8183 displays the point 8183b indicating the specified position together with the error range 8183a of the position.

  Next, when the user moves from the position A and arrives at the position B, the receiver 8183 cannot acquire a signal from the transmitter 8181. At this time, the receiver 8183 estimates its own position by using a 9-axis sensor provided in the receiver 8183. Then, the receiver 8183 displays a point 8183b indicating the estimated position together with an error range 8183a of the position. At this time, since the position is estimated by the 9-axis sensor, the error range 8183a is displayed widely.

  Next, when the user moves from the position B and arrives at the position C, the receiver 8183 identifies the position of the receiver 8183 by acquiring a signal transmitted from another transmitter 8182 whose luminance changes. As a result, the receiver 8183 displays the point 8183b indicating the specified position together with the error range 8183a of the position. Here, the receiver 8183 smoothly displays the point 8183b indicating the position estimated using the 9-axis sensor and the error range 8183a without immediately switching to the position and error range specified as described above and displaying them. Move them to switch. At this time, the error range 8183a becomes small.

  FIG. 69 is a diagram illustrating an example of appearance of a receiver in Embodiment 3.

  The receiver 8183 is configured as, for example, a smartphone (high-function mobile phone), and as illustrated in FIG. ing. The image sensor 8183c acquires an image including a bright line by imaging the subject whose luminance changes as described above. The illuminance sensor 8183d detects the change in luminance of the subject described above. Therefore, the illuminance sensor 8183d can be used in place of the image sensor 8183c depending on the state or situation of the subject. The display 8183e displays an image or the like. Here, the receiver 8183 may have a function as a subject whose luminance changes. In this case, the receiver 8183 transmits a signal by changing the brightness of the display 8183e.

  As shown in FIG. 69B, an image sensor 8183f, an illuminance sensor 8183g, and a flash light emitting unit 8183h are arranged on the back surface of the receiver 8183. The image sensor 8183f is the same as the image sensor 8183c described above, and acquires an image including a bright line by imaging the subject whose luminance changes as described above. The illuminance sensor 8183g is the same as the illuminance sensor 8183d described above, and detects a change in luminance of the subject. Therefore, the illuminance sensor 8183g can be used in place of the image sensor 8183f depending on the state or situation of the subject. The flash light emitting unit 8183h emits a flash for imaging. Here, the receiver 8183 may have a function as a subject whose luminance changes. In this case, the receiver 8183 transmits a signal by changing the luminance of the flash light emitting unit 8183h.

  FIG. 70 is a diagram illustrating an example of operation of a transmitter, a receiver, and a server in Embodiment 3.

  For example, the transmitter 8185 configured as a smartphone displays information indicating “coupon 100 yen discount”, for example, by changing the luminance of the display 8185a except for the barcode portion 8185b, that is, by visible light communication. Send. The transmitter 8185 displays the barcode on the barcode portion 8185b without changing the luminance of the barcode portion 8185b. This bar code indicates the same information as the information transmitted by the visible light communication described above. Furthermore, the transmitter 8185 displays a character or a picture indicating information transmitted by visible light communication, for example, the character “coupon 100 yen discount” on a portion of the display 8185a excluding the barcode portion 8185b. By displaying such characters or pictures, the user of the transmitter 8185 can easily grasp what information is being transmitted.

  The receiver 8186 acquires the information transmitted by visible light communication and the information indicated by the barcode by imaging, and transmits the information to the server 8187. The server 8187 determines whether or not these pieces of information match or relate to each other. When it determines that these pieces of information match or relate to each other, the server 8187 executes processing according to the pieces of information. Alternatively, the server 8187 transmits the determination result to the receiver 8186, and causes the receiver 8186 to execute processing according to the information.

  The transmitter 8185 may transmit a part of information indicated by the barcode by visible light communication. The barcode may indicate the URL of the server 8187. Further, the transmitter 8185 may acquire information associated with the ID by acquiring the ID as a receiver and transmitting the ID to the server 8187. The information associated with this ID is the same as the information transmitted by the above visible light communication or the information indicated by the barcode. Further, the server 8187 may transmit an ID associated with information (visible light communication information or barcode information) transmitted from the transmitter 8185 via the receiver 8186 to the transmitter 8185.

  71 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3. FIG.

  For example, the transmitter 8185 configured as a smartphone transmits a signal by changing the luminance of the display 8185a. The receiver 8188 includes a light-shielding cone-shaped container 8188b and an illuminance sensor 8188a. The illuminance sensor 8188a is stored inside the container 8188b and is disposed near the tip of the container 8188b. When a signal is transmitted from the transmitter 8185 by visible light communication, the opening (bottom) of the container 8188b in the receiver 8188 is directed to the display 8185a. Thereby, since light other than the light from the display 8185a does not enter the container 8188b, the illuminance sensor 8188a of the receiver 8188 appropriately receives the light from the display 8185a without being affected by noise light. It can receive light. As a result, the receiver 8188 can appropriately receive the signal from the transmitter 8185.

  FIG. 72 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  The transmitter 8190 is configured, for example, as a sign tower at a bus stop, and transmits operation information indicating the operation status of the bus to the receiver 8183 by changing the luminance. For example, the destination of the bus, the time at which the destination bus arrives at the bus stop, and operation information indicating the current location of the bus are transmitted to the receiver 8183. When receiving the operation information, the receiver 8183 displays the content indicated by the operation information on the display.

  Here, for example, when buses with different destinations stop at the bus stop, the transmitter 8190 transmits operation information regarding the destination buses. When the receiver 8183 receives the operation information, the receiver 8183 selects the operation information of the destination bus frequently used by the user from the operation information, and displays the contents indicated by the operation information on the display. To do. Specifically, the receiver 8183 specifies the destination of the bus used by the user by using, for example, GPS, and records the destination history. By referring to this history, the receiver 8183 selects operation information of a destination bus frequently used by the user. Alternatively, the receiver 8183 may display the content indicated by the operation information selected by the user's operation from the operation information on the display. Alternatively, the receiver 8183 may preferentially display the operation information of the destination bus that is frequently selected by the user's operation.

  FIG. 73 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  For example, the transmitter 8191 configured as a signage transmits information on a plurality of stores to the receiver 8183 by changing the luminance. This information is a collection of information related to a plurality of stores, and is not unique to each store. Therefore, when the receiver 8183 receives the information by imaging, the receiver 8183 can display information on a plurality of stores as well as one store. Here, the receiver 8183 selects information regarding a store (for example, “B shop”) included in the imaging range from information regarding the plurality of stores, and displays the selected information. In addition, when displaying the information, the receiver 8183 translates a language for expressing the information into a pre-registered language, and displays the information in the translated language. Further, the transmitter 8191 may display a message prompting the user to take an image with the image sensor (camera) of the receiver 8183 in characters or the like. Specifically, when a dedicated application program is activated and imaging is performed with a camera, a message (for example, “GET information with camera”) indicating that information can be received is displayed on the transmitter 8191. The

  FIG. 74 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.

  For example, the receiver 8183 images a subject including a plurality of persons 8197 and street lamps 8195. The streetlight 8195 includes a transmitter 8195a that transmits information according to a change in luminance. By this imaging, the receiver 8183 acquires an image in which the image of the transmitter 8195a appears as the bright line pattern described above. Furthermore, the receiver 8183 acquires the AR object 8196a associated with the ID indicated by the bright line pattern from, for example, a server. Then, the receiver 8183 superimposes the AR object 8196a on a normal captured image 8196 obtained by normal imaging, and displays a normal captured image 8196 on which the AR object 8196a is superimposed.

  FIG. 75A is a diagram illustrating an example of a structure of information transmitted by a transmitter in Embodiment 3.

  For example, the information transmitted by the transmitter includes a preamble part, a fixed-length data part, and a check part. The receiver checks the data part using the check part, and normally receives information including these parts. Here, when the receiver receives the preamble part and the data part and fails to receive the check part, the check using the check part is omitted. Even in the case where such a check is omitted, the receiver can normally receive the information composed of these parts.

  FIG. 75B is a diagram illustrating another example of a structure of information transmitted by a transmitter in Embodiment 3.

  For example, the information transmitted by the transmitter includes a preamble part, a check part, and a variable length data part. The next information transmitted by the transmitter also includes a preamble part, a check part, and a variable length data part. Here, when the receiver receives the preamble part during reception and further receives the next preamble part, the information from the previous preamble part to immediately before the next preamble part has one meaning. Recognize as information. Further, the receiver may specify the end of the data part received next to the check part by using the check part. In this case, even if the receiver cannot receive the above-described next preamble part (all or a part of the preamble part), it appropriately receives the information that makes one meaning transmitted immediately before. be able to.

  FIG. 76 is a diagram illustrating an example of a 4-level PPM modulation scheme by a transmitter in Embodiment 3.

  The transmitter modulates a signal to be transmitted into a luminance change pattern by a four-value PPM modulation method. At this time, the transmitter can keep the brightness of the light whose luminance changes constant regardless of the signal to be transmitted.

  For example, when the brightness is kept at 75%, the transmitter transmits each of the signals to be transmitted “00”, “01”, “10”, and “11” to any one of four consecutive slots. One shows luminance L (Low), and the remaining three modulate to a luminance change pattern showing luminance H (High). Specifically, the transmitter transmits a signal “00” to be transmitted, a luminance change pattern (L, H, H, etc.) in which the first slot indicates luminance L and the second to fourth slots indicate luminance H. H). That is, in this luminance change, there is a rise in luminance between the first slot and the second slot. Similarly, the transmitter transmits a signal “01” to be transmitted, a luminance change pattern (H, H) in which the second slot indicates luminance L, and the first, third, and fourth slots indicate luminance H. L, H, H). That is, in this luminance change, there is a rise in luminance between the second slot and the third slot.

  When the brightness is maintained at 50%, the transmitter transmits each of the signals to be transmitted “00”, “01”, “10”, and “11” to any two of the four slots. The luminance L (Low) is indicated, and the remaining two are modulated into a luminance change pattern indicating luminance H (High). Specifically, the transmitter transmits a signal “00” to be transmitted, a luminance change pattern (L, L) in which the first and fourth slots indicate luminance L, and the second and third slots indicate luminance H. H, H, L). That is, in this luminance change, there is a rise in luminance between the first slot and the second slot. Similarly, the transmitter transmits a signal “01” to be transmitted, a luminance change pattern (L, L) in which the first and second slots indicate luminance L, and the third and fourth slots indicate luminance H. L, H, H). Alternatively, the transmitter transmits a signal “01” to be transmitted, a luminance change pattern (H, L, H) in which the second and fourth slots indicate luminance L and the first and third slots indicate luminance H. , L). That is, in these luminance changes, there is a rise in luminance between the second slot and the third slot.

  When the brightness is kept at 25%, the transmitter transmits each of the signals to be transmitted “00”, “01”, “10”, and “11” to any three of the four slots. The luminance L (Low) is indicated, and the remaining one is modulated into a luminance change pattern indicating the luminance H (High). Specifically, the transmitter transmits a signal “00” to be transmitted, a luminance change pattern (L, L) in which the first, third and fourth slots indicate luminance L, and the second slot indicates luminance H. H, L, L). That is, in this luminance change, there is a rise in luminance between the first slot and the second slot. Similarly, the transmitter transmits the signal “01” to be transmitted to a luminance change pattern (L, L, in which the first, second, and fourth slots indicate luminance L, and the third slot indicates luminance H). L, H, L). That is, in this luminance change, there is a rise in luminance between the second slot and the third slot.

  The transmitter can suppress flickering and can easily adjust the brightness step by step by the four-value PPM modulation method as described above. In addition, the receiver can appropriately demodulate the luminance change pattern by specifying the position where the luminance rises. Note that the receiver ignores the presence or absence of a rise in luminance at the boundary between the slot group consisting of four slots and the next slot group without using it for demodulating the luminance change pattern.

  77 is a diagram illustrating an example of a PPM modulation scheme by a transmitter in Embodiment 3. FIG.

  The transmitter modulates the signal to be transmitted into a luminance change pattern as in the 4-value PPM modulation method shown in FIG. 76, but performs PPM modulation without switching the luminance between L and H for each slot. Also good. That is, the transmitter performs PPM modulation by switching the rising edge of the luminance in the time width (hereinafter referred to as unit time width) of four consecutive slots shown in FIG. 76 according to the signal to be transmitted. For example, as shown in FIG. 77, the transmitter modulates the transmission target signal “00” into a luminance change pattern in which the luminance rises at a position of 25% of the unit time width. Similarly, as shown in FIG. 77, the transmitter modulates the signal to be transmitted “01” into a luminance change pattern in which the luminance rises at a position of 50% of the unit time width.

  Further, when the transmitter keeps the brightness at 75%, the transmission target signal “00” indicates the luminance L at the position of 0 to 25% in the above unit time width, and the position of 25 to 100%. To modulate to a luminance change pattern indicating luminance H. Here, when the transmitter keeps the brightness at 99%, the transmission target signal “00” indicates the luminance L at the position of 24 to 25% in the above unit time width, and is 0 to 24%. The pattern is modulated into a luminance change pattern indicating the luminance H at the position and the position of 25 to 100%. Similarly, when the transmitter keeps the brightness at 1%, the transmitter sets the signal “00” to be transmitted to the luminance L at the position of 0 to 25% and the position of 26 to 100% in the above unit time width. It is modulated into a luminance change pattern indicating luminance H at a position of 25 to 26%.

  In this way, the brightness can be continuously adjusted by switching the luminance to L and H at any position in the unit time width without switching the luminance to L and H for each slot.

  FIG. 78 is a diagram illustrating an example of a PPM modulation scheme in a transmitter in Embodiment 3.

  The transmitter performs modulation in the same manner as the PPM modulation method shown in FIG. 77. However, regardless of the signal to be transmitted, the transmitter always indicates luminance H at the beginning of the unit time width, And at the end of the unit time width, it is always modulated into a luminance change pattern indicating the luminance L. Thereby, since a rise in luminance occurs at the boundary between the unit time width and the next unit time width, the receiver can appropriately specify the boundary. Therefore, the receiver and the transmitter can correct the clock shift.

  FIG. 79A is a diagram showing an example of a luminance change pattern corresponding to the header (preamble portion) in the third embodiment.

  For example, when transmitting the header (preamble portion) shown in FIGS. 75A and 75B, the transmitter changes in luminance according to the pattern shown in FIG. 79A. That is, when the header is composed of 7 slots, the transmitter changes in luminance according to a pattern indicated by L, H, L, H, L, H, and H. When the header is composed of 8 slots, the transmitter changes in luminance according to a pattern indicated by H, L, H, L, H, L, H, and H. Since these patterns can be distinguished from the luminance change patterns shown in FIG. 76, it is possible to clearly notify the receiver that the signal indicated by these patterns is a header.

  FIG. 79B is a diagram showing an example of a luminance change pattern in Embodiment 3.

  As shown in FIG. 76, in the four-value PPM modulation method, when the signal “01” to be transmitted included in the data portion is modulated with the brightness maintained at 50%, the transmitter Is modulated into one of two patterns. That is, it modulates to the 1st pattern shown by L, L, H, and H, or the 2nd pattern shown by H, L, H, and L.

  Here, when the luminance change pattern corresponding to the header is a pattern as shown in FIG. 79A, the transmitter uses the first transmission signal “01” indicated by L, L, H, and H as described above. It is desirable to modulate the pattern. For example, the transmission target signal “11, 01, 11” included in the above-described data portion is “H, H, L, L, L, L, H, H, when the first pattern is used. , H, H, L, L ". On the other hand, when the second pattern is used, the transmission target signals “11, 01, 11” included in the data portion are “H, H, L, L, H, L, H, L , H, H, L, L ". In this case, the pattern “H, H, L, L, H, L, H, L, H, H, L, L” has the same pattern as the header pattern configured by 7 slots shown in FIG. 79A. Is included. Therefore, in order to clarify the distinction between the header and the data portion, it is desirable to modulate the above-described transmission target signal “01” into the first pattern.

  FIG. 80A is a diagram showing an example of a luminance change pattern in Embodiment 3.

  As shown in FIG. 76, in the quaternary PPM modulation method, when the signal “11” to be transmitted is modulated, the transmitter converts the signal to “H, H, H, so that no rise in luminance occurs. , L ”pattern,“ H, H, L, L ”pattern, or“ H, L, L, L ”pattern. However, as shown in FIG. 80A, the transmitter transmits the signal “11” to be transmitted to the pattern “H, H, H, H” or “L, L, L, L,” in order to adjust the brightness. May be modulated into a pattern.

  FIG. 80B is a diagram showing an example of a luminance change pattern in Embodiment 3.

  As shown in FIG. 76, in the 4-level PPM modulation method, when the signal “11,0” to be transmitted is modulated with the brightness maintained at 75%, the transmitter converts the signal to “H, H , H, L, L, H, H, H ”. However, when the luminance L is to be generated continuously, luminances other than the last luminance L among the continuous luminances L may be changed to H so that the luminance L does not continue. In this case, the transmitter modulates the signal “11.00” into a pattern of “H, H, H, H, L, H, H, H”.

  Thereby, since the luminance L is not continuous, the load on the transmitter can be suppressed. Moreover, the capacity | capacitance of the capacitor | condenser with which a transmitter is equipped can be made small, and the volume of a control circuit can be made small. Furthermore, since the load on the light source of the transmitter is small, the light source can be easily made. Furthermore, the power efficiency of the transmitter can be increased. Further, since it is guaranteed that the luminance L is not continuous, the receiver can easily demodulate the luminance change pattern.

(Summary of this embodiment)
The information communication method according to the present embodiment is an information communication method for transmitting a signal according to a luminance change, wherein a determination step for determining a luminance change pattern by modulating a signal to be transmitted and a light emitter are determined. A transmission step of transmitting the signal to be transmitted by changing the luminance according to the pattern, wherein the luminance change pattern includes two different luminances at arbitrary positions in a predetermined time width. In the pattern in which one of the values appears, in the determination step, the luminance change position that is the rising or falling position of the luminance in the time width is different for each of the different signals to be transmitted. In addition, the integrated value of the luminance of the luminous body in the time width is the same according to the preset brightness As a value to determine the pattern of the luminance change.

  For example, as shown in FIG. 77, for each of the signals “00”, “01”, “10”, and “11” that are different from each other, the luminance rising position (luminance change position) is different from each other The luminance change so that the integrated value of the luminance of the light emitter in a predetermined time width (unit time width) becomes the same value according to a predetermined brightness (for example, 99% or 1%). Pattern is determined. As a result, the brightness of the illuminant can be kept constant for each signal to be transmitted, flicker can be suppressed, and the receiver that images the illuminant is based on the luminance change position. The brightness change pattern can be demodulated appropriately. In addition, the luminance change pattern is a pattern in which one of two different luminance values (luminance H (High) or luminance L (Low)) appears at any position in the unit time width. The brightness of the body can be changed continuously.

  The information communication method further includes an image display step of sequentially switching and displaying each of the plurality of images. In the determination step, the displayed image is displayed each time the image is displayed in the image display step. The luminance change pattern for the identification information is determined by modulating the identification information corresponding to the signal to be transmitted. In the transmission step, the image is displayed every time the image is displayed in the image display step. The identification information may be transmitted by the luminance change of the light emitter according to the luminance change pattern determined with respect to the identification information corresponding to the existing image.

  As a result, for example, as shown in FIG. 65, each time an image is displayed, identification information corresponding to the displayed image is transmitted. Therefore, the user receives the received information on the receiver based on the displayed image. The identification information to be made can be easily selected.

  In the transmission step, each time an image is displayed in the image display step, the light emitter is further radiated according to a luminance change pattern determined for identification information corresponding to an image displayed in the past. The identification information may be transmitted by changing.

  As a result, for example, as shown in FIG. 66, even if the receiver cannot receive the identification signal transmitted before switching because the displayed image is switched, the identification information corresponding to the currently displayed image is displayed. At the same time, since the identification information corresponding to the images displayed in the past is also transmitted, the identification information transmitted before the switching can be properly received again by the receiver.

  In the determination step, each time an image is displayed in the image display step, the identification information corresponding to the displayed image and the time when the image is displayed are modulated as the signal to be transmitted. Thus, a pattern of luminance change with respect to the identification information and the time is determined, and in the transmission step, each time an image is displayed in the image display step, the identification information and time corresponding to the displayed image are determined. The identification information and the time are transmitted when the luminous body changes in luminance according to the luminance change pattern determined in the above, and the identification information and the time corresponding to an image displayed in the past are further determined. Even if the identification information and the time are transmitted by the luminance change of the light emitter according to the luminance change pattern. There.

  As a result, for example, as shown in FIG. 66, each time an image is displayed, a plurality of ID time information (information including identification information and time) is transmitted. From the information, an identification signal that has been transmitted in the past and cannot be received can be easily selected based on the time included in each of the ID time information.

  In addition, each of the light emitters has a plurality of regions that emit light, light in regions adjacent to each other among the plurality of regions interfere with each other, and only one of the plurality of regions is determined. When the luminance changes according to the luminance change pattern, in the transmission step, only the region arranged at the end of the plurality of regions may change in luminance according to the determined luminance change pattern.

  As a result, for example, as shown in FIG. 59B (a), only the region arranged at the end (light emitting portion) changes in luminance, so that only the region arranged other than the end changes in luminance. The influence of the light from the region on the luminance change can be suppressed. As a result, the receiver can appropriately capture the luminance change pattern by photographing.

  In addition, when only two of the plurality of regions change in luminance according to the determined luminance change pattern, the transmission step includes: a region disposed at an end of the plurality of regions; The area adjacent to the area arranged at the end may change in luminance according to the determined luminance change pattern.

  As a result, for example, as shown in FIG. 59B (b), the luminance changes between the region arranged at the end (light emitting portion) and the region adjacent to the region arranged at the end (light emitting portion). Compared to the case where the luminance changes in a distant area, the area of the range in which the luminance continuously changes can be kept large. As a result, the receiver can appropriately capture the luminance change pattern by photographing.

  The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, a location information transmission step for transmitting location information indicating a location of an image sensor used for photographing the subject, and the location information. A list receiving step for receiving an ID list including a plurality of pieces of identification information associated with the position indicated by, and an image obtained by photographing the subject by the image sensor corresponding to an exposure line included in the image sensor An exposure time setting step for setting an exposure time of the image sensor so that a bright line to be generated is generated according to a change in luminance of the subject, and the subject in which the image sensor changes in luminance is photographed at the set exposure time. An image acquisition step of acquiring a bright line image including the bright line, An information acquisition step of acquiring information by demodulating data specified by the bright line pattern included in the bright line image, and a search step of searching identification information including the acquired information from the ID list. .

  Thereby, as shown in FIG. 61, for example, since the ID list is received in advance, even if the acquired information “bc” is only a part of the identification information, the appropriate identification information “ abcd "can be specified.

  If the identification information including the acquired information is not uniquely specified in the search step, new information is acquired by repeatedly performing the image acquisition step and the information acquisition step, and the information communication The method may further include a re-search step of searching the ID list for identification information including the acquired information and the new information.

  Thereby, as shown in FIG. 61, for example, even when the acquired information “b” is only a part of the identification information and the identification information is not uniquely specified by the information alone, the new information “b” Since “c” is acquired, the appropriate identification information “abcd” can be specified based on the new information and the ID list.

  The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, and an bright line corresponding to an exposure line included in the image sensor is included in an image obtained by photographing the subject with an image sensor. An exposure time setting step for setting an exposure time of the image sensor so as to occur according to a change in luminance of the subject, and the image sensor photographing the subject whose luminance changes with the set exposure time, An image acquisition step of acquiring a bright line image including the bright line, an information acquisition step of acquiring identification information by demodulating data specified by the pattern of the bright line included in the acquired bright line image, and A transmission step of transmitting the identification information and position information indicating the position of the image sensor Error reception for receiving error notification information for notifying an error when there is no acquired identification information in an ID list that includes a plurality of identification information associated with the position indicated by the position information Steps.

  Thus, for example, as shown in FIG. 63, when the acquired identification information is not in the ID list, the error notification information is received, so that the user of the receiver that has received the error notification information has acquired the error notification information. It can be easily understood that information associated with the identification information cannot be obtained.

(Embodiment 4)
In the present embodiment, an application example for each situation using a receiver such as a smartphone in the above first to third embodiments and a transmitter that transmits information as a blinking pattern such as an LED or an organic EL will be described.

<Situation: In front of the store>
First, an application example in a situation where there is a user carrying a receiver in front of a store that has a billboard for advertisement configured as a transmitter will be described with reference to FIGS.

  FIG. 81 is a diagram illustrating an example of the operation of the receiver in the situation in front of the store.

  For example, a user finds a store sign 8301 when walking while carrying a receiver 8300 (terminal device) configured as a smartphone. The sign 8301 is a transmitter (subject) that transmits a signal by luminance change like the transmitter of any of the first to third embodiments. Here, when the user is interested in the store and determines that the sign 8301 is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to perform the visible light communication of the receiver 8300. Application software (hereinafter referred to as communication application).

  FIG. 82 is a diagram illustrating another example of operation of the receiver 8300 in the store front situation.

  The receiver 8300 may automatically start the communication application without accepting an operation by the user. For example, the receiver 8300 detects its current location by using a GPS or a 9-axis sensor, and determines whether or not the current location has entered a specific area predetermined for the signboard 8301. This specific area is an area around the signboard 8301. When the receiver 8300 determines that the current location of the receiver 8300 has entered the specific area, the receiver 8300 activates the communication application. Further, the receiver 8300 may detect an operation of projecting the receiver 8300 or an operation of rotating the receiver 8300 by using a built-in 9-axis sensor or the like, and may activate a communication application. Thereby, a user's operation can be omitted and usability can be improved.

  FIG. 83 is a diagram illustrating an example of next operation of the receiver 8300 in the store front situation.

  The receiver 8300 that has activated the communication application as described above captures an image (visible light imaging) of the signboard 8301 configured as a transmitter that transmits a signal according to a change in luminance. That is, the receiver 8300 performs visible light communication with the sign 8301.

  FIG. 84 is a diagram illustrating an example of next operation of the receiver 8300 in the situation in front of the store.

  The receiver 8300 acquires an image including a bright line by the above-described imaging. Then, the receiver 8300 acquires the device ID of the sign 8301 by demodulating the data specified by the bright line pattern. That is, the receiver 8300 acquires the device ID from the signboard 8301 by visible light imaging or visible light communication in the first to third embodiments. Furthermore, the receiver 8300 transmits the device ID to the server, and acquires advertisement information (service information) associated with the device ID from the server.

  Note that the receiver 8300 may acquire advertisement information associated with the device ID from a plurality of advertisement information held in advance. In this case, when the receiver 8300 determines that the current location of the receiver 8300 has entered the specific area, the receiver 8300 notifies the server of the specific area or the current position, and all the device IDs corresponding to the specific area and The advertisement information associated with each of the device IDs is acquired in advance from the server and held (cached). Thus, when the receiver 8300 acquires the device ID of the signboard 8301 within the specific area, the receiver 8300 associates the advertisement information corresponding to the device ID with each device ID held in advance without requesting the server. The advertisement information associated with the device ID of the sign 8301 can be quickly acquired from the advertisement information.

  When the receiver 8300 acquires the advertisement information associated with the device ID of the signboard 8301, the receiver 8300 displays the advertisement information. For example, the receiver 8300 displays the coupon of the store indicated by the sign 8301, the vacant seat status, and a bar code indicating the same content as them.

  Here, the receiver 8300 may acquire not only the device ID but also the privilege data from the signboard 8301 by visible light communication. The privilege data indicates, for example, a random ID (random number) or a time or a time zone when the privilege data is transmitted. When receiving the privilege data, the receiver 8300 transmits the privilege data together with the device ID to the server. Then, the receiver 8300 acquires advertisement information associated with the device ID and privilege data from the server. Thereby, the receiver 8300 can receive different advertising information according to privilege data. For example, if the time zone when the signboard 8301 is imaged is early morning, the receiver 8300 can acquire and display advertisement information indicating an early morning discount coupon. That is, the advertisement according to the same signboard can be changed according to privilege data (such as time zone). As a result, the user can receive a service suitable for a time zone. In the present embodiment, presentation (display) of information such as service information to a user is referred to as service provision.

  The receiver 8300 may acquire three-dimensional information indicating the spatial arrangement of the sign 8301 with high accuracy (within an error of 1 m) from the sign 8301 together with the device ID by visible light communication. Alternatively, the receiver 8300 may acquire three-dimensional information associated with the device ID from the server. In addition, the receiver 8300 may acquire size information indicating the size of the sign 8301 instead of or together with the three-dimensional information. When the receiver 8300 acquires the size information, the receiver 8300 receives the size information from the receiver 8300 based on the difference between the size of the sign 8301 indicated by the size information and the size of the sign 8301 displayed in the image obtained by imaging. A distance up to 8301 can be calculated.

  Further, when transmitting the device ID acquired by visible light communication to the server, the receiver 8300 may transmit the holding information (attached information) held in advance by itself to the server together with the device ID. For example, the retained information is personal information (such as gender or age) of the user of the receiver 8300 or a user ID. The server that has received such retained information together with the device ID receives the advertisement information associated with the retained information (personal information or user ID) from at least one piece of advertisement information associated with the device ID. Send to. That is, the receiver 8300 can receive store advertisement information that matches personal information, store information corresponding to the user ID, and the like. As a result, the user can be provided with a more useful service.

  Alternatively, the holding information indicates reception conditions set in advance in the receiver 8300. This reception condition is, for example, the number of visitors when the store is a restaurant. The server that has received such retained information together with the device ID transmits, to at least one piece of advertisement information associated with the device ID, advertisement information associated with the reception condition (number of visitors) to the receiver 8300. . That is, the receiver 8300 can receive the advertisement information of the store that matches the number of visitors, specifically, information indicating the vacant seat status for the number of visitors. In addition, the store can display the advertisement information with the discount rate changed according to the number of customers, the day of the week, and the time zone, thereby optimizing customer attraction and profit.

  Alternatively, the holding information indicates the current location detected in advance by the receiver 8300. The server that has received such retained information together with the device ID is not limited to the advertisement information associated with the device ID, but also other device IDs corresponding to the current location (current location and its surroundings) indicated by the retained information, and others. The advertisement information associated with the device ID is transmitted to the receiver 8300. Thereby, the receiver 8300 can cache the other device ID and the advertisement information associated with the other device ID. Therefore, when the receiver 8300 performs visible light communication with another transmitter at the current location (current location and its surroundings), the advertisement associated with the device ID of the other transmitter without accessing the server. Information can be acquired quickly.

  FIG. 85 is a diagram illustrating an example of next operation of the receiver 8300 in the store front situation.

  When the receiver 8300 acquires the advertisement information from the server as described above, for example, the receiver 8300 displays a button described as “vacant seats” as the vacant seat status indicated by the advertisement information. Here, when the user performs an operation of touching the button “available seat” with the finger, the receiver 8300 notifies the server of the operation result. Upon receiving the notification, the server makes a temporary reservation for the store of the signboard 8301 and notifies the receiver 8300 that the temporary reservation has been completed. Upon receiving the notification from the server, the receiver 8300 displays a character string “temporary reservation” indicating that the temporary reservation has been completed, instead of the button “vacant seats”. The receiver 8300 stores, as a pre-acquired image, an image including a store coupon indicated by the signboard 8301, a character string “provisional reservation” certifying that the provisional reservation has been made, and a bar code indicating the same content as the pre-acquired image. Store it.

  Here, the server can log information related to visible light communication performed between the sign 8301 and the receiver 8300 by the operation described with reference to FIGS. 84 and 85. That is, the server provides privilege data indicating the device ID of the transmitter (signboard) that performed visible light communication, the location where visible light communication was performed (current location of the receiver 8300), the time zone during which visible light communication was performed, and the like. And personal information of the user of the receiver 8300 that performed visible light communication can be logged. The server can analyze the value of the sign 8301, that is, the degree of contribution of the sign 8301 to the advertisement and promotion of the store as an advertising effect using at least one of these pieces of logged information.

<Situation: In-store>
Next, application examples in situations where the user carrying the receiver 8300 enters a store corresponding to the displayed advertisement information (service information) will be described with reference to FIGS. 86 to 94.

  FIG. 86 is a diagram showing an example of the operation of the display device in the in-store situation.

  For example, the user of the receiver 8300 that has performed visible light communication with the above-described sign 8301 enters a store corresponding to the displayed advertisement information. At this time, the receiver 8300 detects that the user has entered the store corresponding to the advertisement information displayed using visible light communication (entering the store). For example, after performing visible light communication with the sign 8301, the receiver 8300 acquires store information indicating the store location associated with the device ID of the sign 8301 from the server. Then, the receiver 8300 determines whether or not the current location of the receiver 8300 obtained by using a GPS or a 9-axis sensor has entered the store location indicated by the store information. Here, the receiver 8300 detects the above-mentioned entrance by determining that the current location has entered the store location.

  Then, when the receiver 8300 detects the entry, the receiver 8300 notifies the display device 8300b of the entry via, for example, a server. Alternatively, the receiver 8300 notifies the display device 8300b that the store has entered by visible light communication or wireless communication. Upon receiving the notification, the display device 8300b acquires product service information indicating a product or service menu provided at the store, and displays the menu indicated by the product service information. Note that the display device 8300b may be a portable terminal carried by a user of the receiver 8300 or a store clerk, or a device provided in the store.

  FIG. 87 is a diagram illustrating an example of the next operation of the display device 8300b in the in-store situation.

  The user selects a desired product from the menu displayed on the display device 8300b. In other words, the user performs an operation of touching a part of the menu where the name of the desired product is displayed. Thereby, the display device 8300b receives the operation result of the product selection.

  FIG. 88 is a diagram illustrating an example of the next operation of the display device 8300b in the in-store situation.

  The display device 8300b that has received the operation result of the product selection displays an image representing the selected product and the price of the product. As a result, the display device 8300b prompts the user to confirm the selected product. In addition, the image showing the above-mentioned goods, the information which shows the price of goods, etc. are contained in the above-mentioned goods service information, for example.

  FIG. 89 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The user who is prompted for confirmation performs an operation for ordering the product. When the operation is performed, the receiver 8300 notifies the POS (Pint of Sale) system of the store via the display device 8300b or the server of payment information necessary for electronic payment. Furthermore, the receiver 8300 determines whether or not there is the above-described pre-acquired image acquired and stored using visible light communication with the sign 8301 of the store. When the receiver 8300 determines that there is the pre-acquired image, the receiver 8300 displays the pre-acquired image.

  Note that although the display device 8300b is used in this situation, the receiver 8300 may perform processing by the display device 8300b instead of using the display device 8300b. In this case, when the receiver 8300 detects entry, the receiver 8300 acquires product service information indicating a product or service menu provided at the store from the server, and displays the menu indicated by the product service information. Further, when receiving an operation for ordering a product, the receiver 8300 notifies the POS system of the store of the ordered product and payment information necessary for electronic payment via the server.

  FIG. 90 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The store clerk applies the barcode scanner 8302 of the POS system to the barcode of the pre-acquired image displayed on the receiver 8300. The barcode scanner 8302 reads the barcode of the pre-acquired image. As a result, the POS system performs electronic settlement according to the coupon indicated by the barcode. Then, the barcode scanner 8302 of the POS system transmits payment completion information indicating that the electronic payment is completed to the receiver 8300 by the luminance change. That is, the barcode scanner 8302 also has a function as a transmitter for visible light communication. When the payment completion information is acquired by visible light communication, the receiver 8300 displays the payment completion information. This settlement completion information indicates, for example, a message “Thank you for your purchase” and the amount of the settlement. By performing such an electronic payment, the POS system, the server, and the receiver 8300 allow the service indicated by the advertisement information at the store corresponding to the advertisement information (service information) displayed in front of the store. It can be determined that it has been used.

  As described above, merchandise is ordered in the store by the operations of the receiver 8300 and the POS system as shown in FIGS. Therefore, when the user enters the store, the user can place an order for a product from the store menu that is automatically displayed on the display device 8300b or the receiver 8300. That is, the store clerk does not need to show the menu to the user and directly accept an order for the product from the user. As a result, the burden on the store clerk can be greatly reduced. In the above example, the barcode scanner 8302 reads the barcode, but the barcode scanner 8302 may not be used. For example, the receiver 8300 may transmit information indicated by the barcode to the POS system via the server. Then, the receiver 8300 may acquire the payment completion information from the POS system via the server. Thereby, the work by the store clerk can be further reduced, and the user can place an order for the product without going through the store clerk. Alternatively, the display device 8300b and the receiver 8300 may exchange ordering and billing data by visible light communication, or the data may be exchanged by wireless communication using a key exchanged by visible light communication.

  In addition, the sign 8301 may be put out by one of a plurality of stores belonging to the chain store. In such a case, the advertisement information acquired using visible light communication with the sign 8301 can be used at all stores belonging to the chain store. However, even in the same chain store, the service received by the user may be different between a store (advertisement store) where a signboard 8301 is displayed and a store (non-advertisement store) where the signboard 8301 is not displayed. For example, when a user enters a non-advertising store, the user receives a service with a discount rate (for example, 20%) according to the coupon indicated in the pre-acquired image. Receive a service with a discount rate (for example, 30%) higher than the coupon discount rate. That is, when the receiver 8300 detects entering the advertising store, the receiver 8300 acquires additional service information indicating a further discount of 10% from the server, and a discount rate of 30% (20% + 10%). Is displayed instead of the pre-acquired image shown in FIG. The receiver 8300 detects whether the user has entered an advertising store or a non-advertising store based on the above-described store information acquired from the server. The store information indicates the location of each of a plurality of stores belonging to the chain store and whether the stores are advertising stores or non-advertising stores.

  Moreover, when there are a plurality of non-advertisement stores in the same chain store, a difference may be provided in the service received by the user in each non-advertisement store. For example, a service corresponding to the distance from the current location of the receiver 8300 when performing visible light communication with the sign 8301 to the non-advertisement store is provided to the user who entered the non-advertisement store Is done. Alternatively, a service corresponding to the difference (time difference) between the time when the receiver 8300 and the sign 8301 perform visible light communication and the time when the user enters the non-advertisement store is provided to the user who enters the non-advertisement store. Is done. That is, the receiver 8300 acquires additional service information indicating a further discount, which differs depending on the above-described distance (the position of the signboard 8301) and the time difference, from the server, and a discount rate (for example, 30) reflecting the further discount. %) Is displayed instead of the pre-acquired image shown in FIG. Such a service is determined by the server or the POS system or by mutual cooperation. In addition, such a service may be applied to all stores belonging to the chain store, regardless of whether the store is an advertisement store or a non-advertisement store.

  When a user enters a non-advertisement store and places an order using advertisement information, the POS system at the non-notification store returns a portion of the amount obtained by the order to the POS system at the advertisement store. May be.

  Further, each time the advertisement information is displayed, the server can determine whether or not the advertisement information has been used, and analyze the advertisement effect of the sign 8301 by accumulating the determined results. Can do. The server further includes at least one of the position of the sign 8301, the time when the advertisement information is displayed, the position of the store where the advertisement information is used, the time when the advertisement information is used, and the time when the user enters the store. By collecting the two, it is possible to improve the analysis accuracy of the advertising effect of the sign 8301 and to find the position of the sign 8301 having the highest advertising effect.

  In addition, the receiver 8300 acquires additional service information indicating a further discount for the number of uses for which the advertisement information has been used for the product order from the server, and a discount rate that reflects the further discount for the number of uses ( For example, an image showing 30%) may be displayed instead of the pre-acquired image shown in FIG. For example, the server may provide a service for further increasing the discount rate in cooperation with the POS system if the number of uses is large.

  In addition, the server receives the advertisement information associated with each of the device IDs of all the signboards 8301 provided by the store by the receiver 8300 (when acquisition of all advertisement information is completed). The profitable service may be provided to the user who enters the store of the signboard 8301. The profitable service is, for example, a service with a very high discount rate or a service that provides products other than ordered products free of charge. That is, when the receiver 8300 detects the user's entry, the server determines whether the receiver 8300 has performed processing including visible light communication or the like for each of all the signboards associated with the store. judge. If it is determined that the processing has been performed, the receiver 8300 acquires additional service information indicating further discount from the server as the above-mentioned profitable service, and a discount rate that reflects the further discount ( For example, an image indicating 50%) is displayed instead of the pre-acquired image illustrated in FIG.

  Further, the receiver 8300 receives additional service information indicating a further discount rate that differs depending on the difference between the time when the advertisement information is displayed by performing visible light communication with the sign 8301 and the time when the user enters the store. An image obtained from the server and showing a discount rate (for example, 30%) reflecting a further discount may be displayed instead of the pre-acquired image shown in FIG. For example, the receiver 8300 acquires additional service information indicating a higher discount rate as the difference is smaller from the server.

  FIG. 91 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The receiver 8300 that has completed the order and the electronic payment receives a signal transmitted by a luminance change from a transmitter configured as a lighting device in the store, and transmits the signal to the server, whereby the seat position of the user ( For example, a guide map in a store indicating a black circle) is acquired. Furthermore, the receiver 8300 specifies the position of the receiver 8300 using the received signal, as in any of the first to third embodiments. Then, the receiver 8300 displays the specified position of the receiver 8300 (for example, an asterisk) in the guide map. Thereby, the user can easily grasp how the user can go to his / her seat in the store.

  In addition, the receiver 8300 identifies the position of the receiver 8300 as described above by performing visible light communication with a nearby transmitter configured as a lighting device in the store even when the user is moving. Is going from time to time. Therefore, the receiver 8300 sequentially updates the position (for example, star) of the displayed receiver 8300. Thereby, a user can be appropriately guided to a seat.

  FIG. 92 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  When the user arrives at the seat, the receiver 8300 identifies the position of the receiver 8300 by performing visible light communication with a transmitter 8303 configured as a lighting device, and determines that the position is at the seat position of the user. To do. Then, the receiver 8300 notifies the terminal in the store that the user has arrived at the seat together with the user name or nickname via the server. Thereby, the salesclerk can grasp which user is sitting in which seat.

  FIG. 93 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The transmitter 8303 transmits a signal including a customer ID and a message notifying that an order item has been made by changing the luminance. Note that the receiver 8300 acquires the above-described customer ID from the server and holds the product ID, for example, when acquiring the product service information indicating the menu of the product from the server. The receiver 8300 receives the above-described signal by photographing the transmitter 8303 with visible light. Furthermore, the receiver 8300 determines whether or not the customer ID included in the signal matches a customer ID that is held in advance. Here, when the receiver 8300 determines that they match, the receiver 8300 displays a message (for example, “Product is ready”) included in the signal.

  FIG. 94 is a diagram illustrating an example of next operation of the receiver 8300 in the in-store situation.

  The clerk who delivered the ordered product to the user's seat points the handy terminal 8302a to the receiver 8300 in order to prove that the ordered product has been delivered. The handy terminal 8302a is configured as a transmitter, and transmits a signal indicating that the ordered product has been delivered to the receiver 8300 by a change in luminance. The receiver 8300 receives the signal by imaging the handy terminal 8302a, and displays a message (for example, “Please enjoy your meal”) indicated by the signal.

<Situation: Finding a store>
Next, an application example in a situation where a user carrying the receiver 8300 is searching for a store of interest will be described with reference to FIGS. 95 to 97.

  FIG. 95 is a diagram illustrating an example of operation of the receiver 8300 in the store search situation.

  The user finds a signage 8304 that lists restaurants of interest. At this time, if the user determines that the signage 8304 is transmitting a signal due to a change in luminance, the communication application of the receiver 8300 is activated by operating the receiver 8300 as in the example shown in FIG. Let Note that, similarly to the example illustrated in FIG. 82, the receiver 8300 may automatically start the communication application.

  FIG. 96 is a diagram illustrating an example of next operation of the receiver 8300 in the store search situation.

  The receiver 8300 obtains an ID for identifying the signage 8304 or the restaurant by imaging the entire signage 8304 or a portion of the signage 8304 where a restaurant in which the user is interested is posted. Receive.

  FIG. 97 is a diagram illustrating an example of next operation of the receiver 8300 in the store search situation.

  When receiving the above-mentioned ID, the receiver 8300 transmits the ID to the server, acquires advertisement information (service information) associated with the ID from the server, and displays it. At this time, the receiver 8300 may notify the server of the number of persons (attached information) scheduled to enter the restaurant together with the ID. Thereby, the receiver 8300 can acquire the advertisement information according to the number of people. For example, the receiver 8300 can acquire advertisement information indicating whether or not there are vacant seats for the notified number of people in the restaurant.

<Situation: Movie advertisement>
Next, application examples in situations where a user carrying the receiver 8300 is in front of a signage on which an interesting movie advertisement is posted will be described with reference to FIGS. 98 to 101.

  FIG. 98 is a diagram illustrating an example of operation of the receiver 8300 in the movie advertisement situation.

  The user finds a signage 8305 on which a movie advertisement of interest is posted and a signage 8306 configured as, for example, a liquid crystal display and displaying a moving image for movie advertisement. The signage 8305 includes, for example, a transparent film on which an image showing a movie advertisement is drawn, and a plurality of LEDs that are arranged on the back side of the film and illuminate the film. That is, the signage 8305 displays the image drawn on the film brightly as a still image by the light emission of the plurality of LEDs. The signage 8305 is configured as a transmitter that transmits a signal by changing luminance.

  Here, if the user determines that the signage 8305 is transmitting a signal due to a change in luminance, the communication application of the receiver 8300 is activated by operating the receiver 8300 as in the example shown in FIG. Let Note that, similarly to the example illustrated in FIG. 82, the receiver 8300 may automatically start the communication application.

  FIG. 99 is a diagram illustrating an example of next operation of the receiver 8300 in the movie advertisement situation.

  The receiver 8300 acquires the ID of the signage 8305 by imaging the signage 8305. Then, the receiver 8300 transmits the ID to the server, downloads the moving image data for movie advertisement associated with the ID as service information from the server, and reproduces it.

  FIG. 100 is a diagram illustrating an example of next operation of the receiver 8300 in the movie advertisement situation.

  The moving image displayed by reproducing the moving image data downloaded as described above is the same as the moving image displayed by the signage 8306, for example. Therefore, when the user wants to see a movie advertisement moving image, the user can view the moving image at an arbitrary place without stopping in front of the signage 8306.

  FIG. 101 is a diagram illustrating an example of next operation of the receiver 8300 in the movie advertisement situation.

  The receiver 8300 may download not only the moving image data but also the moving image data together with the moving image data showing the screening information indicating the movie showing time as service information. Thus, the receiver 8300 can display the contents of the screening information and notify the user, and can share the screening information with other terminals (such as other smartphones).

<Situation: Museum>
Next, an application example in a situation where the user carrying the receiver 8300 enters the museum and appreciates the exhibits in the hall will be described with reference to FIGS.

  FIG. 102 is a diagram illustrating an example of operation of the receiver 8300 in the museum situation.

  For example, when a user tries to enter a museum, the user finds a guidance bulletin board 8307 hung at the entrance of the museum. At this time, if the user determines that the guidance bulletin board 8307 is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to change the communication application of the receiver 8300 as in the example shown in FIG. Start. Note that, similarly to the example illustrated in FIG. 82, the receiver 8300 may automatically start the communication application.

  FIG. 103 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  The receiver 8300 acquires the ID of the guidance bulletin board 8307 by imaging the guidance bulletin board 8307. Then, the receiver 8300 transmits the ID to the server, downloads the museum guidance application program (hereinafter referred to as the museum application) from the server as service information associated with the ID, and activates it.

  FIG. 104 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  When the museum application is activated, the receiver 8300 displays a guide map in the museum according to the museum application. Furthermore, the receiver 8300 specifies the position of the receiver 8300 in the museum as in any of Embodiments 1 to 3. Then, the receiver 8300 displays the specified position of the receiver 8300 (for example, an asterisk) in the guide map.

  In order to specify the position as described above, the receiver 8300 acquires form information indicating the size and shape of the guidance bulletin board 8307 from the server, for example, when downloading a museum application. The receiver 8300 then performs triangulation based on the size and shape of the guidance bulletin board 8307 indicated by the form information and the size and shape of the guidance bulletin board 8307 displayed on the image obtained by the above imaging. The relative position of the receiver 8300 with respect to the guidance bulletin board 8307 is specified in accordance with the above method.

  FIG. 105 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  As described above, when the user enters the museum, the receiver 8300 that has activated the museum application performs visible light communication with a nearby transmitter configured as a lighting device in the museum, and thus the receiver 8300 The position is specified at any time. For example, the receiver 8300 acquires the ID of the transmitter 8308 from the transmitter 8308 by capturing an image of the transmitter 8308 configured as a lighting device. Then, the receiver 8300 acquires position information indicating the position of the transmitter 8308 and form information indicating the size and shape of the transmitter 8308 associated with the ID from the server. The receiver 8300 then performs triangulation based on the size and shape of the transmitter 8308 indicated by the form information and the size and shape of the transmitter 8308 displayed in the image obtained by the above imaging. The relative position of the receiver 8300 with respect to the transmitter 8308 is estimated according to the above method. In addition, the receiver 8300 determines whether the receiver 8300 is in the museum based on the position of the transmitter 8308 indicated by the position information acquired from the server and the relative position of the receiver 8300 estimated as described above. Identify the location.

  Then, every time the position of the receiver 8300 is specified, the receiver 8300 moves the displayed star to the latest specified position. As a result, a user in the museum can easily grasp where he / she is in the museum by looking at the guide map and the star displayed on the receiver 8300.

  FIG. 106 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  When the user who has entered the museum finds an exhibit 8309 of interest, the user performs an operation of holding the receiver 8300 over the exhibit 8309 so that the receiver 8300 can capture an image of the exhibit 8309. Here, the exhibit 8309 is illuminated by light from the lighting device 8310. The lighting device 8310 is used exclusively for the exhibit 8309, and is configured as a transmitter that transmits a signal according to a change in luminance. Therefore, the exhibit 8309 is illuminated by light that changes in luminance, and indirectly transmits a signal from the lighting device 8310.

  When the receiver 8300 detects an operation of holding the receiver 8300 over the exhibit 8309 based on, for example, an output from a built-in 9-axis sensor, the receiver 8300 captures the signal from the lighting device 8310 by imaging the exhibit 8309. Receive. This signal indicates, for example, the ID of the exhibit 8309. Then, the receiver 8300 acquires the introduction information (service information) of the exhibit 8309 associated with the ID from the server. This introduction information shows a diagram for introducing the exhibit 8309, and the introduction text is shown in the language of each country such as Japanese, English, and French.

  When the introduction information is acquired from the server, the receiver 8300 displays a diagram and a text indicated by the introduction information. Here, when displaying a sentence, the receiver 8300 extracts a sentence in a language preset by the user from sentences in the language of each country, and displays only the sentence in that language. The receiver 8300 may change the language by a selection operation by the user.

  FIG. 107 is a diagram illustrating an example of next operation of the receiver 8300 in the museum situation.

  When the display of the introduction information figure and the text is terminated by the user's operation, the receiver 8300 again performs visible light communication with a nearby transmitter configured as a lighting device (for example, the lighting device 8311). Then, the position of the receiver 8300 is specified. Then, when the new position of the receiver 8300 is specified, the receiver 8300 moves the displayed star to the specified new position. Accordingly, the user who has viewed the exhibition 8309 can easily move to the next exhibition to view by viewing the guide map and the star displayed on the receiver 8300.

<Situation: Bus stop>
Next, an application example in a situation where the user carrying the receiver 8300 is at a bus stop will be described with reference to FIGS.

  FIG. 108 is a diagram illustrating an example of operation of the receiver 8300 in a bus stop situation.

  The user goes to a bus stop, for example, to get on a bus. Therefore, when the user determines that the sign tower 8312 at the bus stop is transmitting a signal due to a change in luminance, the user operates the receiver 8300 to perform the operation of the receiver 8300 as in the example shown in FIG. Start the communication application. Note that, similarly to the example illustrated in FIG. 82, the receiver 8300 may automatically start the communication application.

  FIG. 109 is a diagram illustrating an example of next operation of the receiver 8300 in the bus stop situation.

  The receiver 8300 images the sign tower 8312 to obtain the ID of the bus stop where the sign tower 8312 is located. Then, the receiver 8300 transmits the ID to the server, and acquires operation status information associated with the ID from the server. The operation status information is information indicating traffic conditions, and is service information indicating services provided to the user.

  Here, the server manages the operation status of these buses by collecting information from each bus operating in the area including the bus stop. Therefore, when the server acquires the ID of the bus stop from the receiver 8300, the server estimates the time until the bus arrives at the bus stop of that ID based on the managed operation status, and the estimated time Is transmitted to the receiver 8300.

  The receiver 8300 that has acquired the operation status information displays the time indicated by the operation status information, for example, “10 minutes after arrival”. Thereby, the user can grasp | ascertain easily the operation condition of a bus | bath.

(Supplement)
When the scanning direction on the imaging side is the vertical direction (vertical direction) of the mobile terminal, when imaging is performed with a short exposure time, the ON / OFF state of the entire LED illumination device is as shown in FIG. In addition, it is possible to image bright lines that are white and black patterns in the same direction as the scanning direction. In (a) of FIG. 110, since the long side direction of the vertically long LED lighting device is imaged so as to be perpendicular to the scanning direction on the imaging side (left and right direction of the portable terminal), the same direction as the scanning direction In addition, a large number of white / black bright lines can be imaged. That is, the amount of information that can be transmitted and received can be increased. On the other hand, as shown in FIG. 110B, when a vertically long LED illumination device is imaged so as to be parallel to the scanning direction on the imaging side (vertical direction of the portable terminal), the white / black pattern that can be imaged There are fewer bright lines. That is, the amount of information that can be transmitted is reduced.

  As described above, when a large number of white and black pattern bright lines can be imaged depending on the orientation of the LED illumination device with respect to the scanning direction on the imaging side (the long side direction of the vertically long LED illumination device is perpendicular to the scanning direction on the imaging side And a case where only a few white / black pattern bright lines can be imaged (when the long side direction of the vertically long LED illumination device is made parallel to the scanning direction on the imaging side). Arise.

  In the present embodiment, a description will be given of a control method of an illumination device that can image a large number of bright lines even when only a small number of bright lines of white / black patterns can be captured.

  FIG. 111 shows an example of a lighting device in which a plurality of LEDs are arranged in the vertical direction and a drive signal thereof. FIG. 111A shows an illumination device in which a plurality of LEDs are arranged in the vertical direction. It is assumed that each LED element corresponds to a minimum unit of horizontal stripes that encodes a visible light communication signal and corresponds to an encoded ON / OFF signal. In this way, by generating a white / black pattern, turning on / off each LED element, and illuminating, it is assumed that the scanning direction on the imaging side and the long side direction of the vertically long LED illumination device are parallel The white / black pattern of the LED element unit can be photographed.

  111 (c) and (d) show an example in which a white / black pattern is generated and each LED element is turned on / off for illumination. When the illumination device illuminates as a white / black pattern, the light may be uneven even for a short time. Therefore, an example in which an antiphase pattern is generated and alternately illuminated is shown in (c) and (d) of FIG. 111 (c), the element that was turned on is turned off in FIG. 111 (d), and the element that was turned off in FIG. 111 (c) is turned off in FIG. In d), it is ON. In this way, by illuminating the white / black pattern alternately with the positive phase pattern and the reverse phase pattern one after the other, without causing light unevenness, the scanning direction on the imaging side and the illumination A lot of information can be transmitted and received in visible light communication without being affected by the relationship with the orientation of the apparatus. In addition, two types of patterns of a positive phase pattern and an antiphase pattern are alternately generated and illuminated, and it is also conceivable to generate and illuminate three or more types of patterns. FIG. 112 shows an example of sequentially illuminating four types of patterns.

  Normally, a configuration in which the entire LED illumination is blinked ((b) in FIG. 111), a white / black pattern is generated for a predetermined time, and illumination is performed in units of LED elements is also conceivable. For example, the transmission / reception time of a predetermined data unit may blink in the whole LED illumination, and then illuminate a white / black pattern in LED element units in a short time. Here, the predetermined data unit refers to, for example, a data unit from the first header to the next second header. At this time, when the image is taken in the direction of (a) in FIG. 110, a signal is received from the bright line obtained by imaging the blinking of the entire LED illumination, and when received in the direction of (b) in FIG. A signal is received from the light emission pattern.

  In addition, this Embodiment is not limited to an LED lighting apparatus, What kind of element may be sufficient if it is an element which can control ON / OFF by a small element unit like a LED element. Further, the apparatus is not limited to a lighting device, and may be a device such as a television, a projector, or a signage.

  In this embodiment, an example in which illumination is performed with a white / black pattern has been described, but a color may be used instead of the white / black pattern. For example, among RGB, RG may be always lit, and only B may be used and blinked. Using B alone rather than R and G is less likely to be recognized by humans, and flicker can be suppressed. As another example, ON / OFF is displayed by using colors (red and cyan patterns, green and magenta patterns, yellow and blue patterns, etc.) that are complementary colors in additive color mixing instead of white and black patterns. Also good. By using a complementary color in additive color mixing, flicker can be suppressed.

  Further, in the present embodiment, the description has been given by using the example in which the LED elements are arranged one-dimensionally. However, the LED elements are not arranged in one dimension but arranged in two dimensions and displayed like a two-dimensional barcode. May be performed.

(Summary of this embodiment)
The service providing method according to the present embodiment is a service providing method for providing a service to a user of the terminal device using a terminal device including an image sensor having a plurality of exposure lines, and each exposure line of the image sensor. Exposure times of 1/480 seconds or less so that the exposure times of the respective exposure lines are partially overlapped in time with adjacent exposure lines. An image acquisition step of acquiring image data by photographing the subject with a visible light communication for acquiring identification information of the subject by demodulating a bright line pattern corresponding to each of the exposure lines appearing in the image data And a service presentation step of presenting service information associated with the identification information of the subject to the user. .

  As a result, service information related to the subject is presented to the user of the terminal device by utilizing the fact that the subject and the terminal device communicate with each other as a transmitter and a receiver, respectively. It can be provided to the user in various forms. For example, in the service presentation step, information indicating a store advertisement, availability or reservation status related to the subject, information indicating a discount rate of a product or service price, a moving image for movie advertisement, and a screening time , Information for guiding the inside of the building, information for introducing exhibits, and information indicating traffic conditions are presented as the service information.

  The service providing method may further include an identification information transmitting step in which the terminal device transmits identification information of the subject to a server, and the terminal device transmits the service information associated with the identification information of the subject. In the service presenting step, the terminal device may present the obtained service information to the user.

  Accordingly, since the service information can be managed by the server in association with the identification information of the subject, maintenance such as updating of the service information can be facilitated.

  In the identification information transmission step, the accessory information is transmitted to the server together with the subject identification information, and in the service acquisition step, the service information associated with the subject identification information and the accessory information is acquired. May be.

  Accordingly, it is possible to provide a more appropriate service for the user according to the attached information. For example, as in the operation described with reference to FIGS. 84 and 97, in the identification information transmission step, the personal information of the user, the identification information of the user, the number information indicating the number of groups including the user, or Position information indicating the position of the terminal device is transmitted as the attached information.

  The service providing method may further include a position transmission step in which the terminal apparatus transmits position information indicating the position of the terminal apparatus to a server, and at least one in a predetermined range including the position indicated by the position information. A pre-acquisition step in which the terminal device acquires and holds from the server at least one service information associated with each of the identification information, and the service presentation step includes: The terminal device selects service information associated with the identification information of the subject from the at least one service information held in the pre-article acquisition step, and presents the service information to the user. Also good.

  Thus, for example, when the terminal device acquires the identification information of the subject as in the operation described with reference to FIG. 82, the terminal device does not communicate with the server or the like and then stores at least one service information held in advance. Therefore, the service information associated with the identification information of the subject can be acquired and presented. Therefore, the service provision can be speeded up.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. When the entry determination step determines that the user has entered the store, the terminal device acquires product service information related to the store product or service from the server and presents it to the user. A presentation step.

  Thereby, for example, as in the operation described with reference to FIGS. 86 to 90, when the user enters the store, the store menu or the like can be automatically presented to the user as the merchandise service information. Therefore, the store clerk does not need to present a menu or the like to the user, and the user can easily make an order to the store.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. And when it is determined in the store entry step that the user has entered the store, the location of the subject and the time of the store that differs depending on at least one of the time when the service information was presented An additional service presenting step in which the terminal device presents the additional service information to the user may be included.

  Thus, for example, as the processing described with reference to FIGS. 86 to 90, the closer the subject is to the store where the user entered, or the time when the user entered the store and the time when the service information was presented (or Service information that is more useful to the user can be presented to the user as additional service information as the time of shooting of the subject is closer. Specifically, each of a plurality of stores belonging to the chain store is a store corresponding to the service information presented in the service presentation step, and a signboard that is a subject by one of the stores (advertisement store) May have been issued. In such a case, the advertising store among the plurality of stores belonging to the chain store is typically located closest to the subject (signboard). Therefore, the closer the subject is to the store where the user entered, or the closer the time when the user entered the store and the time when the service information was presented, the more likely the store where the user entered was an advertising store. . Therefore, when there is a high possibility that the user has entered the advertising store, service information more useful to the user can be presented to the user as additional service information.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. When the store entry step determines that the user has entered the store, additional services of the store differ according to the number of times the user has used the service indicated by the service information at the store. An additional service presenting step in which the terminal device presents the information to the user may be included.

  As a result, service information that is more useful to the user can be presented to the user as additional service information as the number of times the service is used, for example, as described with reference to FIGS. For example, if the usage count of service information indicating a 20% discount on the product price exceeds a threshold, additional service information indicating a further discount of 10% can be presented to the user.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. When the entry determination step determines that the user has entered the store, the image acquisition step is performed for each of all other subjects associated with the store other than the subject. A complete determination step for determining whether or not processing including the visible light communication step and the service presentation step has been performed; and when it is determined in the complete determination step that the processing has been performed, An additional service presenting step in which the terminal device presents the service information to the user. Good.

  Thereby, for example, as in the operation described with reference to FIGS. 86 to 90, for example, the store puts out several subjects as signboards, and the image acquisition step, the visible light communication step, and the service presentation for all of the signboards. When the step is performed, service information most useful to the user can be presented to the user as additional service information.

  The service providing method further includes a store entry determining step of determining whether or not the user has entered a store corresponding to the service information presented in the service presenting step by specifying the position of the user. And when it is determined in the store entry determining step that the user has entered the store, the store differs according to a difference between a time at which the service information is presented and a time at which the user enters the store. The additional service information step may be included in which the terminal device presents the additional service information to the user.

  Accordingly, for example, as in the operation described with reference to FIGS. 86 to 90, the difference between the time when the service information is presented (or the time when the subject is photographed) and the time when the user enters the store is The smaller the service information, the more useful service information for the user can be presented to the user as additional service information. That is, a more useful service can be additionally provided to a user who takes a short time from entering the store after receiving service information by imaging the subject.

  The service providing method further includes a use determination step of determining whether or not the user has used the service indicated by the service information at a store corresponding to the service information presented in the service presentation step; Each time the service information is presented, the results determined in the use determination step may be accumulated, and an analysis step of analyzing the advertising effect of the subject based on the accumulated contents may be included.

  For example, when a service such as a 20% discount on the product price is indicated in the service information as in the operation described with reference to FIGS. 86 to 90, it is determined whether or not the service is used by, for example, electronic payment. Is done. In other words, every time a service is provided to the user at the time of imaging a subject, it is determined whether or not that service has been used. As a result, for example, when it is often determined that the subject has been used, it can be analyzed that the advertising effect of the subject is high. That is, the advertising effect of the subject can be appropriately analyzed based on the use result.

  In the analyzing step, together with the result determined in the use determining step, the position of the subject, the time when the service information is presented, the position of the store, and the time when the user enters the store The advertisement effect of the subject may be analyzed based on the accumulated contents.

  Thereby, the advertising effect of the subject can be analyzed in more detail. For example, when the position of the subject is changed, the advertising effect before and after the position change can be compared, and as a result, the subject can be placed at a position with a high advertising effect.

  The service providing method further includes a use determination step of determining whether or not the user has used the service indicated by the service information at a store corresponding to the service information presented in the service presentation step; When it is determined that the service is used in the use determination step, a store determination step of determining whether a use store that is a store where the service is used is a specific store associated with the subject; When it is determined in the store determination step that the use store is not the specific store, at least a part of the amount settled using the service at the use store is returned to the specific store using electronic commerce. A reduction step.

  Thereby, for example, even when the service is not used at a specific store (for example, an advertising store that has a signboard as a subject) as in the operation described with reference to FIGS. For example, a profit can be obtained as a price for installing a signboard as a subject.

  In the service presentation step, the terminal device presents the service information for introducing the subject to the user when the subject illuminated by light whose luminance changes in the image acquisition step is photographed. When the lighting device that changes in luminance is photographed as the subject in the image acquisition step, the terminal device presents the service information for guiding the inside of the building where the subject is arranged to the user. Also good.

  Accordingly, for example, as in the operation described with reference to FIGS. 105 and 106, it is possible to appropriately provide a user with a guidance service in a museum or the like and an introduction service for an exhibit that is a subject.

  The information communication method according to the present embodiment is an information communication method for acquiring information from a subject having a plurality of light emitting elements, and is included in the image sensor in an image obtained by photographing the subject with an image sensor. The exposure time setting step for setting the exposure time of the image sensor so that the bright line corresponding to the exposure line is generated according to the luminance change of the subject, and the plurality of the luminance lines according to the luminance change pattern for indicating the first information. An image acquisition step of acquiring a bright line image, which is an image including the bright lines, by the image sensor capturing the subject whose luminance changes in the same manner in all of the light emitting elements in the image sensor for the set exposure time; By demodulating the data specified by the bright line pattern included in the bright line image, A first information acquisition step of acquiring first information and each of the plurality of light emitting elements is obtained by photographing the subject that emits light with one of two different luminance values. And a second information acquisition step of acquiring second information by demodulating data indicated by an array of brightness intensities along a direction parallel to the exposure line, which is indicated in the image.

  Alternatively, the information communication method according to the present embodiment is an information communication method for transmitting a signal by a luminance change, and determining a pattern of the luminance change by modulating the first signal to be transmitted; A first transmission step of transmitting the first signal when all of the plurality of light emitting elements included in the light emitter change in luminance in the same manner according to the determined luminance change pattern, and the plurality of light emission Each of the elements emits light with one of the two different brightness values, so that a light brightness / darkness array appears in the space in which the light emitters are arranged, whereby the second transmission target. And a second transmission step of transmitting the signal.

  Thereby, for example, as in the operation described with reference to FIGS. 110 to 112, even if the illumination device that is a subject or a light emitter is an elongated device including a plurality of LEDs arranged in a row, the receiver Regardless of the direction of shooting, information or signals from the lighting device can be appropriately acquired. In other words, when the exposure line (the operation direction on the imaging side) of the image sensor provided in the receiver and the arrangement direction of the plurality of LEDs are not parallel, the receiver detects information from the luminance change of the entire lighting device. Or a signal can be acquired appropriately. Further, even when the exposure line and the above-described arrangement direction are parallel to each other, the receiver can appropriately acquire information or a signal from the bright and dark arrangement of luminance along the direction parallel to the exposure line. it can. In other words, the dependence of the imaging direction on the reception of information can be suppressed.

(Embodiment 5)
In this embodiment, an application example using a receiver such as a smartphone in Embodiments 1 to 4 and a transmitter that transmits information as a blinking pattern such as an LED or an organic EL will be described.

  FIG. 113 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

  Each of transmitter 8321, transmitter 8322, and transmitter 8323 has the same function as that of any of the transmitters in Embodiments 1 to 4, and is configured as a lighting device that transmits a signal (changes in visible light communication) by luminance change. Has been. These transmitters 8321 to 8323 transmit signals by changing in luminance at different frequencies. For example, the transmitter 8321 transmits the ID “1000” of the transmitter 8321 by changing the luminance at the frequency a (eg, 9200 Hz), and the transmitter 8322 changes the luminance at the frequency b (eg, 9600 Hz). The transmitter 8322 transmits the ID “2000”, and the transmitter 8323 transmits the ID “3000” of the transmitter 8322 by changing the luminance at the frequency c (for example, 10000 Hz).

  The receiver images (transmits visible light) the transmitters 8321 to 8323 in the same manner as in the first to fourth embodiments so that these transmitters 8321 to 8323 are all included in the angle of view. In the image obtained by the imaging, a bright line pattern corresponding to each transmitter appears. From the bright line pattern, the frequency of the luminance change of the transmitter corresponding to the bright line pattern can be specified.

  Here, if the frequencies of the transmitters 8321 to 8323 are the same, the frequencies specified from the bright line patterns corresponding to the transmitters are also the same. Furthermore, when the bright line patterns are adjacent to each other, the frequencies specified from the bright line patterns are the same, so it is difficult to distinguish the bright line patterns.

  Therefore, as described above, when the transmitters 8321 to 8323 change in luminance at different frequencies, the receiver can easily distinguish these bright line patterns, and the data specified by each bright line pattern can be identified. By demodulating, the IDs of the transmitters 8321 to 8323 can be appropriately acquired. That is, the receiver can appropriately distinguish the signals from the transmitters 8321 to 8323.

  In addition, each frequency of the transmitters 8321 to 8323 may be set by a remote controller or may be set at random. In addition, each of the transmitters 8321 to 8323 may communicate with an adjacent transmitter and automatically set the frequency of its own transmitter so as to be different from the frequency of the adjacent communication device.

  114 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  In the above example, each of the transmitters changes in luminance at a different frequency. However, when there are five or more transmitters, each of the transmitters may not change in luminance at different frequencies. That is, each of the five or more transmitters only needs to change in luminance at any one of the four types of frequencies.

  For example, as shown in FIG. 114, even in a situation where bright line patterns (rectangular regions in FIG. 114) corresponding to each of five or more transmitters are adjacent to each other, the frequency types are the same as the number of transmitters. If there are four types (frequency a, b, c, d), the frequencies of the bright line patterns adjacent to each other can be reliably varied. This is reasoned by the four-color problem or the four-color theorem.

  That is, in this embodiment, each of the plurality of transmitters changes in luminance at any one of at least four types of frequencies, and two or more light emitters of the plurality of transmitters are the same. The luminance changes at the frequency of. In addition, when a plurality of transmitters are projected on the light receiving surface of the image sensor of the receiver, the luminance changes between all transmitters (bright line patterns that are images of the transmitter) adjacent to each other on the light receiving surface. Each of the plurality of transmitters changes in luminance so that their frequencies are different.

  FIG. 115 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

  The transmitter changes in luminance by outputting high-luminance light (H) or low-luminance light (L) for each predetermined time unit (slot), and transmits a signal according to the luminance change. Here, the transmitter transmits a signal for each block including a header and a body. The header is expressed as (L, H, L, H, L, H, H) using, for example, 7 slots as shown in FIG. 79A. The body is composed of a plurality of symbols (00, 01, 10 or 11), and each symbol is expressed using 4 slots (4-value PPM). A block is expressed using a predetermined number of slots (19 in the example of FIG. 115). The ID is obtained by combining bodies included in each of four blocks, for example. A block may be expressed using 33 slots.

  The bright line pattern obtained by imaging by the receiver includes a pattern (header pattern) corresponding to the header and a pattern (data pattern) corresponding to the body. The data pattern does not include the same pattern as the header pattern. Therefore, the receiver can easily find the header pattern from the bright line pattern, and can measure the number of pixels (the number of exposure lines corresponding to the block) between the header pattern and the next header pattern. In the receiver, the number of slots in one block (19 in the example of FIG. 115) is set to a fixed number regardless of the frequency, and therefore the frequency of the transmitter (the time of one slot is determined according to the measured number of pixels. The reciprocal of the width) can be specified. That is, the receiver specifies a lower frequency as the number of pixels is larger, and identifies a higher frequency as the number of pixels is smaller.

  Thus, the receiver can acquire the ID of the transmitter and identify the frequency of the transmitter by imaging the transmitter. Here, the receiver can determine whether or not the acquired ID is appropriate by using the frequency specified in this way, that is, can perform error detection of the ID. Specifically, the receiver calculates a hash value for the ID, and compares the hash value with the identified frequency. When the hash value and the frequency match, the receiver determines that the acquired ID is appropriate, and when it does not match, the acquired ID is inappropriate (error). Judge that there is. For example, the receiver treats the remainder obtained by dividing the ID by a predetermined divisor as a hash value. In other words, the transmitter transmits the ID of the transmission target by changing the luminance at the same frequency as the hash value for the transmission target ID (the reciprocal of the time width of one slot).

  116 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5. FIG.

  The transmitter may use an arbitrary frequency instead of using the same frequency as the hash value as described above, and change the luminance at the arbitrary frequency. In this case, the transmitter transmits a signal indicating a value different from the ID of the transmitter. For example, when the ID of the transmitter is “100” and the transmitter uses 2 kHz as an arbitrary frequency, the transmitter transmits a signal “1002” in which the ID and the frequency are combined. Similarly, when the ID of another transmitter is “110” and the other transmitter uses 1 kHz as an arbitrary frequency, the other transmitter uses a signal “1101” that is a combination of the ID and the frequency. Send.

  In this case, the receiver uses the first digit of the signal transmitted and acquired from the transmitter for error detection, and extracts the remaining digits as the transmitter ID. Then, the receiver compares the frequency specified from the luminance pattern with the numerical value of the lower first digit included in the acquired signal. The receiver determines that the extracted ID is appropriate when the numerical value of the first digit matches the frequency, and determines that the extracted ID is inappropriate when the frequency does not match. (Error) is determined.

  As a result, it is possible to increase the degree of freedom in setting the frequency of the luminance change in the transmitter while enabling error detection in the receiver.

  117 is a diagram illustrating an example of operation of a receiver in Embodiment 5. FIG.

  As shown in FIG. 117, in an image obtained by imaging (visible light imaging) with a receiver, a part of each of the bright line pattern 8327a and the bright line pattern 8327b may overlap. In such a case, the receiver does not demodulate data from the portion 8327c where the bright line pattern 8327a and the bright line pattern 8327b overlap, and demodulates data from portions other than the respective portions 8327c of the bright line pattern 8327a and the bright line pattern 8327b. Do. Thereby, the receiver can acquire an appropriate ID from each of the bright line pattern 8327a and the bright line pattern 8327b.

  118 is a diagram illustrating an example of operation of a receiver in Embodiment 5. FIG.

  As shown in (a) of FIG. 118, the transmitter switches the frequency of the luminance change for transmitting each block, for example, for each block. As a result, the receiver can more easily discriminate between the blocks.

  Also, as shown in FIG. 118 (b), for example, the transmitter makes the frequency of the luminance change for transmitting the header of the block different from the frequency of the luminance change for transmitting the body of the block. Thereby, it can suppress that the same pattern as a header appears in a body. As a result, the receiver can more appropriately determine the distinction between the header and the body.

  FIG. 119 is a diagram illustrating an example of operation of a system including a transmitter, a receiver, and a server in Embodiment 5.

  The system in this embodiment includes a transmitter 8331, a receiver 8332, and a server 8333. The transmitter 8331 has the same function as that of any of the transmitters of Embodiments 1 to 4, and is configured as an illumination device that transmits the ID of the transmitter 8331 by changing the luminance (visible light communication). The receiver 8332 has the function as the receiver in any of Embodiments 1 to 4 above, and acquires the ID of the transmitter 8331 from the transmitter 8331 by imaging the transmitter 8331 (visible light imaging). To do. The server 8333 communicates with the transmitter 8331 and the receiver 8332 via a network such as the Internet.

  In the present embodiment, the ID of transmitter 8331 is fixed without being changed. On the other hand, the frequency used for the luminance change (visible light communication) of the transmitter 8331 can be arbitrarily changed by setting.

  In such a system, the transmitter 8331 first registers the frequency used in the luminance change (visible light communication) with the server 8333. Specifically, transmitter 8331 transmits its ID, registered frequency information indicating the frequency, and related information related to transmitter 8331 to server 8333. When the server 8333 receives the ID of the transmitter 8331, the registered frequency information, and the related information, the server 8333 records them in association with each other. That is, the ID of the transmitter 8331, the frequency used for the luminance change of the transmitter 8331, and the related information are recorded in association with each other. Thereby, the frequency used for the luminance change of the transmitter 8331 is registered.

  Next, the transmitter 8331 transmits the ID of the transmitter 8331 according to the luminance change of the registered frequency. The receiver 8332 acquires the ID by imaging the transmitter 8331 and specifies the frequency of the luminance change of the transmitter 8331 as described above.

  Next, the receiver 8332 transmits the acquired ID and specific frequency information indicating the specified frequency to the server 8333. When the server 8333 receives the ID and specific frequency information transmitted by the receiver 8332, the server 8333 searches for a frequency (frequency indicated by the registered frequency information) recorded in association with the ID, and the recorded frequency and Then, it is determined whether or not the frequency indicated by the specific frequency information matches. If it is determined that they match, the server 8333 transmits related information (data) recorded in association with the ID and frequency to the receiver 8332.

  Accordingly, if the frequency specified by the receiver 8332 does not match the frequency registered in the server 8333, the related information is not transmitted from the server 8333 to the receiver 8332. Therefore, if the receiver 8332 obtains the ID from the transmitter 8331 even once, the frequency registered in the server 8333 indicates that the receiver 8332 can receive related information from the server 8333 at any time. Can be prevented by changing from time to time. That is, the transmitter 8331 can prohibit the acquisition of related information by the receiver 8332 that has acquired the ID before the change by changing the frequency registered in the server 8333 (that is, the frequency used for the luminance change). . In other words, an expiration date can be set for acquisition of related information by changing the frequency. For example, when the user of the receiver 8332 stays at a hotel where the transmitter 8331 is installed, the hotel administrator changes the frequency after staying. Accordingly, the receiver 8332 can acquire related information only on the day the user stays, and can prohibit the receiver 8332 from acquiring related information after staying.

  Note that the server 8333 may register a plurality of frequencies associated with one ID. For example, each time the server 8333 acquires the registered frequency information from the receiver 8332, the server 8333 always registers the frequency indicated by the latest four registered frequency information in association with the ID. Thereby, even if the receiver 8332 has acquired the ID in the past, the related information can be acquired from the server 8333 until the frequency is changed three times. Further, the server 8333 may manage the time or the time zone when the frequency is set in the transmitter 8331 for each registered frequency. In this case, when the server 8333 receives the ID and specific frequency information from the receiver 8332, the server 8333 confirms the time zone managed for the frequency indicated by the specific frequency information, and the receiver 8333 The time zone when 8332 acquired the ID can be specified.

  120 is a block diagram illustrating a configuration of a transmitter in Embodiment 5. In FIG.

  The transmitter 8334 has the same function as that of any of the transmitters of Embodiments 1 to 4, and includes a frequency storage unit 8335, an ID storage unit 8336, a check value storage unit 8337, a check value comparison unit 8338, a check value. A calculation unit 8339, a frequency calculation unit 8340, a frequency comparison unit 8341, a transmission unit 8342, and an error notification unit 8343 are provided.

  The frequency storage unit 8335 stores a frequency used for luminance change (visible light communication). The ID storage unit 8336 stores the ID of the transmitter 8334. The check value storage unit 8337 stores a check value for determining whether the ID stored in the ID storage unit 8336 is appropriate.

  The check value calculation unit 8339 reads the ID stored in the ID storage unit 8336 and applies a predetermined function to the ID to calculate a check value (calculation check value) for the ID. The check value comparison unit 8338 reads the check value stored in the check value storage unit 8337, and compares the check value with the calculated check value calculated by the check value calculation unit 8339. If the check value comparison unit 8338 determines that the calculated check value is different from the check value, the check value comparison unit 8338 notifies the error notification unit 8343 of an error. For example, the check value storage unit 8337 stores a value “0” indicating that the ID stored in the ID storage unit 8336 is an even number as a check value. The check value calculation unit 8339 reads the ID stored in the ID storage unit 8336 and divides it by the value “2”, thereby calculating the remainder as a calculation check value. Then, the check value comparison unit 8338 compares the check value “0” with the calculated check value that is the remainder of the above division.

  The frequency calculation unit 8340 reads the ID stored in the ID storage unit 8336 via the check value calculation unit 8339 and calculates a frequency (calculated frequency) from the ID. For example, the frequency calculation unit 8340 calculates the remainder as a frequency by dividing the ID by a predetermined value. The frequency comparison unit 8341 compares the frequency (storage frequency) stored in the frequency storage unit 8335 with the calculated frequency. If the frequency comparison unit 8341 determines that the calculated frequency is different from the storage frequency, the frequency comparison unit 8341 notifies the error notification unit 8343 of an error.

  The transmission unit 8342 transmits the ID stored in the ID storage unit 8336 by changing the luminance at the calculated frequency calculated by the frequency calculation unit 8340.

  When an error is notified from at least one of the check value comparison unit 8338 and the frequency comparison unit 8341, the error notification unit 8343 notifies the error by a buzzer sound, blinking, or lighting. Specifically, the error notification unit 8343 includes a lamp for error notification, and notifies the error by lighting or blinking the lamp. Alternatively, when the power switch of the transmitter 8334 is turned ON, the error notification unit 8343 sets a light source that changes in luminance to transmit a signal such as an ID for a predetermined period (for example, 10 seconds). An error is notified by blinking at a recognizable cycle.

  Thereby, since it is checked whether the ID stored in the ID storage unit 8336 and the frequency calculated from the ID are appropriate, transmission of an incorrect ID and a change in luminance at the wrong frequency Can be prevented.

  FIG. 121 is a block diagram illustrating a configuration of a receiver in Embodiment 5. In FIG.

  The receiver 8344 has the same function as the receiver of any of Embodiments 1 to 4 above, and also includes a light receiving unit 8345, a frequency detecting unit 8346, an ID detecting unit 8347, a frequency comparing unit 8348, and a frequency calculating unit 8349. It has.

  The light receiving unit 8345 includes, for example, an image sensor, and acquires an image including a bright line pattern by imaging (visible light imaging) a transmitter that changes in luminance. The ID detection unit 8347 detects the transmitter ID from the image. That is, the ID detection unit 8347 acquires the ID of the transmitter by demodulating data specified by the bright line pattern included in the image. The frequency detector 8346 detects the frequency of the luminance change of the transmitter from the image. That is, the frequency detection unit 8346 specifies the frequency of the transmitter from the bright line pattern included in the image as in the example described with reference to FIG.

  The frequency calculation unit 8349 calculates the frequency of the transmitter by dividing the ID from the ID detected by the ID detection unit 8347, for example, as described above. The frequency comparison unit 8348 compares the frequency detected by the frequency detection unit 8346 with the frequency calculated by the frequency calculation unit 8349. Here, if these frequencies are different, the frequency comparison unit 8348 determines that the detected ID is an error, and causes the ID detection unit 8347 to detect the ID again. Thereby, it can prevent acquiring wrong ID.

  122 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter may distinguish and transmit each of the symbols “00, 01, 10, 11” by changing the position where the luminance change occurs in a predetermined time unit.

  For example, when transmitting the symbol “00”, the transmitter transmits the symbol “00” by changing the luminance only in the first interval which is the first interval in the time unit. Further, when transmitting the symbol “01”, the transmitter transmits the symbol “01” by changing the luminance only in the second interval which is the second interval in the time unit. Similarly, when transmitting the symbol “10”, the transmitter transmits the symbol “10” and transmits the symbol “11” by changing the luminance only in the third interval which is the third interval in the time unit. When transmitting, the transmitter transmits the symbol “11” by changing the luminance only in the fourth interval, which is the fourth interval in the time unit.

  As described above, in this embodiment, when any symbol is transmitted, the luminance changes within one section, so that it is lower than the above-described transmitter that reduces the entire luminance of one section (slot). , Flicker can be suppressed.

  FIG. 123 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

  The transmitter may distinguish and transmit each of the symbols “0, 1” by changing the presence / absence of a luminance change in a predetermined time unit. For example, when transmitting the symbol “0”, the transmitter transmits the symbol “0” by not changing the luminance in time units. Further, when transmitting the symbol “1”, the transmitter transmits the symbol “1” by changing the luminance in time units.

  124 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter may distinguish and transmit each of the symbols “00, 01, 10, 11” by changing the intensity (frequency) of the luminance change in a predetermined time unit. For example, when transmitting the symbol “00”, the transmitter transmits the symbol “00” by not changing the luminance in time units. Further, when transmitting the symbol “01”, the transmitter transmits the symbol “01” by changing the luminance in a time unit (changing the luminance at a low frequency). Further, when transmitting the symbol “10”, the transmitter transmits the symbol “10” when the luminance changes drastically in a time unit (luminance changes at a high frequency). Then, when transmitting the symbol “11”, the transmitter transmits the symbol “11” by changing the luminance more intensely in time units (by changing the luminance at a higher frequency).

  125 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter may distinguish and transmit each of the symbols “0, 1” by changing the phase of the luminance change in a predetermined time unit. For example, when transmitting the symbol “0”, the transmitter transmits the symbol “0” by changing the luminance at a predetermined phase in a time unit. Further, when transmitting the symbol “1”, the transmitter transmits the symbol “1” by changing the luminance in a phase opposite to the above phase in a time unit.

  FIG. 126 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.

  When transmitting a signal such as an ID, the transmitter may change the luminance for each color such as red, green, and blue. Thus, the transmitter can transmit a signal having a larger amount of information to a receiver that can recognize a luminance change for each color. Further, the luminance change of any color may be used for clock adjustment. For example, a red luminance change may be used for clock adjustment. In this case, the red luminance change serves as a header. As a result, it is not necessary to use a header for luminance changes of colors other than red (green and blue), and redundant data transmission can be suppressed.

  127 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter may express the luminance of the combined color (for example, white) by combining a plurality of colors such as red, green, and blue. That is, the transmitter expresses the luminance change of the composite color by changing the luminance for each color such as red, green, and blue. A signal to be transmitted is transmitted by the luminance change of the composite color, similarly to the above-described visible light communication. Here, the luminance of one or more of red, green, and blue may be used for adjustment for expressing a predetermined luminance of the composite color. Accordingly, a signal can be transmitted by a luminance change of the composite color, and a signal can be transmitted by a luminance change of any two colors of red, green, and blue. In other words, the transmitter can transmit a signal to a receiver that can recognize only the luminance change of the composite color (for example, white) as described above, and can also recognize each color such as red, green, and blue. For such a receiver, more signals can be transmitted as incidental information, for example.

  128 is a diagram illustrating an example of operation of a transmitter in Embodiment 5. FIG.

  The transmitter has four light sources. Each of these four light sources (for example, LED lamps) emits light of colors represented by different positions 8351a, 8351b, 8352a, and 8352b in the CIExy chromaticity diagram shown in FIG.

  The transmitter transmits each signal by switching between the first lighting transmission and the second lighting transmission. Here, the first lighting transmission is a process of transmitting a signal “0” by turning on a light source that emits light of the color at position 8351a and a light source that emits light of the color at position 8351b among the four light sources. It is. The second lighting transmission is a process of transmitting the signal “1” by turning on the light source that emits the light of the color at the position 8352a and the light source that emits the light of the color at the position 8352b. The image sensor of the receiver can identify the colors represented by each of the positions 8351a, 8351b, 8352a, 8352b, so that the receiver can properly receive the signal “0” and the signal “1”. it can.

  Here, when the first lighting transmission is performed, the color represented by the position in the middle of the positions 8351a and 8351b in the CIExy chromaticity diagram is seen by human eyes. Similarly, when the second lighting transmission is performed, a color represented by a position in the middle of the positions 8352a and 8352b in the CIExy chromaticity diagram is seen by human eyes. Accordingly, by appropriately adjusting the colors and luminances of the four light sources, the position between the positions 8351a and 8351b can be matched with the position between the positions 8352a and 8352b (to the position 8353). As a result, even if the first lighting transmission and the second lighting transmission are switched, it appears to the human eye that the light emission color of the transmitter is fixed, so that it is possible to suppress the human from feeling flickering. .

  FIG. 129 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5.

  The transmitter includes an ID storage unit 8361, a random number generation unit 8362, an addition unit 8363, an encryption unit 8364, and a transmission unit 8365. The ID storage unit 8361 stores the ID of the transmitter. The random number generation unit 8362 generates different random numbers every certain time. Adder 8363 combines the latest random number generated by random number generator 8362 with the ID stored in ID storage unit 8361, and outputs the result as an edit ID. The encryption unit 8364 generates an encrypted edit ID by encrypting the edit ID. The transmission unit 8365 transmits the encrypted edit ID to the receiver by changing the luminance.

  The receiver includes a receiving unit 8366, a decoding unit 8367, and an ID acquisition unit 8368. The receiving unit 8366 receives the encrypted edit ID from the transmitter by imaging the transmitter (visible light imaging). The decryption unit 8367 restores the edit ID by decrypting the received encrypted edit ID. The ID acquisition unit 8368 acquires the ID by extracting the ID from the restored editing ID.

  For example, the ID storage unit 8361 stores the ID “100”, and the random number generation unit 8362 generates the latest random number “817” (Example 1). In this case, the adding unit 8363 generates and outputs the edit ID “100817” by combining the random number “817” with the ID “100”. The encryption unit 8364 generates an encrypted edit ID “abced” by encrypting the edit ID “100817”. The decryption unit 8367 of the receiver restores the edit ID “100817” by decrypting the encrypted edit ID “abced”. Then, the ID acquisition unit 8368 extracts the ID “100” from the restored editing ID “100817”. In other words, the ID acquisition unit 8368 acquires the ID “100” by deleting the last three digits of the edit ID.

  Next, the random number generation unit 8362 generates a new random number “619” (example 2). In this case, the adding unit 8363 generates and outputs the edit ID “100619” by combining the random number “619” with the ID “100”. The encryption unit 8364 generates an encrypted edit ID “diffia” by encrypting the edit ID “100619”. The decryption unit 8367 of the transmitter restores the edit ID “100619” by decrypting the encrypted edit ID “diffia”. Then, the ID acquisition unit 8368 extracts the ID “100” from the restored editing ID “100619”. In other words, the ID acquisition unit 8368 acquires the ID “100” by deleting the last three digits of the edit ID.

  In this way, the transmitter simply encrypts a combination of random numbers that change every fixed time without simply encrypting the ID, so the ID can be easily decrypted from the signal transmitted from the transmitter 8365. Can be prevented. In other words, when a simple encrypted ID is transmitted from the transmitter to the receiver several times, even if the ID is encrypted, if the ID is the same, the transmitter to the receiver. Since the transmitted signal is the same, the ID may be deciphered. However, in the example shown in FIG. 129, a random number that is changed at regular intervals is combined with the ID, and the ID combined with the random number is encrypted. Therefore, even when the same ID is transmitted to the receiver several times, the signals transmitted from the transmitter to the receiver can be made different if the transmission timings of these IDs are different. As a result, it is possible to prevent the ID from being easily decoded.

  FIG. 130 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5.

  The transmitter includes an ID storage unit 8371, a timing unit 8372, an addition unit 8373, an encryption unit 8374, and a transmission unit 8375. The ID storage unit 8371 stores the ID of the transmitter. The timer unit 8372 keeps time and outputs the current date and time (current date and time). The adding unit 8373 combines the ID stored in the ID storage unit 8371 with the current date and time output from the time measuring unit 8372 as the transmission date and time, and outputs the result as the edit ID. The encryption unit 8374 generates an encrypted edit ID by encrypting the edit ID. The transmission unit 8375 transmits the encrypted edit ID to the receiver by changing the luminance.

  The receiver includes a receiving unit 8376, a decoding unit 8377, a validity determining unit 8378, and a time measuring unit 8379. The receiving unit 8376 receives the encrypted editing ID from the transmitter by imaging the transmitter (visible light imaging). The decryption unit 8377 restores the edit ID by decrypting the received encrypted edit ID. The timer unit 8379 measures time and outputs the current date and time (current date and time). The validity determination unit 8378 acquires the ID by extracting the ID from the restored editing ID. Further, the validity determination unit 8378 extracts the transmission date / time from the restored edit ID, and compares the transmission date / time with the current date / time output from the time measuring unit 8379 to determine the validity of the ID. For example, the validity determination unit 8378 determines that the ID is invalid when the difference between the transmission date and time and the current date and time is longer than a predetermined time, or when the transmission date and time is newer than the current date and time. .

  For example, the ID storage unit 8371 stores the ID “100”, and the time measuring unit 8372 outputs the current date and time “201305011200” (May 1, 2013 12:00) as the transmission date and time (Example 1). In this case, the adding unit 8373 generates and outputs the edit ID “100201305011200” by combining the transmission date “2013030501200” with the ID “100”. The encryption unit 8374 generates an encrypted edit ID “ei39ks” by encrypting the edit ID “100201305011200”. The decryption unit 8377 of the receiver restores the edit ID “100201305011200” by decrypting the encrypted edit ID “ei39ks”. Then, the validity determination unit 8378 extracts the ID “100” from the restored edit ID “100201305011200”. In other words, the validity determination unit 8378 acquires the ID “100” by deleting the last 12 digits of the edit ID. Further, the validity determination unit 8378 extracts the transmission date and time “201305011200” from the restored editing ID “100201305011200”. If the transmission date and time “201305011200” is older than the current date and time output from the timing unit 8379 and the difference between the transmission date and time and the current date and time is within 10 minutes, for example, the validity determination unit 8378 has the ID “100 Is determined to be valid.

  On the other hand, the ID storage unit 8371 stores the ID “100”, and the time measuring unit 8372 outputs the current date and time “201401101730” (January 1, 2014, 17:30) as the transmission date (Example 2). In this case, the adding unit 8373 generates and outputs the edit ID “100201401101730” by combining the transmission date and time “201401101730” with the ID “100”. The encryption unit 8374 generates the encrypted edit ID “002jflk” by encrypting the edit ID “100201401411730”. The decryption unit 8377 of the receiver restores the edit ID “100201401101730” by decrypting the encrypted edit ID “002jflk”. Then, the validity determination unit 8378 extracts the ID “100” from the restored editing ID “1002014014101730”. In other words, the validity determination unit 8378 acquires the ID “100” by deleting the last 12 digits of the edit ID. Further, the validity determination unit 8378 extracts the transmission date and time “201401101730” from the restored editing ID “100201401101730”. Then, the validity determination unit 8378 determines that the ID “100” is invalid if the transmission date and time “201401101730” is newer than the current date and time output from the time measuring unit 8379.

  In this way, the transmitter encrypts a combination of the current date and time that is changed at regular intervals without simply encrypting the ID, so that the ID can be easily obtained from the signal transmitted from the transmission unit 8375. It can be prevented from being deciphered. In other words, when a simple encrypted ID is transmitted from the transmitter to the receiver several times, even if the ID is encrypted, if the ID is the same, the transmitter to the receiver. Since the transmitted signal is the same, the ID may be deciphered. However, in the example shown in FIG. 130, the current date and time that are changed at regular intervals are combined with the ID, and the ID that is combined with the current date and time is encrypted. Therefore, even when the same ID is transmitted to the receiver several times, the signals transmitted from the transmitter to the receiver can be made different if the transmission timings of these IDs are different. As a result, it is possible to prevent the ID from being easily decoded.

  Furthermore, since it is determined whether or not the acquired ID is valid by comparing the transmission date and time when the encrypted edit ID is transmitted with the current date and time, the validity of the ID is managed based on the transmission and reception time. be able to.

  Note that the receiver shown in FIG. 129 and FIG. 130 may acquire the encrypted edit ID, transmit the encrypted edit ID to the server, and acquire the ID from the server.

(Guidance at the station)
FIG. 131 shows an example of a usage form of the present invention in a train platform. The user holds the portable terminal over an electronic bulletin board or lighting, and obtains information displayed on the electronic bulletin board, train information of a station where the electronic bulletin board is installed, information on the premises of the station, or the like by visible light communication. Here, the information itself displayed on the electronic bulletin board may be transmitted to the portable terminal by visible light communication, or ID information corresponding to the electronic bulletin board is transmitted to the portable terminal, and the ID information acquired by the portable terminal The information displayed on the electronic bulletin board may be acquired by inquiring the server. When the ID information is transmitted from the mobile terminal, the server transmits the content displayed on the electronic bulletin board to the mobile terminal based on the ID information. The train ticket information stored in the memory of the mobile terminal is compared with the information displayed on the electronic bulletin board, and the ticket information corresponding to the user's ticket is displayed on the electronic bulletin board. An arrow indicating the destination to the home where the user's scheduled train arrives is displayed on the display. When getting off, the route to the vehicle near the exit or the transfer route may be displayed. If a seat has been designated, the route to that seat may be displayed. When the arrow is displayed, it can be displayed more easily by displaying the arrow using the same color as the color of the train route in the map or the train guide information. In addition to the arrow display, the user's reservation information (home number, vehicle number, departure time, seat number) can also be displayed. By displaying the user reservation information together, it is possible to prevent erroneous recognition. If the ticket information is stored in the server, query the server from the mobile terminal to obtain and compare the ticket information, or compare the ticket information with the information displayed on the electronic bulletin board on the server side. Thus, information related to the ticket information can be acquired. The target route may be estimated from the history of the user performing a transfer search, and the route may be displayed. Further, not only the contents displayed on the electronic bulletin board but also the train information / premises information of the station where the electronic bulletin board is installed may be acquired and compared. Information related to the user may be highlighted with respect to the display of the electronic bulletin board on the display, or may be rewritten and displayed. When the user's boarding schedule is unknown, an arrow for guiding to the boarding place on each route may be displayed. When station premises information is acquired, an arrow for guiding to a store or restroom may be displayed on the display. The user's behavior characteristics may be managed in advance by a server, and an arrow for guiding the user to a store / restroom may be displayed on the display when the user often stops at a store / restaurant in the station. Only users who have behavioral characteristics of stopping at a store / restroom will display an arrow that directs them to a store / restroom, etc., and will not be displayed to other users, so the amount of processing can be reduced. . The color of the arrow leading to a store / restroom may be different from the arrow guiding the destination to the home. When both arrows are displayed simultaneously, it is possible to prevent erroneous recognition by using different colors. Note that although an example of a train is shown in FIG. 131, it is possible to perform display with a similar configuration even on an airplane or a bus.

(Translation of signage)
FIG. 132 shows an example in the case where information is acquired by visible light communication from an electronic guidance display board installed in an airport, a station premises, a sightseeing spot, a hospital, or the like. Information on the display content is acquired from the electronic guidance display board by visible light communication, the display content information is translated into language information set in the mobile terminal, and then displayed on the display of the mobile terminal. Since it is translated into the user's language and displayed, the user can easily understand the information. Language translation may be performed in the mobile terminal or in the server. When translating in the server, the display content information acquired by visible light communication and the language information of the portable terminal are transmitted to the server, the translation is performed in the server, the transmitted to the portable terminal, and the display on the portable terminal May be displayed. Moreover, when ID information is acquired from an electronic guidance display board, the structure which transmits ID information to a server and acquires the display content information corresponding to ID information from a server may be sufficient. Furthermore, based on nationality information, ticket information, and baggage deposit information stored in the mobile terminal, an arrow that guides the user to the next place may be displayed.

(Coupon pop-up)
FIG. 133 shows an example in which coupon information acquired by visible light communication is displayed or a popup is displayed on the display of the mobile terminal when the user approaches the store. A user acquires coupon information of a store from an electronic bulletin board etc. by visible light communication using a portable terminal. Next, when the user enters the predetermined range from the store, the coupon information of the store or a pop-up is displayed. Whether or not the user has entered the predetermined range from the store is determined using the GPS information of the mobile terminal and the store information included in the coupon information. Not only coupon information but also ticket information may be used. Since it automatically alerts when a store where coupons and tickets can be used approaches, the user can use coupons and tickets appropriately.

  FIG. 134 shows an example in which coupon information / ticket information or a pop-up is displayed on the display of the portable terminal at a cash register or a ticket gate. Display is performed when position information is acquired from visible light communication from lighting installed in a cash register or ticket gate, and the acquired position information matches information included in coupon information / ticket information. The barcode reader may have a light emitting unit, and the position information may be acquired by performing visible light communication with the light emitting unit, or the position information may be acquired from the GPS of the mobile terminal. A transmitter is installed near the cash register, and coupons and payment information may be displayed on the receiver display by the user holding the receiver over the transmitter, or the receiver communicates with the server. Payment processing may be performed. When coupon information / ticket information includes Wi-Fi information installed in a store or the like, and the user's portable terminal acquires the same information as the Wi-Fi information included in the coupon information / ticket information In addition, display may be performed.

(Launch operation application)
FIG. 135 illustrates an example in which a user acquires information from a home appliance by visible light communication using a mobile terminal. When ID information or information related to the home appliance is acquired from the home appliance by visible light communication, an application for operating the home appliance is automatically started. FIG. 135 shows an example using a television. With such a configuration, it is possible to start an application for operating a home appliance simply by holding the portable terminal over the home appliance.

(Stop transmission while the barcode reader is operating)
FIG. 136 shows an example in which data communication for visible light communication is stopped near the barcode reader 8405a when the barcode reader 8405a reads a barcode of a product. By stopping the visible light communication when reading the barcode, the barcode reader 8405a can be prevented from erroneously recognizing the barcode. The barcode reader 8405a transmits a transmission stop signal to the visible light signal transmitter 8405b when the barcode reading button is pressed. When the finger is released from the button or when the finger is released from the button, a predetermined signal is transmitted. When the time has elapsed, a transmission resumption signal is transmitted to the visible light signal transmitter 8405b. The transmission stop signal and the transmission restart signal are performed by wired / wireless communication, infrared communication, or sound wave communication. The barcode reader 8405a transmits a transmission stop signal when it is estimated that the barcode reader 8405a is moved from the measurement value of the acceleration sensor provided in the barcode reader 8405a, and the barcode reader 8405a is not moved for a predetermined time. A transmission resumption signal may be transmitted when estimated. The barcode reader 8405a transmits a transmission stop signal when it is estimated that the barcode reader 8405a is grasped from the measured value of the electrostatic sensor or the illuminance sensor provided in the barcode reader 8405a, and when it is estimated that the hand is released. A transmission resumption signal may be transmitted. The barcode reader 8405a detects that the barcode reader 8405a has been lifted because the switch provided on the grounding surface of the barcode reader 8405a is released from the pressed state, and transmits a transmission stop signal. It may be detected that the barcode reader 8405a has been placed because it is pressed, and a transmission resumption signal may be transmitted. The bar code reader 8405a detects that the bar code reader 8405a has been lifted from the measured values of the bar code reader storage switch and the infrared sensor, transmits a transmission stop signal, and detects that the bar code reader 8405a has been returned. Then, a transmission resumption signal may be transmitted. The cash register 8405c may transmit a transmission stop signal when the operation is started, and may transmit a transmission restart signal when the settlement operation is performed.

  For example, the transmitter 8405b configured as illumination stops signal transmission when receiving a transmission stop signal, and operates so that a ripple (luminance change) of 100 Hz to 100 kHz becomes small. Alternatively, signal transmission is continued by reducing the luminance change of the signal pattern. Alternatively, the carrier period is set longer than the barcode reading time of the barcode reader 8405a, or the carrier period is set shorter than the exposure time of the barcode reader 8405a. Thereby, malfunction of the barcode reader 8405a can be prevented.

  As shown in FIG. 137, for example, the transmitter 8406b configured as illumination detects that there is a person near the barcode reader 8406a with a human sensor or camera, and stops signal transmission. Alternatively, the transmitter 8405b performs the same operation as when a transmission stop signal is received. The transmitter 8406b resumes signal transmission when detecting that there is no person in the vicinity of the barcode reader 8406a. The transmitter 8406b may detect an operation sound of the barcode reader 8406a and stop signal transmission for a predetermined time.

(Information transmission from PC)
FIG. 138 shows an example of a usage form of the present invention.

  For example, the transmitter 8407a configured as a personal computer transmits a visible light signal through a display device such as an attached display, a connected display, or a projector. The transmitter 8407a transmits the URL of the website displayed by the browser, information on the clipboard, and information determined by the application having the focus. For example, coupon information acquired on a website is transmitted.

(Database)
FIG. 139 shows an example of the configuration of a database held by a server that manages IDs transmitted by a transmitter.

  The database has an ID-data table that holds data to be passed in response to an inquiry using an ID as a key and an access log table that stores a record of the inquiry using an ID as a key. The ID-data table clears the ID transmitted by the transmitter, the data passed in response to the inquiry using the ID as a key, the condition for passing the data, the number of times of access using the ID as a key, and the condition. Have the number of times. The conditions for passing data include the date and time, the number of accesses, the number of successful accesses, the information of the terminal of the inquiry source (terminal model, the application that made the inquiry, the current location of the terminal, etc.), and the user information of the inquiry source ( Age, gender, occupation, nationality, language, religion, etc.). By setting the number of successful accesses as a condition, a service method such as “1 yen per access, but 100 yen as the upper limit and no data returned thereafter” becomes possible. When there is access using ID as a key, the log table clears the ID, requested user ID, time, other incidental information, whether or not the data is passed, and the passed data Record the contents of.

(Reception start gesture)
FIG. 140 shows an example of a gesture operation for starting reception by this communication method.

  For example, the user protrudes a receiver configured as a smartphone, and starts reception by an operation of rotating the wrist to the left and right. The receiver detects this operation from the measurement value of the 9-axis sensor and starts reception. The receiver may start reception when detecting at least one of these operations. The operation of projecting the receiver has an effect that the transmitter is closer to the transmitter, so that the transmitter is imaged larger, and the reception speed and accuracy are improved. The operation of rotating the wrist to the left and right has the effect of changing the angle of the receiver to eliminate the angle dependency of this method and enable stable reception.

  This operation may be performed only when the home screen is in the foreground. Thereby, it is possible to prevent this communication from being performed against the user's intention while using another application.

  Note that the image sensor may be activated when the operation of the receiver protruding is detected, and reception may be stopped if there is no operation of shaking the wrist to the left or right. Since activation of the image sensor takes approximately several hundred milliseconds to 2 seconds, this can further improve the responsiveness.

(Transmitter control by power line)
FIG. 141 shows an example of the transmitter of the present invention.

  The signal control unit 8410g controls the transmission state of the transmitter 8410a (contents of signals to be transmitted, presence / absence of transmission, intensity of luminance change used for transmission, etc.). The signal control unit 8410g sends the control content of the transmitter 8410a to the power distribution control unit 8410f. The distribution control unit 8410f changes the voltage, current, and frequency supplied to the power supply unit 8410b of the transmitter 8410a, thereby transmitting the control contents in the form of the magnitude of change and the time of change. Since the power supply unit 8410b performs a constant output without being affected by a slight change in voltage, current, or frequency, a signal that exceeds the stabilization capability of the power supply unit 8410b, for example, a signal depending on the timing or time width at which the power supply is cut off. The signal is transmitted by expressing The luminance control unit 8410d receives the content transmitted by the power distribution control unit 8410f in consideration of conversion by the power supply unit 8410b, and changes the luminance change pattern of the light emitting unit.

(Encoding method)
FIG. 142 is a diagram for explaining one of the visible light communication image encoding methods.

  In this encoding method, since the ratio of white and black is approximately the same, the average luminance of the normal phase image and the reverse phase image is approximately the same, and there is an advantage that it is difficult for humans to feel flicker.

(Encoding method that can receive light even when imaged from oblique direction)
FIG. 143 is a diagram for explaining one of the encoding methods of the visible light communication image.

  The image 1001a is an image in which white and black lines are displayed with a uniform width. When this image 1001a is imaged obliquely, in the image 1001b obtained by the imaging, the left line appears thin and the right line appears thick. When the image 1001a is projected onto a curved surface, lines with different thicknesses appear in the image 1001i obtained by the imaging.

  Therefore, a visible light communication image is created by the following encoding method. The visible light communication image 1001c includes, from the left, a white line, a black line that is three times as thick as the white line, and a white line that is one-third the thickness of the black line. In this way, the preamble is encoded as an image in which a line three times as thick as the left adjacent line and a line one third as thick as the left adjacent line are continued. Like the visible light communication images 1001d and 1001e, a line having the same thickness as the left adjacent line is encoded as “0”. Like the visible light communication images 1001f and 1001g, a line that is twice as thick as the left adjacent line or half as thick as the left adjacent line is encoded as “1”. That is, when the thickness of the encoding target line is different from the thickness of the left adjacent line, the encoding target line is encoded as “1”. As an example using this encoding method, a signal including “0101010001011” following the preamble is represented by an image such as a visible light communication image 1001h. In this example, the line having the same thickness as the left adjacent line is encoded as “0”, and the line having a different thickness from the left adjacent line is encoded as “1”, but the line having the same thickness as the left adjacent line is encoded. May be encoded as “1”, and a line having a different thickness from the left adjacent line may be encoded as “0”. Further, not only the comparison with the left adjacent line, but also the comparison with the right adjacent line may be performed. That is, “1” and “0” may be encoded depending on whether the thickness of the encoding target line is the same as or different from the thickness of the line on the right. In this way, on the transmitter side, “0” is encoded by using a color different from that of the encoding target line and by making the thickness of the adjacent line and the encoding target line the same. By doing so, “1” is encoded.

  On the receiver side, a visible light communication image is captured, and the thickness of a white or black line is detected in the captured visible light communication image. The color is different from the line to be decoded, and the thickness of the line adjacent to the line to be decoded (left adjacent or right adjacent) is compared with the thickness of the line to be decoded. The decoding target line is decoded as “0”, and when the thicknesses are different, the decoding target line is decoded as “1”. It may be decoded as “1” when the thickness is the same, and as “0” when the thickness is different. Based on the 1 and 0 data strings that have been decoded, the data is finally decoded.

  This encoding method uses a local line thickness relationship. As seen in the image 1001b and the image 1001i, the ratio of the thicknesses of the neighboring lines does not change greatly. Therefore, even if the image is taken from an oblique direction or projected onto a curved surface, it is created by this encoding method. Visible light communication images can be correctly decoded.

  In this encoding method, since the ratio of white and black is approximately the same, the average luminance of the normal phase image and the reverse phase image is approximately the same, and there is an advantage that it is difficult for humans to feel flicker. In addition, since this encoding method is not different between black and white, there is an advantage that it can be decoded by the same algorithm for any visible light communication image of a normal phase signal and a reverse phase signal.

  Moreover, there is an advantage that it is easy to add a code. For example, the visible light communication image 1001j is a combination of a four-times-thick line on the left and a quarter line on the left. In this way, there are many unique patterns such as “5x and 1/5 next to the left” and “3x and 2/3 next to the left”. It can be defined. For example, a visible light communication image can represent a single piece of data, but visible light can be used as a cancel signal to indicate that some of the data received so far has become invalid because the data to be transmitted has been changed. It is conceivable to use the communication image 1001j. The color is not limited to white and black, and any color may be used as long as it is different. For example, complementary colors may be used.

(Encoding method with different amount of information depending on distance)
144 and 145 are diagrams illustrating one of the visible light communication image encoding methods.

  As shown in (a-1) of FIG. 144, when a 2-bit signal is expressed by setting one portion of the four divided images to black and the remaining portion to white, the average luminance of the image is When white is 100% and black is 0%, it is 75%. If the white and black portions are reversed as shown in FIG. 144 (a-2), the average luminance is 25%.

  In the image 1003a, the white portion of the visible light communication image created by the encoding method of FIG. 143 is represented by the image of (a-1) in FIG. 144, and the black portion is represented by the image of (a-2) of FIG. 144. This is a visible light communication image. This visible light communication image represents a signal A encoded by the encoding method of FIG. 143 and a signal B encoded by (a-1) and (a-2) of FIG. 144. When the nearby receiver 1003b captures the visible light communication image 1003a, a fine image 1003d is obtained, and both the signal A and the signal B can be received. When the remote receiver 1003c captures the visible light communication image 1003a, a small image 1003e is obtained. In the image 1003e, the detailed part cannot be confirmed, and the part (a-1) in FIG. 144 is white and the part (a-2) in FIG. 144 is black. Therefore, only the signal A can be received. . Thereby, as the distance between the visible light communication image and the receiver is shorter, more information can be transmitted. As a method for encoding the signal B, a combination of (b-1) and (b-2) in FIG. 144 or a combination of (c-1) and (c-2) in FIG. 144 may be used.

  By using the signals A and B, the signal A can represent basic and important information, and the signal B can represent additional information. Further, when the receiver transmits the signals A and B as ID information to the server and the server transmits information corresponding to the ID information to the receiver, the information transmitted by the server may be changed depending on the presence or absence of the signal B. It becomes possible.

(Encoding method that divides data)
FIG. 146 is a diagram for explaining one of the coding methods of the visible light communication image.

  A signal 1005a to be transmitted is divided into a plurality of data pieces 1005b, 1005c, and 1005d. An address indicating the position of the data piece is added to each data piece, and further, a preamble, an error detection / correction code, a frame type description, and the like are added to form frame data 1005e, 1005f, and 1005g. Visible light communication images 1005h, 1005i, and 1005j are generated by encoding the frame data and displayed. When the display area is sufficiently large, a visible light communication image 1005k obtained by connecting a plurality of visible light communication images is displayed.

  As shown in FIG. 146, as a method for inserting a visible light communication image into a video, in the case of a display device using a solid light source, a visible light communication signal image is normally displayed and only displayed. A fixed light source is turned on, and a solid-state light source is turned off during other periods, thereby realizing a wide range of display devices, for example, projectors using DMD, projectors using liquid crystals such as LCOS, Of course, display devices using MEMS can be applied. Further, even in a display device that divides and displays an image display into subframes, for example, a display device that does not use a light source such as a backlight such as a PDP or an EL, some subframes are converted into visible light communication images. Adaptation is possible by replacing. In addition, as a solid light source, a semiconductor laser, an LED light source, etc. can be considered.

(Effect of inserting reverse phase image)
FIG. 147 and FIG. 148 are diagrams for explaining one of the visible light communication image encoding methods.

  As (1006a) in FIG. 147, the transmitter inserts a black image between the video and the visible light communication image (normal phase image). An image obtained by capturing the image with the receiver is as shown in (1006b) of FIG. Since it is easy for the receiver to search for a portion where the line of simultaneously exposed pixels is black, the position where the visible light communication image is captured is easily set as the position of the pixel exposed at the next timing. Can be specified.

  As shown in (1006a) of FIG. 147, the transmitter displays a visible light communication image (normal phase image), and then displays a reverse running visible light communication image with black and white reversed. The receiver obtains a signal-to-noise ratio that is twice that obtained when only the normal phase image is used by obtaining the difference between the pixel values of the normal phase image and the reverse phase image. Conversely, when the same S / N ratio is ensured, the luminance difference between black and white can be halved, and flickering when viewed by humans can be suppressed. Also, as shown in (1007a) and (1007b) in FIG. 148, the moving average of the expected value of the difference in luminance between the video and the visible light communication image is canceled for the normal phase image and the reverse phase image. Since the time resolution of human vision is about 1/60 second, it is possible to make humans feel that no visible light communication image is displayed by setting the time for displaying a visible light communication image to be less than this. .

  As illustrated in (1006c) of FIG. 147, the transmitter may further insert a black image between the normal phase image and the reverse phase image. In this case, an image shown in (1006d) in FIG. 147 is obtained by imaging by the receiver. In the image shown in (1006b) of FIG. 147, since the normal phase image pattern and the reverse phase image pattern are adjacent to each other, pixel values may be averaged at the boundary portion. Such an issue does not occur in the image shown in FIG.

(Super-resolution)
FIG. 149 is a diagram illustrating one method for encoding a visible light communication image.

  As shown in FIG. 149 (a), when video data and signal data to be transmitted by visible light communication are separated, super-resolution processing is performed on the video data, and visible light communication is performed on the obtained super-resolution image. Superimpose the image. That is, the super-resolution processing is not performed on the visible light communication image. When the visible light communication image is already superimposed on the video data as shown in FIG. 149 (b), (1) the edge of the visible light communication image (data such as white and black is indicated by data). (2) The super-resolution processing is performed so that the average image of the normal phase image and the reverse phase image of the visible light communication image has a uniform luminance. Thus, it is possible to perform visible light communication more appropriately (decrease the error rate) by changing the processing for the visible light communication image depending on whether or not the visible light communication image is superimposed on the video data. It becomes.

(Indication of support for visible light communication)
FIG. 150 is a diagram illustrating one operation of the transmitter.

  The transmitter 8500a superimposes and displays on the image to be projected or displayed that it is compatible with visible light communication. This display is displayed only for a predetermined time after starting the transmitter 8500a, for example.

  The transmitter 8500a transmits to the connected device 8500c that it is compatible with visible light communication. The device 8500c displays that the transmitter 8500a is compatible with visible light communication. For example, the display of the device 8500c displays that the transmitter 8500a is compatible with visible light communication. When the connected transmitter 8500a supports visible light communication, the device 8500c transmits data for visible light communication to the transmitter 8500a. The indication that the transmitter 8500a is compatible with visible light communication may be displayed when the device 8500c is connected to the transmitter 8500a, or the device 8500c may transmit visible light communication to the transmitter 8500a. It may be displayed when data is transmitted. In the case where data is displayed when visible light communication data is transmitted from the device 8500c, the transmitter 8500a acquires identification information indicating visible light communication from the data, and the identification information is included in the data. If it is included, it may be displayed that the transmitter 8500a is compatible with visible light communication.

  In this way, by displaying on the projection screen or the display of the device that the transmitter (illumination, projector, video display device) is compatible with visible light communication or whether it is compatible or not. The user can easily grasp whether the transmitter is compatible with visible light communication. Therefore, it is possible to prevent a malfunction that the visible light communication cannot be performed even though the data for visible light communication is transmitted from the device to the transmitter.

(Information acquisition using visible light communication signals)
FIG. 151 is a diagram illustrating one application example of visible light communication.

  The transmitter 8501a receives video data and signal data from the device 8501c, and displays a visible light communication image 8501b. The receiver 8501d captures the visible light communication image 8501b and receives a signal included in the visible light communication image. The receiver 8501d communicates with the device 8501c from information (address, password, etc.) included in the received signal, and the video itself displayed by the transmitter 8501a and its accompanying information (video ID, URL, password, SSID, Translation data, voice data, hash tags, product information, purchase information, coupons, vacancy information, etc.). The device 8501c may transmit the transmission status to the transmitter 8501a to the server 8501e, and the receiver 8501d may obtain the information from the server 8501e.

(data format)
FIG. 152 is a diagram illustrating one format of visible light communication data.

  The data shown in (a) of FIG. 152 has a video address table indicating the position of video data in the storage area and a position address table indicating the position of signal data transmitted by visible light communication. In a video display device that does not support visible light communication, only the video address table is referred to. Therefore, even if a signal address table and signal data are included in the input, video display is not affected. Thereby, the backward compatibility with respect to the video display apparatus which does not support visible light communication is maintained.

  In the data format shown in FIG. 152 (b), an identifier indicating that the following data is video data is arranged in front of the video data, and an identifier indicating that the subsequent data is signal data is included in the signal data. Arranged in front. By using the identifier, it is inserted into the data only when there is video data or signal data, so that the entire code amount can be reduced. Also, identification information indicating whether it is video data or signal data may be arranged. Further, the program information may include identification information indicating whether or not data for visible light communication is included. By including identification information indicating whether or not data for visible light communication is included in the program information, the user can determine whether or not visible light communication is possible when searching for a program. The program information may include an identifier indicating that data for visible light communication is included. Furthermore, by adding an identifier / identification information for each data, it is possible to switch processing such as switching of luminance for each data and switching of super-resolution, thereby reducing the error rate during visible light communication. It becomes possible.

  The data format shown in (a) of FIG. 152 is suitable for a situation in which data is read from a storage medium such as an optical disk, and the data format shown in (b) of FIG. 152 is suitable for streaming data such as television broadcasting. Yes. The signal data includes the value of the signal transmitted by visible light communication, the transmission start time, the transmission end time, the location used for transmission on the display or projection surface, the brightness of the visible light communication image, the bar of the visible light communication image. Contains information such as code orientation.

(Estimate solid shape and receive)
FIG. 153 and FIG. 154 are diagrams illustrating one application example of visible light communication.

  As shown in FIG. 153, for example, a transmitter 8503a configured as a projector projects a distance measurement image in addition to a video and a visible light communication image. The dot pattern shown in the distance measurement image is a dot pattern in which the positional relationship between a predetermined number of dots in the vicinity of an arbitrary dot is different from the positional relationship between other arbitrary combinations of dots. The receiver can identify a local dot pattern by capturing an image for distance measurement, and can estimate the three-dimensional shape of the projection plane 8503b. The receiver restores the visible light communication image distorted by the three-dimensional shape of the projection surface to a flat image and receives a signal. Note that the distance measurement image and the visible light communication image may be projected with infrared rays that cannot be perceived by humans.

  As shown in FIG. 154, for example, a transmitter 8504a configured as a projector includes an infrared projector 8504b that projects a distance measurement image using infrared rays. The receiver estimates the three-dimensional shape of the projection plane 8504c from the distance measurement image, restores the distortion of the visible light communication image, and receives the signal. Note that the transmitter 8504a may project an image with visible light and project a visible light communication image with infrared light. Note that the infrared projector 8504b may project the visible light communication image with infrared rays.

(Stereoscopic projection)
FIG. 155 and FIG. 156 are diagrams illustrating one method for displaying a visible light communication image.

  When performing stereoscopic projection or displaying a visible light communication image on a cylindrical display surface, visible light communication images 8505a to 8505f can be displayed as shown in FIG. 155 to enable reception from a wide angle. Become. By displaying the visible light communication images 8505a and 8505b, it is possible to receive from a wide angle in the horizontal direction. By combining the visible light communication images 8505a and 8505b, reception is possible even when the receiver is tilted. The visible light communication image 8505a and the visible light communication image 8505b may be displayed alternately, or the visible light communication image 8505f that is an image obtained by combining these images may be displayed. Further, by adding the visible light communication image 8505c and the visible light communication image 8505d, it becomes possible to receive from a wide angle in the vertical direction. Like the visible light communication image 8505e, the visible light communication image segment may be expressed by providing a portion that is projected with an intermediate color or a portion that is not projected. By rotating the visible light communication images 8505a to 8505f, it is possible to receive from a wider angle. In FIG. 155, the visible light communication image is displayed on the cylindrical display surface. However, the visible light communication image may be displayed on the cylindrical display surface.

  When stereoscopic projection is performed or when a visible light communication image is displayed on a spherical display surface, visible light communication images 8506a to 8506d can be displayed as shown in FIG. 156 to enable reception from a wide angle. Become. In the visible light communication image 8506a, since the receivable area in the horizontal direction is wide but the receivable area in the vertical direction is narrow, the visible light communication image 8506a is displayed in combination with the visible light communication image 8506b having the opposite property. The visible light communication image 8506a and the visible light communication image 8506b may be displayed alternately, or the visible light communication image 8506c, which is an image obtained by combining these images, may be displayed. The portion where the barcode is concentrated, such as the upper portion of the visible light communication image 8506a, is finely displayed, and there is a high possibility of receiving a signal in error. Therefore, a reception error can be prevented by displaying this portion in an intermediate color as in the visible light communication image 8506d or by projecting nothing.

(Different communication protocols for each zone)
FIG. 157 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 5.

  The receiver 8420a receives zone information from the base station 8420h, recognizes in which zone it is located, and selects a reception protocol. The base station 8420h is configured as, for example, a mobile phone communication base station, a Wi-Fi access point, an IMES transmitter, a speaker, or a wireless transmitter (Bluetooth (registered trademark), ZigBee, specific low-power wireless station, etc.). Note that the receiver 8420a may specify a zone based on position information obtained from GPS or the like. As an example, it is assumed that communication is performed at a signal frequency of 9.6 kHz in zone A, and communication is performed at a signal frequency of 15 kHz for ceiling lighting and signage of 4.8 kHz in zone B. At the position 8420j, the receiver 8420a recognizes that the current location is zone A from the information of the base station 8420h, performs reception at a signal frequency of 9.6 kHz, and receives signals transmitted from the transmitters 8420b and 8420c. The receiver 8420a recognizes that the current location is zone B from the information of the base station 8420i at the position 8420l, and is further trying to receive a signal from the ceiling lighting because the in-camera is directed upward. Is received at a signal frequency of 15 kHz, and signals transmitted by the transmitters 8420e and 8420f are received. At the position 8420m, the receiver 8420a recognizes that the current location is the zone B from above the base station 8420i, and further estimates that it is trying to receive a signal transmitted by the signage from the movement of the out camera. Receive at a signal frequency of .8 kHz and receive the signal transmitted by transmitter 8420g. The receiver 8420a receives the signals of both the base station 8420h and the base station 8420i at the position 8420k, and cannot determine whether the current location is the zone A or the zone B. Therefore, the reception process is performed at both 9.6 kHz and 15 kHz. I do. It should be noted that the part where the protocol differs depending on the zone is not limited to the frequency, but may be the server that inquires about the modulation method, signal format, and ID of the transmission signal. The base stations 8420h and 8420i may transmit the protocol in the zone to the receiver, or may transmit only the ID indicating the zone, and the receiver may acquire the protocol information from the server using the zone ID as a key. Good.

  Transmitters 8420b to 8420f receive zone IDs and protocol information transmitted from base stations 8420h and 8420i, and determine a signal transmission protocol. A transmitter 8420d capable of receiving signals transmitted by both base station 8420h and base station 8420i utilizes a base station zone protocol with stronger signal strength, or alternately uses both protocols.

(Zone recognition and services for each zone)
FIG. 158 is a diagram illustrating an example of operation of a receiver and a transmitter in Embodiment 5.

  The receiver 8421a recognizes the zone to which it belongs from the received signal. The receiver 8421a provides services (coupon distribution, point assignment, route guidance, etc.) determined for each zone. As an example, the receiver 8421a receives a signal transmitted from the left side of the transmitter 8421b and recognizes that it is in the zone A. Here, the transmitter 8421b may transmit different signals depending on the transmission direction. Further, the transmitter 8421b may transmit a signal such that a different signal is received according to the distance to the receiver by using a signal having a light emission pattern such as 2217a. In addition, the receiver 8421a may recognize the positional relationship with the transmitter 8421b from the direction and size in which the transmitter 8421b is imaged, and may recognize the zone in which the receiver 8421a is located.

  A part of the signals indicating that they are located in the same zone may be shared. For example, IDs representing zone A transmitted from the transmitter 8421b and the transmitter 8421c are common to the first half. Accordingly, the receiver 8421a can recognize the zone where the receiver 8421a is located only by receiving the first half of the signal.

(Summary of this embodiment)
The information communication method in the present embodiment is an information communication method for transmitting a signal by a luminance change, and a determination step for determining a plurality of luminance change patterns by modulating each of a plurality of transmission target signals; Each of the plurality of light emitters changes a luminance according to any one of the determined plurality of luminance change patterns, thereby transmitting a transmission target signal corresponding to any one of the patterns. A transmission step, wherein two or more of the plurality of light emitters each have two types of brightness different from each other for each time unit predetermined for the light emitter. The time unit predetermined for each of the two or more light emitters is set so that either one of the two lights is output. Differently, luminance changes at different frequencies to.

  Thereby, as in the operation described with reference to FIG. 113, each of two or more light emitters (for example, a transmitter configured as a lighting device) changes in luminance at a different frequency. A receiver that receives signals to be transmitted (for example, IDs of light emitters) can easily distinguish and acquire the signals to be transmitted.

  In the transmission step, each of the plurality of light emitters changes in luminance at any one of at least four frequencies, and two or more light emitters of the plurality of light emitters are: The luminance may be changed at the same frequency. For example, in the transmitting step, when the plurality of light emitters are projected onto a light receiving surface of an image sensor for receiving the plurality of transmission target signals, all the light emitters adjacent to each other on the light receiving surface. Each of the plurality of light emitters changes in luminance so that the frequency of the luminance change differs between them.

  Thus, as in the operation described with reference to FIG. 114, if there are at least four types of frequencies used for luminance change, even if there are two or more light emitters that change in luminance at the same frequency, Even if the number of frequency types is less than the number of multiple light emitters, the luminance changes between all the light emitters adjacent to each other on the light receiving surface of the image sensor based on the four-color problem or the four-color theorem. It is possible to reliably vary the frequency of the. As a result, the receiver can easily distinguish and acquire each of the transmission target signals transmitted from the plurality of light emitters.

  In the transmission step, each of the plurality of light emitters may transmit the signal to be transmitted by changing in luminance at a frequency specified by a hash value of the signal to be transmitted.

  Accordingly, as in the operation described with reference to FIG. 113, each of the plurality of light emitters changes in luminance at a frequency specified by the hash value of the signal to be transmitted (for example, the ID of the light emitter). When receiving the signal to be transmitted, the machine can determine whether the frequency specified from the actual luminance change matches the frequency specified by the hash value. That is, the receiver can determine whether or not there is an error in the received signal (for example, the ID of the light emitter).

  The information communication method may further calculate a frequency corresponding to the transmission target signal as a first frequency from the transmission target signal stored in the signal storage unit according to a predetermined function. A calculation step; a frequency determination step for determining whether or not the second frequency stored in the frequency storage unit matches the calculated first frequency; and the first frequency and the second frequency A frequency error notification step of notifying an error when it is determined that the frequency does not match, and a determination step when it is determined that the first frequency and the second frequency match. Then, a luminance change pattern is determined by modulating the transmission target signal stored in the signal storage unit, and in the transmission step, among the plurality of light emitters Any one of the light emitters according to the determined the pattern, by changing the luminance in the first frequency may transmit a signal of the transmission target stored in the signal storage unit.

  Thus, as in the operation described with reference to FIG. 120, the frequency stored in the frequency storage unit and the frequency calculated from the signal to be transmitted stored in the signal storage unit (ID storage unit) are It is determined whether or not they match, and when it is determined that they do not match, an error is notified, so that it is possible to easily detect abnormality of the signal transmission function by the light emitter.

  The information communication method further includes a check value calculation step of calculating a first check value from a transmission target signal stored in the signal storage unit according to a predetermined function, and a check value storage unit. A check value determination step for determining whether or not the stored second check value matches the calculated first check value; and the first check value and the second check value. A check value error notification step of notifying an error when it is determined that they do not match, and a determination step when it is determined that the first check value and the second check value match Then, a luminance change pattern is determined by modulating the transmission target signal stored in the signal storage unit, and in the transmission step, the plurality of light emitters are detected. Any one of the light emitters is by changing luminance in accordance with the determined the pattern, may transmit a signal of the transmission target stored in the signal storage unit.

  Thereby, like the operation described with reference to FIG. 120, the check value calculated from the check value stored in the check value storage unit and the transmission target signal stored in the signal storage unit (ID storage unit). It is determined whether or not the values match, and when it is determined that they do not match, an error is notified, so that the abnormality detection of the signal transmission function by the light emitter can be easily performed.

  The information communication method according to the present embodiment is an information communication method for acquiring information from a subject, and corresponds to a plurality of exposure lines included in the image sensor in an image obtained by photographing the subject by an image sensor. An exposure time setting step for setting an exposure time of the image sensor so that a plurality of bright lines are generated according to a change in luminance of the subject, and the exposure time set for the subject whose luminance is changed by the image sensor. And acquiring information by demodulating data identified by the pattern of the plurality of bright lines included in the acquired bright line image, and an image acquisition step of acquiring a bright line image including the plurality of bright lines Information acquisition step and a plurality of bright line patterns included in the acquired bright line image. Te, and a frequency specifying step of specifying the frequency of the luminance change of the subject. For example, in the frequency specifying step, a plurality of header patterns, which are a plurality of predetermined patterns for indicating a header, are included in the plurality of bright line patterns, and the number of pixels between the plurality of header patterns is determined. Is determined as the frequency of luminance change of the subject.

  Thereby, since the frequency of the luminance change of the subject is specified as in the operation described with reference to FIG. 115, when a plurality of subjects with different luminance change frequencies are photographed, information from those subjects is obtained. Can be easily distinguished and acquired.

  Further, in the image acquisition step, the bright line image including a plurality of patterns each represented by a plurality of bright lines is acquired by photographing a plurality of subjects each changing in luminance, and the information acquisition step acquires the bright line image. When a part of each of the plurality of patterns included in the bright line image overlaps, by demodulating data specified by a portion excluding the part from each of the plurality of patterns, the plurality of patterns Information may be acquired from each of the patterns.

  As a result, since the data is not demodulated from a portion where a plurality of patterns (plural bright line patterns) overlap as in the operation described with reference to FIG. 117, erroneous information is acquired. Can be prevented.

  Further, in the image acquisition step, a plurality of bright line images are acquired by photographing the plurality of subjects a plurality of times at mutually different timings, and in the frequency specifying step, the bright line images are included in the bright line image. A frequency for each of a plurality of patterns is specified, and in the information acquisition step, a plurality of patterns in which the same frequency is specified is searched from a plurality of previous bright line images, and the searched plurality of patterns are combined and combined. The information may be obtained by demodulating data specified by the plurality of patterns.

  Thereby, a plurality of patterns (plural emission line patterns) in which the same frequency is specified are searched from a plurality of emission line images, the plurality of searched patterns are combined, and information is acquired from the combined patterns. Therefore, even when a plurality of subjects are moving, information from the plurality of subjects can be easily distinguished and acquired.

  The information communication method may further include: identifying information on the subject included in the information acquired in the information acquisition step; and the frequency specification for a server in which frequencies are registered for each of the identification information. A transmission step of transmitting specific frequency information indicating the frequency specified in the step; a related information acquisition step of acquiring related information associated with the identification information and the frequency indicated by the specific frequency information from the server; May be included.

  Thereby, as in the operation described with reference to FIG. 119, the identification information (ID) acquired based on the luminance change of the subject (transmitter) and the related information associated with the frequency of the luminance change are acquired. Is done. Therefore, by changing the frequency of the luminance change of the subject and updating the frequency registered in the server to the frequency after the change, the receiver that acquired the identification information before the frequency change acquires the related information from the server. Can be prevented. That is, by changing the frequency registered in the server in accordance with the change in the luminance change frequency of the subject, the receiver that has acquired the subject identification information in the past can acquire the related information from the server indefinitely. It can prevent becoming a state.

  The information communication method is further acquired in the identification information acquisition step of acquiring the identification information of the subject by extracting a part from the information acquired in the information acquisition step, and in the information acquisition step. In addition, a setting frequency specifying step of specifying a number indicated by the remaining part other than the part of the information as a setting frequency of the luminance change set for the subject may be included.

  Accordingly, as in the operation described with reference to FIG. 116, the information obtained from the plurality of bright line patterns does not depend on the identification information of the subject and the set frequency of the luminance change set on the subject. Since it can be included, the degree of freedom between the identification information and the set frequency can be increased.

(Embodiment 6)
In this embodiment, each application example using a receiver such as a smartphone in Embodiments 1 to 5 and a transmitter that transmits information as a blinking pattern of LEDs will be described.

  FIG. 159 is a diagram illustrating an example of a transmission signal according to the sixth embodiment.

  The transmission signal D is divided into data pieces Dx of a predetermined size (for example, Dx = D1, D2, D3), and a frame check sequence FCS for error detection / correction calculated from each data piece and a header Hdr are obtained for each data piece. Append to Further, a frame check sequence FCS2 for error detection / correction calculated from the original data and a header Hdr2 are added. Data composed of Hdr, Dx, and FCS is configured to be received by the image sensor. Since the image sensor is suitable for receiving continuous data in a short time, Hdr, Dx, and FCS transmit continuously. The data composed of Hdr2, Dx, and FCS2 is configured to be received by the illuminance sensor. Although it is desirable that the Hdr and FCS received by the image sensor be short, the Hdr2 and FCS2 received by the illuminance sensor can be longer signal sequences. By using a long signal sequence for Hdr2, header detection accuracy can be increased. By making FCS2 longer, a code that can detect and correct many bit errors can be adopted, and the performance of error detection and correction can be improved. Note that Hdr2 and FCS2 may not be transmitted, and instead Hdr and FCS may be received by the illuminance sensor. The illuminance sensor may receive both Hdr and Hdr2, or both FCS and FCS2.

  FIG. 160 is a diagram illustrating an example of a transmission signal according to the sixth embodiment.

  FCS2 has a long signal length, and if it is frequently inserted, the reception efficiency of the image sensor deteriorates. Therefore, the insertion frequency of FCS2 is reduced, and a signal PoFCS2 indicating the location of FCS2 is inserted instead. As an example, when 4PPM having an information amount of 2 bits per unit time is used for signal representation, if CRC32 is used for FCS2, 16 units of transmission time are required, but PoFCS2 with a range of 0 to 3 is 1 unit time Can be sent. Since the transmission time is shortened compared with the case where only the FCS 2 is inserted, the efficiency of image sensor reception can be improved. The illuminance sensor receives PoFCS2 following the transmission signal D, specifies the transmission time of FCS2 from PoFCS2, and receives FCS2. Further, the PoFCS2 following the FCS2 is received, the transmission time of the next FCS2 is specified, and the next FCS2 is received. If the FCS2 received earlier and the FCS2 received later are the same, it is estimated that the receiver is receiving the same signal.

  161A to 161C are diagrams each illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6.

  In the captured image shown in FIG. 161A, the number of bright lines is small because the image is small. Therefore, only a small amount of data can be received from this captured image at a time. The captured image illustrated in FIG. 161B is an image captured using zoom and has a large number of bright lines because the captured image is large. Therefore, if imaging is performed using the zoom, a large amount of data can be received at one time. In addition, it can receive data from a distance and can receive signals from small transmitters. As a zoom method, optical zoom or Ex zoom is used. Optical zoom is zooming by increasing the focal length of a lens. Ex zoom is a zoom method that enlarges a part of a captured image by capturing an image using only a part rather than the entire image sensor when the image is captured at a resolution lower than the capability of the image sensor. The captured image illustrated in FIG. 161C is an image captured using electronic zoom (image enlargement). Although the image becomes larger, the bright line becomes thicker when enlarged by electronic zoom, and the number of bright lines is the same as before zooming, so the reception characteristics are the same as before zooming.

  162A and 162B are diagrams illustrating an example of a captured image (bright line image) of a receiver in Embodiment 6. FIG.

  The captured image illustrated in FIG. 162A is an image captured with the subject focused, and the captured image illustrated in FIG. 162B is an image captured with the focus out. In the captured image shown in FIG. 162B, since the image is blurred, it is possible to observe bright lines up to the periphery of the actual transmitter, and more bright lines can be observed. Therefore, if the image is taken out of focus, a large amount of data can be received at one time, and data can be received from a greater distance. An image similar to the captured image shown in FIG. 162B can also be captured by capturing using the macro mode.

  163A to 163C are diagrams each illustrating an example of a captured image (bright line image) of the receiver in Embodiment 6.

  When the exposure time is set longer than the visible light communication mode and shorter than the normal imaging mode, an image as shown in FIG. 163A is obtained. An imaging mode in which such an image is obtained is called a bright line detection mode (intermediate mode). In the image shown in FIG. 163A, the bright line of the transmitter can be observed in the left center, and a dark normal captured image appears in the other part. By displaying this image on the receiver, it is possible to make it easy for the user to capture the image of the receiver toward the target transmitter. In the bright line detection mode, an image is captured darker than in the normal imaging mode. Therefore, by capturing an image in the high sensitivity mode, it is possible to capture an image that is close to the normal imaging mode and has a brightness that is easily visible to humans. If the sensitivity is set too high, the dark part of the bright line becomes bright, so the sensitivity is set to such an extent that the bright line can be observed. The receiver shifts to the visible light communication mode and receives the transmission signal of the transmitter that is imaged at a portion designated by means such as a user touching an image. The receiver may automatically shift to the visible light communication mode and receive a signal when a bright line (transmission signal) is found in the captured image.

  The receiver finds the transmission signal from the bright line in the captured image, and highlights and displays the portion as shown in FIG. 163B, so that the portion where the signal is transmitted can be presented to the user in an easy-to-understand manner. The bright line may be observed not only by the transmission signal but also by the pattern of the subject. Therefore, instead of determining the presence or absence of a transmission signal from the bright line of one image, it may be determined that there is a transmission signal when the position of the bright line changes in a plurality of images.

  Since an image captured in the bright line detection mode is darker and less visible than an image captured in the normal imaging mode, an image with improved visibility by image processing may be displayed. The image illustrated in FIG. 163C is an example of an image in which an edge is extracted to emphasize the boundary of the imaging target.

  164 is a diagram illustrating an example of a captured image (bright line image) of a receiver in Embodiment 6. FIG. Specifically, FIG. 164 is a diagram in which a transmitter with a signal transmission period of 1/9600 seconds is imaged at the exposure time ratio shown in the lower part of the figure. When the exposure time is shorter than 1/9600 seconds, which is the transmission cycle, the captured images are almost the same and a clear bright line can be captured. As the exposure time becomes longer, the outline of the bright line becomes blurred. However, in this signal expression example, if the exposure time is up to about 1.5 times the transmission cycle, the bright line pattern can be observed and the signal can be received. In this signal representation example, the bright line can be observed if the exposure time is up to about 20 times the transmission cycle, and the exposure time in this range can be used as the exposure time in the bright line detection mode.

  To what extent the signal can be received differs depending on the signal expression method. If a signal expression rule in which the number of bright lines is small and the interval between the bright lines is long, the transmission efficiency is lowered, but the signal can be received even with a longer exposure time, and the bright lines can be observed even with a longer exposure time.

(Exposure time in intermediate imaging mode)
From FIG. 164, a clear bright line can be observed if the exposure time is up to about three times the modulation period. Since the modulation frequency is 480 Hz or more, it is desirable that the exposure time in the intermediate imaging mode (intermediate mode) be 1/160 seconds or less.

  In addition, when the exposure time is 1 / 10,000 seconds or less, it is difficult to see non-light-emitting objects even in the high sensitivity mode under illumination light. Therefore, the exposure time in the intermediate imaging mode is preferably 1 / 10,000 seconds or more. However, this limitation can be relaxed by improving the sensitivity of the image sensor in the future.

  FIG. 165 is a diagram illustrating an example of a transmission signal in Embodiment 6.

  Since the receiver integrates a plurality of received data pieces and receives a series of signals, if the transmission signal is suddenly changed, the data pieces before and after the change are mixed and the signals cannot be correctly integrated. . Therefore, as shown in FIG. 165 (a), when the transmission signal is changed, the transmitter performs normal lighting for a predetermined time as a buffer band and does not transmit a signal. When the receiver cannot receive a signal for a predetermined time T2 shorter than the predetermined time T1, the receiver discards the data pieces received so far to avoid mixing the data pieces before and after the change. can do. Alternatively, as shown in FIG. 165 (b), the transmitter repeatedly transmits a signal X for notifying it when the transmission signal is changed. Repeated transmission prevents the transmission signal change notification X from being missed. Alternatively, as shown in FIG. 165 (c), the transmitter repeatedly transmits the preamble when the transmission signal is changed. When the receiver receives a preamble with a period shorter than the period in which the preamble appears in a normal signal, the receiver discards data pieces received so far.

  FIG. 166 is a diagram illustrating an example of operation of a receiver in Embodiment 6.

  The image shown in FIG. 166 (a) is an image obtained by capturing the transmitter with just focus. The receiver can take an image as shown in FIG. 166 (b) by taking an image out of focus. When the focus is further removed, the captured image looks like the image shown in FIG. In FIG. 166 (c), the bright lines of a plurality of transmitters overlap, and the receiver cannot receive signals. Therefore, the receiver takes an image by adjusting the focus so that the bright lines of the plurality of transmitters do not overlap. When there is only one transmitter in the imaging range, the receiver adjusts the focus so that the size of the transmitter is maximized in the captured image.

  The receiver may compress the captured image in a direction parallel to the bright line, but does not compress the image in a direction perpendicular to the bright line. Alternatively, the receiver reduces the degree of compression in the vertical direction. Thereby, it is possible to prevent a reception error from occurring due to blurring of bright lines due to compression.

  167 and 168 are diagrams illustrating an example of instructions to the user displayed on the screen of the receiver in the sixth embodiment.

  The receiver can estimate the position of the receiver in the manner of triangulation from the position information of each transmitter and the position, size, and angle of each transmitter in the captured image by imaging a plurality of transmitters. Therefore, when the image is captured in a form that only one transmitter can receive, the receiver changes the direction of the receiver or lowers the image to capture images of a plurality of transmitters. Indicates an imaging direction and a moving direction by displaying an image including an arrow or the like. FIG. 167 (a) is a display example of an instruction to image the right transmitter with the receiver pointing to the right, and FIG. 167 (b) is an instruction to image the transmitter in front in the back. It is a display example. FIG. 168 is a display example of an instruction for imaging the other transmitter by shaking the receiver or the like because the position of the other transmitter is unknown to the receiver. Although it is desirable to photograph a plurality of transmitters in one captured image, the positional relationship of the transmitters in the plurality of images may be estimated using image processing or sensor values of a nine-axis sensor. The receiver may use the ID received from one transmitter, inquire the position information of the surrounding transmitters from the server, and instruct the user to image the transmitter that is most easily imaged.

  The receiver detects that the user has moved the receiver from the sensor value of the 9-axis sensor, and displays the screen based on the signal received last after the movement has ended and a predetermined time has elapsed. As a result, when the receiver is directed to the transmitter intended by the user, a signal of another transmitter is received while the receiver is moving, and processing based on the transmission signal of the unintended transmitter is performed. Can be prevented.

  The receiver may continue the reception process even while moving, and may perform a process based on the received signal, for example, acquisition of information from a server using the received signal as a key. In this case, the reception process is continued after the process, and the process based on the last received signal is defined as the final process.

  The receiver may process the signal received a predetermined number of times or notify the user. The receiver may process a signal that has been received the most times while being moved.

  The receiver may include a notification unit that notifies the user when the signal is successfully received or when the presence of the signal is detected in the captured image. The notification means notifies by sound, vibration, display update (pop-up display, etc.), or the like. Thereby, the user can know the presence of the transmitter.

  FIG. 169 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

  For example, a plurality of transmitters configured as displays are arranged adjacent to each other. When transmitting the same signal, the plurality of transmitters synchronize the timing of signal transmission and transmit signals from the entire surface as shown in FIG. With this configuration, since a plurality of displays are observed as one large transmitter in the receiver, the receiver can receive signals at a higher speed and from a longer distance. When a plurality of transmitters transmit different signals, as shown in FIG. 169 (b), the plurality of transmitters transmit signals by providing a buffer band (non-transmission area) in which signals are not transmitted. With this configuration, the receiver can recognize that the plurality of transmitters are different from each other with a buffer band therebetween, and can receive different signals.

  170 is a diagram illustrating an example of a signal transmission method in Embodiment 6. FIG.

  As shown in FIG. 170 (a), the liquid crystal display is provided with a backlight extinction period, and by changing the state of the liquid crystal while the backlight is extinguished, the image in the state change is made invisible and dynamic resolution is performed. A feeling can be heightened. For a liquid crystal display performing such backlight control, a signal is superimposed in accordance with the lighting cycle of the backlight, as shown in FIG. Reception efficiency can be improved by continuously transmitting a set of data (Hdr, Data, FCS). In addition, the light emitting portion is in a bright state (Hi) at the beginning and the end of the backlight lighting period. This is because if the light emitting unit is in a dark state (Lo) continuously with the backlight extinction period, the receiver cannot determine whether Lo is transmitted as a signal or whether the signal is in the dark state due to the backlight extinction period.

  During the backlight extinction period, a signal with a low average luminance may be superimposed.

  By superimposing the signal, the average luminance changes compared to the case where the signal is not superimposed. Therefore, the average luminance is adjusted to be equal by increasing / decreasing the backlight lighting period or increasing / decreasing the luminance during backlight lighting.

  FIG. 171 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

  The liquid crystal display can reduce a change in luminance of the entire screen by performing backlight control at different timings for each position. This is called backlight scanning. The backlight scan is normally performed so that the backlight is turned on in order from the end as shown in FIG. At this time, a captured image 8802a is obtained. However, in the captured image 8802a, the portion where the bright line exists is divided, and it may not be estimated that the entire screen of the display is one transmitter. Therefore, as shown in FIG. 171 (b), all the portions that emit light (the signal is superimposed) are displayed when the vertical axis represents the spatial axis in the backlight scanning division direction and the horizontal axis represents the time axis. A captured image 8802b can be obtained by setting the order of backlight scanning so as to be connected. In the captured image 8802b, all the bright line portions are connected, and it can be easily estimated that the transmission signal is from one transmitter. In addition, since the number of bright lines that can be continuously received increases, signals can be received quickly and remotely. In addition, since the size of the transmitter can be easily estimated, the position of the receiver can be accurately estimated from the position, size, and angle of the transmitter in the captured image.

  FIG. 172 is a diagram illustrating an example of a signal transmission method in Embodiment 6.

  In a time-division backlight scan, the backlight lighting period is short, and light is emitted on the graph with the vertical axis representing the spatial axis in the backlight scan division direction and the horizontal axis representing the time axis (the signal is superimposed). ) Portions cannot be connected, a signal is superimposed on each light emitting portion in accordance with the light emission timing of the backlight as in the case of FIG. At this time, by controlling the backlight so that the distance from the other backlight lighting portion on the graph is maximized, it is possible to prevent the bright lines in the adjacent portions from being mixed.

  FIG. 173 is a diagram for describing a use case in the sixth embodiment. The system in this embodiment includes a lighting device 100 that performs visible light communication, a wearable device 101 having a visible light communication function, a smartphone 102, and a server 103.

  An object of the present embodiment is to reduce a time required for shopping by omitting a series of troubles when a user performs shopping at a store using visible light communication. Conventionally, when purchasing a product at a store, it is necessary to search the store site in advance to obtain coupon information. In addition, there is a problem that it takes time to search for a product that is the coupon target in the store.

  As shown in FIG. 173, the lighting apparatus 100 transmits its own illumination ID information using visible light communication periodically in front of a store (assuming a home appliance mass retailer as an example). When receiving the illumination ID information, the wearable device 101 of the user transmits the illumination ID information to the smartphone 102 using short-range wireless communication. The smartphone 102 transmits user information and lighting ID information to the server 103 using a mobile line or the like. The smartphone 102 receives point information of a store in front of the user, coupon information, and the like from the server 103. The user browses the information received from the server 103 with the wearable device 101 or the smartphone 102. The user can purchase the product information of the displayed store on the spot or receive guidance to an exhibition place in the store. Hereinafter, it demonstrates in detail using figures.

  FIG. 174 is a diagram illustrating an information table transmitted from the smartphone 102 to the server 103. In addition to the membership number of the store, the store ID information, the transmission time, and the location information held in the smartphone 102, the user's preference information and biometric information, search history, and action history information stored in the smartphone 102 are displayed on the smartphone 102. Will send.

  FIG. 175 is a block diagram of the server 103. The transmission / reception unit 201 receives information transmitted from the smartphone 102. The control unit 202 performs overall control. The member information DB 203 stores a member number, the name and date of birth of the user of the member number, point information, purchase history, and the like. The store DB 204 stores store information such as a store ID, product information sold at the store, store display information, and store map information. The notification information generation unit 205 generates coupon information and recommended product information according to the user's preference.

  FIG. 176 is a flowchart showing the overall processing of the system. The wearable device 101 receives the illumination ID from the luminaire 100 (S301). Next, the wearable device 101 transmits the illumination ID to the smartphone 102 using close proximity wireless communication such as Bluetooth (registered trademark) (S302). Next, the smartphone 102 transmits the history information of the user, the membership number, and the lighting ID shown in FIG. 174 to the server 103 (S303). When the server 103 receives the data, the data is first transmitted to the control unit 202 (S304). Next, the control unit 202 refers to the member information DB 203 for the member number, and acquires member information (S305). Next, the control unit 202 inquires of the store DB 204 for the illumination ID and acquires store information (S306). The store information includes product information in stock at the store, information on products that the store wants to promote, coupon information, store map information, and the like. The control unit 202 transmits member information and store information to the notification information generation unit (S307). The notification information generation unit 205 generates advertisement information suitable for the user from the member information and the store information, and transmits it to the control unit 202 (S308). The control unit 202 transmits member information and advertisement information to the transmission / reception unit 201 (S309). Member information includes user point information, expiration date information, and the like. The transmission / reception unit 201 transmits member information and advertisement information to the smartphone 102 (S310). The smartphone 102 displays the received information on the display screen (S311).

  Furthermore, the smartphone 102 transfers the information received from the server 103 to the wearable device 101 (S312). If the notification setting of the wearable device 101 is ON, the wearable device 101 displays information (S314). In addition, when a wearable device displays information, it is desirable to urge the user to be alerted by vibration or the like. This is because the user does not necessarily enter the store, and therefore the user may not notice even if coupon information is transmitted.

  FIG. 177 is a diagram illustrating an information table transmitted from the server 103 to the smartphone 102. The store map DB is in-store guidance information such as which product is displayed at which position in the store. The product information of a store includes information on products in stock at the store and price information of products. The user member information includes user point information and membership card expiration date information.

  FIG. 178 is a diagram illustrating a screen flow displayed on the wearable device 101 from when the user receives information from the server 103 in front of the store until when the user actually purchases the product. In front of the store, points given when the user visits the store and coupon information are displayed. When the user taps the coupon information, the information transmitted from the server 103 is displayed according to the user's preference. For example, when the user taps the TV, recommended TV information is displayed. When the purchase button is pressed here, a receiving method selection screen is displayed, and it is possible to select delivery to home or receiving in the store. In this embodiment, since the store where the user is located is known, there is an advantage that it can be received at the store. When the guidance to the sales floor is selected in the flow 403, the wearable device 101 shifts to the Guide Mode. In this mode, the user is guided to a specific place using an arrow or the like, and the user can be guided to a place where the selected product is actually displayed. When guided in front of the product shelf, the wearable device 101 transitions to a screen for inquiring whether purchase is necessary. The user can make a purchase decision after trying the size, color, and convenience by actually viewing the product.

  The visible light communication in the present invention can specify the position of the user with high accuracy. For this reason, for example, as shown in FIG. 179, a warning can be given when the user is likely to enter a dangerous area in the factory. Furthermore, since it is possible to determine whether or not to issue a warning on the wearable device side, it is possible to construct a warning system with a high degree of freedom, such as giving a warning to a child under a specific age, for example.

(Embodiment 7)
FIG. 180 is a diagram showing a service providing system using the reception method described in the embodiment already described.

  First, another company B or an individual ex8001 requests the company A ex8000 managing the server ex8002 to distribute information to the mobile terminal. For example, the mobile terminal that has made visible light communication with signage is requested to distribute detailed advertisement information, coupon information, or map information. The company A ex8000 that manages the server manages information distributed to the mobile terminal in association with arbitrary ID information. The portable terminal ex8003 acquires ID information from the subject ex8004 by visible light communication, and transmits the acquired ID information to the server ex8002. The server ex8002 transmits information corresponding to the ID information to the portable terminal and counts the number of times the information corresponding to the ID information is transmitted. The company A ex8000 managing the server charges the requested company B and the individual ex8001 for a fee corresponding to the counted number. For example, the larger the count number, the larger the amount to be charged.

  FIG. 181 is a flowchart illustrating a service provision flow.

  In Step ex8000, the company A that manages the server receives a request for information distribution from another company B. Next, in Step ex8001, in the server managed by the company A, the information received in the distribution request is associated with specific ID information. In Step ex8002, the mobile terminal receives specific ID information from the subject by visible light communication, and transmits the specific ID information to a server managed by the company A. The details of the visible light communication method have already been described in other embodiments, and will be omitted. The server transmits information corresponding to the specific ID information transmitted from the mobile terminal to the mobile terminal. In Step ex8003, the server counts the number of times of information distribution. Finally, in Step ex8004, the company B is charged a fee according to the count number of information distribution. In this way, by charging according to the count number, it becomes possible to charge Company B with an appropriate fee according to the advertising effect of information distribution.

  FIG. 182 is a flowchart showing service provision in another example. The description of the same steps as those in FIG. 181 is omitted.

  In Step ex8008, it is determined whether or not a predetermined time has elapsed from the start of information distribution. If it is determined that the time is within the predetermined time period, the company B is not charged in Step ex8011. On the other hand, if it is determined that the predetermined period has elapsed, Step ex8009 counts the number of times the information has been distributed. In Step ex8010, the company B is charged a fee corresponding to the information distribution count. Thus, since information is distributed free of charge within a predetermined period, company B can receive a billing service after confirming the advertising effect and the like.

  FIG. 183 is a flowchart illustrating service provision in another example. The description of the same steps as those in FIG. 182 is omitted.

  In Step ex8014, the number of times information is distributed is counted. If it is determined in Step ex8015 that the predetermined period has not elapsed since the start of information distribution, charging is not performed in Step ex8016. On the other hand, if it is determined that the predetermined period has elapsed, in Step ex8017, it is determined whether or not the number of times information is distributed is equal to or greater than a predetermined value. When the number of times of distributing information is less than a predetermined value, the count number is reset and the number of times of distributing information is counted again. In this case, the company B is not charged for a predetermined period in which the number of times information is distributed is less than a predetermined value. In Step ex8017, if the count number is equal to or larger than the predetermined value, the count number is reset once in Step ex8018, and the count is restarted again. In Step ex8019, the company B is charged a fee corresponding to the count number. As described above, when the number of counts during the period in which the distribution is performed free of charge is small, the company B can receive the billing service at an appropriate timing by providing a period of free distribution again. Further, when the company A has a small number of counts, the information content is analyzed. For example, when the information does not correspond to the season, the company A may propose to the company B to change the information content. It becomes possible. In addition, when providing a free information delivery period again, it is good also as a period shorter than the initial predetermined period. By making it shorter than the initial predetermined period, the burden on the company A can be reduced. Moreover, it is good also as a structure which provides a free delivery period again after a fixed period. For example, in the case of information affected by the season, a free delivery period can be provided again after a certain period until the season changes.

  The charging fee may be changed according to the amount of data regardless of the number of times of information distribution. A certain amount of data may be distributed free of charge, and a predetermined amount of data or more may be charged. Further, the billing fee may be increased as the amount of data increases. Further, a management fee may be charged when managing information in association with specific ID information. By charging as a management fee, it is possible to determine the fee when the information distribution is requested.

(Embodiment 8)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

(Modulation method that is easy to receive)
184A, 184B, and 185 are diagrams illustrating an example of signal coding in Embodiment 8. FIG.

The transmission signal is composed of a header (H) and a body (Body). The header includes a unique signal pattern. The receiver finds this unique pattern from the received signal, recognizes which part of the received signal represents the header or body based on the position, and receives the data.

  When the transmission signal is modulated with the pattern of FIG. 184A (a), the receiver can receive data when the header and the subsequent body are received in succession. The length of time that the receiver can continuously receive the signal is determined by the size of the transmitter shown in the captured image (captured image). When the transmitter is small or when the transmitter is imaged from a distance, the time during which the receiver can continuously receive signals is short. If the time that the receiver can continuously receive a signal (continuous reception time) is the same as the transmission time of one block including the header and the body, it is received only when the transmission start time of the header is equal to the reception start time. The machine can receive data. FIG. 184A (a) shows a case where the continuous reception time is slightly longer than the transmission time of one block including the header and the body. The arrow line represents the continuous reception time, and data can be received when a signal is received at the timing indicated by the thick line, but when a signal is received at the timing indicated by the thin line, Since the header and the body are not aligned, data cannot be received.

  Therefore, by modulating the transmission signal with the pattern of (b) in FIG. 184A, it is possible to receive data with wider reception timing. The transmitter transmits a modulated signal with “body, header, body” as a set. Here, two bodies in the same set have the same signal. Even if the receiver does not receive all the signals contained in the body continuously, the body can be restored by connecting the parts of the body before and after the header, so all the signals contained in the header are continuous. Data can be received if it can be received. In FIG. 184A, the reception timing at which data can be received is indicated by a thick line. In the case of (b), it can be seen that data can be received at a wider reception timing than in the case of (a).

  In the case of the modulation scheme in FIG. 184A (b), if the signal length of the body is a fixed value, the receiver can restore the body. Alternatively, if the header includes the signal length information of the body, the receiver can restore the body.

  That is, as illustrated in FIG. 184B, the receiver first finds a header including a unique bright line pattern from a captured image (bright line image) including the bright line. Then, the receiver sequentially reads each signal of the body that follows the header (direction (1) in FIG. 184B). At this time, each time a signal is read, the receiver determines whether or not the signal of the body has been read by the amount corresponding to the signal length of the body. That is, it is determined whether all the signals included in the body have been read. When it is determined that the signal has not been read, the receiver reads the signal that follows the read signal. However, when there is no subsequent signal, the receiver sequentially reads each signal of the body that follows the front of the header (direction (2) in FIG. 184B). Thereby, all signals included in the body are read out. Here, if the signal length of the body is a fixed length, the receiver holds the signal length in advance, and makes the above determination using the signal length. Alternatively, the receiver specifies the signal length of the body from the header, and makes the above determination using the signal length.

  In addition, even when the signal length of the body is variable, by defining the modulation method so that the bodies modulated by the same transmitter have the same signal length, the receiver can determine the signal length between the two headers. The body can be restored by estimating the signal length of the body. At this time, in the modulation method of FIG. 184A (b), signals for two headers and two bodies must be received at a time. However, with the modulation scheme shown in FIG. 185, the signal length of the body can be estimated only by receiving signals for two headers and one body. Note that in FIG. 185, the modulation scheme is a set of “body header header body header 2 (H2)”, and the receiver can receive data if it can continuously receive all the signals included in the header. is there.

  As described above, the transmitter according to the present embodiment includes a second luminance change pattern corresponding to a body that is a part of a signal to be transmitted and a header for specifying the body. A luminance change pattern, and a first luminance change pattern, a second luminance change pattern, and a first luminance change pattern in that order, and the luminance changes according to the respective patterns, whereby the header and the body Send. The transmitter further determines a third luminance change pattern indicating another header different from the header, and the first luminance change pattern, the second luminance change pattern, and the first luminance change pattern. The header, the body, and other headers may be transmitted by changing the luminance according to each pattern in the order of the third luminance change pattern.

(Combination of communication using bright lines and image recognition)
FIG. 186 is a diagram illustrating an example of a captured image in the eighth embodiment.

  The receiver can read a signal from the bright line in the captured image and analyze a part other than the bright line by image processing. Thereby, as an example, the receiver receives a signal from a transmitter configured as digital signage, for example, and even if the same signal is received, it varies depending on the image shown on the transmitter screen at that time An advertisement can be displayed.

  Since bright lines become noise in image processing, image processing may be performed after interpolating the pixel values of the bright line portion from the left and right pixels of the bright line. Further, image processing may be performed on an image excluding the bright line portion.

(How to use an image sensor suitable for receiving visible light signals)
187A to 187C are diagrams each illustrating an example of a structure and operation of a receiver in Embodiment 8.

  As shown in FIG. 187A, 8910a is an image sensor of the receiver. The imaging element includes an effective pixel that is a part that captures an image, an optical black for measuring noise such as dark current, and an invalid area 8910b. Optical black further includes VOB for measuring noise in the vertical direction and HOB for measuring noise in the horizontal direction. Since the bright line is generated in the direction of 8910c (horizontal direction), the bright line cannot be obtained during the exposure time to the VOB or the invalid area 8910b, and a signal cannot be received. Therefore, at the time of visible light communication, the time during which a signal can be received can be increased by switching to an imaging mode in which the VOB and the invalid area 8910b are not used or used at a minimum.

  As shown in FIG. 187B, by not using the VOB and the invalid area 8910b, it is possible to extend the exposure time in the effective pixel area that is an area including the effective pixel. Specifically, as shown in FIG. 187B (a), during normal shooting, one shot image is acquired between time t0 to t10, time t10 to t20, and time t20 to t30. In addition, since the VOB and the invalid area 8910b are also used to acquire each captured image, the exposure time in the effective pixel area (the time during which charges are read out, the shaded portion in FIG. 187B) ) Are times t3 to t10, times t13 to t20, and times t23 to t30.

  On the other hand, in the visible light communication, by not using the VOB and the invalid area 8910b, as shown in FIG. 187B (b), the effective pixel area is only used for the time during which the VOB and the invalid area 8910b are used. The exposure time can be extended. That is, it is possible to increase the time that can be received by visible light communication. As a result, many signals can be received.

  As shown in FIG. 187C (a), during normal photographing, exposure of each exposure line in the effective pixel region is started after a predetermined time m has elapsed since the exposure of the adjacent exposure line is started. On the other hand, as shown in FIG. 187C (b), when the time of exposure in the effective pixel area is extended during visible light communication, the exposure of each exposure line in the effective pixel area is the exposure of the adjacent exposure line. Is started after a predetermined time n (n> m) has elapsed.

  As described above, when performing normal shooting, the receiver according to the present embodiment applies to each of the plurality of exposure lines in the area including the optical black of the image sensor with respect to the exposure line adjacent to the exposure line. The charge is read after a predetermined time has elapsed since the charge was read. Then, when performing the visible light communication, the receiver does not use the optical black for reading out the electric charge, and for each of the plurality of exposure lines in the area other than the optical black in the image sensor, The charge is read after a time longer than the predetermined time from the time when the charge is read to the adjacent exposure line.

  Further, during visible light communication, by setting an imaging mode in which the number of vertical pixels is not reduced by processing such as demosaicing or clipping, the time during which signals can be received can be further increased.

  When imaging is performed in a mode that does not use the VOB or the invalid area 8910b and does not reduce the vertical pixels, the exposure timings of the lower end of the captured image and the upper end of the captured image of the next frame are temporally continuous, and the signal is continuous Can be received automatically. Even when VOB or the like cannot be completely invalidated, the signal can be continuously received by modulating the transmission signal by a method capable of error correction.

  In FIG. 187A, a horizontal bright line appears because a horizontal photodiode is simultaneously exposed. During visible light communication, the horizontal and vertical emission lines can be obtained by alternately repeating this exposure mode and the exposure mode in which the vertical photodiodes are simultaneously exposed. Even if it exists, it can receive a signal stably.

(Continuous signal reception)
FIG. 187D is a diagram illustrating an example of a signal reception method in Embodiment 8.

  The imaging device includes an effective pixel that is a pixel that makes the intensity of received light an image, and an invalid pixel that does not make an image of the intensity of received light, and is used as, for example, a dark current intensity reference. For this reason, in the normal imaging mode, as shown in (a), there is a time during which only invalid pixels receive light, and during that time, signals cannot be received. Therefore, in the visible light communication mode, reception is possible by minimizing the time for which only invalid pixels receive light as shown in (b) or by setting so that effective pixels always receive light as shown in (c). Can take a long time. This also allows continuous reception. In the case of (b), there is a time during which reception is not possible, but by using an error correction code for transmission data, the entire signal can be estimated even when some signals cannot be received.

(Receiving method of signal from transmitter imaged small)
FIG. 187E is a flowchart illustrating an example of a signal reception method in Embodiment 8.

  As shown in FIG. 187E, start is made at step 9000a, a signal is received at step 9000b, and a header is detected at step 9000c. In step 9000d, it is checked whether the data size of the body following the header is known. If YES, the process proceeds to step 9000f. If NO, the process proceeds to step 9000e, the data size of the body following the header is read from the header, and the process proceeds to step 9000f. In step 9000f, it is confirmed whether or not all the signals indicating the body have been received following the header. If YES, the process proceeds to step 9000g, the body part is read from the signal received following the header, and the process ends in step 9000p. To do. If NO, the process proceeds to step 9000h, and it is confirmed whether the data length of the body received after the header and the part received before the header is sufficient. If YES, the process proceeds to step 9000i. The body part is read out by connecting the received part following the header and the received part before the header, and the process ends at step 9000p. If NO, the process proceeds to step 9000j to check whether there is a means for imaging many bright lines from the transmitter. If YES, the process proceeds to step 9000n, where the setting is changed so that many bright lines can be imaged. Return to. If NO, proceed to step 9000k, notify that the transmitter is present but the image size is insufficient, notify the direction of the transmitter in step 9000m, and receive it when approaching there. This is notified, and the process ends at step 9000p.

  By this method, a signal can be stably received even when there are few exposure lines passing through the transmitter in the captured image.

(Captured image size suitable for receiving visible light signals)
188 and 189A are diagrams illustrating an example of a reception method in Embodiment 8.

  When the effective pixel area of the image sensor is 4: 3, when the image is captured at 16: 9, the upper and lower portions of the image are clipped. When the bright line appears in the horizontal direction, the bright line is lost by this clipping, and the time during which the signal can be received is shortened. Similarly, when the effective pixel area of the image sensor is 16: 9, if the image is captured at 4: 3, the left and right portions of the image are clipped, and the time for receiving a signal is shortened when the bright line appears in the vertical direction. Therefore, the aspect ratio at which clipping does not occur, that is, 4: 3 in FIG. 188 and 16: 9 in FIG. 189A is the imaging aspect ratio in the visible light communication mode. Thereby, the receivable time can be increased.

  That is, the receiver in the present embodiment further sets the ratio between the vertical width and the horizontal width of the image obtained by the image sensor. Then, when performing the visible light communication, the receiver determines whether or not the end of the image in the direction perpendicular to the exposure line (bright line) is clipped according to the set ratio, and the end is clipped. If it is determined, the set ratio is changed to a non-clipping ratio that is a ratio at which the end is not clipped. Then, the image sensor of the receiver acquires a bright line image of the non-clipping ratio by photographing the subject whose luminance changes.

  FIG. 189B is a flowchart illustrating an example of a reception method in Embodiment 8.

  In this reception method, the reception time is extended and an imaging aspect ratio for receiving a signal from a small transmitter is set.

  That is, as shown in FIG. 189B, start is made at step 8911Ba, and the imaging mode is changed to the visible light communication mode at step 8911Bb. In step 8911Bc, it is confirmed whether the aspect ratio setting of the captured image is the setting closest to the aspect ratio of the effective pixel. In the case of YES, the process proceeds to Step 8911Bd, and the aspect ratio setting of the captured image is set to the setting closest to the aspect ratio of the effective pixel. The process ends at step 8911Be. In the case of NO, the process proceeds to Step 8911Be and ends. By setting the aspect ratio in this way in the visible light communication mode, the time during which reception is not possible can be reduced. In addition, a signal from a small transmitter or a remote transmitter can be received.

  FIG. 189C is a flowchart illustrating an example of a reception method in Embodiment 8.

  In this reception method, an imaging aspect ratio for increasing the number of samples per time is set.

  That is, as shown to FIG. 189C, it starts at step 8911Ca and changes an imaging mode to visible light communication mode at step 8911Cb. In step 8911Cc, the bright line of the exposure line can be confirmed, but it is confirmed whether the number of samples per time is small and signals cannot be received. In the case of YES, the process proceeds to Step 8911Cd, and the aspect ratio setting of the captured image is set to the setting most different from the aspect ratio of the effective pixel. In step 8911Ce, the imaging frame rate is increased, and the process returns to step 8911Cc. If NO, the process proceeds to step 8911Cf, receives a signal, and ends.

  By setting the aspect ratio in this way in the visible light communication mode, a signal having a high frequency can be received. Also, it can be received even in a noisy environment.

(Reception of visible light signal using zoom)
190 is a diagram illustrating an example of a reception method in Embodiment 8. FIG.

  The receiver finds an area where bright lines exist from the captured image 8913a, and performs zooming so that as many bright lines as possible are generated. As in the captured image 8913b, the number of bright lines can be maximized by enlarging the bright line region in the direction perpendicular to the bright line direction until the bright line region reaches the upper and lower ends of the screen.

  In addition, the receiver may find an area where the bright line is clearly displayed, and perform zooming so that the portion appears large like 8913c.

  In addition, when a some bright line area | region exists in a captured image, it is good also as performing said process sequentially about each bright line area | region. Further, the above processing may be performed for the bright line region designated by the user from the captured image.

(Image data size compression method suitable for visible light signal reception)
FIG. 191 is a diagram illustrating an example of a reception method in Embodiment 8.

  When the image data size needs to be compressed when the captured image (a) is transmitted from the image capturing unit to the image processing unit or transmitted from the image capturing terminal (receiver) to the server, as shown in (c) If the reduction or pixel thinning is performed in a direction horizontal to the bright line, the data size can be compressed without reducing the information amount of the bright line. When reduction or pixel thinning is performed as in (b) or (d), the number of bright lines is reduced or it is difficult to recognize bright lines. Even during image compression, reduction in reception efficiency can be prevented by not compressing in the direction perpendicular to the bright line, or by making the compression rate in the vertical direction smaller than the compression rate in the horizontal direction. Note that the moving average filter may be applied in both the vertical direction and the horizontal direction, and is effective for both data size reduction and noise reduction.

  That is, the receiver in the present embodiment further generates a compressed image by compressing the bright line image in a direction parallel to each of the multiple bright lines included in the bright line image, and transmits the compressed image. .

(Modulation method with high reception error detection accuracy)
FIG. 192 is a diagram illustrating an example of a signal modulation method in Embodiment 8.

  Since error detection using a parity bit detects a 1-bit reception error, it is impossible to detect a mistake between “01” and “10” and a difference between “00” and “11”. In the modulation method of (a), “01” and “10” are easy to be mistaken because only one position of L is different. However, in the modulation method of (b), “01”, “10”, “00” There are two different positions of L for “11”. By using the modulation scheme (b), reception errors can be detected with high accuracy. The same applies to the modulation schemes shown in FIGS.

  That is, in the present embodiment, two luminance change patterns whose timings at which a predetermined luminance value (for example, L) appears are assigned to the same parity signal unit (for example, “01” and “10”). As is not shown, luminance change patterns having different timings are assigned in advance to signal units having different timings. The transmitter according to the present embodiment determines a luminance change pattern assigned to each signal unit included in the signal to be transmitted.

(Change in receiver operation due to different circumstances)
193 is a diagram illustrating an example of operation of a receiver in Embodiment 8. FIG.

  The receiver 8920a operates differently depending on the situation where reception is started. For example, when activated in Japan, a signal modulated by a 60 kHz phase shift keying is received, and data is downloaded from the server 8920d using the received ID as a key. When activated in the United States, a signal modulated by a 50 kHz frequency shift keying is received, and data is downloaded from the server 8920e using the received ID as a key. In the situation where the operation of the receiver is changed, the location (country or building) where the receiver 8920a exists, the base station or the wireless access point (Wi-Fi, Bluetooth (registered trademark) that communicates with the receiver 8920a is registered. ), IMES, etc.) and time.

  For example, the receiver 8920a uses the location information and the last accessed wireless base station (a base station such as a carrier communication network, Wi-Fi, Bluetooth (registered trademark), or IMES) or the last received ID by visible light communication. To server 8920f. The server 8920f estimates the position of the receiver 8920a based on the received information, and receives the transmission algorithm of the transmitter near the position, and ID management that manages the ID of the transmitter near the position Send server information (URI, etc.). The receiver 8920a receives signals from the transmitters 8920b and 8920c using the received algorithm, and makes an inquiry to the received ID management servers 8920d and 8920e using the ID as a key.

  By this method, different types of communication can be performed in units such as countries, regions, and buildings. In addition, when receiving a signal, the receiver 8920a in this embodiment switches a server to be accessed or switches a reception algorithm in accordance with a frequency used for modulation of the signal, as illustrated in FIG. The signal modulation method may be switched.

(Well-known communication of visible light to humans)
194 is a diagram illustrating an example of operation of a transmitter in Embodiment 8. FIG.

  As shown in FIG. 194 (a), the light emitting unit of the transmitter 8921a repeats blinking visible to humans and visible light communication. By performing blinking that is visible to humans, it is possible to inform humans that visible light communication is possible. The user notices that visible light communication is possible because the transmitter 8921a is blinking, performs visible light communication with the receiver 8921b directed to the transmitter 8921a, and performs user registration of the transmitter 8921a.

  That is, the transmitter according to the present embodiment alternately and repeatedly performs a step in which the light emitter transmits a signal according to a change in luminance and a step in which the light emitter blinks so as to be visually recognized by human eyes.

  The transmitter may separately provide a visible light communication unit and a blinking unit (communication status display unit) as shown in FIG.

  The transmitter operates as shown in (c) of FIG. 194 using the modulation method shown in FIG. 77 or 78, so that the light emitting unit blinks for humans while performing visible light communication. Can show. That is, for example, the transmitter repeatedly performs high luminance visible light communication with a brightness of 75% and low luminance visible light communication with a brightness of 1% alternately. For example, when an abnormality or the like occurs in the transmitter and a signal different from usual is transmitted, the operation shown in (c) of FIG. 194 is performed to alert the user without stopping the visible light communication. Can do.

(Expansion of reception range with diffuser)
FIG. 195 is a diagram illustrating an example of a receiver in Embodiment 8.

  FIG. 195 (a) shows the normal mode of the receiver 8922a, and FIG. 195 (b) shows the visible light communication mode of the receiver 8922a. The receiver 8922a includes a diffuser plate 8922b in front of the imaging unit. The receiver 8922a moves the diffuser plate 8922b in front of the imaging unit in the visible light communication mode so that the light source spreads and images are taken. At this time, the position of the light diffusing plate 8922b is adjusted so that light from a plurality of light sources does not overlap. Note that a macro lens or a zoom lens may be used instead of the diffuser plate 8922b. Thereby, the signal of a distant transmitter and a small transmitter can be received.

  Note that the imaging direction of the imaging unit may be moved instead of moving the diffuser plate 8922b.

  Note that the area of the image sensor on which the diffuser plate 8922b is reflected may not be used in the normal imaging mode, but only in the visible light communication mode. As a result, the above-described effect can be obtained without moving the diffuser plate 8922b and the imaging unit.

(Synchronization method of signal transmission from multiple transmitters)
196 and 197 are diagrams illustrating an example of a transmission system according to the eighth embodiment.

  When a plurality of projectors are used for projection mapping or the like, in order to avoid interference, only one projector needs to transmit a signal for one projection or the signal transmission timings of a plurality of projectors must be synchronized. FIG. 196 shows a mechanism for synchronization of transmission timing.

  Projector A and projector B that project onto the same projection surface transmit signals as shown in FIG. The receiver captures and receives this, calculates the time difference between the signal a and the signal b, and adjusts the signal transmission timing of each projector.

  Since the projector A and the projector B are not synchronized at the start of operation, a time during which neither of the signals is transmitted (total pause time) is provided to prevent the signals a and b from being overlapped and cannot be received. As the timing adjustment of the projector proceeds, the signal transmitted by the projector may be changed. For example, the timing can be adjusted efficiently by increasing the total pause time at the start of the operation and shortening the total pause time as the timing adjustment proceeds.

  In order to accurately adjust the timing, it is desirable that the signal a and the signal b are included in one captured image. The imaging frame rate of the receiver is often from 60 fps to 7.5 fps. By setting the signal transmission period to 1 / 7.5 second or less, the signal a and the signal b can be contained in an image captured at 7.5 fps. Further, by setting the signal transmission cycle to 1/60 second or less, the signal a and the signal b can be reliably contained in an image captured at 30 fps.

  FIG. 197 is a diagram illustrating a case where transmitters configured by a plurality of displays are synchronized. An image is captured so that the synchronized display fits in one image, and the timing is adjusted.

(Reception of visible light signal by illuminance sensor and image sensor)
198 is a diagram illustrating an example of operation of a receiver in Embodiment 8. FIG.

  Since the image sensor has higher power consumption than the illuminance sensor, the receiver 8940a activates the image sensor 8940b to receive the signal when the illuminance sensor 8940c detects the signal. As shown in FIG. 198 (a), the receiver 8940a receives the signal transmitted from the transmitter 8940d by the illuminance sensor 8940c. Thereafter, the receiver 8940a activates the image sensor 8940b, receives the transmission signal of the transmitter 8940d by the image sensor, and recognizes the position of the transmitter 8940d. At this time, if the image sensor 8940b receives a part of the signal and if the part of the signal is the same as the signal received by the illuminance sensor 8940c, the receiver 8940a temporarily determines that the same signal has been received. For example, display of the current position. The determination is completed when all the signals are received by the image sensor 8940b.

  At the time of provisional determination, it may be displayed that the determination is not completed. For example, the display of the current position is made translucent, or the position error is displayed.

  The part of the signal can be determined as, for example, 20% of the entire signal length or an error detection code part.

  In the situation shown in FIG. 198 (b), the illuminance sensor 8940c cannot receive the signal due to interference, but can recognize that the signal exists. For example, when the sensor value of the illuminance sensor 8940c is Fourier-transformed, if a peak appears in the modulation frequency of the transmission signal, it can be estimated that the signal exists. When the receiver 8940a estimates that a signal is present from the sensor value of the illuminance sensor 8940c, the receiver 8940a activates the image sensor 8940b and receives signals from the transmitters 8940e and 8940f.

(Reception start trigger)
199 is a diagram illustrating an example of operation of a receiver in Embodiment 8. FIG.

  Since power is consumed when an image sensor or illuminance sensor (hereinafter collectively referred to as a light receiving sensor) is activated, it is stopped when not needed and activated when necessary to improve power consumption efficiency. Can be improved. Since the power consumption of the illuminance sensor is less than that of the image sensor, the illuminance sensor may be always activated and control only the image sensor.

  In FIG. 199 (a), the receiver 8941a detects movement from the sensor value of the 9-axis sensor, activates the light receiving sensor, and starts reception.

  In FIG. 199 (b), the operation of tilting the receiver is detected from the sensor values of the 9-axis sensor, and the light receiving sensor on the side facing upward is activated to start reception.

  In FIG. 199 (c), the operation of protruding the receiver is detected from the sensor values of the 9-axis sensor, and the light receiving sensor in the protruding direction is activated to start reception.

  In (d) of FIG. 199, from the sensor values of the 9-axis sensor, an operation of turning the receiver upward or a motion of shaking is detected, and the light receiving sensor on the side directed upward is activated to start reception.

  That is, the receiver in the present embodiment further determines whether or not the receiver (receiving device) has been moved in a predetermined manner, and determines that the receiver has been moved in a predetermined manner. Start the sensor.

(Reception start gesture)
FIG. 200 is a diagram illustrating an example of a gesture operation for starting reception according to the communication method.

  For example, the receiver 8942a configured as a smartphone detects, from the sensor value of the 9-axis sensor, an operation of standing the receiver and sliding it in the horizontal direction or an operation of repeating the sliding in the horizontal direction. Thereafter, the receiver 8942a starts reception, and acquires the position of the transmitter 8942b based on the received ID. Thereafter, the receiver 8942a acquires its position from the relative positional relationship between the plurality of transmitters 8942b and itself. By sliding the receiver 8942a, the plurality of transmitters 8942b can be stably imaged, and the self-position can be estimated with high accuracy by the triangulation method.

  This operation may be performed only when the home screen is in the foreground. Thereby, it is possible to prevent this communication from being performed against the user's intention while using another application.

(Application examples for car navigation systems)
201 and 202 are diagrams illustrating an example of an application example of the transmission and reception system in the eighth embodiment.

  For example, the transmitter 8950b configured as a car navigation system transmits information for wireless connection to the transmitter 8950b, such as Bluetooth (registered trademark) pairing information, a Wi-Fi SSID and password, and an IP address. For example, the receiver 8950a configured as a smartphone establishes a wireless connection with the transmitter 8950b based on the received information, and subsequent exchanges are performed via this wireless connection.

  For example, the user inputs a destination, shop information to be searched, or the like to the smartphone 8950a, the smartphone 8950a transmits the input information to the car navigation 8950b via a wireless connection, and the car navigation 8950b displays route information. For example, the smartphone 8950a operates as a controller of the car navigation 8950b, and controls music and video that the car navigation 8950b reproduces. Further, for example, music and video held by the smartphone 8950a are reproduced by the car navigation system 8950b. Further, for example, the car navigation system 8950b acquires peripheral store information and road congestion information and displays the information on the smartphone 8950a. For example, when an incoming call is received, the smartphone 8950a performs a call process using a microphone and a speaker of the car navigation 8950b that are wirelessly connected. Note that the smartphone 8950a may perform the above operation by establishing a wireless connection when an incoming call is received.

  The car navigation system 8950b automatically establishes a wireless connection with a registered terminal when the wireless connection is set to the automatic connection mode. When not in the automatic connection mode, the car navigation system 8950b transmits connection information using visible light communication and waits for a connection. Note that the car navigation system 8950b may also wait for a connection by transmitting connection information using visible light communication even in the automatic connection mode. When the car navigation system is manually connected, the automatic connection mode may be canceled, and further, the connection with the automatically connected terminal may be disconnected.

(Application examples for content protection)
FIG. 203 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  For example, the transmitter 8951b configured as a television transmits content protection information held by itself or a connected device 8951c. For example, the receiver 8951a configured as a smartphone receives the content protection information, and after that, for a predetermined time, the content protection is performed so that the content protected by the content protection information of the transmitter 8951b or the device 8951c can be played back. I do. Thereby, the content held in another device owned by the user can be reproduced by the receiver.

  The transmitter 8951b may store the content protection information in the server, and the receiver 8951a may acquire the content protection information from the server using the received ID of the transmitter 8951b as a key.

  Note that the receiver 8951a may transmit the acquired content protection information to another device.

(Application example as an electronic lock)
FIG. 204A is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8952a receives the ID transmitted from the transmitter 8952b and transmits it to the server 8952c. When the ID of the transmitter 8952b is sent from the receiver 8952a, the server 8952c unlocks the door 8952d, opens the automatic door, or moves to the floor registered in the receiver 8952a. Call to the floor with 8952a. Accordingly, the receiver 8952a serves as a key, and the user can perform unlocking or the like before reaching the door 8952d.

  That is, the receiver in the present embodiment acquires a first bright line image that is an image including a plurality of bright lines by photographing a subject whose luminance changes (for example, the above-described transmitter), and acquires the first bright line image acquired. First transmission information (for example, subject ID) is acquired by demodulating data specified by a plurality of bright line patterns included in the bright line image. Then, after the first transmission information is acquired, the receiver causes the door opening / closing drive device to open the door by transmitting a control signal (for example, a subject ID).

  In order to prevent malicious operation, the server 8952c may confirm that the communication partner is the receiver 8952a by using security protection such as a secure element of the receiver 8952a. Also, to confirm that the receiver 8952a is indeed near the transmitter 8952b, when the server 8952c receives the ID of the transmitter 8952b, it issues a command to send a different signal to the transmitter 8952b, The unlocking may be performed when the signal is sent from the receiver 8952a.

  When a plurality of transmitters 8952b configured as lighting devices are arranged along the path toward the door 8952d, the receiver 8952a receives the ID from the transmitters 8952b, thereby receiving the receivers. It can be determined whether 8952a is approaching door 8952d. For example, when the values indicated by the IDs are reduced in the order in which the IDs are acquired, the receiver determines that the door is approaching. Or a receiver pinpoints the position of each transmitter 8952b based on those ID, Furthermore, based on the position of those transmitters 8952b, and the position of those transmitters 8952b reflected in a picked-up image Estimate your position. Then, the receiver determines whether or not the door 8952d is approaching by comparing the position of the door 8952d held in advance with the estimated position of the door 8952d as needed. When the receiver determines that the door 8952d is approaching, the receiver transmits one of the acquired IDs to the server 8952c. As a result, the server 8952c performs processing for opening the door 8952d.

  That is, the receiver in the present embodiment acquires a second bright line image that is an image including a plurality of bright lines by photographing another subject whose luminance changes, and is included in the acquired second bright line image. Second transmission information (for example, IDs of other subjects) is acquired by demodulating data specified by the plurality of bright line patterns. The receiver determines whether or not the receiver is approaching the door based on the acquired first and second transmission information, and determines that the receiver is approaching the door. , ID of any subject).

  FIG. 204B is a flowchart of the information communication method in this embodiment.

  The information communication method in the present embodiment is an information communication method for acquiring information from a subject, and includes steps SK21 to SK24.

  That is, in this information communication method, a plurality of bright lines corresponding to each exposure line included in the image sensor are displayed on the image obtained by photographing the first subject, which is the subject, by the image sensor. A first exposure time setting step SK21 for setting a first exposure time of the image sensor so as to occur according to a change, and the first subject for which the image sensor has changed in luminance is set as the first exposure time. A first bright line image acquisition step SK22 for acquiring a first bright line image, which is an image including the plurality of bright lines, by photographing with one exposure time, and the first bright line image included in the acquired first bright line image A first information acquisition step SK23 for acquiring first transmission information by demodulating data specified by a plurality of bright line patterns; After the transmission information is acquired, and by sending a control signal, and a door control step SK24 to open the door against the opening and closing devices of the door.

  FIG. 204C is a block diagram of the information communication apparatus in this embodiment.

  The information communication device K20 in the present embodiment is an information communication device that transmits a signal by a change in luminance, and includes constituent elements K21 to K24.

  That is, the information communication device K20 is configured so that a plurality of bright lines corresponding to each exposure line included in the image sensor are generated according to a change in luminance of the subject in an image obtained by photographing the subject by an image sensor. The exposure time setting unit K21 that sets the exposure time of the image sensor, and the bright-line image that is an image including the plurality of bright lines is acquired by photographing the subject that changes in brightness at the set exposure time. Bright line image acquisition unit K22 having an image sensor, information acquisition unit K23 that acquires transmission information by demodulating data specified by the plurality of bright line patterns included in the acquired bright line image, and the transmission information Is acquired, the door is opened with respect to the door opening and closing drive equipment by sending a control signal And a that door control unit K24.

  In such an information communication method and information communication apparatus K20 shown in FIGS. 204B and 204C, for example, as shown in FIG. 204A, a receiver including an image sensor can be used like a door key, An electronic lock can be dispensed with. As a result, it is possible to perform communication between various devices including a device having a small computing power.

  In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. For example, the program causes the computer to execute the information communication method shown by the flowchart in FIG. 204B.

(Application example for store visit information transmission)
FIG. 205 is a diagram illustrating an example of application of the transmission and reception system in Embodiment 8.

  The receiver 8953a transmits the ID transmitted from the transmitter 8953b to the server 8953c. The server 8953c notifies the clerk 8953d of the order information associated with the receiver 8953a. The store clerk 8953d prepares goods and the like based on the order information. Thereby, since an order is prepared when a user enters a store etc., the user can receive goods etc. quickly.

(Application example of order control according to location)
FIG. 206 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8954a displays a screen on which an order can be placed only when the transmission signal of the transmitter 8954b is received. Thereby, the store can avoid the order of the customer who is not near.

  Alternatively, the receiver 8954a places an order by adding the ID of the transmitter 8954b to the order information. Thereby, the store can grasp the position of the orderer and grasp the position where the product is delivered. Alternatively, the store can estimate the time until the orderer visits the store. Note that the receiver 8954a may add the travel time to the store calculated from the travel speed to the order information. Further, the receiver may not accept the purchase for an unnatural purchase (for example, purchase of a boarding pass that departs from other than the station at the current location) as judged from the current location.

(Example of application to directions)
FIG. 207 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8955a receives, for example, the transmission ID of the transmitter 8955b configured as a guide board, acquires map data displayed on the guide board from the server, and displays the map data. At this time, the server may transmit an advertisement suitable for the user of the receiver 8955a, and the receiver 8955a may also display this advertisement information. The receiver 8955a displays a route from the current location to a location designated by the user.

(Application example for location communication)
FIG. 208 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8956a receives, for example, an ID transmitted from a transmitter 8956b configured as home or school lighting, and transmits position information acquired using the ID as a key to the terminal 8956c. Thus, the guardian with the terminal 8906c can know that the child with the receiver 8956a has gone home or has arrived at school. Further, the supervisor having the terminal 8956c can grasp the current position of the worker having the receiver 8956a.

(Application log storage and application examples)
FIG. 209 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  The receiver 8957a receives, for example, an ID transmitted from the transmitter 8957b configured as a signboard, acquires coupon information from the server, and displays the coupon information. The receiver 8957a stores subsequent user actions such as saving a coupon, moving to a store displayed on the coupon, shopping at the store, and leaving without saving the coupon. Save to 8957c. As a result, the subsequent behavior of the user who has acquired information from the sign 8957b can be analyzed, and the advertising value of the sign 8957b can be estimated.

(Application example for screen sharing)
210 and 211 are diagrams illustrating an example of application examples of the transmission and reception system in the eighth embodiment.

  For example, the transmitter 8960b configured as a projector or a display transmits information (SSID, wireless connection password, IP address, password for operating the transmitter) for wireless connection to itself. Alternatively, an ID serving as a key for accessing these pieces of information is transmitted. For example, the receiver 8960a configured as a smartphone, a tablet, a laptop computer, or a camera receives the signal transmitted from the transmitter 8960b, acquires the information, and establishes a wireless connection with the transmitter 8960b. This wireless connection may be connected via a router, or may be directly connected by Wi-Fi Direct, Bluetooth (registered trademark), Wireless Home Digital Interface, or the like. The receiver 8960a transmits a screen displayed by the transmitter 8960b. Thereby, the image of the receiver can be easily displayed on the transmitter.

  When the transmitter 8960b is connected to the receiver 8960a, the transmitter 8960b notifies the receiver 8960a that a password is required in addition to the information transmitted by the transmitter in order to display the screen. If is not sent, the transmitted screen may not be displayed. At this time, the receiver 8960a displays a password input screen such as 8960d and allows the user to input the password.

  FIG. 211 is a diagram illustrating an example in which the screen of the transmitter 8961c is displayed on the transmitter 8960b via the receiver 8960a. For example, the transmitter 8961c configured as a notebook personal computer transmits information for connection to itself or an ID associated with the information. The receiver 8960a receives the signal transmitted from the transmitter 8960b and the signal transmitted from the transmitter 8961c, establishes a connection with each transmitter, and displays an image to be displayed on the transmitter 8960b from the transmitter 8961c. Let it transmit. The transmitter 8960b and the transmitter 8961c may directly communicate with each other, or may communicate with each other via the receiver 8960a or a router. Accordingly, even when the signal transmitted from the transmitter 8960b cannot be received by the transmitter 8961c, an image of the transmitter 8961c can be easily displayed on the transmitter 8960b.

  Note that the receiver 8960a may perform the above operation only when the difference between the time when the signal transmitted by the transmitter 8960b is received and the time when the signal transmitted by the transmitter 8961c is received is equal to or less than a predetermined time. Good.

  Note that the transmitter 8961c may transmit an image to the transmitter 8960b only when the correct password is received from the receiver 8960a.

(Application example of position estimation using wireless access point)
FIG. 212 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  For example, the receiver 8963a configured as a smartphone receives the ID transmitted by the transmitter 8963b. The receiver 8963a acquires position information of the transmitter 8963b using the received ID as a key, and estimates its own position from the position and direction of the transmitter 8963b in the captured image. The receiver 8963a receives a signal from a wireless transmitter 8963c configured as, for example, a Wi-Fi access point. The receiver 8963a estimates its own position from the position information of the wireless transmitter 8963c and the radio wave transmission direction information included in the signal. The receiver 8963a can estimate the self-position with high accuracy by estimating the self-position using a plurality of means.

  A method for estimating the self-location using information of the wireless transmitter 8963c will be described. The wireless transmitter 8963c transmits synchronized signals in different directions from a plurality of antennas. In addition, the wireless transmitter 8963c sequentially changes the signal transmission direction. The receiver 8963a estimates that the radio wave transmission direction when the radio wave intensity is the strongest is the direction from the radio transmitter 8963c to itself. Further, a path difference is calculated from a difference in arrival times of radio waves transmitted from different antennas and respectively passing through paths 8963d, 8963e, and 8963f, and the radio transmitter 8963c and itself are calculated from the radio wave transmission angle differences θ12, θ13, and θ23. Calculate the distance. Further, by using the surrounding electric field information and radio wave reflector information, the self-position can be estimated with higher accuracy.

(Position estimation by visible light communication and wireless communication)
FIG. 213 is a diagram illustrating a configuration for performing position estimation using visible light communication and wireless communication. That is, FIG. 213 shows a configuration for performing terminal position estimation using visible light communication and wireless communication.

  The portable terminal (smart phone terminal) acquires the ID of the light emitting unit by performing visible light communication with the light emitting unit. The server inquires of the acquired ID for the position information of the light emitting unit. Thereby, the mobile terminal acquires an actual distance L1 and an actual distance L2, which are distances in the x direction and the y direction, respectively, between the MIMO (multiple-input and multiple-output) access point and the light emitting unit. Further, as already described in other embodiments, the mobile terminal detects the inclination θ1 of the mobile terminal using a gyro sensor or the like.

  On the other hand, when beam forming is performed from the MIMO access point toward the mobile terminal, the beam form angle θ2 is set by the MIMO access point and is a known value. Therefore, the mobile terminal acquires the beam forming angle θ2 by wireless communication or the like.

  As a result, the portable terminal uses the actual distance L1 and the actual distance L2, the inclination θ1 of the portable terminal, and the beam forming angle θ2 to determine the coordinate position (x1, y1) of the portable terminal relative to the MIMO access point. Can be calculated. Since a MIMO access point can form a plurality of beams, it is possible to estimate a position with higher accuracy by using a plurality of beam forming.

  As described above, according to the present embodiment, it is possible to further improve the accuracy of position estimation by using both position estimation by visible light communication and position estimation by wireless communication.

  As described above, the information communication method according to one or more aspects has been described based on the embodiment. However, the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

  In addition, as illustrated in FIGS. 214, 215, and 216, an information communication method according to one embodiment of the present invention may be applied.

  FIG. 214 is a diagram illustrating an example of application of the transmission / reception system in Embodiment 8.

  A camera configured as a receiver for visible light communication performs imaging in a normal imaging mode (Step 1). By this imaging, the camera acquires an image file configured in a format such as EXIF (Exchangeable image file format). Next, the camera performs imaging in the visible light communication imaging mode (Step 2). Based on the bright line pattern in the image obtained by this imaging, the camera acquires a signal (visible light communication information) transmitted by the visible light communication from the transmitter that is the subject (Step 3). Furthermore, the camera obtains information corresponding to the key from the server by using the signal (reception information) as a key and accessing the server (Step 4). The camera transmits a signal (visible light reception data) transmitted from the subject by visible light communication, information acquired from the server, and data indicating a position where the transmitter as the subject is projected in the image indicated by the image file. And the data indicating the time when the signal transmitted by the visible light communication is received (the time in the moving image) are stored as metadata in the above-described image file. Note that when a plurality of transmitters are projected as subjects in an image (image file) obtained by imaging, the camera stores, for each transmitter, some metadata corresponding to the transmitter. Save to file.

  When a display or projector configured as a transmitter for visible light communication displays an image indicated by the above-described image file, the display or projector transmits a signal corresponding to metadata included in the image file by visible light communication. For example, the display or the projector may transmit the metadata itself by visible light communication, or may transmit a signal associated with the transmitter displayed in the image as a key.

  A portable terminal (smart phone) configured as a receiver for visible light communication receives a signal transmitted by visible light communication from the display or projector by capturing an image of the display or projector. If the received signal is the above-described key, the portable terminal uses the key to acquire the transmitter metadata associated with the key from the display, projector, or server. In addition, if the received signal is a signal (visible light reception data or visible light communication information) transmitted from an actual transmitter by visible light communication, the portable terminal receives the visible light from a display, a projector, or a server. Information corresponding to received light data or visible light communication information is acquired.

  FIG. 215 is a flowchart illustrating operation of the camera (receiver) of the transmission / reception system according to the eighth embodiment.

  First, when the camera detects that the imaging button is pressed (step S901), the camera performs imaging in the normal imaging mode (step S902). Then, the camera performs imaging in the visible light imaging mode by increasing the shutter speed to a certain speed or higher, that is, by setting an exposure time shorter than that in the normal imaging mode (step S903). Thereby, the camera acquires a signal transmitted from the subject by visible light communication.

  Thereafter, the camera treats a signal (information) obtained by visible light communication as a key, thereby obtaining information associated with the key from the server (step S905). Next, the camera stores the signal, each piece of information, and each piece of data in a metadata area (such as an area in which EXIF meter data is stored) of an image file obtained by imaging in the normal imaging mode (step S905). . That is, the camera captures a signal acquired by visible light communication, information acquired from a server, and an image of a transmitter that is a subject that has transmitted the signal by visible light communication (an image obtained by imaging in a normal imaging mode). Stores position data indicating the position inside.

  Then, the camera determines whether or not to capture a moving image (step S906). If it is determined to capture a moving image (Y in step S906), the process from step S902 is repeatedly executed. On the other hand, if the camera determines not to capture a moving image (N in step S906), the imaging process ends.

  FIG. 216 is a flowchart illustrating operation of the display (transmitter) of the transmission / reception system according to the eighth embodiment.

  First, the display checks the metadata area of the image file to determine whether there is one or more transmitters shown in the image indicated by the image file (step S911). Here, when the display determines that there are a plurality of transmitters (a plurality of steps S911), the display further determines whether or not the split transmission mode is set as the visible light communication mode (step S912). When the display determines that the divided transmission mode is set (Y in step S912), the display divides the display area (transmission portion) of the display and transmits a signal from each display area (step S914). Specifically, for each transmitter, the display treats the area in which the transmitter is projected, or the area in which the transmitter and its surroundings are projected, as a display area. A signal corresponding to the machine is transmitted by visible light communication.

  On the other hand, when the display determines in step S912 that the divided transmission mode is not set (N in step S912), signals corresponding to each of the plurality of transmitters are transmitted by visible light communication from the entire display area of the display. Transmit (step S913). That is, the display transmits keys associated with a plurality of pieces of information from the entire screen.

  When the display determines that there is one transmitter in step S911 (one of step S911), the display transmits a signal corresponding to the single transmitter from the entire display area of the display through visible light communication. (Step S915). That is, the display transmits from the entire screen.

  Furthermore, when the display accepts, for example, from a portable terminal (smartphone) an access that uses a signal (transmission information) transmitted by visible light communication as a key after any of steps S913 to S915, the display receives the key in the image file. Corresponding metadata is passed to the mobile terminal that is the access source (step S916).

(Summary of this embodiment etc.)
The information communication method in the present embodiment is an information communication method for acquiring information from a subject, and each exposure included in the image sensor is included in an image obtained by photographing the first subject that is the subject by an image sensor. A first exposure time setting step for setting a first exposure time of the image sensor so that a plurality of bright lines corresponding to a line are generated according to a change in luminance of the first subject; A first bright line image acquisition step of acquiring a first bright line image that is an image including the plurality of bright lines by photographing the first subject that changes with the set first exposure time; The first transmission information is acquired by demodulating data specified by the pattern of the plurality of bright lines included in the acquired first bright line image. An information obtaining step of, after said first transmission information is obtained by sending a control signal, and a door control steps to open the door against the opening and closing devices of the door.

  Thus, for example, as shown in FIG. 204A, a receiver including an image sensor can be used like a door key, and a special electronic lock can be dispensed with. As a result, it is possible to perform communication between various devices including a device having a small computing power.

  The information communication method may further include a second image in which the image sensor includes a plurality of bright lines by photographing the second subject whose luminance changes with the set first exposure time. Second transmission information is acquired by demodulating data specified by a pattern of the plurality of bright lines included in the acquired second bright line image and a second bright line image acquiring step of acquiring a bright line image A second information acquisition step; and an approach determination step of determining whether or not a receiving device including the image sensor is approaching the door based on the acquired first and second transmission information. In the door control step, the control signal may be transmitted when it is determined that the receiving device is approaching the door.

  Thereby, for example, as shown in FIG. 204A, the door can be opened only when the receiving device (receiver) approaches the door, that is, only at an appropriate timing.

  Further, in the information communication method, a second exposure time setting step for setting a second exposure time longer than the first exposure time, and the image sensor sets a third subject. A normal image acquisition step of acquiring a normal image on which the third subject is projected by photographing at the second exposure time, and the normal image acquisition step includes an area including optical black of the image sensor For each of the plurality of exposure lines, the charge is read after a predetermined time has elapsed from the time when the charge is read for the exposure line adjacent to the exposure line, and the first bright line image obtaining step is performed. Then, other than the optical black in the image sensor, without using the optical black for reading out charges. For each of the plurality of exposure lines in the region, the charge may be read after elapse of a time longer than the predetermined time from the time when the charge is read for the exposure line adjacent to the exposure line. Good.

  As a result, for example, as shown in FIGS. 187A to 187E, when the first bright line image is acquired, the readout (exposure) of the charge from the optical black is not performed, and thus the area other than the optical black in the image sensor. It is possible to lengthen the time required for reading (exposure) the charge from the effective pixel region. As a result, it is possible to increase the time for receiving a signal in the effective pixel region, and it is possible to acquire many signals.

  Further, in the information communication method, the length in the direction perpendicular to each of the plurality of bright lines in the plurality of bright line patterns included in the first bright line image is less than a predetermined length. A length determination step for determining whether or not there is a frame rate of the image sensor when the pattern length is determined to be less than the predetermined length; A frame rate changing step of changing to a second frame rate that is slower than the first frame rate when the image is acquired; and the image sensor detects the first subject whose luminance changes at the second frame rate. And a third bright line image acquisition step of acquiring a third bright line image that is an image including a plurality of bright lines by photographing at the set first exposure time, and acquisition May include a third information obtaining step of obtaining the first transmission information by demodulating the data identified by said plurality of emission lines pattern included in the third bright line image.

  Thus, for example, as shown in FIG. 224A, when the signal length indicated by the bright line pattern (bright line region) included in the first bright line image is less than one block of the transmitted signal, for example, The rate is lowered, and the bright line image is acquired again as the third bright line image. As a result, the length of the bright line pattern included in the third bright line image can be increased, and the transmitted signal can be acquired for one block.

  The information communication method further includes a ratio setting step for setting a ratio between a vertical width and a horizontal width of an image obtained by the image sensor, and the first bright line image acquisition step includes the set ratio. A clipping determination step for determining whether or not an end in a direction perpendicular to each exposure line in the image is clipped, and when it is determined that the end is clipped, the ratio set in the ratio setting step Changing the ratio to a non-clipping ratio which is a ratio at which the edge is not clipped, and the image sensor captures the first bright line with the non-clipping ratio by photographing the first subject whose luminance changes. An acquisition step of acquiring an image.

  Accordingly, as shown in FIGS. 188 and 189A to 189C, for example, the ratio of the horizontal width to the vertical width of the effective pixel region of the image sensor is 4: 3, and the ratio of the horizontal width to the vertical width of the image is 16: When the bright line along the horizontal direction appears, that is, when the exposure line is along the horizontal direction, it is determined that the upper end and the lower end of the image are clipped. That is, it is determined that the end of the first bright line image is missing. In this case, the ratio of the image is changed to 4: 3, which is a ratio that is not clipped. As a result, it is possible to prevent the end of the first bright line image from being lost, and a large amount of information can be acquired from the first bright line image.

  The information communication method further includes a compression step of generating a compressed image by compressing the first bright line image in a direction parallel to each of the plurality of bright lines included in the first bright line image. And a compressed image transmission step of transmitting the compressed image.

  Thereby, as shown in FIG. 191, for example, the first bright line image can be appropriately compressed without missing information indicated by a plurality of bright lines.

  The information communication method further determines that the receiving device including the image sensor has been moved in a predetermined manner, and a gesture determination step for determining whether or not the receiving device has been moved in a predetermined manner. In some cases, an activation step of activating the image sensor may be included.

  Accordingly, for example, as shown in FIG. 199, the image sensor can be easily activated only when necessary, and the power consumption efficiency can be improved.

(Embodiment 9)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  FIG. 217 is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9.

  The robot 8970 has, for example, a function as a self-propelled cleaner and a function as a receiver in each of the above embodiments. The lighting devices 8971a and 8971b each have a function as a transmitter in each of the above embodiments.

  For example, the robot 8970 performs cleaning while moving in the room and photographs the lighting device 8971a that illuminates the room. The lighting device 8971a transmits the ID of the lighting device 8971a by changing the luminance. As a result, the robot 8970 receives the ID from the lighting device 8971a and estimates its own position (self-position) based on the ID as in the above embodiments. That is, the robot 8970 moves itself based on the detection result by the 9-axis sensor, the relative position of the lighting device 8971a reflected in the image obtained by photographing, and the absolute position of the lighting device 8971a specified by the ID. Is estimated.

  Further, when the robot 8970 moves away from the lighting device 8971a, the robot 8970 transmits a signal (turn-off command) to instruct the lighting device 8971a to turn off. For example, when the robot 8970 leaves the lighting device 8971a by a predetermined distance, the robot 8970 transmits a turn-off command. Alternatively, the robot 8970 transmits a turn-off command to the lighting device 8971a when the lighting device 8971a does not appear in the image obtained by shooting or when another lighting device appears in the image. When the lighting device 8971a receives a turn-off command from the robot 8970, the lighting device 8971a turns off according to the turn-off command.

  Next, the robot 8970 detects that the lighting device 8971b has been approached based on the estimated self-position while moving and cleaning. That is, the robot 8970 holds information indicating the position of the lighting device 8971b, and when the distance between the self position and the position of the lighting device 8971b is equal to or less than a predetermined distance, the lighting device 8971b. Detecting that you are approaching Then, the robot 8970 transmits a signal (lighting command) for instructing lighting to the lighting device 8971b. When the lighting device 8971b receives the lighting command, the lighting device 8971b lights up in accordance with the lighting command.

  Thus, the robot 8970 can easily perform cleaning by brightening only the surroundings of the robot 8970 while moving.

  FIG. 218 is a diagram illustrating an example of application of the transmitter in Embodiment 9.

  For example, as shown in FIG. 218 (a), a plurality of light emitting areas A to F are displayed side by side on the display, and a signal is transmitted by changing the luminance of each of the light emitting areas A to F. Here, in the example shown to (a) of FIG. 218, the light emission area | regions AF are each rectangular shape, and are arranged along the horizontal direction and the vertical direction. In such a case, a non-luminance changing area that does not change in luminance crosses the display through the light emitting areas A, B, and C and the light emitting areas D, E, and F along the horizontal direction of the display. Yes. Further, another non-luminance changing area that does not change in luminance crosses the display along the vertical direction of the display, passing between the light emitting areas A and D and the light emitting areas B and E. Further, other non-luminance changing areas that do not change in luminance cross the display through the light emitting areas B and E and the light emitting areas C and F along the vertical direction of the display.

  Therefore, when the receiver in each of the above embodiments photographs the display with the exposure line of the receiver oriented in the horizontal direction, the non-luminance change along the horizontal direction of the image (captured image) obtained by the imaging A bright line does not appear in the portion corresponding to the region. That is, in the captured image, the region where the bright line appears (bright line region) becomes discontinuous. Further, when the receiver captures the display with the exposure line oriented in the vertical direction, no bright line appears in the portion corresponding to the two non-luminance change regions along the vertical direction in the captured image. That is, even at this time, the bright line region becomes discontinuous in the captured image. When the bright line region becomes discontinuous in this way, it becomes difficult to receive a signal transmitted due to a change in luminance.

  Therefore, the display 8972 in this embodiment has a function as a transmitter in each of the above embodiments, and each of the plurality of light emitting areas A to F is arranged so as to be shifted so that the bright line areas are continuous.

  For example, as shown in FIG. 218 (b), the display 8972 arranges the upper light emitting areas A, B, and C and the lower light emitting areas D, E, and F in the horizontal direction. Or as shown to (c) of FIG. 218, the display 8972 displays the light emission area | region AF of parallelogram or a rhombus, respectively. Accordingly, it is possible to eliminate the non-luminance change region that crosses the display 8972 through the light emitting regions A to F along the vertical direction of the display 8972. As a result, even if the receiver captures the display 8972 with the exposure line oriented in the vertical direction, the bright line region can be continued in the captured image.

  Furthermore, as shown in (d) and (e) of FIG. 218, the display 8972 may be arranged by shifting each of the light emitting areas A to F in the vertical direction. Thereby, it is possible to eliminate the non-luminance changing region that crosses the display 8972 through the light emitting regions A to F along the horizontal direction of the display 8972. As a result, even if the receiver captures the display 8972 with the exposure line oriented in the horizontal direction, the bright line region can be continued in the captured image.

  As shown in FIG. 218 (f), the display 8972 may display hexagonal light emitting areas A to F so that the sides of the respective areas are parallel to each other. Even in such a case, the non-luminance change region that crosses the display 8972 through the light emitting regions A to F along the horizontal direction and the vertical direction of the display 8972 can be eliminated in the same manner as described above. As a result, even if the receiver shoots the display 8972 with the exposure line oriented in the horizontal direction or the display 8972 shoots with the exposure line oriented in the vertical direction, The region can be continuous.

  FIG. 219 is a flowchart of the information communication method in this embodiment.

  The information communication method in the present embodiment is an information communication method for transmitting a signal according to a luminance change, and includes steps SK11 and SK12.

  That is, in this information communication method, by modulating the signal to be transmitted, a determination step SK11 for determining a luminance change pattern, and by changing the luminance of the plurality of light emitters according to the determined luminance change pattern, And a transmission step SK12 for transmitting the signal to be transmitted. In the surface on which the plurality of light emitters are arranged, a non-luminance changing region that is outside the plurality of light emitters and does not change in luminance is along the at least one of the vertical direction and the horizontal direction of the surface. The plurality of light emitters are disposed on the surface so as not to cross the surface through the plurality of light emitters.

  FIG. 220 is a block diagram of the information communication apparatus in this embodiment.

  The information communication device K10 in the present embodiment is an information communication device that transmits a signal by a change in luminance, and includes constituent elements K11 and K12.

  In other words, the information communication device K10 modulates the signal to be transmitted to determine a luminance change pattern, and a plurality of light emitters change in luminance according to the determined luminance change pattern. And a transmission unit K12 for transmitting the transmission target signal. In the surface on which the plurality of light emitters are arranged, a non-luminance changing region that is outside the plurality of light emitters and does not change in luminance is along the at least one of the vertical direction and the horizontal direction of the surface. The plurality of light emitters are disposed on the surface so as not to cross the surface through the plurality of light emitters.

  In the information communication method and the information communication apparatus K10 shown in FIGS. 219 and 220 as described above, for example, as shown in FIG. 218, the information is acquired by photographing the surface (display) by the image sensor provided in the receiver. In the captured image, the bright line region can be made continuous. As a result, it is possible to easily receive a signal to be transmitted, and it is possible to perform communication between various devices including a device having a small calculation power.

  In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. For example, the program causes the computer to execute the information communication method shown by the flowchart in FIG.

  FIG. 221A is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9.

  The receiver 8973 is configured as a smartphone having a function as a receiver in each of the above embodiments. As shown in FIG. 221A (a), the receiver 8973 captures the display 8972 and tries to read the bright line appearing in the captured image. Here, when the display 8972 is dark, the receiver 8973 may not be able to read the bright line and may not be able to receive a signal from the display 8972. At this time, the receiver 8973 emits a flash at a predetermined rhythm, as shown in FIG. 221A (b). When the display 8972 receives the flash, as shown in FIG. 221A (c), the display 8972 raises the luminance and performs a bright display. As a result, the receiver 8973 can read the bright line appearing in the photographed image and can receive a signal from the display 8972.

  FIG. 221B is a flowchart illustrating operation of receiver 8973 in Embodiment 9.

  The receiver 8973 first determines whether or not an operation or gesture by the user for starting reception has been received (step S831). Here, when the receiver 8973 determines that it has been received (Y in step S831), the receiver 8973 starts reception by photographing using an image sensor (step S832). Next, the receiver 8973 determines whether or not a predetermined time has elapsed from the start of reception without completion of reception (step S833). Here, when the receiver 8973 determines that the predetermined time has elapsed (Y in step S833), the flashing light is blinked at a predetermined rhythm (step S834), and the processing from step S833 is repeatedly executed. Note that in the case where the process of step S833 is repeated, the receiver 8973 determines whether or not a predetermined time has elapsed since the flashing was blinked without completion of reception. In step S834, instead of flashing the flash, the receiver 8973 generates a predetermined sound having a frequency that is difficult for humans to hear, or transmits a signal to the display 8972 to inform that it is waiting for reception. You may send to the machine.

  FIG. 222 is a diagram illustrating an example of application of a transmitter and a receiver in Embodiment 9.

  The lighting device 8974 has a function as a transmitter in each of the above embodiments. The lighting device 8974 illuminates a route bulletin board 8975 at a railway station, for example, while changing in luminance. The receiver 8973 pointed to the route bulletin board 8975 by the user photographs the route bulletin board 8975. As a result, the receiver 8973 acquires the ID of the route bulletin board 8975, and acquires detailed information about each route described in the route bulletin board 8975, which is information associated with the ID. The receiver 8973 displays a guide image 8973a indicating the detailed information. For example, the guidance image 8973a indicates the distance to the route described on the route bulletin board 8975, the direction toward the route, and the time when the next train arrives on the route.

  Here, when the guide image 8973a is touched by the user, the receiver 8973 displays a supplementary guide image 8973b. This supplementary guide image 8973b is, for example, a user's selection operation of any one of a railway timetable, information on another route different from the route indicated by the guide image 8973a, and detailed information on the station. It is an image for displaying accordingly.

  FIG. 223 is a diagram illustrating an example of application of the transmitter in Embodiment 9.

  Each of the lighting devices 8976a to 8976c has a function as a transmitter in each of the above embodiments, and illuminates a store sign 8977. Here, as shown to (a) of FIG. 223, the illuminating devices 8976a-8976c may each transmit the same ID by changing a brightness | luminance synchronously. Further, as shown in FIG. 223 (b), only the lighting devices 8976a and 8976c arranged at both ends transmit the same ID by changing the luminance in synchronization, and are arranged between the lighting devices. The illuminated lighting device 8976b may illuminate the sign 8977 without transmitting an ID due to a change in luminance. Further, as shown in FIG. 223 (c), in a state where the lighting device 8976b does not transmit the ID, the lighting device 8976a and the lighting device 8976c arranged at both ends transmit different IDs by changing the luminance. May be. In this case, since the lighting device 8976b between the lighting devices 8976a and 8976c does not change the luminance for transmitting the ID, signals from the lighting device 8976a and the lighting device 8976c interfere with each other. Can be prevented. Note that the ID transmitted by the lighting device 8976a and the ID transmitted by the lighting device 8976c are different from each other, but the same information may be associated with these IDs.

  FIG. 224A is a diagram illustrating an example of application of the transmitter and the receiver in Embodiment 9.

  The lighting device 8978 has a function as a transmitter in each of the above embodiments, and constantly transmits a signal by changing the luminance as illustrated in (1) of FIG. 224A.

  The receiver in this embodiment images the lighting device 8978. At this time, as illustrated in FIG. 224A, the imaging range 8979 of the receiver includes the lighting device 8978 and a portion other than the lighting device 8978. That is, the upper range “a” and the lower range “c” in the shooting range 8979 each include a portion other than the lighting device 8978, and the central range “b” in the shooting range 8979 includes the lighting device 8978.

  As shown in (2) and (3) of FIG. 224A, the receiver acquires a photographed image (bright line image) including a plurality of bright lines generated by the luminance change of the lighting device 8978 by photographing with the lighting device 8978. In this bright line image, no bright line appears in the portion corresponding to the upper range a and lower range c, and a bright line appears only in the portion corresponding to the central range b.

  Here, when the receiver captures the lighting device 8978 at a frame rate of 30 fps, for example, as shown in (2) of FIG. 224A, the length b of the bright line area in the bright line image is short, and the receiver has, for example, 15 fps. When the illumination device 8978 is photographed at the frame rate of 1, the length b of the bright line region in the bright line image is long as shown in (3) of FIG. The length of the bright line region (the bright line pattern) is a length in a direction perpendicular to each bright line included in the bright line region.

  Therefore, the receiver in the present embodiment captures the lighting device 8978 at a frame rate of 30 fps, for example, and determines whether or not the length b of the bright line area in the bright line image is less than a predetermined length. The predetermined length is, for example, a length corresponding to one block of a signal transmitted by the luminance change of the lighting device 8978. When the receiver determines that the length is less than the predetermined length, the receiver changes the frame rate to, for example, 15 fps. Accordingly, the receiver can collectively receive signals for one block from the lighting device 8978.

  FIG. 224B is a flowchart illustrating operation of the receiver in Embodiment 9.

  First, the receiver determines whether or not a bright line is included in the photographed image, that is, whether or not a stripe by the exposure line is photographed (step S841). Here, if the receiver determines that the image is being captured (Y in step S841), the receiver determines which imaging mode (imaging mode) is set (step S842). When the receiver determines that the imaging mode is the intermediate imaging mode (intermediate mode) or the normal imaging mode (normal imaging mode), the receiver changes the imaging mode to the visible light imaging mode (visible light communication mode) (step S843). .

  Next, the receiver determines whether or not the length in the direction perpendicular to the bright line in the bright line region (bright line pattern) is equal to or longer than a predetermined length (step S844). That is, the receiver determines whether or not there is a stripe region having a predetermined size or more in a direction perpendicular to the exposure line. If it is determined that the length is not greater than or equal to the predetermined length (N in step S844), the receiver determines whether the optical zoom is available (step S845). If it is determined that the optical zoom can be used (Y in step S845), the receiver performs the optical zoom so that the bright line area becomes longer, that is, the fringe area is enlarged (step S846). ). On the other hand, if it is determined that the optical zoom cannot be used (N in step S845), the receiver determines whether or not the Ex zoom (Ex optical zoom) can be used (step S847). If it is determined that the Ex zoom can be used (Y in step S847), the Ex zoom is performed so that the bright line region becomes longer, that is, the fringe region is enlarged (step S848). On the other hand, if it is determined that the Ex zoom cannot be used (N in step S847), the receiver decreases the imaging frame rate (step S849). Then, the receiver receives the signal by photographing the lighting device 8978 at the set frame rate (step S850).

  In the example shown in FIG. 224B, the frame rate is decreased when the optical zoom and the Ex zoom cannot be used. However, the frame rate may be decreased when the zoom can be used. Ex zoom is a function that limits the use area of the image sensor, narrows the angle of view, and makes the apparent focal length telephoto.

  225 is a diagram illustrating operation of a receiver in Embodiment 9. FIG.

  When the illumination device 8978 which is a transmitter is small in the captured image 8980a, the receiver can acquire a captured image 8980b in which the illumination device 8978 is greatly projected by using optical zoom or Ex zoom. it can. That is, the receiver can acquire a bright line image (photographed image) having a bright line region that is long in a direction perpendicular to the bright line by using optical zoom or Ex zoom.

  226 is a diagram illustrating an example of application of a transmitter in Embodiment 9. FIG.

  The transmitter 8981 has a function as a transmitter in each of the above embodiments, and communicates with the operation panel 8982, for example. The operation panel 8982 includes a transmission switch 8982a and a power switch 8982b.

  When the transmission switch 8982a is turned on, the operation panel 8982 instructs the transmitter 8981 to perform visible light communication. Upon receiving the instruction, the transmitter 8981 transmits a signal by changing the luminance. When transmission switch 8982a is turned off, operation panel 8982 instructs transmitter 8981 to stop visible light communication. When the transmitter 8981 accepts the instruction, the transmitter 8981 stops the signal transmission without changing the luminance.

  When the power switch 8982b is turned on, the operation panel 8982 instructs the transmitter 8981 to turn on the transmitter 8981. Upon receiving the instruction, transmitter 8981 turns on its own power supply. For example, when the transmitter 8981 is configured as a lighting device, the transmitter 8981 illuminates the surroundings when the power is turned on, and when the transmitter 8981 is configured as a television, the transmitter 8981 is imaged when the power is turned on. Etc. are displayed. Further, when the power switch 8982b is turned off, the operation panel 8982 instructs the transmitter 8981 to turn off the power of the transmitter 8981. When the transmitter 8981 receives the instruction, the transmitter 8981 turns off its own power supply and enters a standby state.

  FIG. 227 is a diagram illustrating an example of application of the receiver in Embodiment 9.

  A receiver 8973 configured as a smartphone has a function as a transmitter in each of the above embodiments, for example, and acquires an authentication ID and an expiration date from the server 8983. If the current time is within the expiration date, the receiver 8773 transmits the authentication ID to the peripheral device 8984 by changing the brightness of the display, for example. The peripheral device 8984 is, for example, a camera, a barcode reader, or a personal computer.

  When the peripheral device 8984 receives the authentication ID from the receiver 8973, the peripheral device 8984 transmits the authentication ID to the server 8983 to request verification. The server 8983 compares the authentication ID transmitted from the peripheral device 8984 with the authentication ID held by itself and transmitted to the receiver 8973, and if they match, notifies the peripheral device 8984 of the match. When the peripheral device 8984 receives a notification of matching from the server 8983, the peripheral device 8984 performs processing such as releasing the lock set for itself, performing payment processing for electronic payment, and logging in.

  FIG. 228A is a flowchart illustrating an example of operation of a transmitter in Embodiment 9.

  The transmitter in the present embodiment has a function as the transmitter in each of the above embodiments, and is configured as, for example, a lighting device or a display. Such a transmitter determines, for example, whether or not the dimming level (brightness level) is below a predetermined level (step S861a). Here, if the transmitter determines that the level is lower than the predetermined level (Y in step S861a), the transmitter stops transmitting a signal due to a luminance change (step S861b).

  FIG. 228B is a flowchart illustrating an example of operation of a transmitter in Embodiment 9.

  The transmitter in this embodiment determines whether or not the dimming level (brightness level) exceeds a predetermined level (step S862a). If the transmitter determines that the predetermined level is exceeded (Y in step S862a), the transmitter starts transmitting a signal due to a change in luminance (step S862b).

  FIG. 229 is a flowchart illustrating an example of operation of a transmitter in this embodiment.

  The transmitter in the present embodiment determines whether or not a predetermined mode is selected (step S863a). For example, the predetermined mode is an eco mode or a power saving mode. Here, when the transmitter determines that the predetermined mode is selected (Y in step S863a), the transmitter stops transmitting the signal due to the luminance change (step S863b). On the other hand, when the transmitter determines that the predetermined mode is not selected (N in step S863a), the transmitter starts transmitting a signal due to a luminance change (step S863c).

  FIG. 230 is a flowchart illustrating an example of operation of the imaging device according to the ninth embodiment.

  The imaging device in the present embodiment is, for example, a video camera, and determines whether or not recording is in progress (step S864a). If the imaging device determines that recording is in progress (Y in step S864a), the imaging device transmits a visible light transmission stop command to the transmitter that transmits a signal due to a change in luminance (step S864b). The transmitter that has received this visible light transmission stop command stops signal transmission (visible light transmission) due to luminance change. On the other hand, when the imaging device determines that the recording is not in progress (N in step S864a), the imaging device further determines whether the recording is stopped, that is, whether the recording is immediately after the recording (step S864c). Here, when the imaging device determines that the recording is stopped (Y in step S864c), the imaging device transmits a visible light transmission start command to the above-described transmitter (step S864d). Upon receiving this visible light transmission start command, the transmitter starts signal transmission (visible light transmission) due to luminance change.

  FIG. 231 is a flowchart illustrating an example of operation of the imaging device according to the ninth embodiment.

  The imaging device in the present embodiment is, for example, a digital still camera, and determines whether the imaging button (shutter button) is half-pressed or whether focus is being performed (step S865a). . Next, the imaging device determines whether or not light appears in a direction along the exposure line of the image sensor provided in the imaging device (step S865b). Here, when it is determined that light and shade appear (Y in step S865b), there is a possibility that a transmitter that transmits a signal due to a luminance change is near the imaging device. Therefore, the imaging device transmits a visible light transmission stop command to the transmitter (step S865c). Next, the imaging device acquires a captured image by performing imaging (step S865d). Then, the imaging device transmits a visible light transmission start command to the above-described transmitter (step S865e). Thereby, the imaging device can acquire a captured image without being affected by the luminance change by the transmitter. In addition, the period in which the transmission of the signal due to the luminance change is stopped is limited to a very short period in which imaging is performed by the imaging device, and thus it is possible to suppress a period during which visible light communication is not possible.

  FIG. 232 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9.

  The transmitter in this embodiment has a function as the transmitter in each of the above embodiments, and transmits a signal by outputting high luminance light (Hi) or low luminance light (Lo) for each slot. To do. Specifically, the slot is a time unit of 104.2 μs. Further, the transmitter transmits a signal indicating 1 by outputting Hi, and transmits a signal indicating 0 by outputting Lo.

  FIG. 233 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9.

  The transmitter described above sequentially outputs PHY (physical layer) frames as signal units by outputting Hi or Lo for each slot. The PHY frame includes an 8-slot preamble, a 2-slot FCS (Frame Check Sequence), and a 20-slot body. Each part included in the PHY frame is transmitted in the order of preamble, FCS, and body.

  The preamble corresponds to a PHY frame header, and includes, for example, “010110111”. The preamble may be composed of 7 slots. In this case, the preamble includes “01011011”. The FCS includes “01” when the number of 1 included in the body is an even number, and includes “11” when the number of 1 included in the body is an odd number. The body includes five symbols consisting of four slots. The symbol includes “0111”, “1011”, “1101”, or “1110” in the case of 4PPM modulation.

  234 is a diagram illustrating an example of signals transmitted by a transmitter in Embodiment 9. FIG.

  The above symbols are converted into 2-bit values by the receiver. For example, the symbols “0111”, “1011”, “1101”, and “1110” are converted into “00”, “01”, “10”, and “11”, respectively. Therefore, the body (20 slots) of the PHY frame is converted into a 10-bit signal. This 10-bit body includes a 3-bit TYPE indicating the type of the PHY frame, a 2-bit ADDR indicating the address of the PHY frame or the body, and a 5-bit DATA indicating the substance of the data. For example, when the type of the PHY frame is TYPE1, TYPE indicates “000”. ADDR indicates “00”, “01”, “10”, or “11”.

  In the receiver, DATA included in the bodies of the four PHY frames is combined. The above ADDR is used for this connection. That is, the receiver includes the DATA included in the body of the PHY frame having ADDR “00”, the DATA included in the body of the PHY frame having ADDR “01”, and the body of the PHY frame having ADDR “10”. 20 bits of data is generated by combining the DATA included in the body and the DATA included in the body of the PHY frame having ADDR “11”. Thereby, four PHY frames are decoded. The generated data includes 16-bit valid data and 4-bit CRC (Cyclic Redundancy Check).

  FIG. 235 is a diagram illustrating an example of signals transmitted by the transmitter in Embodiment 9.

  The types of PHY frames described above include TYPE1, TYPE2, TYPE3, and TYPE4. The length of the body, the length of ADDR, the length of DATA, the number of concatenated DATA (number of concatenations), the length of effective DATA, and the type of CRC differ for each type.

  For example, in the case of TYPE1, TYPE (TYPEBIT) indicates “000”, the body length is 20 slots, the ADDR length is 2 bits, and the DATA length is 5 bits. The number is 4, the length of effective DATA is 16 bits, and the type of CRC is CRC-4. On the other hand, in the case of TYPE2, TYPE (TYPEBIT) indicates “001”, the body length is 24 slots, the ADDR length is 4 bits, and the DATA length is 5 bits. The number is 8, the length of effective DATA is 32 bits, and the type of CRC is CRC-8.

  Visible light communication can be appropriately performed by such signals shown in FIGS. 232 to 235.

  FIG. 236 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9.

  The system in this embodiment includes a transmitter 8991 having the same function as the transmitter in each of the above embodiments, a receiver 8973 configured as, for example, a smartphone, a content sharing server 8992, and an ID management server 8993. With.

  For example, the content creator uploads content such as audio video data indicating a still image or moving image for introducing a product, and product information indicating a manufacturer, a production area, a raw material, or a specification of the product to the content sharing server 8992. To do. Then, the content sharing server 8992 registers the product information in the ID management server 8993 in association with the content ID for identifying the content.

  Next, the transmitter 8991 downloads the content and the content ID from the content sharing server 8992 in accordance with the operation by the user, displays the content, changes the luminance, that is, by visible light communication, that is, sets the content ID. Send. When the user views the content and is interested in the product introduced by the content, the user points the receiver 8973 toward the transmitter 8991 and performs shooting. The receiver 8973 receives the content ID by photographing the content displayed on the transmitter 8991.

  Next, the receiver 8973 accesses the ID management server 8993 and makes an inquiry about the content ID to the ID management server 8993. As a result, the receiver 8973 acquires the product information associated with the content ID from the ID management server 8993 and displays the product information. Here, when receiving an operation prompting the purchase of a product corresponding to the product information, the receiver 8973 accesses the manufacturer of the product and executes a process for purchasing the product.

  Next, the ID management server notifies inquiry information indicating the number of inquiries made to the content ID or the number of accesses to the manufacturer indicated by the product information associated with the content ID. Upon receiving the inquiry information, the manufacturer electronically sends an affiliate reward according to the number of inquiries indicated by the inquiry information to the content creator specified by the content ID via the ID management server 8993 and the content sharing server 8992. Pay by settlement.

  FIG. 237 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9.

  In the case of the example shown in FIG. 236, when content and product information are uploaded, the content sharing server 8992 registers product information in the ID management server 8993 in association with the content ID, but performs such registration. It does not have to be. For example, as shown in FIG. 237, the content sharing server 8992 searches the ID management server for a product ID for identifying the product of the uploaded product information, and embeds the product ID in the uploaded content.

  Next, the transmitter 8991 downloads the content ID and the content ID embedded with the product ID from the content sharing server 8992 in accordance with the operation by the user, displays the content, and changes the luminance, that is, visible light. The content ID and the product ID are transmitted by communication. When the user views the content and is interested in the product introduced by the content, the user points the receiver 8973 toward the transmitter 8991 and performs shooting. The receiver 8973 receives the content ID and the product ID by photographing the content displayed on the transmitter 8991.

  Next, the receiver 8973 accesses the ID management server 8993 and makes an inquiry about the content ID and the product ID to the ID management server 8993. As a result, the receiver 8973 acquires the product information associated with the product ID from the ID management server 8993 and displays the product information. Here, when receiving an operation prompting the purchase of a product corresponding to the product information, the receiver 8973 accesses the manufacturer of the product and executes a process for purchasing the product.

  Next, the ID management server notifies the maker indicated by the product information associated with the product ID of the inquiry information indicating the number of inquiries made for the content ID and the product ID or the number of accesses. To do. Upon receiving the inquiry information, the manufacturer electronically sends an affiliate reward according to the number of inquiries indicated by the inquiry information to the content creator specified by the content ID via the ID management server 8993 and the content sharing server 8992. Pay by settlement.

  238 is a diagram illustrating an example of a system configuration including a transmitter and a receiver in Embodiment 9. In FIG.

  The system in the present embodiment includes a content sharing server 8992a instead of the content sharing server 8992 shown in FIG. 237, and further includes an SNS server 8994. This SNS server 8994 is a server that performs a social networking service, and performs part of the processing performed by the content sharing server 8992 shown in FIG.

  Specifically, the SNS server 8994 acquires the content and product information uploaded from the content creator, searches for the product ID corresponding to the product information, and embeds the product ID in the content. Then, the SNS server 8994 transfers the content in which the product ID is embedded to the content sharing server 8992a. The content sharing server 8992a receives the content transferred from the SNS server 8994, and transmits the content in which the product ID is embedded and the content ID to the transmitter 8991.

  That is, in the example shown in FIG. 238, a unit including the SNS server 8994 and the content sharing server 8992a plays a role as the content sharing server 8992 shown in FIG.

  In the systems shown in FIGS. 236 to 238, an appropriate affiliate reward can be paid appropriately for an advertisement (content) inquired using visible light communication.

  As described above, the information communication method according to one or more aspects has been described based on the embodiment. However, the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

  Hereinafter, the present embodiment will be supplemented.

(Mixed modulation method)
FIG. 239 and FIG. 240 are diagrams illustrating an example of operation of a transmitter in Embodiment 9.

  As shown in FIG. 239, the transmitter modulates the transmission signal with a plurality of modulation schemes, and transmits the modulated signals alternately or simultaneously.

  By modulating and transmitting the same signal with a plurality of modulation schemes, a receiver that supports only one of the modulation schemes can be received. In addition, for example, by using a modulation method with a high transmission rate, a modulation method resistant to noise, or a modulation method with a long communication distance, reception can be performed in an optimum manner according to the environment on the reception side.

  When the receiver supports reception of a plurality of modulation schemes, the receiver receives a signal modulated by a plurality of methods. When the same signal is modulated, the transmitter assigns the same signal ID and transmits the modulated signal. Thereby, the receiver can recognize that the same signal is modulated by different modulation schemes by checking the signal ID, and synthesizes a signal having the same signal ID from a plurality of types of modulation signals. Thus, a signal can be received quickly and accurately.

  For example, the transmitter includes a signal division unit and modulation units 1 to 3. The signal dividing unit divides the transmission signal into the partial signal 1 and the partial signal 2, associates the partial signal 1 with the signal ID, and associates the partial signal 2 with another signal ID. The modulation unit 1 generates a signal indicating a sine wave by performing frequency modulation on the partial signal 1 accompanied by the signal ID. The modulation unit 2 generates a signal indicating a rectangular wave by performing frequency modulation different from that of the modulation unit 1 on the partial signal 1 accompanied by the signal ID. On the other hand, the modulation unit 3 generates a signal indicating a rectangular wave by performing pulse position modulation on the partial signal 2 accompanied by another signal ID.

  As shown in FIG. 240, the transmitter transmits signals modulated by a plurality of modulation schemes together. In the example of FIG. 240, the receiver can receive only the signal modulated by the frequency modulation method using a low frequency by setting the exposure time to be long. The receiver can receive a pulse position modulation method using a high frequency band by setting the exposure time short. At this time, the receiver averages the luminance in the direction perpendicular to the bright line, thereby averaging the received light intensity over time, and can obtain a signal when the exposure time is long. .

(Transmission signal verification and digital modulation)
241 and 242 are diagrams illustrating an example of a structure and operation of a transmitter in Embodiment 9.

  As illustrated in FIG. 241, the transmitter includes a signal storage unit, a signal verification unit, a signal modulation unit, a light emitting unit, an abnormality notification unit, an original key storage unit, and a key generation unit. The signal storage unit stores a transmission signal and a signal conversion value obtained by converting the transmission signal using a verification key described later. A one-way function is used for this conversion. The original key storage unit stores an original key, which is a key source value, for example, as a circuit constant such as a resistance value or a time constant. The key generation unit generates a verification key from the original key.

  The signal verification unit obtains a signal conversion value by converting the transmission signal stored in the signal storage unit using the verification key. Whether or not the signal has been tampered with is verified based on whether or not the signal conversion value obtained here is equal to the signal conversion value stored in the signal storage unit. As a result, simply copying the signal stored in the signal storage unit to another transmitter prevents the transmitter from being forged because the verification key is different in the other transmitter, and the transmitter can be prevented from forgery. I can do it.

  If the signal has been tampered with, the abnormality notification unit displays that fact. As the method, for example, there is a method of blinking the light emitting unit at a period that can be visually recognized by a human or sounding a sound. By limiting the abnormality notification to a predetermined time immediately after the power is turned on, the transmitter can be used for purposes other than transmission even when there is an abnormality in the signal.

  If the signal has not been tampered with, the signal modulator modulates the signal into a light emission pattern. Various modulation methods can be used for this modulation method. Modulation schemes that can be used here include, for example, amplitude shift keying (ASK), phase shift keying (PSK), frequency shift keying (FSK), quadrature amplitude modulation (QAM), and delta modulation (DM). , Minimum deviation modulation (MSK), complementary code modulation (CCK), orthogonal frequency division multiplexing (OFDM), amplitude modulation (AM), frequency modulation (FM), phase modulation (PM), pulse width modulation (PWM) , Pulse amplitude modulation (PAM), pulse density modulation (PDM), pulse position modulation (PPM), pulse code modulation (PCM), frequency hopping spread spectrum (FHSS), direct sequence spread spectrum (DSSS), etc. Select according to the characteristics (analog or digital, whether continuous data transmission, etc.) and required characteristics (transmission speed, noise resistance, transmission distance) . Furthermore, a modulation scheme combining these can be used.

  In the first to ninth embodiments, the same effect can be obtained even when a signal modulated by the modulation method described here is used.

  As illustrated in FIG. 242, the transmitter may include a signal demodulation unit instead of the signal verification unit. In this case, the signal storage unit holds the encrypted transmission signal obtained by encrypting the transmission signal using the encryption key that is paired with the decryption key generated by the key generation unit. The signal demodulation unit decrypts the encrypted transmission signal using the decryption key. With this configuration, it is possible to make it difficult to forge the transmitter, that is, to create a transmitter that transmits an arbitrary signal.

(Embodiment 10)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

(Reception of signals from multiple directions by multiple light receiving units)
FIG. 243 is a diagram illustrating a timepiece equipped with an optical sensor.

  This timepiece is configured as a receiver for visible light communication, and includes a plurality of optical sensors and a condensing lens corresponding to each of the plurality of optical sensors. Specifically, as shown in the cross-sectional view of FIG. 243, a condensing lens is disposed on the upper surface of the optical sensor. In FIG. 243, the condenser lens is disposed with a predetermined inclination. The shape of the condensing lens is not limited to this, and may be other shapes as long as the condensing lens can be condensed. With such a configuration, the optical sensor can collect and receive light from an external light source by the lens. Therefore, even a small optical sensor mounted on a watch can perform visible light communication. In FIG. 243, it is divided into 12 regions, 12 photosensors are mounted, and a condensing lens is arranged on the upper surface of each photosensor. Thus, by dividing the inside of the watch into a plurality of regions and arranging a plurality of photosensors, information from a plurality of light sources can be acquired. For example, in FIG. 243, the light from the light source 1 can be received by the first optical sensor, and the light from the light source 2 can be received by the second optical sensor. Moreover, it is also possible to use a solar cell as an optical sensor. By using a solar battery as an optical sensor, it is possible to perform visible light communication at the same time as performing photovoltaic power generation with a single optical sensor, thereby reducing costs and making it compact. It becomes possible. Furthermore, when a plurality of optical sensors are arranged, information from a plurality of light sources can be acquired at the same time, so that the position estimation accuracy can be improved. In this embodiment, the optical sensor is provided in the timepiece. However, the present invention is not limited thereto, and the optical sensor may be provided in another device as long as the terminal is movable, such as a mobile phone or a mobile terminal.

  FIG. 244 is a diagram illustrating an example of a receiver in Embodiment 10.

  For example, the receiver 9020a configured as a wristwatch includes a plurality of light receiving units. For example, as shown in FIG. 244, the receiver 9020a is arranged in the vicinity of the character indicating 12 o'clock at the light receiving portion 9020b arranged at the upper end portion of the rotating shaft that supports the long hand and the short hand of the watch. Light receiving portion 9020c. The light receiving unit 9020b receives light traveling toward the light receiving unit 9020b along the direction of the rotation axis described above, and the light receiving unit 9020c transmits light traveling toward the light receiving unit 9020c along the direction connecting the rotation axis and the character indicating 12:00. Receive. Accordingly, when the user holds the receiver 9020a in front of the chest as when checking the time, the light receiving unit 9020b can receive light from above. As a result, the receiver 9020a can receive a signal from the ceiling lighting. Further, when the user holds the receiver 9020a in front of the chest as when checking the time, the light receiving unit 9020c can receive light from the front direction. As a result, the receiver 9020a can receive a signal from a signage or the like at the front.

  These light receiving units 9020b and 9020c have directivity, so that signals can be received without interference even when there are a plurality of transmitters at close positions.

  FIG. 245 is a diagram illustrating an example of a receiver in Embodiment 10.

  For example, as shown in FIG. 245 (a), a receiver 9021 configured as a wristwatch includes 17 light receiving elements (light receiving units). These light receiving elements are arranged on the dial. Of these light receiving elements, twelve light receiving elements are arranged at positions corresponding to 1 to 12 o'clock on the dial, and the remaining five are arranged in the central portion on the dial. Yes. These 17 light receiving elements have directivities different from each other, and receive light (signals) in directions corresponding to the respective light receiving elements. Thus, by arranging a plurality of light receiving elements having directivity, the receiver 9021 can estimate the direction of the received signal. Further, as shown in FIG. 245 (b), a prism for guiding light to the light receiving element may be arranged in front of the light receiving element. In other words, the receiver 9021 includes eight light receiving elements arranged at equal intervals on the periphery of the dial and a plurality of prisms that guide light to at least one of the light receiving elements. By providing such a prism, the fine direction of the transmitter can be estimated even with a small number of light receiving elements. For example, when only the light receiving element 9021d among the eight light receiving elements receives light, the receiver 9021 can estimate that there is a transmitter in the direction connecting the center of the dial and the prism 9021a, and the light receiving element 9021d and the light receiving element When 9021e receives the same signal, it can be estimated that there is a transmitter in the direction connecting the center of the dial and the prism 9021b. The watch windshield may be provided with a directivity function or a prism function.

  FIG. 246A is a flowchart of an information communication method according to an aspect of the present invention.

  An information communication method according to an aspect of the present invention is an information communication method in which a mobile terminal acquires information, and includes steps SE11 and SE12.

  That is, in this information communication method, at least one solar battery among the plurality of solar batteries each having directivity provided in the mobile terminal is in a direction according to the directivity of the solar battery. A light receiving step (SE11) for receiving visible light emitted along the line, and an information acquiring step (SE12) for acquiring information by demodulating a signal specified by the received visible light.

  FIG. 246B is a block diagram of a mobile terminal according to one embodiment of the present invention.

  A mobile terminal E10 according to an aspect of the present invention is a mobile terminal that acquires information, and includes a plurality of solar batteries E11 and an information acquisition unit E12 each having directivity. The information acquisition unit E12 receives visible light emitted along a direction according to the directivity of the solar battery E11 by at least one of the solar batteries E11. In some cases, information is obtained by demodulating a signal specified by the received visible light.

  In such an information communication method and portable terminal E10 shown in FIGS. 246A and 246B, the solar battery E11 can be used for power generation while being used as an optical sensor for visible light communication. The cost of the mobile terminal E10 to be acquired can be reduced, and the mobile terminal E10 can be made compact. Moreover, since each of the plurality of solar cells E11 has directivity, the direction in which the transmitter emitting the visible light is present can be estimated based on the directivity of the solar cells E11 that have received visible light. . Further, since each of the plurality of solar cells E11 has directivity, visible light emitted from the plurality of transmitters can be distinguished and received, and information is appropriately acquired from each of the plurality of transmitters. be able to.

  Further, in the light receiving step (SE11), as shown in FIG. 245 (b), the solar battery E11 (9021d, 9021e) is a prism (9021a, 9021b or 9021c) provided in the portable terminal E10 (9021). ) May be received. Thereby, it is possible to estimate with high accuracy the direction in which the transmitter emitting visible light is present while suppressing the number of solar cells E11 provided in the portable terminal E10. Furthermore, as shown in FIG. 245, the portable terminal E10 is a wristwatch, and a plurality of solar cells E11 (light receiving elements) are arranged along the periphery of the dial of the wristwatch, and the plurality of solar cells E11. The directions of visible light received by each of these may be different from each other. Thereby, information can be appropriately acquired by a wristwatch.

(Cooperation between watch-type receiver and smartphone)
FIG. 247 is a diagram illustrating an example of a reception system in Embodiment 10.

  For example, a receiver 9022b configured as a wristwatch is connected to a smartphone 9022a and a glasses-type display 9022c via wireless communication such as Bluetooth (registered trademark). When receiving a signal or confirming the presence of the signal, the receiver 9022b displays information indicating that the signal has been received on the display 9022c. The receiver 9022b transmits the received signal (reception signal) to the smartphone 9022a. The smartphone 9022a acquires data associated with the received signal from the server 9022d, and displays the acquired data on the glasses-type display 9022c.

(Wayway guidance using a watch-type display)
FIG. 248 is a diagram illustrating an example of a reception system in Embodiment 10.

  For example, the receiver 9023b configured as a wristwatch is connected to the smartphone 9022a via wireless communication such as Bluetooth (registered trademark). The receiver 9023b has a dial made up of a display such as a liquid crystal display, and can display information other than the time. The smartphone 9022a recognizes the current location from the signal received by the receiver 9023b, and displays the route and distance to the destination on the display surface of the receiver 9023b.

(Frequency shift keying and frequency multiplexing)
249A, 249B, and 249C are diagrams each illustrating an example of a modulation scheme in Embodiment 10.

  FIG. 249A (a) expresses a specific signal as a specific modulation frequency. The receiving side performs frequency analysis of the light pattern (light source luminance change pattern) to obtain a dominant modulation frequency, and restores the signal.

  As shown in FIG. 249C (a), many values can be expressed by temporally changing the modulation frequency. Since the imaging frame rate of a general image sensor is 30 fps, it can be reliably received by continuing one modulation frequency for 1/30 second or more. Further, as shown in FIG. 249C (b), when changing the frequency, by providing a time during which no signal is superimposed, the receiver can easily recognize the change in the modulation frequency. The light pattern during which no signal is superimposed can be distinguished from the signal superimposed portion by making the brightness constant or setting a specific modulation frequency. If the specific modulation frequency used here is an integer multiple of 30 Hz, the difference image is less likely to appear and the reception process is not hindered. Reception is facilitated by setting the length of time during which no signal is superimposed to be equal to or longer than the signal having the longest period in the optical pattern used for the signal. As an example, if the light pattern with the lowest modulation frequency is 100 Hz, the length of time during which no signal is superimposed is set to 1/100 second or more.

  (B) of FIG. 249A is an example (1) in which a specific bit and a specific modulation frequency are associated with each other, and an optical pattern is expressed as a waveform obtained by superimposing modulation frequencies having a corresponding bit of 1. Specifically, when transmitting information whose first bit is 1, the transmitter changes in luminance with a light pattern having a frequency f1 = 1000 Hz. Further, when transmitting information whose second bit is 1, the transmitter changes in luminance with a light pattern having a frequency f2 = 1100 Hz. Further, when transmitting information whose third bit is 1, the transmitter changes in luminance with a light pattern having a frequency f3 = 1200 Hz. Therefore, for example, when transmitting information consisting of a bit string of “110”, the transmitter changes in luminance with the optical pattern of frequency f2 at time T2, and with the optical pattern of frequency f1 at time T1 longer than time T2. Change. For example, when transmitting information consisting of a bit string of “111”, the transmitter changes in luminance with an optical pattern of frequency f2 at time T2, and with an optical pattern of frequency f3 at time T3 shorter than time T2. Furthermore, the luminance changes with the light pattern of frequency f1 at time T1. In this case, a higher CN ratio (Carrier to Noise Ratio) is required as compared with the modulation method of (a), but more values can be expressed. In example (1), when the number of bits that are turned on is large, that is, when the waveform includes many frequencies, the energy per frequency is reduced and a higher CN ratio is required. There is a problem.

  Therefore, in the example (2) expressing the light pattern, the number of frequencies included in the waveform is limited to a predetermined number or less, that is, the number of frequencies can be varied below the predetermined number. Alternatively, in the example (3) expressing the light pattern, the number of frequencies included in the waveform is limited to a predetermined number. Thereby, the above-mentioned problem can be avoided. In Example (3), since the number of included frequencies is determined, the signal and noise can be separated more easily than in Examples (1) and (2), and the method is most resistant to noise. ing.

When a signal is expressed using n types of frequencies, in the example (1), 2 ⁿ −1 signals can be expressed. Further, in Example (2) to limit the types of frequency by m different, represent a _{(Σ (k = 1~m) n} C k) -1 ways, in Example _(3), signals as n _{C m} be able to.

As a method of superimposing a plurality of modulation frequencies, (i) a method of simply adding the respective waveforms, (ii) a method of performing weighted averaging with weights of the respective waveforms, and (iii) a waveform of each frequency. There is a method to repeat in order. When performing frequency analysis such as discrete cosine series expansion on the receiving side, the peak tends to be smaller at higher frequencies. Therefore, in (ii), weighting is performed by adjusting the peak of each frequency to the same size. Do the average. That is, the higher the frequency, the better. In (iii), instead of repeating outputting the waveform of each frequency once (one cycle at a time), the size of the frequency peak at the time of reception is adjusted by adjusting the ratio of the number of outputs (cycle number). Can be adjusted. The number of cycles to be output may be increased as the frequency is higher, or the output time may be increased as the frequency is higher. By this adjustment, it is possible to make the reception processing easier by aligning the sizes of the frequency peaks, and it is possible to express additional information by giving meaning to the difference in the size of the frequency peaks. For example, when the order of the size of the frequency peak is given, if the number of included frequencies is n, the information amount of log ₂ (n!) Bits can be added. The frequency may be changed every cycle, the frequency may be changed every cycle or every half cycle, the frequency may be changed every constant multiple of a half cycle, or at regular intervals. The frequency may be changed. The timing of changing the frequency may be when the luminance is the highest, when the luminance is the lowest, or when the value is an arbitrary value. Flickering can be suppressed by equalizing the luminance before and after changing the frequency (= changing the luminance continuously). For that purpose, it is only necessary to output the frequency for a time that is an integral multiple of the half wavelength of each frequency to be transmitted. At this time, the time for outputting each frequency is different. In addition, by outputting a signal of a certain frequency for a time that is an integral multiple of a half cycle, it is possible to easily recognize that the frequency is included in the signal on the receiving side by frequency analysis even in the case of digital output. Can do. Flickering is less likely to be perceived by the human eye or camera if the same frequency is sent continuously rather than continuously. For example, when outputting the cycle T ₁ twice, T ₂ twice, and T ₃ once, T ₁ T ₂ T ₃ T ₂ T rather than T ₁ T ₁ T ₂ T ₂ T _{3 1} is better. Instead of repeating output in a predetermined order, it may be output while changing the order. By giving meaning in this order, additional information can also be expressed. This order does not appear in the frequency peak, but this information can be acquired by analyzing the order of the frequencies. When analyzing the frequency order rather than analyzing the frequency peak, it is necessary to set the exposure time short. Therefore, the exposure time may be set short only when additional information is required, or the exposure time may be shortened. Only the receiver that can be set may acquire this additional information.

  FIG. 249B shows a case where the signal of FIG. 249A is expressed by a binary light pattern. As a method of overlapping frequencies, the methods (i) and (ii) have a complicated analog waveform, and even if the analog waveform is binarized, a complicated shape cannot be expressed. For this reason, the receiver cannot obtain an accurate frequency peak, and reception errors increase. In the method (iii), since the analog waveform does not have a complicated shape, the influence of binarization is small, and a relatively accurate frequency peak can be obtained. Therefore, the method (iii) is excellent when using a light pattern digitized with two values or a small number of values. This modulation method can be interpreted as a kind of frequency modulation, focusing on the fact that the signal is expressed by the frequency of the light pattern, and the signal is expressed by adjusting the length of the pulse width. Focusing on this point, it can be interpreted as a type of PWM modulation.

  By setting the unit of time when the luminance changes to a discrete value, transmission and reception can be performed in the same manner as pulse modulation. The average luminance can be increased by setting the interval where the luminance is low as the shortest time unit, not limited to the length of the frequency cycle to be transmitted. At this time, the longer the frequency of the transmission frequency is, the higher the average luminance is. Therefore, the average luminance can be increased by increasing the number of times of output of the frequency having the longer frequency. When the frequency analysis is performed by setting the length of the high luminance section to the length of the transmission frequency cycle minus the length of the low luminance section even if the low luminance section has the same length Shows a frequency peak in the transmission frequency. Therefore, a signal can be received by using a frequency analysis method such as discrete cosine transform without setting the exposure time of the receiver so short.

  As shown in (c) of FIG. 249C, many values can be expressed by temporally changing the superposition of modulation frequencies in the same manner as (a) of FIG. 249C.

  A signal with a high modulation frequency cannot be received unless the exposure time is set short, but a modulation frequency with a certain height can be used without setting the exposure time. In the case of a terminal in which all terminals can receive a signal expressed at a low modulation frequency by transmitting a signal modulated using a frequency from a low modulation frequency to a high modulation frequency, and the exposure time can be set short. By receiving signals up to a high modulation frequency, more information can be received quickly from the same transmitter. Alternatively, when a low frequency modulation signal is found in the normal imaging mode, the entire transmission signal including the high frequency modulation signal may be received in the visible light communication mode.

  Since frequency shift keying and frequency division multiplexing have the effect of not causing flickering in the human eye even when using a lower modulation frequency than expressing the signal depending on the pulse position, use many frequency bands. Can do.

  In the first to tenth embodiments, the same effect can be obtained even when a signal modulated by the reception method / modulation method described here is used.

(Mixed signal separation)
249D and 249E are diagrams illustrating an example of mixed signal separation in the tenth embodiment.

  The receiver has the function (a) of FIG. The light receiving unit receives the light pattern. The frequency analysis unit maps a signal to the frequency domain by performing a Fourier transform on the light pattern. The peak detector detects the peak of the frequency component of the light pattern. If no peak is detected by the peak detector, the subsequent processing is interrupted. The peak time change analysis unit analyzes the time change of the peak frequency. The signal source specifying unit specifies which combination of modulation frequencies of signals transmitted from the same transmitter is detected when a plurality of frequency peaks are detected.

  Thereby, even when a plurality of transmitters are arranged close to each other, reception can be performed while avoiding signal interference. In addition, when light from a transmitter receives light reflected from a floor, wall, ceiling, etc., light from multiple transmitters is often mixed, but even in this case, avoid signal interference. Can be received.

  As an example, when the receiver receives an optical pattern in which the signal from the transmitter A and the signal from the transmitter B are mixed, a frequency peak as shown in FIG. 249D (b) is obtained. Since fA1 disappears and fA2 appears, it can be specified that fA1 and fA2 are signals from the same transmitter. Similarly, it can be specified that fA1, fA2, and fA3 are signals from the same transmitter, and fB1, fB2, and fB3 are signals from the same transmitter.

  By making the time interval at which one transmitter changes the modulation frequency constant, it is possible to easily identify signals from the same transmitter.

  When the timings at which the modulation frequencies of a plurality of transmitters change are equal, the above method cannot identify signals from the same transmitter. Therefore, by changing the time interval for changing the modulation frequency of the transmitter for each individual transmitter, it is possible to avoid the situation where the timings at which the modulation frequencies of the multiple transmitters change are always the same. It becomes possible to specify the signal.

  As shown in (c) of FIG. 249D, by setting the time from when the transmitter changes the modulation frequency to the next change to a value obtained from the current modulation frequency and the modulation frequency before the change, Even when a plurality of transmitters change the modulation frequency at the same timing, it is possible to specify which modulation frequency signal is transmitted from the same transmitter.

  The transmitter may recognize a transmission signal of another transmitter and adjust the modulation frequency change timing so as not to be equal.

  The above method is the same in the same way not only in the case of frequency shift key modulation in which one transmission signal is composed of one modulation frequency, but also in the case where one transmission signal is composed of a plurality of modulation frequencies. The effect is obtained.

  As shown in (a) of FIG. 249E, when the optical pattern is not changed in time by the frequency multiplexing modulation method, signals from the same transmitter cannot be specified, but as shown in (b) of FIG. 249E. In addition, it is possible to specify a signal from the same transmitter from the time change of the peak by including a section without a signal or changing to a specific modulation frequency.

  FIG. 249F is a flowchart illustrating processing of the information processing program in the tenth embodiment.

  This information processing program is a program for changing the luminance of the light emitter (or light emitting unit) of the transmitter described above with the light pattern shown in FIG. 249A (b) or FIG. 249B (b).

  In other words, this information processing program is an information processing program for causing a computer to process information to be transmitted in order to transmit the information to be transmitted by a luminance change. Specifically, the information processing program encodes information to be transmitted to determine the luminance change frequency SA11 and to change the luminance according to the luminance change frequency determined by the light emitter. The computer executes an output step SA12 that outputs a signal indicating the determined frequency of the luminance change so as to transmit the information to be transmitted. In the determination step SA11, a first frequency (for example, frequency f1) and a second frequency (for example, frequency f2) different from the first frequency are respectively determined as the luminance change frequencies. In the output step SA12, the luminous body changes in luminance according to the first frequency at a first time (for example, time T1), and after the first time has elapsed, a second time (for example, time) that is different from the first time. In T2), the signals indicating the first and second frequencies are output as signals indicating the determined frequency of the luminance change so that the luminance changes according to the second frequency.

  Thereby, the information to be transmitted can be appropriately transmitted by the visible light signals having the first and second frequencies. Further, by making the first time different from the second time, transmission according to various situations can be performed. As a result, communication between various devices can be enabled.

  For example, as shown in FIGS. 249A and 249B, the first time is a time corresponding to one cycle of the first frequency. The second time is a time corresponding to one cycle of the second frequency.

  Further, in the output step SA12, the signal indicating the first frequency and the signal indicating the second frequency so that the number of outputs of the signal indicating the first frequency and the signal indicating the second frequency are different. At least one of and may be repeatedly output. Thereby, transmission according to various situations can be performed.

  In addition, in the output step SA12, the signal indicating the first frequency so that the number of outputs of the low frequency signal among the signals indicating the first and second frequencies is larger than the number of outputs of the high frequency signal. And at least one of the signals indicating the second frequency may be repeatedly output.

  Thereby, when the luminous body changes in luminance according to the frequency indicated by each output signal, the luminous body can transmit information to be transmitted with bright luminance. For example, it is assumed that the luminance change time according to the first frequency, which is a low frequency, and the luminance change value according to the second frequency, which is a high frequency, have the same duration of dark luminance. In this case, the luminance change according to the first frequency (that is, the low frequency) takes a longer time for the bright luminance to continue than the luminance change according to the second frequency (that is, the high frequency). Therefore, when many signals indicating the first frequency are output, the light emitter can transmit information to be transmitted with bright luminance.

  In addition, in the output step SA12, the signal indicating the first frequency so that the number of outputs of the high-frequency signal among the signals indicating the first and second frequencies is larger than the number of outputs of the low-frequency signal. And at least one of the signals indicating the second frequency may be repeatedly output. For example, as shown in FIGS. 249A and 249B, the number of outputs of the signal of frequency f2 is greater than the number of outputs of the signal of frequency f1.

  As a result, when the luminous body changes in luminance according to the frequency indicated by each output signal, the reception efficiency of the information transmitted by the luminance change can be increased. For example, when information to be transmitted is transmitted to a receiver by a visible light signal expressed by a plurality of frequencies, the receiver performs frequency analysis such as Fourier transform on an image obtained by imaging. The frequency peak contained in the visible light signal is detected. At this time, peak detection is more difficult as the frequency becomes higher. Therefore, as described above, each signal is output so that the number of outputs of the high-frequency signal among the signals indicating the first and second frequencies is greater than the number of outputs of the low-frequency signal. , High frequency peak detection can be facilitated. As a result, reception efficiency can be improved.

  In the output step SA12, at least one of the signal indicating the first frequency and the signal indicating the second frequency may be repeatedly output so that signals indicating the same frequency are not continuously output. Good. For example, as shown in FIGS. 249A and 249B, the signal indicating the frequency f1 is not continuously output, and the signal indicating the frequency f2 is not continuously output.

  Accordingly, when the luminance of the light emitter changes according to the frequency indicated by each output signal, flickering of light from the light emitter can be made difficult to be caught by human eyes or a camera.

  FIG. 249G is a block diagram of an information processing device in Embodiment 10.

  This information processing apparatus A10 is an apparatus for changing the luminance of the light emitter of the transmitter described above with the light pattern shown in FIG. 249A (b) or FIG. 249B (b).

  That is, the information processing apparatus A10 is an apparatus that processes information to be transmitted in order to transmit the information to be transmitted by a change in luminance. Specifically, the information processing apparatus A10 encodes information to be transmitted, thereby changing the luminance according to the frequency change unit A11 that determines the frequency of the luminance change and the luminance change frequency for which the light emitter is determined. And an output unit A12 that outputs a signal indicating the determined luminance change frequency so as to transmit information to be transmitted. Here, the frequency determination unit A11 determines the first frequency and the second frequency different from the first frequency as the luminance change frequencies. In the output unit A12, the luminance of the light emitter changes in accordance with the first frequency in the first time, and the luminance in accordance with the second frequency in a second time different from the first time after the first time has elapsed. In order to change, a signal indicating the first and second frequencies is output as a signal indicating the frequency of the determined luminance change. Such an information processing apparatus A10 can achieve the same effects as the information processing program described above.

(Operation of home appliances via illumination by visible light communication)
FIG. 250A is a diagram illustrating an example of a visible light communication system in Embodiment 10.

  For example, a transmitter configured as ceiling lighting (illumination equipment) has a wireless communication function such as Wi-Fi or Bluetooth (registered trademark). The transmitter transmits information (such as a light emitting device ID and an authentication ID) for connecting to the transmitter by wireless communication by visible light communication. For example, the receiver A configured as a smartphone (mobile terminal) performs wireless communication with the transmitter based on the received information. The receiver A may be connected to the transmitter using other information, and in that case, the receiver A may not have a reception function. The receiver B is configured as an electronic device (control target device) such as a microwave oven. The transmitter transmits the information of the paired receiver B to the receiver A. Receiver A displays information of receiver B as an operable device. The receiver A transmits an operation command (control signal) of the receiver B to the transmitter through wireless communication, and the transmitter transmits the operation command to the receiver B through visible light communication. Thereby, the user can operate the receiver B via the receiver A. A device connected to the receiver A via the Internet or the like can operate the receiver B via the receiver A.

  Since the receiver B has a transmission function and the transmitter has a reception function, bidirectional communication can be performed. The transmission function may be realized as visible light by light emission, or communication by sound may be performed. For example, the transmitter includes a sound collection unit and can recognize the state of the receiver B by recognizing the sound emitted by the receiver B. For example, the operation end sound of the receiver B is recognized and transmitted to the receiver A, and the receiver A can notify the user by displaying the operation end of the receiver B on the display.

  The receiver A and the receiver B are provided with NFC. The receiver A receives the signal from the transmitter, and then communicates with the receiver B via NFC. The receiver B can receive the signal from the transmitter that transmitted the signal received immediately before. It is registered in the receiver A and the transmitter. This is called pairing of the transmitter and the receiver B. The receiver A registers cancellation of pairing with the transmitter when the receiver B is moved. When receiver B is paired with another transmitter, the newly paired transmitter communicates that information to the previously paired transmitter and releases the previous pairing.

  FIG. 250B is a diagram for describing a use case in Embodiment 10. With reference to FIG. 250B, an embodiment in which the receiving unit 1028 using the modulation method of the PPM method, the FDM method, the FSK method, or the frequency allocation method of the present invention is used will be described.

  First, the light emission operation of the light emitter 1003 which is a lighting device will be described. In a light emitting device 1003 such as a lighting fixture or a TV monitor attached to a ceiling or a wall, an authentication ID is first generated by an authentication ID generation unit 1010 using a random number generation unit 1012 that changes with time. When there is no interrupt process (step 1011) for the ID of the light emitter 1003 and the authentication ID 1004, the light emitter 1003 determines that there is no “transmission data string” sent from the portable terminal 1020, and (1) light emission A machine ID, (2) an authentication ID, and an identifier for identifying whether there is a transmission data string 1009 sent from the electronic device 1040 that is a control target device via the portable terminal 1020, that is, (3) a transmission data string Three flags = 0 are sent from the light emitting unit 1016 such as an LED to the outside continuously or intermittently.

  The transmitted optical signal is received by the photo sensor 1041 of the electronic device 1040 (step 1042), and the electronic device 1040 receives the device ID and authentication ID (device authentication ID and light emitter ID) of the electronic device 1040 in step 1043. Check if it is genuine. If the confirmation result is YES (regular), the electronic device 1040 checks whether the transmission data string flag = 1 (step 1051). Only when the result of the check is YES (transmission data string flag = 1), the electronic device 1040 executes a data command of the transmission data string, for example, a user command such as cooking recipe execution (step 1045).

  Here, a mechanism for optical transmission using the optical modulation method of the present invention of the electronic device 1040 will be described. The electronic device 1040 displays the device ID, the authentication ID for authenticating the device, and the light emitting device ID of the light emitting device 1003 received by the electronic device 1040 as described above, that is, reliably received, on the display unit 1047. It is sent using the LED backlight unit 1050 or the like (step 1046).

  Since the optical signal of the present invention is sent from a display unit 1047 such as a liquid crystal of a microwave oven or a POS machine at a modulation frequency of 60 Hz or more without flickering, it is transmitted to a general consumer with light. I don't know the signal is being sent. Therefore, the display unit 1047 can independently display a menu of a microwave oven, for example.

(ID detection method of light emitting device 1003 that electronic device 1040 can receive)
A user who intends to use a microwave oven or the like receives an optical signal from the light emitter 1003 by the in-camera unit 1017 of the portable terminal 1020, and receives a light-emitting device ID and a light-emitting device authentication ID by the in-camera processing unit 1026. (Step 1027). The light-emitting device ID that can be received by the electronic device 1040 includes the position information using 3G mobile phone radio waves and Wi-Fi, and the light-emitting device ID existing at the position recorded in the cloud 1032 or the mobile terminal. It may be detected (step 1025).

  When the user points the out camera 1019 of the portable terminal 1020 toward the display unit 1047 of the microwave oven 1040, for example, the optical signal 1048 of the present invention can be demodulated using a MOS camera. Faster data can be received by increasing the shutter speed. The reception unit 1028 receives the device ID, authentication ID, service ID, or service cloud URL converted from the device ID of the electronic device 1040 or the device status.

In step 1029, the 3G Wi-Fi communication unit 1031 is used to connect to the external cloud 1032 inside the terminal or using the received URL, and send the service ID and device ID. In the cloud 1032, data corresponding to the device ID and service ID in the database 1033 is searched and sent to the mobile terminal 1020. Based on this data, video data, command buttons, etc. are displayed on the screen of the portable terminal. The user who sees this inputs a desired command by an input method such as pressing a button on the screen (step 1030). If YES (input), BTLE (Blue Tooth (registered trademark))
The transmitter 1022 of the transmission / reception unit 1021 transmits a transmission data string including a device ID such as the electronic device 1040, a device authentication ID, a light emitter ID, a light emitter authentication ID, and a user command in step 1030.

  The light emitting device 1003 receives the “transmission data string” by the reception unit 1007 of the BTLE transmission / reception unit 1004 and detects that the “transmission data string” is received by the interrupt processing unit 1011 (YES in Step 1013). Data of “transmission data string + ID + transmission data flag = 1” is modulated by the modulation unit of the present invention, and light is transmitted from the light emitting unit 1016 such as an LED. When it is not detected that the “transmission data string” has been received (NO in step 1013), the light emitter 1003 continuously transmits the light emitter ID and the like.

  As described above, since the electronic device 1040 has actually received and confirmed that the signal from the light emitter 1003 can be received, the electronic device 1040 can reliably receive the signal.

  In this case, since the light emitting device ID is included in the transmission data string in the interrupt processing unit 1011, it can be seen that there is an electronic device to be transmitted within the light irradiation range of the light emitting device with that ID. Therefore, since the electronic device is sent only from the light emitting device in a very narrow position without sending a signal from another light emitting device, the radio wave space can be used efficiently.

  If this method is not adopted, the Bluetooth signal reaches far away, so that an optical signal is sent from a light emitter located at a different position from the electronic device. Therefore, since a certain light emitter cannot transmit light to other electronic devices to transmit during the light emission period, or interferes with it, this type of solution is effective.

  Next, countermeasures against malfunction of electronic devices are described.

  The photosensor 1041 receives an optical signal in step 1042. First, since the light emitting device ID is checked, the light emission signal of another light emitting device ID can be removed, thereby reducing malfunctions.

  In the present invention, the transmission data string 1009 includes the device ID and device authentication ID of the electronic device to be received. Accordingly, since it is checked in step 1043 whether the device authentication ID and the device ID are the ID of the electronic device 1040, no malfunction occurs. There is an effect that malfunction of a microwave oven or the like due to erroneous processing of a signal transmitted from the electronic device 1040 to another electronic device can be prevented.

  A method for preventing a malfunction in execution of a user command will be described.

  In step 1044, when the transmission data flag = 1, it is determined that there is a user command, and when the transmission data flag = 0, the process stops. When the transmission data flag = 1, the device ID and the authentication ID of the user data string are authenticated, and then the transmission data string such as a user command is executed. For example, the electronic device 1040 takes out the recipe and displays it on the screen. If the user presses the button, the recipe, i.e., 600w for 3 minutes, 200w for 1 minute, and steam cooking for 2 minutes are started without malfunction.

  When the user command is executed, electromagnetic noise of 2.4 GHz is generated in the case of the microwave oven. In order to reduce this, when receiving a command via Bluetooth or Wi-Fi via a smartphone and operating, the intermittent drive unit 1061 stops the output of the microwave oven intermittently, for example, about 100 ms in 2 seconds. During this time, communication such as Bluetooth or Wi-Fi 802.11n becomes possible. For example, when the range is not stopped, sending a stop command from the smartphone to the light emitter 1003 by BTLE is hindered. On the other hand, in the present invention, it is possible to send without being affected by the jamming wave, and the range can be stopped by the optical signal, or the recipe of the range can be changed.

  Since the feature of this embodiment is that a photosensor 1041 with a cost of several yen is added to an electronic device with a display unit, bidirectional communication with a smartphone linked with the cloud can be achieved. There is an effect that it can be installed in white goods and converted into smart appliances. However, although white goods were used as an embodiment, a POS terminal with a display unit may be used, and a similar effect can be obtained with a supermarket electronic price tag or a personal computer.

  Further, in this embodiment, the light emitting device ID can be received only from the illuminator at the top of the electronic device. Since the reception area is narrow, by defining a small zone ID such as Wi-Fi for each light-emitting device and assigning the following ID of the position in each zone, there is also an effect of reducing the number of digits of the ID of the light emitting unit. is there. In this case, the number of digits of the ID of the light emitting device that transmits using the aforementioned PPM, FSK, or FDM of the present invention is reduced, so that an optical signal can be received from a small light source, an ID can be quickly acquired, Can be received.

  FIG. 250C is a diagram illustrating an example of a signal transmission / reception system in Embodiment 10.

  The signal transmission / reception system includes a smartphone (smartphone) that is a multi-function mobile phone, an LED light-emitting device that is a lighting device, a home appliance such as a refrigerator, and a server. The LED light emitter performs communication using BTLE (Bluetooth (registered trademark) Low Energy) and visible light communication using an LED (Light Emitting Diode). For example, the LED light emitter controls a refrigerator or communicates with an air conditioner by BTLE. In addition, the LED light emitter controls a power source of a microwave oven, an air purifier, a television (TV), or the like by visible light communication.

  The television includes a solar power generation element, for example, and uses the solar power generation element as an optical sensor. That is, when a signal is transmitted when the brightness of the LED light emitter changes, the television detects a change in the brightness of the LED light emitter based on a change in the power generated by the solar power generation element. Then, the television acquires the signal transmitted from the LED light emitter by demodulating the signal indicated by the detected luminance change. When the signal is a command indicating that the power is on, the television switches its main power source to ON, and when the signal is a command indicating that the power source is OFF, the television switches its main power source to OFF.

The server can communicate with the air conditioner via the router and the specific low-power radio station (extra-small). Furthermore, since the air conditioner can communicate with the LED light emitter via BTLE, the server can communicate with the LED light emitter. Therefore, the server can switch the power of the TV ON and OFF via the LED light emitter. In addition, the smartphone can control the power supply of the TV via the server by communicating with the server via, for example, Wi-Fi (Wireless Fidelity).

  As shown in FIGS. 250A to 250C, the information communication method according to the present embodiment is such that the mobile terminal (smartphone) uses a control signal (transmission data string) by wireless communication (such as BTLE or Wi-Fi) different from visible light communication. Or a wireless communication step for transmitting a user command) to a lighting device (light emitter), a visible light communication step in which the lighting device performs visible light communication by changing the luminance according to the control signal, and a control target device (electronic An execution step of detecting a luminance change of the lighting device, acquiring a control signal by demodulating a signal specified by the detected luminance change, and executing a process according to the control signal; Including. Thereby, even if the portable terminal cannot change the luminance for visible light communication, the luminance of the lighting device can be changed instead of the portable terminal by wireless communication, and the control target device is appropriately controlled. can do. The mobile terminal may be a wristwatch instead of a smartphone.

(Reception without interference)
FIG. 251 is a flowchart illustrating a reception method in which interference is eliminated in the tenth embodiment.

  First, start is performed in step 9001a, and it is confirmed whether there is a periodic change in the intensity of light received in step 9001b. If YES, the process proceeds to step 9001c. In the case of NO, the process proceeds to Step 9001d to receive a wide range of light by setting the lens of the light receiving unit to a wide angle, and returns to Step 9001b. In step 9001c, it is confirmed whether the signal can be received. If YES, the process proceeds to step 9001e, the signal is received, and the process ends in step 9001g. In the case of NO, the process proceeds to Step 9001f, the lens of the light receiving unit is telephoto, and light in a narrow range is received, and the process returns to Step 9001c.

  By this method, signals from transmitters in a wide direction can be received while eliminating interference of signals from a plurality of transmitters.

(Estimation of transmitter orientation)
FIG. 252 is a flowchart illustrating a transmitter direction estimation method according to the tenth embodiment.

  First, start is performed in step 9002a, the lens of the light receiving unit is set to the maximum telephoto in step 9002b, and it is checked whether there is a periodic change in the intensity of light received in step 9002c. Proceed to 9002d. In the case of NO, the process proceeds to Step 9002e, where a wide range of light is received by setting the lens of the light receiving unit to a wide angle, and the process returns to Step 9002c. In step 9002d, the signal is received, in step 9002f, the lens of the light receiving unit is set to the maximum telephoto position, the light receiving direction is changed along the boundary of the light receiving range, and the direction in which the light receiving intensity is maximized is detected. Estimating that it is in the direction, the process ends at step 9002d.

  With this method, the direction in which the transmitter is present can be estimated. The maximum wide angle may be set first, and the telephoto may be gradually increased.

(Start receiving)
FIG. 253 is a flowchart illustrating a reception start method in Embodiment 10.

  First, start is performed in step 9003a, and it is confirmed in step 9003b whether a signal from a base station such as Wi-Fi, Bluetooth (registered trademark) or IMES is received. If YES, the process proceeds to step 9003c. If NO, the process returns to step 9003b. In step 9003c, it is confirmed whether or not the base station is registered in the receiver or server as a trigger for starting reception. If YES, the process proceeds to step 9003d, starts receiving a signal, and ends in step 9003e. . If NO, the process returns to step 9003b.

  By this method, reception can be started without the user performing an operation to start reception. Further, power consumption can be suppressed as compared with the case where reception is always performed.

(Generation of ID using information from other media)
FIG. 254 is a flowchart illustrating an ID generation method using information of another medium according to the tenth embodiment.

  First, start in step 9004a, ID of carrier communication network, Wi-Fi, Bluetooth (registered trademark), etc. connected in step 9004b, or position information obtained from the ID, position information obtained from GPS, etc. Is transmitted to the upper bit ID index server. In step 9004c, the upper bits of the visible light ID are received from the upper bit ID index server, and in step 9004d, the signal from the transmitter is received as the lower bits of the visible light ID. In step 9004e, the upper and lower bits of the visible light ID are combined and transmitted to the ID resolution server, and the process ends in step 9004f.

  With this method, it is possible to obtain upper bits that are commonly used in the vicinity of the receiver, and to reduce the amount of data transmitted by the transmitter. In addition, the receiving speed of the receiver can be increased.

  Note that the transmitter may transmit both the upper and lower bits. In this case, the receiver using this method can synthesize the ID when the lower bit is received, and the receiver not using this method obtains the ID by receiving the entire ID from the transmitter. .

(Selection of reception method by frequency separation)
FIG. 255 is a flowchart illustrating a reception method selection method based on frequency separation in the tenth embodiment.

  First, in step 9005a, the optical signal received in step 9005b is applied to a frequency filter circuit, or discrete Fourier series expansion is performed to perform frequency decomposition. In step 9005c, it is confirmed whether or not a low frequency component exists. If YES, the process proceeds to step 9005d, a signal expressed in a low frequency region such as frequency modulation is decoded, and the process proceeds to step 9005e. If NO, the process proceeds to step 9005e. In step 9005e, it is confirmed whether or not the base station is registered in the receiver or server as a trigger for starting reception. If YES, the process proceeds to step 9005f, and a signal expressed in a high frequency region such as pulse position modulation. And the process proceeds to Step 9005g. If NO, the process advances to step 9005g. In step 9005g, signal reception is started, and in step 9005h, the process ends.

  By this method, a signal modulated by a plurality of modulation schemes can be received.

(Signal reception when exposure time is long)
FIG. 256 is a flowchart showing a signal reception method when the exposure time is long in the tenth embodiment.

  First, start is made in step 9030a, and if the sensitivity can be set in step 9030b, the sensitivity is set to the maximum. If the exposure time can be set in step 9030c, the exposure time is set shorter than that in the normal shooting mode. In step 9030d, two images are picked up and a difference in luminance is obtained. If the position or direction of the imaging unit changes during the imaging of two images, the change is canceled and an image as if it was captured from the same position and direction is generated to obtain the difference. In Step 9030e, a value obtained by averaging the luminance values in the direction parallel to the difference image or the exposure line of the captured image is obtained. The averaged values in step 9030f are arranged in the direction perpendicular to the exposure line, and discrete Fourier transform is performed. In step 9030g, it is recognized whether there is a peak near a predetermined frequency, and the process ends in step 9030h.

  With this method, a signal can be received even when the exposure time is long, such as when the exposure time cannot be set or when a normal image is taken simultaneously.

  When the exposure time is automatically set, when the camera is directed to a transmitter configured as illumination, the exposure time is set to about 1/60 second to about 1/480 second by the automatic exposure correction function. If the exposure time cannot be set, a signal is received under this condition. In the experiment, when the illumination is blinked periodically, if the time of one cycle is about 1/16 or more of the exposure time, stripes can be visually recognized in the direction perpendicular to the exposure line. I was able to recognize it. At this time, since the brightness is too high in the portion where the illumination is reflected and it is difficult to confirm the stripe, it is preferable to obtain the period of the signal from the portion where the illumination light is reflected.

  When using a method of periodically turning on / off the light emitting unit, such as a frequency shift keying method or a frequency multiplexing modulation method, even if the modulation frequency is the same as that of the pulse position modulation method, It is difficult for the viewer to see the flicker, and it is difficult for the flicker to appear in the video shot with the video camera. Therefore, a low frequency can be used as the modulation frequency. Since the temporal resolution of human vision is about 60 Hz, a frequency higher than this frequency can be used as the modulation frequency.

  When the modulation frequency is an integral multiple of the imaging frame rate of the receiver, pixels at the same position in the two images are imaged when the light pattern of the transmitter is in the same phase, so a bright line appears in the difference image. It is difficult to receive. Since the imaging frame rate of the receiver is usually 30 fps, reception is easy if the modulation frequency is set to a value other than an integral multiple of 30 Hz. Also, since there are various imaging frame rates of the receiver, two disjoint modulation frequencies are assigned to the same signal, and the transmitter receives signals by alternately using the two modulation frequencies for transmission. The machine can easily restore the signal by receiving at least one signal.

  FIG. 257 is a diagram illustrating an example of a transmitter dimming (adjusting brightness) method.

By adjusting the ratio between the high luminance section and the low luminance section, the average luminance changes and the brightness can be adjusted. At this time, to keep the period T ₁ which repeats high and low brightness constant, it is possible to keep the frequency peak constant. For example, in all of (a), (b), and (c) of FIG. 257, transmission is performed while the time T1 between the first luminance change that becomes brighter than the average luminance and the second luminance change is kept constant. When dimming the machine darkly, the time for lighting brighter than the average brightness is shortened. On the other hand, when the transmitter is dimmed brightly, the illumination time is set longer than the average luminance. (B) and (c) in FIG. 257 are dimmed darker than (a), and (c) in FIG. 257 is dimmed the darkest. Thereby, dimming can be performed while transmitting signals having the same meaning.

  The average brightness may be changed by changing the brightness value of the section with high brightness, the brightness of the section with low brightness, or both brightness values.

  FIG. 258 is a diagram illustrating an example of a method of configuring the dimming function of the transmitter.

  Since the accuracy of the component parts is limited, even if the same dimming setting is performed, the brightness is slightly different from that of another transmitter. However, when transmitters are arranged side by side, if the brightness of adjacent transmitters is different, unnaturalness can be felt. Therefore, the user adjusts the brightness of the transmitter by operating the dimming correction operation unit. The dimming correction unit holds the correction value, and the dimming control unit controls the brightness of the light emitting unit according to the correction value. When the degree of dimming is changed by the user operating the dimming operation unit, the dimming control unit uses the changed dimming setting value and the correction value held in the dimming correction unit. In addition, the brightness of the light emitting unit is controlled. The dimming control unit transmits the dimming setting value to another transmitter through the interlocking dimming unit. When the dimming setting value is transmitted from another device through the linked dimming unit, the dimming control unit, based on the dimming setting value and the correction value held in the dimming correction unit, To control the brightness.

  According to one embodiment of the present invention, there is provided a control method for controlling an information communication device that transmits a signal by changing luminance of a light emitter, and includes a plurality of different signals for a computer of the information communication device. A determination step for determining a luminance change pattern of a different frequency for each different signal by modulating a signal to be transmitted; and a luminance change obtained by modulating a single signal at a time corresponding to a single frequency. A control method may include a transmission step of transmitting a signal to be transmitted by changing the luminance of the light emitter so as to include only the pattern.

  For example, when a time corresponding to a single frequency includes a luminance change pattern obtained by modulating a plurality of signals, the waveform of the luminance change over time becomes complicated, and it is difficult to receive it appropriately. However, by performing control so that only a luminance change pattern obtained by modulating a single signal is included at a time corresponding to a single frequency, reception can be performed more appropriately.

  According to one embodiment of the present invention, the determining step is configured such that the number of transmissions for transmitting one signal among a plurality of different signals is different from the number of transmissions for transmitting other signals within a predetermined time. In addition, the number of transmissions may be determined.

  Since the number of transmissions for transmitting one signal is different from the number of transmissions for transmitting another signal, flickering during transmission can be prevented.

  According to one embodiment of the present invention, the determining step may increase the number of transmissions of a signal corresponding to a high frequency within a predetermined time, compared to the number of transmissions of other signals.

  When frequency conversion is performed on the receiving side, a signal corresponding to a high frequency has low luminance, but by increasing the number of transmissions, it is possible to increase the luminance value when performing frequency conversion.

  According to one embodiment of the present invention, the luminance change pattern may be a pattern in which the waveform of the luminance change over time is any one of a rectangular wave, a triangular wave, and a sawtooth wave.

  By using a rectangular wave or the like, reception can be performed more appropriately.

  According to one embodiment of the present invention, when increasing the value of the average luminance of the illuminant, the time during which the luminance of the illuminant is greater than a predetermined value at a time corresponding to a single frequency, You may lengthen with respect to the case where the value of the average luminance of the said light-emitting body is made small.

  By adjusting the time during which the luminance of the light emitter is greater than a predetermined value at the time corresponding to a single frequency, it is possible to transmit a signal and adjust the average luminance of the light emitter. For example, when the light emitter is used as illumination, it is possible to transmit a signal while making the overall brightness darker or brighter.

  The receiver can set the exposure time to a predetermined value by using an API for setting the exposure time (which stands for application programming interface and indicates a means for using the function of the OS). The visible light signal can be received stably. Further, the receiver can set the sensitivity to a predetermined value by using an API for setting the sensitivity, and stably receives a visible light signal even when the brightness of the transmission signal is dark or bright. be able to.

(Embodiment 11)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

(Exposure time setting)
259A to 259D are flowcharts illustrating an example of operation of a receiver in Embodiment 11.

  In order to receive a visible light signal by the image sensor using the method of the present invention, it is necessary to set the exposure time to be shorter than a predetermined time. The predetermined time is determined by the modulation method and the modulation frequency of the visible light signal. Generally, it is necessary to shorten the exposure time as the modulation frequency increases.

  As the exposure time is shortened, the bright line can be clearly observed. On the other hand, when the exposure time is shortened, the amount of received light is reduced and the entire captured image becomes dark. That is, the signal intensity is attenuated. Therefore, the reception performance (reception speed and error rate) can be improved by shortening the exposure time within a range in which the presence of the visible light signal can be detected.

  As illustrated in FIG. 259A, the receiver first sets the imaging mode to the visible light imaging mode (step S9201). At this time, the receiver determines whether or not to receive a signal having a monochrome imaging function and modulated only with luminance information (step S9202). Here, when it is determined that there is a monochrome imaging function and a signal modulated only with luminance information is received (Y in step S9202), the receiver sets the mode related to the color among the imaging modes to the monochrome imaging function. Is set to a monochrome imaging mode using (step S9203). As a result, when receiving a signal modulated only with luminance information, that is, when receiving a visible light signal representing information only by luminance change, the processing speed can be improved by not handling color information. it can. On the other hand, when it is determined in step S9202 that there is a monochrome imaging function and a signal modulated only with luminance information is not received (N in step 9202), that is, visible light is used using color information. When the signal is expressed, the receiver sets the color-related mode in the imaging mode to the color imaging mode (step S9204).

  Next, the receiver determines whether or not the function for specifying the exposure time is provided in the imaging unit including the above-described image sensor (step S9205). If it is determined that the image is provided (Y in step S9205), the receiver sets the exposure time to a time shorter than the predetermined time using the function so that a bright line appears in the captured image. (Step S9206). Note that the receiver may set the exposure time so that the exposure time is as short as possible in the range where the transmitter transmitting the visible light signal can be seen in the captured image.

  On the other hand, if it is determined in step S9205 that the function for specifying the exposure time is not provided (N in step S9205), the receiver further determines whether the imaging unit has a function for setting sensitivity (step S9205). Step S9207). If it is determined that the function for setting the sensitivity is provided (Y in step S9207), the receiver sets the sensitivity to the maximum by the function (step S9208). As a result, when a captured image is obtained by performing imaging with the maximized sensitivity, the captured image becomes bright. Therefore, in the receiver, if the automatic exposure is set to be effective, the exposure time is set to be short so that the exposure is within a certain range by the automatic exposure. In automatic exposure, every time imaging is performed, a captured image obtained by the imaging is used for input of automatic exposure. Based on the captured image, the exposure time is set so that the exposure falls within a certain range. Adjusted from time to time. Details of the automatic exposure will be described later.

  Further, the receiver determines whether or not the imaging unit has a function of setting the F value (aperture) (step S9209). If it is determined that the function for setting the F value is provided (Y in step S9209), the receiver sets the F value to the minimum (open the aperture) by using the function (step S9210). As a result, when a captured image is obtained by performing imaging with the minimized F value, the captured image becomes bright. Therefore, in the receiver, if the automatic exposure is set to be effective, the exposure time is set to be short so that the exposure is within a certain range by the automatic exposure.

  Further, the receiver determines whether or not the imaging unit has a function for designating an exposure correction value (step S9211). If it is determined that the function for specifying the value of exposure correction is provided (Y in step S9211), the receiver sets the value of exposure correction to the minimum by the function (step S9212). As a result, in the receiver, if the automatic exposure is set to be effective, the exposure time is set short to reduce the exposure by the automatic exposure.

  A scene mode (high-speed scene mode) for imaging a subject that moves at high speed is generally defined by the names “sports” and “action”.

  As shown in FIG. 259B, the receiver has a function for setting the high-speed scene mode after step S9211 or step S9212 and the sensitivity is set to the scene mode by setting the high-speed scene mode. Judgment whether or not the condition that the F value is set lower than before, the F value is set higher than before the scene mode setting, or the exposure correction value is not set higher than before the scene mode setting is satisfied. (Step S9213). If it is determined that the above condition is satisfied (Y in step S9213), the receiver sets the scene mode to the high-speed scene mode (step S9214). As a result, in the receiver, when the automatic exposure is set to be effective, the exposure time is set to be short by the automatic exposure in order to capture an image of a subject moving at high speed without blurring.

  Next, the receiver sets automatic exposure to be effective (step S9215), and captures an image of the subject (step S9216).

  As illustrated in FIG. 259C, the receiver determines whether or not the image capturing unit has a zoom function after step S9215 (step S9217). If it is determined that the zoom function is provided (Y in step S9217), the receiver can further specify whether or not the center position of the zoom can be designated, that is, the center position can be arbitrarily set in the captured image. It is determined whether or not the position can be set (step S9218). If it is determined that the center position can be designated (Y in step S9218), the receiver designates the center position of the zoom in the bright part in the captured image, so that the subject corresponding to the bright part becomes large at the center. Zooming is performed so that an image is captured (step S9219). On the other hand, if it is determined that the center position cannot be specified (N in step S9218), the receiver determines whether the center of the captured image is brighter than a predetermined value of brightness, It is determined whether or not the brightness is brighter than the average brightness at the location (step S9220). If it is determined that the image is bright (Y in step S9220), the receiver performs zooming (step S9221). That is, also at this time, the subject corresponding to the bright part is imaged largely at the center.

  Generally, many devices with an imaging unit use center-weighted metering for photometry, so if there is a bright part in the center of the captured image, the bright part even if the photometric position is not designated as another position The exposure is adjusted with reference to. Thereby, the exposure time is set short. In addition, the area of the bright portion is increased by zooming, and the exposure is adjusted based on a brighter screen, so the exposure time is set short.

  Next, the receiver determines whether or not there is a function for designating a photometric position or a focus position (step S9222). If it is determined that the function is present (Y in step S9222), the receiver performs processing for finding a bright place in the captured image. That is, the receiver performs processing for finding out the location of an area having a predetermined shape and size that is brighter than the predetermined brightness from the captured image. Specifically, the receiver first determines whether or not an exposure evaluation calculation formula for automatic exposure is known (step S9224). If it is determined that it is known (Y in step S9224), the receiver evaluates the brightness of each region in the captured image using the same calculation formula as the known calculation formula, thereby A bright area is found (step S9226). On the other hand, when it is determined that the calculation formula for the exposure evaluation is unknown (N in step S9224), the receiver calculates the average value of the brightness of each pixel in a region having a predetermined shape and size. By using the calculation formula, the brightness of each area in the captured image is evaluated to find the location of the above-described bright area (step S9225). The predetermined shape is, for example, a rectangle, a circle, or a cross shape. Moreover, the area | region may be comprised from the non-continuous several area | region. In addition, the above average value may be calculated not by a simple average but by a weighted average with a greater weight at the center.

  The receiver determines whether the total area of all the bright areas found is smaller than a predetermined area (step S9227). If it is determined that the image is small (Y in step S9227), the receiver performs zooming so that the total area of the bright regions is imaged with a size equal to or larger than a predetermined area (step S9228). Next, the receiver determines whether or not the photometric position can be designated (step S9229). If it is determined that the photometric position can be designated (Y in step S9229), the receiver designates the brightest area as the photometric position (step S9230). In automatic exposure, the exposure is adjusted according to the brightness of the photometric position. Therefore, by specifying the place of the brightest area as the photometric position, the exposure time is set short by automatic exposure. On the other hand, if it is determined in step S9229 that the photometric position cannot be designated (N in step S9229), that is, if the focus position can be designated, the receiver designates the location of the brightest area as the focus position. Various imaging units can be mounted on the receiver, and in the automatic exposure of some imaging units among the various imaging units, the exposure is adjusted by the brightness of the focus position. Therefore, by specifying the brightest area as the focus position, the exposure time is set short by automatic exposure. The location specified here may be different from the location of the region used for the brightness evaluation or calculation, and is set in accordance with the setting method of the imaging unit. As an example, if the specified format is a format that specifies a center point, the receiver specifies the center of the brightest area, and if the specified format is a format that specifies a rectangular area, the receiver Specifies a rectangular area that includes the center of the bright area.

  Next, the receiver determines whether there is an area brighter than the area of the place designated as the photometric position or the focus position in the captured image (step S9232). Here, if it is determined that the area is present (Y in step S9232), the receiver repeatedly executes the processing from step S9217. On the other hand, if it is determined that there is no such area (N in step S9232), the receiver images the subject (step S9233).

  Next, the receiver determines whether or not the automatic exposure should be terminated based on the captured image obtained by the imaging in step S9233, or whether or not a certain time has elapsed since the automatic exposure was set to be effective. (Step S9234). Here, for example, when it is determined that the automatic exposure should not be terminated (N in Step S9234), the receiver further determines the position or the imaging direction of the imaging unit based on the captured image obtained by the imaging in Step S9233. It is determined whether or not there has been a change (step S9235). If it is determined that the position of the imaging unit or the imaging direction has changed (Y in step S9235), the receiver performs the processing from step S9217 again. Thereby, even when the place designated as the photometric position or the focus position moves in the captured image, the place of the brightest region at that time can be designated again. On the other hand, if it is determined in step S9235 that the position of the imaging unit or the imaging direction has not changed (N in step S9235), the receiver repeatedly performs the processing from step S9232. Note that the receiver may search for the brightest area every time a single captured image is acquired by imaging, and respecify the photometric position and focus position.

  If it is determined in step S9234 that the automatic exposure should be terminated, if the exposure time does not change, or if it is determined that a certain time has elapsed (Y in step S9234), or exposure is performed in step S9206. When the time is set, the receiver sets the automatic exposure to invalid (step S9236) and images the subject (step S9237). Then, the receiver determines whether a visible light signal has been received by the imaging (step S9238). If it is determined that no visible light signal has been received (N in step S9238), the receiver further determines whether or not a predetermined time has elapsed (step S9239). If it is determined that the predetermined time has not elapsed (N in step S9239), the receiver repeatedly executes the processing from step S9237. On the other hand, when it is determined that the predetermined time has elapsed (Y in step S9239), that is, when the visible light signal cannot be received even after the predetermined time has elapsed, the receiver By repeatedly executing the processing from step S9232, the search for the brightest region is performed again.

  Note that when the receiver uses the zoom function, the receiver may turn off the zoom at an arbitrary timing. That is, when the zoom is off, the receiver may detect whether there is a bright subject in the range where the image is not captured during the zoom but is not zoomed. Note that this bright subject is likely to be a transmitter that transmits a visible light signal by a change in luminance. Thereby, the signal from the transmitter which exists in a wide range can be received.

  Here, the automatic exposure and photometry method will be described.

  Hereinafter, automatic exposure in FIGS. 259A to 259D will be described. Automatic exposure is an operation, process, or function in which the imaging unit of the receiver adjusts the exposure time, sensitivity, and aperture, and automatically adjusts the photometric result to a predetermined value.

  Photometric methods for obtaining a photometric result include methods such as average photometry (entire photometry), center-weighted photometry, spot photometry (partial photometry), and split photometry. In average photometry, the average brightness of the entire image obtained by imaging is calculated. In center-weighted metering, a weighted average value of brightness is calculated by increasing the weight closer to the center (or designated portion) of the image. In spot photometry, the average value (or weighted average value) of the brightness in a predetermined range (or several locations) defined around the center or specified portion of the image is calculated. In split photometry, an image is divided into a plurality of parts, and photometry is performed on each part to calculate a total brightness value.

  In an imaging unit having an automatic exposure function, the exposure time can be indirectly set by the automatic exposure function even if the exposure time cannot be set to be shortened directly. For example, if the sensitivity is set high (for example, at the maximum value), the image obtained by imaging becomes bright if the other conditions are the same. Therefore, the exposure time can be set short by automatic exposure. Further, when the aperture is set to be opened (that is, opened), similarly, the exposure time can be set short. If a value indicating the level of exposure correction is set low (for example, to a minimum value), an image obtained by imaging becomes dark due to automatic exposure. That is, the exposure time is set short. By specifying the brightest place in the image as the photometry position, the exposure time can be set short. If the photometric method can be specified, the exposure time can be set short by specifying spot photometry. If the photometric range can be designated, the exposure time can be set short by designating the photometric range to the minimum. When the area of the bright part in the image is large, the exposure time can be set short by specifying the photometric range as large as possible so as not to exceed that part. If a plurality of photometric positions can be designated, the exposure time can be set short by designating the same place as the photometric position several times. By zooming in on a bright place in the image and designating that place as the photometric position, the exposure time can be set short.

  Here, the EX zoom will be described.

  FIG. 260 is a diagram for explaining the EX zoom.

  The zoom in FIG. 259C, that is, a method for obtaining a large image includes optical zoom for changing the size of the image captured on the image sensor by adjusting the focal length of the lens, and digital processing for interpolating the image captured on the image sensor. There are a digital zoom that obtains a large image and an EX zoom that obtains a large image by changing a plurality of image sensors used for imaging. The EX zoom can be used when the number of image sensors included in the image sensor is larger than the resolution of the captured image.

  For example, in the image sensor 10080a shown in FIG. 260, 32 × 24 imaging elements are arranged in a matrix. That is, 32 image sensors are arranged horizontally and 24 elements are arranged vertically. When an image having a horizontal 16 × longitudinal resolution is obtained by imaging by the image sensor 10080a, as shown in FIG. 260 (a), among the 32 × 24 imaging elements included in the image sensor 10080a, the image sensor Only 16 × 12 image sensors (for example, image sensors indicated by black squares in the image sensor 1080a in FIG. 260A) that are uniformly distributed throughout 10080a are used for imaging. That is, only an odd-numbered or even-numbered image sensor is used for imaging among a plurality of image sensors arranged in the vertical direction and the horizontal direction. As a result, an image 10080b having a desired resolution is obtained. In FIG. 260, a subject appears on the image sensor 1008a, but this is for easy understanding of the correspondence between each image sensor and an image obtained by imaging.

  The receiver including the image sensor 10080a captures a wide range, and searches for a transmitter or receives information from many transmitters in the image sensor 10080a. An image is picked up using only a part of the image pickup elements arranged in a distributed manner.

  When the receiver performs the EX zoom, as shown in FIG. 260 (b), in the image sensor 10080a, some image pickup devices (for example, FIG. 260 (b)) arranged locally densely. Only 16 × 12 image sensors indicated by black squares in the image sensor 1080a in FIG. As a result, a part of the image 10080b corresponding to a part of the imaging elements is zoomed, and an image 10080d is obtained. By such an EX zoom, it is possible to receive a visible light signal for a long time by capturing a large image of the transmitter, the reception speed is improved, and a visible light signal can be received from a distance.

  In digital zoom, the number of exposure lines that receive visible light signals cannot be increased, and the reception time of visible light signals does not increase. Therefore, it is better to use other zooms as much as possible. The optical zoom requires a physical movement time of the lens and the image sensor. However, since the EX zoom is performed only by electronic setting change, there is an advantage that the time required for the zoom is short. From this viewpoint, the priority order of each zoom is (1) EX zoom, (2) optical zoom, and (3) digital zoom. The receiver may select and use any one or a plurality of zooms according to the priority order and the necessity of the zoom magnification. Note that in the imaging methods shown in FIGS. 260A and 260B, image noise can be suppressed by using an unused imaging device.

  FIG. 261A is a flowchart illustrating processing of a reception program in the tenth embodiment.

  This reception program is a program that causes a computer provided in the receiver to execute, for example, the processes shown in FIGS. 259A to 260.

  That is, this reception program is a reception program for receiving information from the light emitter. Specifically, the reception program sets the exposure time of the image sensor using automatic exposure, and causes the image sensor to image the illuminant whose luminance changes with the set exposure time. A bright line image acquisition step SA22 for acquiring a bright line image that is an image including a bright line corresponding to each of a plurality of exposure lines included in the image sensor; and decoding a plurality of bright line patterns included in the acquired bright line image. The information acquisition step SA23 for acquiring information is executed by the computer. In exposure time setting step SA21, as in step S9208 of FIG. 259A, the sensitivity of the image sensor is set to the maximum value in a predetermined range for the image sensor, and the sensitivity is set according to the sensitivity of the maximum value. Set the exposure time by automatic exposure.

  As a result, even if the exposure time of the image sensor cannot be set directly, the exposure time is short enough to obtain an appropriate bright line image using the automatic exposure function installed in a general camera. Can be set. That is, in the automatic exposure, the exposure is adjusted based on the brightness of the image acquired by imaging by the image sensor. Therefore, if the sensitivity of the image sensor is set to a large value, the image becomes bright. Therefore, in order to suppress exposure, the exposure time of the image sensor is set short. If the sensitivity of the image sensor is set to the maximum value, the exposure time can be set shorter and an appropriate bright line image can be acquired. That is, information from the light emitter can be appropriately received. As a result, communication between various devices can be enabled. The sensitivity is, for example, ISO sensitivity.

  In the exposure time setting step SA21, as in step S9212 in FIG. 259A, the value indicating the exposure correction level of the image sensor is set to the minimum value in the range preset for the image sensor. The exposure time corresponding to the maximum sensitivity and the minimum level exposure correction is set by automatic exposure.

  As a result, since the value indicating the level of exposure correction is set to the minimum value, the exposure time can be set shorter by automatic exposure processing that attempts to suppress exposure, and a more appropriate bright line image is acquired. be able to. The unit of the value indicating the exposure correction level is, for example, EV.

  In the exposure time setting step SA21, as shown in FIG. 259C, a brighter part than the other part is specified in the first image acquired by imaging the subject including the light emitter by the image sensor. Then, a part of the subject corresponding to the bright part is enlarged by optical zoom. Furthermore, an exposure time is set by using a second image acquired by imaging a part of the enlarged subject by the image sensor for input of automatic exposure. In the bright line image acquisition step SA22, a bright line image is acquired by causing the image sensor to capture a part of the enlarged subject with the set exposure time.

  Thereby, a part of the subject corresponding to the bright part of the first image is enlarged by the optical zoom, that is, the bright light emitter is enlarged by the optical zoom. Can also be brightened overall. Then, since the bright second image is used for the input of automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can.

  Further, in the exposure time setting step SA21, as shown in FIG. 259C, the central portion of the first image acquired by the imaging of the subject including the illuminant is brightness at a plurality of positions in the first image. It is judged whether it is brighter than the average of. If it is determined that the central portion is bright, a part of the subject corresponding to the central portion is enlarged by optical zoom. Furthermore, an exposure time is set by using a second image acquired by imaging a part of the enlarged subject by the image sensor for input of automatic exposure. In the bright line image acquisition step SA22, a bright line image is acquired by causing the image sensor to capture a part of the enlarged subject with the set exposure time.

  Accordingly, a part of the subject corresponding to the bright central portion of the first image is enlarged by the optical zoom, that is, a bright light-emitting body is enlarged by the optical zoom, so that the second image is changed to the first image. Can be brighter overall. Then, since the bright second image is used for the input of automatic exposure, the exposure time can be set shorter by the automatic exposure processing for suppressing the exposure, and an appropriate bright line image can be acquired. it can. Further, in the optical zoom, if the enlargement center position cannot be arbitrarily set, the angle of view or the central portion of the image is enlarged. Therefore, even if the center position cannot be arbitrarily set, if the central portion of the first image is bright, the second image can be brightened as a whole by using the optical zoom. it can. Further, even when the central portion of the first image is dark, if the enlargement by the optical zoom is performed, the second image becomes dark and the exposure time becomes long. Therefore, as described above, it is possible to prevent the exposure time from becoming longer by enlarging with the optical zoom only when it is determined that the central portion is bright.

  Further, in the exposure time setting step SA21, as shown in FIG. 260, among the K imaging elements (K is an integer of 3 or more) included in the image sensor, N that are uniformly distributed in the image sensor. In a first image acquired by imaging a subject including a light emitter using only N (N is an integer of 2 or more and an integer of 2 or more), a brighter part than the other part is specified. Furthermore, by using a second image obtained by imaging with only N dense image pickup elements corresponding to a bright portion among K image pickup elements included in the image sensor for input of automatic exposure, exposure is performed. Set the time. Also, in the bright line image acquisition step SA22, a bright line image is acquired by causing only N dense image pickup elements included in the image sensor to capture images with a set exposure time.

  Thereby, even if the bright part in the first image is not at the center by so-called EX zoom, the second image can be brightened as a whole, and the exposure time can be set shorter.

  In the exposure time setting step SA21, as shown in FIG. 259C, the photometric position in the image acquired by the image sensor capturing the subject is set, and the exposure time according to the brightness of the set photometric position. Is set by automatic exposure.

  As a result, if a bright part in the image acquired by imaging is set as the photometric position, the exposure time can be set shorter by the automatic exposure process to suppress exposure, and an appropriate bright line image can be obtained. Can be acquired.

  The reception program further includes an imaging mode setting step for switching the imaging mode of the image sensor from a color imaging mode for acquiring a color image by imaging to a monochrome imaging mode for acquiring a monochrome image by imaging. It may be executed by a computer. In this case, in the exposure time setting step SA21, the exposure time is set by using an image acquired in the monochrome imaging mode for input of automatic exposure.

  Accordingly, since an image acquired in the monochrome imaging mode is used for automatic exposure input, an appropriate exposure time can be set without being influenced by color information. Further, when the exposure time is set in the monochrome imaging mode, the bright line image is acquired by imaging according to the mode. Therefore, when information is transmitted from the light emitter only by a luminance change, the information can be appropriately acquired.

  In the exposure time setting step SA21, the exposure time of the image sensor is updated by using the acquired image for automatic exposure every time an image is acquired by imaging the light emitter with the image sensor. Here, for example, as shown in step S9234 in FIG. 259D, when the fluctuation range of the exposure time updated as needed is less than or equal to a predetermined range, the exposure time is updated by ending the update of the exposure time by automatic exposure. Set.

  Thereby, when the fluctuation of the exposure time is stabilized, that is, when the brightness of the image acquired by imaging falls within the target brightness by the automatic exposure, the exposure time at that time is the bright line image. It is used for imaging for acquisition. Therefore, an appropriate bright line image can be acquired.

  FIG. 261B is a block diagram of a receiving apparatus in Embodiment 10.

  The receiving device A20 is the above-described receiver that executes the processes shown in FIGS. 259A to 260, for example.

  That is, the receiving device A20 is a device for receiving information from a light emitter, and sets an exposure time setting unit A21 that sets the exposure time of the image sensor using automatic exposure, and a light emitter that changes in luminance. An image sensor A22 that acquires a bright line image that is an image including a bright line corresponding to each of a plurality of exposure lines included in the image sensor by causing the image sensor to capture an image with the exposure time that is included, and included in the acquired bright line image And a decoding unit A23 that acquires information by decoding a plurality of bright line patterns. The exposure time setting unit A21 sets the sensitivity of the image sensor to the maximum value in a range predetermined for the image sensor, and sets the exposure time according to the sensitivity of the maximum value by automatic exposure. This receiving apparatus A20 can achieve the same effects as the above receiving program.

(Embodiment 12)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  In the present embodiment, the exposure time is set for each exposure line or each image sensor.

  262, 263, and 264 are diagrams illustrating an example of a signal reception method in Embodiment 12.

  As shown in FIG. 262, in an image sensor 10010a that is an imaging unit provided in the receiver, an exposure time is set for each exposure line. That is, a long exposure time for normal imaging is set for a predetermined exposure line (white exposure line in FIG. 262), and a visible light imaging for other exposure lines (black exposure line in FIG. 262). A short exposure time is set. For example, a long exposure time and a short exposure time are alternately set for each exposure line arranged in the vertical direction. Thereby, when imaging the transmitter which transmits a visible light signal by a luminance change, normal imaging and visible light imaging (visible light communication) can be performed almost simultaneously. The two exposure times may be alternately set for each line, may be set for every several lines, or different exposure times may be set for the upper part and the lower part of the image sensor 10010a. By using two exposure times in this way, when data obtained by imaging a plurality of exposure lines set to the same exposure time are collected, a normal captured image 10010b and a bright line image showing a plurality of bright line patterns are obtained. A visible light captured image 10010c is obtained. In the normal captured image 10010b, since a portion that has not been captured with a long exposure time (that is, an image corresponding to a plurality of exposure lines set to a short exposure time) is missing, the preview image is obtained by interpolating that portion. 10010d can be displayed. Here, information obtained by visible light communication can be superimposed on the preview image 10010d. This information is information associated with a visible light signal obtained by decoding a plurality of bright line patterns included in the visible light captured image 10010c. The receiver stores the normal captured image 10010b or an image obtained by performing interpolation on the normal captured image 10010b as a captured image, and stores the received visible light signal or information associated with the visible light signal. As additional information, it can also be added to the stored captured image.

  As shown in FIG. 263, an image sensor 10011a may be used instead of the image sensor 10010a. In the image sensor 1011a, the exposure time is set not for each exposure line but for each column (hereinafter, referred to as a vertical line) composed of a plurality of imaging elements arranged along a direction perpendicular to the exposure line. That is, a long exposure time for normal imaging is set for a predetermined vertical line (white vertical line in FIG. 263), and a visible light imaging for other vertical lines (black vertical line in FIG. 263). A short exposure time is set. In this case, in the image sensor 10011a, exposure is started at different timings for each exposure line, as in the image sensor 10010a. However, the exposure time of each image sensor included in the exposure line is different for each exposure line. . The receiver obtains a normal captured image 10011b and a visible light captured image 10011c by imaging with the image sensor 10011a. Further, the receiver generates and displays a preview image 10011d based on the normal captured image 10011b and information associated with the visible light signal obtained from the visible light captured image 10011c.

  In the image sensor 10011a, unlike the image sensor 10010a, all exposure lines can be used for visible light imaging. As a result, since the visible light captured image 10011c obtained by the image sensor 10011a includes more bright lines than the visible light captured image 10010c, the reception accuracy of the visible light signal can be increased.

  As shown in FIG. 264, an image sensor 10012a may be used instead of the image sensor 10010a. In the image sensor 10012a, the exposure time is set for each image sensor so that the same exposure time is not set continuously for each image sensor along the horizontal direction and the vertical direction. That is, the exposure time for each image sensor is such that a plurality of image sensors with a long exposure time set and a plurality of image sensors with a short exposure time are distributed like a grid or checkered pattern. Is set. Also in this case, similarly to the image sensor 10010a, the exposure is started at different timings for each exposure line, but the exposure time of each image sensor included in the exposure line is different for each exposure line. The receiver obtains a normal captured image 10012b and a visible light captured image 10012c by imaging with the image sensor 10012a. Further, the receiver generates and displays a preview image 10012d based on the normal captured image 10012b and information associated with the visible light signal obtained from the visible light captured image 10012c.

  Since the normal captured image 10012b obtained by the image sensor 10012a has data of a plurality of imaging elements arranged in a grid pattern or uniformly, interpolation or more accurately than the normal captured image 10010b and the normal captured image 10011b is performed. You can resize. The visible light captured image 10012c is generated by imaging using all exposure lines of the image sensor 10012a. That is, in the image sensor 10012a, unlike the image sensor 10010a, all exposure lines can be used for visible light imaging. As a result, the visible light captured image 10012c obtained by the image sensor 10012a includes a larger number of bright lines than the visible light captured image 10010c, as in the visible light captured image 10011c. Can be done with precision.

  Here, the interlaced display of the preview image will be described.

  FIG. 265 is a diagram illustrating an example of a screen display method of a receiver in Embodiment 12.

  The receiver including the image sensor 10010a shown in FIG. 262 described above is set to an exposure time set for an odd-numbered exposure line (hereinafter referred to as an odd-numbered line) and an even-numbered exposure line (hereinafter referred to as an even-numbered line). The exposure time is changed every predetermined time. For example, as shown in FIG. 265, at time t1, the receiver sets a long exposure time for each image sensor on the odd lines and sets a short exposure time on each image sensor on the even lines. Imaging is performed using the set exposure time. Further, at time t2, the receiver sets a short exposure time for each image sensor of the odd line, sets a long exposure time for each image sensor of the even line, and sets the set exposure time. The used imaging is performed. Then, at time t3, the receiver performs imaging using each exposure time set similarly to time t1, and uses each exposure time set similarly to time t2 at time t4. Take an image.

  Further, at time t1, the receiver captures an image obtained from each of a plurality of odd lines by imaging (hereinafter referred to as an odd line image) and an image obtained from each of a plurality of even lines by imaging (hereinafter referred to as an even line image). Image1 including the above. At this time, since the exposure time is short in each of the plurality of even lines, the subject is not clearly displayed in each of the even line images. Therefore, the receiver generates a plurality of interpolated line images by interpolating pixel values for the plurality of odd line images. Then, the receiver displays a preview image including a plurality of interpolation line images instead of the plurality of even line images. That is, odd-numbered line images and interpolated line images are alternately arranged in the preview image.

  At time t2, the receiver acquires Image2 including a plurality of odd line images and even line images by imaging. At this time, since the exposure time is short in each of the plurality of odd lines, the subject is not clearly displayed in each of the odd line images. Therefore, the receiver displays a preview image including the odd line image of Image1 instead of the odd line image of Image2. That is, the odd line image of Image1 and the even line image of Image2 are alternately arranged in the preview image.

  Further, at time t3, the receiver acquires Image3 including a plurality of odd line images and even line images by imaging. At this time, similarly to the time t1, since the exposure time is short in each of the plurality of even lines, the subject is not clearly displayed in each of the even line images. Therefore, the receiver displays a preview image including an even line image of Image2 instead of an even line image of Image3. That is, in the preview image, the even line image of Image2 and the odd line image of Image3 are alternately arranged. Then, at time t4, the receiver acquires Image4 including a plurality of odd line images and even line images by imaging. At this time, similarly to the time t2, since the exposure time is short in each of the plurality of odd lines, the subject is not clearly displayed in each of the odd line images. Therefore, the receiver displays a preview image including the odd line image of Image3 instead of the odd line image of Image4. That is, the odd line image of Image3 and the even line image of Image4 are alternately arranged in the preview image.

  In this way, the receiver performs so-called interlaced display that displays an image including even-numbered line images and odd-numbered line images that have different acquired timings.

  Such a receiver can display a fine preview image while performing visible light imaging. Note that the plurality of image sensors set with the same exposure time may be a plurality of image sensors arranged along the horizontal direction of the exposure line as in the image sensor 10010a, or the exposure line as in the image sensor 10011a. A plurality of image sensors arranged along a direction perpendicular to the image sensor may be used, or a plurality of image sensors arranged according to a checkered pattern like the image sensor 10012a. Further, the receiver may save the preview image as imaging data.

  Next, the spatial ratio between normal imaging and visible light imaging will be described.

  266 is a diagram illustrating an example of a signal reception method in Embodiment 12. FIG.

  In the image sensor 10014b provided in the receiver, a long exposure time or a short exposure time is set for each exposure line as in the image sensor 10010a described above. In this image sensor 10014b, the ratio of the number of image sensors for which a long exposure time is set to the number of image sensors for which a short exposure time is set is 1: 1. This ratio is a ratio between normal imaging and visible light imaging, and is hereinafter referred to as a spatial ratio.

  However, in the present embodiment, the space ratio is not necessarily 1: 1. For example, the receiver may include an image sensor 10014a. In this image sensor 10014a, the number of image sensors with a short exposure time is larger than that of images with a long exposure time, and the spatial ratio is 1: N (N> 1). The receiver may include an image sensor 10014c. In this image sensor 10014c, the image sensor with a short exposure time is smaller than the image sensor with a long exposure time, and the spatial ratio is N (N> 1): 1. In addition, instead of the image sensors 10014a to 10014c, the receiver sets the exposure time for each of the above-described vertical lines, and the image sensors 10015a to 10015c have a spatial ratio of 1: N, 1: 1, or N: 1, respectively. Any of these may be provided.

  In such image sensors 10014a and 10015a, since there are many image sensors with a short exposure time, the reception accuracy or reception speed of a visible light signal can be increased. On the other hand, since the image sensors 10014c and 10015c have many image sensors with a long exposure time, a fine preview image can be displayed.

  Further, the receiver may perform interlaced display as illustrated in FIG. 265 using the image sensors 10014a, 10014c, 10015a, and 10015c.

  Next, the time ratio between normal imaging and visible light imaging will be described.

  FIG. 267 is a diagram illustrating an example of a signal reception method in Embodiment 12.

  The receiver may switch the imaging mode between the normal imaging mode and the visible light imaging mode for each frame, as illustrated in FIG. The normal imaging mode is an imaging mode in which a long exposure time for normal imaging is set for all imaging elements of the image sensor of the receiver. The visible light imaging mode is an imaging mode in which a short exposure time for visible light imaging is set for all imaging elements of the image sensor of the receiver. In this way, by switching between long and short exposure times, a preview image can be displayed by imaging with a long exposure time while receiving a visible light signal by imaging with a short exposure time.

  Note that when the long exposure time is determined by automatic exposure, the receiver ignores the image obtained by imaging with a short exposure time, and only determines the brightness of the image obtained by imaging with a long exposure time. Automatic exposure may be performed with reference. Thereby, a long exposure time can be determined as an appropriate time.

  Further, as illustrated in FIG. 267 (b), the receiver may switch the imaging mode between the normal imaging mode and the visible light imaging mode for each set of a plurality of frames. When it takes time to switch the exposure time or when it takes time to stabilize the exposure time, as shown in FIG. 267 (b), by changing the exposure time for each set of a plurality of frames, Visible light imaging (reception of visible light signals) and normal imaging can be made compatible. Further, since the number of exposure time switching is reduced as the number of frames included in the set is increased, power consumption and heat generation in the receiver can be suppressed.

  Here, the number of at least one frame continuously generated by imaging with a long exposure time in the normal imaging mode and the at least one frame continuously generated by imaging with a short exposure time in the visible light imaging mode The ratio (hereinafter referred to as the time ratio) with the number of s is not necessarily 1: 1. That is, in the cases shown in FIGS. 267 (a) and (b), the time ratio is 1: 1, but the time ratio may not be 1: 1.

  For example, as illustrated in (c) of FIG. 267, the receiver may increase the number of frames in the visible light imaging mode than the frames in the normal imaging mode. Thereby, the receiving speed of the visible light signal can be increased. If the frame rate of the preview image is equal to or higher than a predetermined rate, the difference in the preview image depending on the frame rate is not recognized by human eyes. When the imaging frame rate is sufficiently high, for example, when the frame rate is 120 fps, the receiver sets the visible light imaging mode for three consecutive frames, and then the visible light imaging for one frame. Set the mode. Thereby, the receiver can receive a visible light signal at high speed while displaying a preview image at a frame rate of 30 fps, which is sufficiently higher than the above-described predetermined rate. Further, since the number of times of switching is reduced, the effect described with reference to FIG.

  Further, as illustrated in (d) of FIG. 267, the receiver may increase the number of frames in the normal imaging mode than the frames in the visible light imaging mode. Thus, the preview image can be smoothly displayed by increasing the number of frames in the normal imaging mode, that is, the frames obtained by imaging with a long exposure time. In addition, since the number of times of receiving the visible light signal is reduced, there is a power saving effect. Further, since the number of times of switching is reduced, the effect described with reference to FIG.

  Further, as shown in FIG. 267 (e), the receiver first switches the imaging mode for each frame as in the case shown in FIG. 267 (a), and then receives a visible light signal. When completed, the number of frames in the normal imaging mode may be increased as in the case shown in FIG. Thereby, after the reception of the visible light signal is completed, the search for a new visible light signal can be continued while the preview image is displayed smoothly. Further, since the number of times of switching is reduced, the effect described with reference to FIG.

  FIG. 268 is a flowchart illustrating an example of a signal reception method in Embodiment 12.

  The receiver starts visible light reception, which is a process of receiving a visible light signal (step S10017a), and sets the exposure time length setting ratio to a value designated by the user (step S10017b). The exposure time length short / high setting ratio is at least one of the above-described space ratio and time ratio. The user may specify only the space ratio, only the time ratio, or both the space ratio and the time ratio, or the receiver may automatically set regardless of the user's specification.

  Next, the receiver determines whether or not the reception performance is equal to or less than a predetermined value (step S10017c). If it is determined that the value is equal to or less than the predetermined value (Y in step S10017c), the receiver sets a high ratio of the short exposure time (step S10017d). Thereby, reception performance can be improved. Note that the ratio of the short exposure time is the ratio of the number of image sensors set with a short exposure time to the number of image sensors set with a long exposure time in the case of a spatial ratio. It is a ratio of the number of frames generated continuously in the visible light imaging mode to the number of frames generated continuously in the mode.

  Next, the receiver receives at least a part of the visible light signal and determines whether or not a priority is set for at least a part of the received visible light signal (hereinafter referred to as a received signal) ( Step S10017e). When priority is set, an identifier indicating the priority is included in the received signal. When the receiver determines that the priority is set (Y in step S10017e), the receiver sets the exposure time length / short ratio according to the priority (step S10017f). That is, if the priority is high, the receiver sets the ratio of the short exposure time high. For example, an emergency light configured as a transmitter emits an identifier indicating a high priority when the luminance changes. In this case, the receiver can increase the reception speed by increasing the ratio of the short exposure time, and can promptly display the evacuation route and the like.

  Next, the receiver determines whether or not reception of all visible light signals has been completed (step S10017g). Here, when it is determined that it has not been completed (N in step S10017g), the receiver repeatedly executes the processing from step S10017c. On the other hand, when it is determined that the process is completed (Y in step S10017g), the receiver sets the ratio of the long exposure time to a high value and shifts to the power saving mode (step S10017h). Note that the ratio of the long exposure time is the ratio of the number of image sensors set with a long exposure time to the number of image sensors set with a short exposure time in the case of a spatial ratio. This is the ratio of the number of frames generated continuously in the normal imaging mode to the number of frames generated continuously in the imaging mode. As a result, the preview image can be displayed smoothly without receiving unnecessary visible light.

  Next, the receiver determines whether another visible light signal has been found (step S10017i). Here, when it is determined that it has been found (Y in step S10017i), the receiver repeatedly executes the processing from step S10017b.

  Next, simultaneous execution of visible light imaging and normal imaging will be described.

  FIG. 269 is a diagram illustrating an example of a signal reception method in Embodiment 12.

  The receiver may set two or more exposure times for the image sensor. That is, as shown in FIG. 269 (a), each of the exposure lines included in the image sensor is continuously exposed for the longest exposure time among the two or more set exposure times. For each exposure line, the receiver reads out the imaging data obtained by exposure of the exposure line when the above-described two or more set exposure times have elapsed. Here, the receiver does not reset the read image data until the longest exposure time has elapsed. Therefore, the receiver can obtain image data for a plurality of exposure times only by exposure with the longest exposure time by recording the accumulated value of the read image data. Note that the image sensor may or may not record the cumulative value of the imaging data. When the image sensor is not used, the components of the receiver that read data from the image sensor perform accumulation calculation, that is, recording the accumulated value of the imaging data.

  For example, when two exposure times are set, as shown in FIG. 269 (a), the receiver captures visible light imaging data including a visible light signal generated by exposure with a short exposure time. Read out, followed by normal imaging data generated by exposure with a long exposure time.

  Thereby, visible light imaging that is imaging for receiving a visible light signal and normal imaging can be performed simultaneously, and normal imaging can be performed while receiving a visible light signal. Further, by using data of a plurality of exposure times, a signal frequency higher than the sampling theorem can be recognized, and a high-frequency signal or a high-density modulation signal can be received.

  Further, when outputting the imaging data, the receiver outputs a data string including the imaging data as an imaging data body, as shown in FIG. 269 (b). That is, the receiver has an imaging mode identifier indicating an imaging mode (visible light imaging or normal imaging), an imaging element identifier for specifying an imaging element or an exposure line to which the imaging element belongs, and an exposure number of an imaging data body. By adding additional information including an imaging data number indicating whether it is time-based imaging data and an imaging data length indicating the size of the imaging data body to the imaging data body, the above-described data string is generated and output. . In the method for reading image data described with reference to FIG. 269 (a), the respective image data is not necessarily output in the order of exposure lines. Therefore, by adding the additional information shown in FIG. 269 (b), it is possible to specify which exposure line the imaging data is.

  FIG. 270A is a flowchart showing processing of a reception program in the twelfth embodiment.

  This reception program is a program that causes a computer provided in the receiver to execute, for example, the processes illustrated in FIGS. 262 to 269.

  That is, this reception program is a reception program for receiving information from a light emitter that changes in luminance. Specifically, this reception program causes the computer to execute step SA31, step SA32, and step SA33. In step SA31, a first exposure time is set for some of the K image sensors (K is an integer of 4 or more) included in the image sensor, and the K image sensors are set. A second exposure time shorter than the first exposure time is set for the remaining plurality of image sensors. In step SA32, the image sensor captures the subject, which is a light-emitting body that changes in luminance, with the set first and second exposure times, and outputs from the plurality of image sensors having the first exposure time set. And acquiring bright lines corresponding to the plurality of exposure lines included in the image sensor, the images corresponding to the outputs from the plurality of imaging elements set with the second exposure time. A bright line image that is an image to be included is acquired. In step SA33, information is acquired by decoding a plurality of bright line patterns included in the acquired bright line image.

  Thereby, since imaging is performed by the plurality of imaging elements in which the first exposure time is set and the plurality of imaging elements in which the second exposure time is set, a normal image can be obtained by one imaging by the image sensor. And bright line images can be acquired. That is, it is possible to simultaneously capture a normal image and acquire information by visible light communication.

  In the exposure time setting step SA31, the first exposure time is set for a part of a plurality of image sensor rows among L (L is an integer of 4 or more) image sensor rows included in the image sensor. Then, the second exposure time is set for the remaining plurality of image sensor rows in the L image sensor rows. Here, each of the L image sensor rows is composed of a plurality of image sensors arranged in a row included in the image sensor.

  Accordingly, it is possible to set the exposure time for each image pickup device array which is a large unit without individually setting the exposure time for each image pickup device which is a small unit, and to reduce the processing load. .

  For example, each of the L imaging element rows is an exposure line included in the image sensor as shown in FIG. Alternatively, as shown in FIG. 263, each of the L image pickup device arrays includes a plurality of image pickup devices arranged along a direction perpendicular to an exposure line included in the image sensor.

  Further, as shown in FIG. 265, in the exposure time setting step SA31, the same exposure time is used for each of the odd-numbered image sensor rows of the L image sensor rows included in the image sensor. One of the first exposure time and the second exposure time is set, and the first exposure time and the second exposure time are the same exposure time for each of even-numbered image sensor rows of the L image sensor rows. You may set the other of time. When the exposure time setting step SA31, the image acquisition step SA32, and the information acquisition step SA33 are repeated, the repeated exposure time setting step SA31 is set for each of the odd-numbered image sensor rows in the previous exposure time setting step SA31. The exposure time that has been set may be interchanged with the exposure time that has been set for each even-numbered imaging element array.

  As a result, each time a normal image is acquired, the plurality of image sensor arrays used for the acquisition can be switched between an odd number of image sensor arrays and an even number of image sensor arrays. As a result, each of the sequentially acquired normal images can be displayed by interlace. Further, by complementing two consecutively acquired normal images with each other, a new normal image including an image by an odd-numbered plurality of imaging element arrays and an image by an even-numbered plurality of imaging element arrays is generated. be able to.

  As shown in FIG. 266, in the exposure time setting step SA31, the first exposure time is set when the setting mode is switched between the normal priority mode and the visible light priority mode and can be switched to the normal priority mode. The number of image sensors may be larger than the number of image sensors for which the second exposure time is set. In addition, when the mode is switched to the visible light priority mode, the number of image sensors for which the first exposure time is set may be smaller than the number of image sensors for which the second exposure time is set.

  As a result, when the setting mode is switched to the normal priority mode, the image quality of the normal image can be improved. When the setting mode is switched to the visible light priority mode, the reception efficiency of information from the light emitter is improved. can do.

  Further, as shown in FIG. 264, in the exposure time setting step SA31, a plurality of image sensors for which the first exposure time is set and a plurality of image sensors for which the second exposure time is set are in a checkered pattern ( For each image sensor included in the image sensor, the exposure time of the image sensor may be set so as to be distributed as in Checkered pattern.

  As a result, the plurality of image pickup devices for which the first exposure time is set and the plurality of image pickup devices for which the second exposure time is set are uniformly distributed. Not normal images and bright line images can be acquired.

  FIG. 270B is a block diagram of a receiving apparatus in Embodiment 12.

  The receiving device A30 is the above-described receiver that executes, for example, the processes shown in FIGS.

  That is, the receiving device A30 is a receiving device that receives information from a light-emitting body that changes in luminance, and includes a multiple exposure time setting unit A31, an imaging unit A32, and a decoding unit A33. The multiple exposure time setting unit A31 sets the first exposure time for some of the plurality of image pickup elements (K is an integer of 4 or more) included in the image sensor, and K pieces. A second exposure time shorter than the first exposure time is set for the remaining plurality of image sensors. The imaging unit A32 causes the image sensor to capture an image of a subject, which is a light-emitting body that changes in luminance, with the set first and second exposure times, so that a plurality of imaging elements with the first exposure time are set. A bright image corresponding to each of the plurality of exposure lines included in the image sensor, which is an image corresponding to the output from the plurality of imaging elements for which the second exposure time is set, while acquiring a normal image according to the output A bright line image that is an image including The decoding unit A33 acquires information by decoding a plurality of bright line patterns included in the acquired bright line image. Such a receiving apparatus A30 can achieve the same effects as the above-described receiving program.

  Next, the display of the content regarding the received visible light signal is demonstrated.

  FIGS. 271 and 272 are diagrams illustrating an example of display on the receiver when a visible light signal is received.

  As illustrated in FIG. 271 (a), when the receiver captures an image of the transmitter 10020d, the receiver displays an image 10020a on which the transmitter 10020d is projected. Further, the receiver generates and displays an image 10020b by superimposing the object 10020e on the image 10020a. The object 10020e is an image indicating that the image of the transmitter 10020d is present and that a visible light signal is received from the transmitter 10020d. The object 10020e may be an image that varies depending on the reception state of the visible light signal (the state of reception, the state of searching for a transmitter, the degree of progress of reception, the reception speed, or the error rate). For example, the receiver changes the color of the object 1020e, the thickness of the line, the type of line (single line, double line, dotted line, or the like), or the interval between dotted lines. Thereby, a user can be made to recognize a receiving state. Next, the receiver generates and displays an image 10020c by superimposing an image indicating the content of the acquired data on the image 10020a as an acquired data image 10020f. The acquired data is data associated with the received visible light signal or the ID indicated by the received visible light signal.

  When the receiver displays the acquired data image 10020f, as shown in FIG. 271 (a), the receiver displays the acquired data image 10020f like a balloon from the transmitter 10020d or near the transmitter 10020d. The acquired data image 10020f is displayed. Further, as shown in FIG. 271 (b), the receiver may display the acquired data image 10020f so that the acquired data image 10020f gradually approaches the receiver side from the transmitter 10020d. Thereby, the user can recognize which transmitter the received data image 10020f is based on the visible light signal received from. Further, as shown in FIG. 272, the receiver may display the acquired data image 10020f so that the acquired data image 10020f gradually emerges from the end of the receiver display. This makes it possible for the user to easily recognize that the visible light signal has been acquired at that time.

  Next, AR (Augmented Reality) will be described.

  FIG. 273 is a diagram illustrating an example of display of the acquired data image 10020f.

  When the image of the transmitter moves in the display, the receiver moves the acquired data image 10020f in accordance with the movement of the image of the transmitter. This allows the user to recognize that the acquired data image 10020f corresponds to the transmitter. The receiver may display the acquired data image 10020f in association with another image instead of the image of the transmitter. Thereby, AR display can be performed.

  Next, saving and discarding of acquired data will be described.

  FIG. 274 is a diagram illustrating an example of an operation for saving or discarding acquired data.

  For example, as shown in FIG. 274 (a), when the user performs a swipe down on the acquired data image 10020f, the receiver stores the acquired data indicated by the acquired data image 10020f. . The receiver places an acquired data image 10020f indicating the stored acquired data at the end of the acquired data image indicating one or more other already stored acquired data. This allows the user to recognize that the acquisition data indicated by the acquisition data image 10020f is the acquisition data stored last. For example, as shown in FIG. 274 (a), the receiver arranges the acquired data image 10020f in the forefront among the plurality of acquired data images.

  In addition, as illustrated in (b) of FIG. 274, when the user performs a right swipe on the acquired data image 10020f, the receiver discards the acquired data indicated by the acquired data image 10020f. Alternatively, the receiver may discard the acquired data indicated by the acquired data image 10020f when the image of the transmitter is framed out of the display by the user moving the receiver. Note that the same effect as described above can be obtained regardless of whether the swipe direction is up, down, left, or right. The receiver may display the swipe direction corresponding to the save or discard. As a result, the user can recognize that the data can be saved or discarded by the operation.

  Next, browsing of acquired data will be described.

  FIG. 275 is a diagram illustrating a display example when browsing acquired data.

  As shown in FIG. 275 (a), the receiver displays the acquired data images of a plurality of acquired data in a small manner overlapping the lower end of the display. At this time, when the user taps a part of the displayed acquired data image, the receiver displays each of the plurality of acquired data images in a large size as shown in FIG. 275 (b). Thereby, only when it is necessary to view each piece of acquired data, those acquired data images are displayed in a large size, and when not necessary, the display can be used effectively for other displays.

  When the user taps the acquired data image that the user wants to display in the state shown in FIG. 275 (b), the receiver displays the tapped acquired data image in a larger size as shown in FIG. 275 (c). A lot of information is displayed in the acquired data image. When the user taps the back surface display button 10024a, the receiver displays the back surface of the acquired data image, and displays other data related to the acquired data.

  Next, turning off camera shake correction at the time of accident position estimation will be described.

  The receiver disables camera shake correction (off), or converts the captured image according to the correction direction and correction amount of camera shake correction, thereby acquiring the correct imaging direction and accurately performing self-position estimation. Can be done. The captured image is an image obtained by imaging by the imaging unit of the receiver. Self-position estimation means that the receiver estimates its own position. In the self-position estimation, specifically, the receiver specifies the position of the transmitter based on the received visible light signal, and receives based on the size, position, or shape of the transmitter reflected in the captured image. Identify the relative positional relationship between the transmitter and transmitter. Then, the receiver estimates the position of the receiver based on the position of the transmitter and the relative positional relationship between the receiver and the transmitter.

  In addition, when partial reading is performed using only a part of the exposure lines shown in FIG. 262, that is, when the imaging shown in FIG. 262 is performed, the transmitter is out of frame with a slight blurring of the receiver. End up. In such a case, the receiver can continuously receive the signal by enabling the camera shake correction.

  Next, self-position estimation using an asymmetrical light emitting unit will be described.

  FIG. 276 is a diagram illustrating an example of a transmitter in Embodiment 12.

  The above-described transmitter includes a light emitting unit, and transmits a visible light signal by changing the luminance of the light emitting unit. In the above-described self-position estimation, the receiver determines the relative positional relationship between the receiver and the transmitter based on the shape of the transmitter (specifically, the light emitting unit) in the captured image. Find the relative angle to the transmitter. Here, for example, as shown in FIG. 276, when the transmitter includes a light-emitting unit 10090a having a rotationally symmetric shape, as described above, based on the shape of the transmitter in the captured image, The relative angle with the receiver cannot be determined accurately. Therefore, it is desirable that the transmitter includes a light emitting unit having a shape that is not rotationally symmetric. Thereby, the receiver can obtain | require correctly the above-mentioned relative angle. That is, since the error of the measurement result is large in the azimuth sensor for acquiring the angle, the receiver can perform accurate self-position estimation by using the relative angle obtained by the above method.

  Here, as shown in FIG. 276, the transmitter may include a light emitting unit 10090b that is not in a completely rotationally symmetric shape. The shape of the light emitting unit 10090b is symmetric with respect to 90 ° rotation, but is not completely rotationally symmetric. In this case, the receiver obtains a rough angle with the azimuth sensor, and further uses the shape of the transmitter in the captured image to uniquely limit the relative angle between the receiver and the transmitter. And accurate self-position estimation can be performed.

  Further, the transmitter may include a light emitting unit 10090c shown in FIG. The shape of the light emitting unit 10090c is basically a rotationally symmetric shape. However, since a light guide plate or the like is provided in a part of the light emitting unit 10090c, the shape of the light emitting unit 10090c is not rotationally symmetric.

  Further, the transmitter may include a light emitting unit 10090d shown in FIG. Each of the light emitting units 10090d includes rotationally symmetric illumination. However, the overall shape of the light emitting unit 10090d configured by combining them is not a rotationally symmetric shape. Therefore, the receiver can perform accurate self-position estimation by imaging the transmitter. In addition, it is not necessary for all illumination included in the light emitting unit 10090d to be illumination for visible light communication that changes in luminance in order to transmit a visible light signal, and only some illumination is illumination for visible light communication. May be.

  Further, the transmitter may include a light emitting unit 10090e and an object 10090f illustrated in FIG. Here, the object 10090f is an object (for example, a fire alarm or a pipe) configured so that the positional relationship with the light emitting unit 10090e does not change. Since the shape of the combination of the light emitting unit 10090e and the object 10090f is not a rotationally symmetric shape, the receiver can accurately perform self-position estimation by imaging the light emitting unit 10090e and the object 10090f.

  Next, a time series process for self-position estimation will be described.

  Each time the receiver captures an image, the receiver can perform self-position estimation from the position and shape of the transmitter in the captured image. As a result, the receiver can estimate the moving direction and distance of the receiver being imaged. Further, the receiver can perform more accurate self-position estimation by performing triangulation using a plurality of frames or images. By integrating estimation results using a plurality of images and estimation results using a plurality of different combinations of images, the receiver can perform self-position estimation more accurately. At this time, the receiver can perform self-position estimation more accurately by emphasizing and summing up the results estimated from recent captured images.

  Next, skipping of optical black will be described.

  FIG. 277 is a diagram illustrating an example of a reception method in Embodiment 12. Note that the horizontal axis of the graph shown in FIG. 277 indicates time, and the vertical axis indicates the position of each exposure line in the image sensor. Furthermore, a solid line arrow in the graph indicates a time (exposure timing) at which exposure of each exposure line in the image sensor is started.

  During normal imaging, the receiver reads the horizontal optical black signal in the image sensor as shown in FIG. 277 (a), but skips the horizontal optical black signal as shown in FIG. 277 (b). May be. Thereby, a continuous visible light signal can be received.

  Horizontal optical black is optical black in the horizontal direction on the exposure line. The vertical optical black is a portion of the optical black other than the horizontal optical black.

  Since the receiver adjusts the black level based on the signal read from the optical black, the black level can be adjusted using the optical black at the start of the visible light imaging similarly to the normal imaging. When vertical optical black can be used, the receiver can perform continuous reception and black level adjustment by performing black level adjustment using only vertical optical black. When visible light imaging continues, the receiver may adjust the black level using horizontal optical black every predetermined time. In the case where the normal imaging and the visible light imaging are alternately performed, the receiver skips the horizontal optical black signal when continuously performing the visible light imaging, and reads the horizontal optical black signal otherwise. Then, the receiver can adjust the black level while continuously receiving the visible light signal by adjusting the black level based on the read signal. The receiver may adjust the black level by setting the darkest part of the visible light captured image as black.

  As described above, the optical black from which the signal is read out is only vertical optical black, so that continuous visible light signals can be received. In addition, by providing a mode for skipping horizontal optical black signals, black level adjustment can be performed during normal imaging, and continuous communication can be performed as necessary during visible light imaging. In addition, by skipping the horizontal optical black signal, the difference in timing of starting exposure between exposure lines is increased, so that a visible light signal from a transmitter that is only small can be received.

  Next, an identifier indicating the type of transmitter will be described.

  The transmitter may transmit the transmitter identifier indicating the type of transmitter added to the visible light signal. In this case, when the receiver receives the transmitter identifier, the receiver can perform a receiving operation according to the type of the transmitter. For example, in the case where the transmitter identifier indicates digital signage, the transmitter displays a content ID indicating which content is currently displayed in addition to the transmitter ID for performing individual identification of the transmitter as a visible light signal. As sending. The receiver can display information according to the content currently displayed by the transmitter by separately handling these IDs based on the transmitter identifier. For example, when the transmitter identifier indicates digital signage or an emergency light, the receiver can reduce reception errors by imaging with increased sensitivity.

(Embodiment 13)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  First, the header pattern in the present embodiment will be described.

  FIG. 278 is a diagram illustrating an example of a header pattern in the present embodiment.

  The transmitter divides data to be transmitted into packets and transmits the packets. The packet is composed of a header and a body, for example. The luminance change pattern of the header, that is, the header pattern needs to be a luminance change pattern that does not exist in the body. Thereby, the position of the packet can be specified in the data transmitted continuously.

  For example, data to be transmitted is modulated by a 4PPM modulation scheme. Specifically, in the 4PPM, data to be transmitted is modulated into a luminance change pattern in which one of the four slots indicates “0” and the remaining three slots indicate “1”. Therefore, when continuously transmitted data is modulated, the number of consecutive slots indicating “0” is 2 or less, and the number of slots indicating “0” among the 4 slots and the next 4 slots. Is 2 or less.

  Therefore, in the present embodiment, the header pattern is “111111111000” shown in (a) of FIG. 278, “111111110001” shown in (b), “111111101001” shown in (c), or “111111100101” shown in (d). ". Thereby, a header and the data (namely, body) transmitted continuously can be identified. In other words, in the header pattern shown in FIG. 278 (a), if only the four slots “1000” after the header pattern are viewed, it can be identified that the four slots are a part of the header. In this case, since the number of slots indicating “0” in the header is 3, and the maximum number of consecutive slots indicating “0” is 4, the receiver can easily recognize the luminance change. That is, the receiver can easily receive data from a small transmitter or a remote transmitter.

  Also, in the header pattern shown in FIG. 278 (b), if only 5 slots “10001” after the header pattern are seen, it can be identified that the 5 slots are a part of the header. In this case, since the maximum number of consecutive slots indicating “0” is 3, which is smaller than that in the case of FIG. 278 (a), flicker due to a change in luminance can be suppressed. As a result, the burden on the circuit of the transmitter or the design requirement can be suppressed. That is, the capacitor can be reduced, and power consumption, heat generation, or a load on the power source can be suppressed.

  Also, in the header pattern shown in (c) or (d) of FIG. 278, if only the six slots “101001” or “100101” after the header pattern are seen, the six slots are identified as part of the header. can do. In this case, the maximum number of consecutive slots indicating “0” is 2, which is smaller than that in the case of FIG. 278 (b), and therefore flicker due to the change in luminance can be further suppressed.

  FIG. 279 is a diagram for describing an example of a packet configuration of a communication protocol in this embodiment.

  The transmitter divides data to be transmitted into packets and transmits the packets. The packet includes a header, an address part, a data part, and an error correction code part. By setting the luminance change pattern of the header to a luminance change pattern that does not exist in other portions, the position of the packet in the continuous data can be specified. A part of the divided data is stored in the data part, and an address indicating which part of the data of the data part is stored in the address part. The error correction code unit is a code for detecting or correcting a reception error of a part or all of a packet (specifically, ECC1, ECC2 or ECC3 shown in FIG. 279, which are collectively referred to as an error correction code). ) Is stored.

  ECC1 is an error correction code of the address part. By providing an error correction code for the address portion instead of an error correction code for the entire packet, the reliability of the address portion can be made higher than the overall reliability. Thereby, when a plurality of packets are received, the data portion can be verified by comparing the data portions of the packets having the same address, and the reception error rate can be reduced. Even if the error correction code in the address part is made longer than the error correction code in the data part, the same effect can be obtained.

  ECC2 and ECC3 are error correction codes of the data part. The number of error correction code units may be other than two. By dividing the error correction code of the data part into multiple parts, error correction can be performed using the error correction codes that have been received, and reliable data can be received without receiving the entire packet. Can do.

  The transmitter may transmit a number of error correction codes equal to or less than a specified number. Thereby, the receiver can receive data at high speed. This transmission method is effective for a transmitter having a small light emitting unit and high luminance, such as a downlight. This is because when the luminance is high, the error occurrence probability is low, so that many error correction codes are not necessary. When ECC3 is not sent, since the next header is transmitted from ECC2, the state where the luminance is high continues for four slots or more, and the receiver can recognize that the part is not ECC3.

  As shown in (b) of FIG. 279, the header, the address part, and ECC1 are transmitted at a lower frequency than the data part, ECC2, and ECC3. In other words, the data part, ECC2 and ECC3 are transmitted at a higher frequency than the header, address part and ECC1. As a result, it is possible to reduce the reception error rate for data such as a header, and to transmit large data in the data portion quickly.

  Here, a receiving method for comparing data portions having the same address will be described.

  FIG. 280 is a flowchart illustrating an example of a reception method in this embodiment.

  The receiver receives the packet (step S10101) and performs error correction (step S10102). Then, the receiver determines whether or not a packet having the same address as that of the received packet has already been received (step S10103). If it is determined that the data is received (Y in Step 10103), the receiver compares the data. That is, the receiver determines whether the data parts are equal (step S10104). Here, when it is determined that they are not equal (N in step S10104), the receiver further determines whether the difference in the plurality of data parts is equal to or greater than a predetermined number, specifically, the number of different bits or the luminance. It is determined whether the number of slots with different states is equal to or greater than a predetermined number (step S10105). If it is determined that the number is equal to or greater than the predetermined number (N in step S10105), the receiver discards the packet that has already been received (step S10106). Thereby, when a packet is started to be received from another transmitter, interference with a packet received from a previous transmitter can be avoided. On the other hand, if it is determined that the number is not equal to or greater than the predetermined number (N in step S10105), the receiver sets the data of the data part with the most packets having the same data part as the data of the address (step S10107). Alternatively, the receiver takes the most equal bit as the value of that bit at that address. Alternatively, the receiver sets the luminance state having the largest number of equal luminance states as the luminance state of the slot at the address, and demodulates the data at the address.

  Here, a reception method for demodulating data in the data portion from a plurality of packets will be described.

  FIG. 281 is a flowchart illustrating an example of a reception method in this embodiment.

  First, the receiver receives the packet (step S10111) and performs error correction of the address part (step S10112). At this time, the receiver does not demodulate the data part and holds the pixel value obtained by imaging as it is. Then, the receiver determines whether or not there are a predetermined number or more of packets having the same address among the plurality of packets that have already been received (step S10113). If it is determined that the packet exists (Y in step S10113), the receiver performs demodulation processing by combining pixel values of portions corresponding to data portions of a plurality of packets having the same address. Since the timings at which the plurality of packets are received are different, the pixel values in the data portion are values that reflect the luminance of the transmitter at slightly different times. Therefore, the portion to be demodulated as described above includes a larger amount of data (number of samples) than the data portion of a single packet. Thereby, a data part can be demodulated more correctly. Further, by increasing the number of samples, a signal modulated with a higher modulation frequency can be demodulated.

  As shown in FIG. 279 (b), the data part and its error correction code part are modulated at a higher frequency than the header part, the address part, and the error correction code part of the address part. By the above demodulation method, since the data portion and the subsequent part can be demodulated even if modulated at a high modulation frequency, the transmission time of the whole packet can be shortened by this configuration, and from a smaller light source even from a longer distance. However, a visible light signal can be received faster.

  Next, a reception method for receiving variable length address data will be described.

  FIG. 282 is a flowchart illustrating an example of a reception method in this embodiment.

  The receiver receives the packet (step S10121), and determines whether or not a packet in which all the bits of the data part are 0 (hereinafter referred to as a 0 termination packet) is received (step S10122). Here, if it is determined that it has been received, that is, if it is determined that there is a zero-termination packet (Y in step S10122), the receiver determines whether or not all packets having addresses below the zero-termination packet address are available, that is, It is determined whether or not it has been received (step S10123). Note that the address is set to a value that increases in accordance with the order of transmission of each packet generated by dividing the transmitted data. If it is determined that the receiver is complete (Y in step S10123), the receiver determines that the address of the 0-termination packet is the last address of the packet transmitted from the transmitter. Then, the receiver restores the data by connecting the data of the packets of each address up to the 0 terminal packet (step S10124). Further, the receiver performs an error check on the restored data (step S10125). This makes it possible to send and receive variable-length address data even when the data to be transmitted is not divided into several parts, that is, when the address is not fixed length but variable length. More IDs can be transmitted and received with high efficiency.

  Next, a reception method using an exposure time longer than the modulation frequency period will be described.

  283 and 284 are diagrams for describing a reception method in which the receiver according to the present embodiment uses an exposure time longer than the period of the modulation frequency (modulation period).

  For example, as shown in FIG. 283 (a), if the exposure time is set to a time equal to the modulation period, the visible light signal may not be received correctly. The modulation period is the time of one slot described above. That is, in such a case, there are few exposure lines (exposure lines indicated by black in FIG. 283) reflecting the luminance state of a certain slot. As a result, it is difficult to estimate the luminance of the transmitter when a lot of noise is accidentally included in the pixel values of these exposure lines.

  On the other hand, for example, as shown in FIG. 283 (b), when the exposure time is set to a time longer than the modulation period, a visible light signal can be correctly received. That is, in such a case, since there are many exposure lines reflecting the brightness of a certain slot, the brightness of the transmitter can be estimated from the pixel values of many exposure lines, and it is resistant to noise.

  On the other hand, if the exposure time is too long, the visible light signal cannot be received correctly.

  For example, as shown in FIG. 284 (a), when the exposure time is equal to the modulation period, the luminance change (that is, the change in the pixel value of each exposure line) received by the receiver is used for transmission. Follow changes in brightness. However, as shown in FIG. 284 (b), when the exposure time is three times the modulation period, the luminance change received by the receiver can sufficiently follow the luminance change used for transmission. Can not. Further, as shown in FIG. 284 (c), when the exposure time is 10 times the modulation period, the luminance change received by the receiver cannot follow the luminance change used for transmission at all. That is, the longer the exposure time, the higher the noise resistance because the luminance can be estimated from many exposure lines. However, the longer the exposure time, the lower the identification margin or the smaller the identification margin. Due to these balances, the noise resistance can be maximized by setting the exposure time to about 2 to 5 times the modulation period.

  Next, the number of packet divisions will be described.

  FIG. 285 is a diagram illustrating an efficient division number with respect to the size of transmission data.

  When a transmitter transmits data by a change in luminance, if all transmitted data (transmission data) is included in one packet, the data size of the packet is large. However, as described with reference to FIG. 279, when the transmission data is divided into a plurality of partial data and the partial data is included in each packet, the data size of each packet is reduced. Here, the receiver receives the packet by imaging. However, the larger the data size of the packet, the more difficult it is for the receiver to receive the packet by one imaging, and it is necessary to repeat imaging.

  Therefore, as shown in (a) and (b) of FIG. 285, it is desirable for the transmitter to increase the division number of the transmission data as the data size of the transmission data increases. However, if the number of divisions is too large, the transmission data cannot be restored unless all of the partial data is received.

  Therefore, as shown in FIG. 285 (a), the data size of the address (address size) is variable, and the data size of the transmission data is 2-16 bits, 16-24 bits, 24-64 bits, 66- In the case of 78 bits, 78-128 bits, 128 bits or more, transmission data into 1-2 pieces, 2-4 pieces, 4 pieces, 4-6 pieces, 6-8 pieces, 7 pieces or more, respectively. Is divided, transmission data can be efficiently transmitted by a visible light signal. As shown in FIG. 285 (b), the data size of the address (address size) is fixed to 4 bits, and the data size of the transmission data is 2-8 bits, 8-16 bits, 16-30 bits, If 30-64 bits, 66-80 bits, 80-96 bits, 96-132 bits, 132 bits or more, 1-2, 2-3, 2-4, 4-5, When the transmission data is divided into 4-7 pieces, 6 pieces, 6-8 pieces, and 7 or more partial data pieces, the transmission data can be efficiently transmitted by a visible light signal.

  Further, the transmitter sequentially changes the luminance based on each packet including each of the plurality of partial data. For example, the transmitter changes the luminance based on the packets in the order of the addresses of the packets. Further, the transmitter may perform the luminance change based on the plurality of partial data again in an order different from the address order. Thereby, each partial data can be reliably received by the receiver.

  Next, a notification operation setting method by the receiver will be described.

  FIG. 286A is a diagram illustrating an example of a setting method in this embodiment.

  First, the receiver acquires a notification operation identifier for identifying the notification operation and a priority of the notification operation identifier (specifically, an identifier indicating the priority) from a server near the receiver. (Step S10131). Here, in the notification operation, when each packet including each of a plurality of partial data is transmitted due to a change in luminance and received by the receiver, reception is performed to notify the user of the receiver that the packets have been received. The operation of the machine. For example, the operation is sounding, vibration, or screen display.

  Next, the receiver receives a packetized visible light signal, that is, each packet including each of a plurality of partial data (step S10132). Here, the receiver acquires the notification operation identifier and the priority of the notification operation identifier (specifically, an identifier indicating the priority) included in the visible light signal (step S10133).

  Further, the receiver sets the current notification operation setting of the receiver, that is, the notification operation identifier preset in the receiver and the priority of the notification operation identifier (specifically, an identifier indicating the priority). ) Is read out (step S10134). Note that the notification operation identifier set in advance in the receiver is set by a user operation, for example.

  Then, the receiver selects the identifier having the highest priority among the preset notification operation identifiers and the notification operation identifiers acquired in steps S10131 and S10133 (step S10135). Next, the receiver resets the selected notification operation identifier to itself, thereby performing the operation indicated by the selected notification operation identifier, and notifies the user of reception of the visible light signal (step S10136).

  Note that the receiver may select a notification operation identifier having a high priority from the two notification operation identifiers without performing any one of steps S10131 and S10133.

  Note that the priority of notification action identifiers transmitted from servers installed in theaters or museums, or the priority of notification action identifiers included in visible light signals transmitted within those facilities are set high. Also good. Thereby, it is possible to prevent the sound for the reception notification from being sounded in the facility regardless of the setting of the user. In other facilities, by setting the priority of the notification operation identifier low, the receiver can notify the reception by the operation according to the user's setting.

  FIG. 286B is a diagram illustrating another example of the setting method according to the present embodiment.

  First, the receiver acquires a notification operation identifier for identifying the notification operation and a priority of the notification operation identifier (specifically, an identifier indicating the priority) from a server near the receiver. (Step S10141). Next, the receiver receives a packetized visible light signal, that is, each packet including each of a plurality of partial data (step S10142). Here, the receiver acquires the notification operation identifier and the priority of the notification operation identifier (specifically, an identifier indicating the priority) included in the visible light signal (step S10143).

  Further, the receiver sets the current notification operation setting of the receiver, that is, the notification operation identifier preset in the receiver and the priority of the notification operation identifier (specifically, an identifier indicating the priority). ) Is read (step S10144).

  The receiver includes an operation notification identifier indicating an operation for prohibiting the generation of the notification sound in the notification operation identifier set in advance and the notification operation identifier acquired in each of steps S10141 and S10143. It is determined whether or not it is (step S10145). Here, if it is determined that it is included (Y in step S10145), the receiver sounds a notification sound for notifying completion of reception (step S10146). On the other hand, if it is determined that it is not included (N in step S10145), the receiver notifies the user of the completion of reception by, for example, vibration (step S10147).

  Note that the receiver does not perform either one of steps S10141 and S10143, and determines whether or not the two notification operation identifiers include an operation notification identifier indicating an operation that prohibits the generation of the notification sound. May be.

  In addition, the receiver may perform self-position estimation based on an image obtained by imaging, and notify the user of reception through an operation associated with the estimated position or a facility at the position.

  FIG. 287A is a flowchart illustrating processing of the information processing program in the thirteenth embodiment.

  This information processing program is a program for changing the luminance of the light emitter of the transmitter described above according to the number of divisions shown in FIG.

  That is, this information processing program is an information processing program that causes a computer to process information to be transmitted in order to transmit the information to be transmitted by a change in luminance. Specifically, the information processing program includes an encoding step SA41 that generates an encoded signal by encoding information to be transmitted, and the number of bits of the generated encoded signal is in a range of 24 to 64 bits. In some cases, the computer executes a division step SA42 for dividing the encoded signal into four partial signals and an output step SA43 for sequentially outputting the four partial signals. These partial signals are output as packets shown in FIG. 279 (a). The information processing program may cause the computer to specify the number of bits of the encoded signal and determine the number of partial signals based on the specified number of bits. In this case, the information processing program causes the computer to generate the determined number of partial signals by dividing the encoded signal.

  Thereby, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and output. As a result, when the luminance of the illuminant changes according to the four partial signals that are output, the four partial signals are transmitted as visible light signals and received by the receiver. Here, the larger the number of bits of the output signal, the more difficult it is for the receiver to properly receive the signal by imaging, and the reception efficiency decreases. Therefore, it is desirable to divide the signal into signals having a small number of bits, that is, small signals. However, if the signal is too finely divided into many small signals, the receiver cannot receive the original signal unless each small signal is individually received, resulting in a decrease in reception efficiency. Therefore, as described above, when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and sequentially output to indicate the information to be transmitted. The encoded signal can be transmitted as a visible light signal with the best reception efficiency. As a result, communication between various devices can be enabled.

  In the output step SA43, the four partial signals may be output in accordance with the first order, and the four partial signals may be output again in accordance with a second order different from the first order.

  As a result, the four partial signals are repeatedly output in a different order. Therefore, when each output signal is transmitted to the receiver as a visible light signal, the reception efficiency of the four partial signals is increased. It can be further increased. That is, even if the four partial signals are repeatedly output in the same order, the same partial signal may not be received by the receiver. However, by changing the order, the occurrence of such a case can be suppressed.

  Also, as shown in FIGS. 286A and 286B, in the output step SA43, a notification operation identifier may be further attached to the four partial signals. The notification operation identifier is used to identify the operation of the receiver notifying the user of the receiver that the four partial signals are received when the four partial signals are transmitted by the luminance change and received by the receiver. Identifier.

  Thus, when the notification operation identifier is transmitted as a visible light signal and received by the receiver, the receiver receives the four partial signals to the user according to the operation identified by the notification operation identifier. You can be notified. That is, the notification operation by the receiver can be set on the side that transmits the information to be transmitted.

  Further, as shown in FIGS. 286A and 286B, in the output step SA43, a priority identifier for identifying the priority of the notification operation identifier may be further output in association with the four partial signals.

  Thus, when the priority identifier and the notification operation identifier are transmitted as a visible light signal and received by the receiver, the receiver handles the notification operation identifier according to the priority identified by the priority identifier. be able to. That is, when the receiver acquires another notification operation identifier, the receiver is identified by the notification operation identified by the notification operation identifier transmitted as the visible light signal and the other notification operation identifier. One of the notification operations can be selected based on the priority.

  FIG. 287B is a block diagram of an information processing device in Embodiment 13.

  This information processing apparatus A40 is an apparatus for changing the luminance of the light emitter (light emitting unit) of the transmitter described above according to the number of divisions shown in FIG.

  That is, the information processing apparatus A40 is an apparatus that processes the information to be transmitted in order to transmit the information to be transmitted with a luminance change. Specifically, the information processing apparatus A40 includes an encoding unit A41 that generates an encoded signal by encoding information to be transmitted, and a range in which the number of bits of the generated encoded signal is 24 to 64 bits. In this case, a dividing unit A42 that divides the encoded signal into four partial signals and an output unit A43 that sequentially outputs the four partial signals are provided. Such an information processing apparatus A40 can achieve the same effects as the information processing program described above.

  Next, registration of network connection of the electronic device will be described.

  FIG. 288 is a diagram for describing an application example of the transmission / reception system in this embodiment.

  The transmission / reception system includes a transmitter 10131b configured as an electronic device such as a washing machine, a receiver 10131a configured as a smartphone, and a communication device 10131c configured as an access point or a router.

  FIG. 289 is a flowchart showing processing operations of the transmission / reception system according to the present embodiment.

  When the start button is pressed (step S10165), the transmitter 10131b transmits Wi-Fi, Bluetooth (registered trademark) information such as an SSID, a password, an IP address, a MAC address, or an encryption key to connect to the transmitter 10131b. ) Or Ethernet (registered trademark) or the like (step S10166), and waits for connection. The transmitter 10131b may transmit these pieces of information directly or indirectly. When transmitting indirectly, the transmitter 10131b transmits an ID associated with the information. For example, the receiver 10131a that has received the ID downloads information associated with the ID from a server or the like.

  The receiver 10131a receives the information (step S10151), connects to the transmitter 10131b, and connects to the communication device 10131c configured as an access point or router (SSID, password, IP address, MAC address, Alternatively, the encryption key or the like is transmitted to the transmitter 10131b (step S10152). The receiver 10131a registers information (MAC address, IP address, encryption key, etc.) for the transmitter 10131b to connect to the communication device 10131c in the communication device 10131c, and makes the communication device 10131c wait for connection. Further, the receiver 10131a notifies the transmitter 10131b that preparation for connection from the transmitter 10131b to the communication device 10131c is completed (step S10153).

  The transmitter 10131b disconnects from the receiver 10131a (step S10168) and connects to the communication device 10131c (step S10169). If the connection is successful (Y in step S10170), the transmitter 10131b notifies the receiver 10131a of the connection success via the communication device 10131c, and notifies the user of the connection success with a screen display, LED status, voice, or the like. (Step S10171). If the connection fails (N in step S10170), the transmitter 10131b notifies the receiver 10131a of the connection failure through visible light communication, and notifies the user in the same way as when it is successful (step S10172). The connection success may be notified by visible light communication.

  The receiver 10131a connects to the communication device 10131c (step S10154), and if there is no notification of a connection success or failure (N in step S10155 and N in step S10156), the transmitter 10131b can be accessed via the communication device 10131c. It is confirmed whether or not (step S10157). If not (N in step S10157), the receiver 10131a determines whether or not the connection to the transmitter 10131b using the information received from the transmitter 10131b has been made a predetermined number of times or more (step S10158). If it is determined that the predetermined number of times has not been performed (N in step S10158), the receiver 10131a repeats the process from step S10152. On the other hand, if it is determined that the predetermined number of times has been performed (Y in step S10158), the receiver 10131a notifies the user of the processing failure (step S10159). If the receiver 10131a determines in step S10156 that there has been a notification of a successful connection (Y in step S10156), the receiver 10131a notifies the user of the processing success (step S10160). That is, the receiver 10131a notifies the user whether or not the transmitter 10131b has been able to connect to the communication device 10131c by screen display, voice, or the like. Accordingly, the transmitter 10131b can be connected to the communication device 10131c without requiring complicated input by the user.

  Next, registration of a network connection of an electronic device (when connecting via another electronic device) will be described.

  FIG. 290 is a diagram for describing an application example of the transmission and reception system in this embodiment.

  This transmission / reception system includes an air conditioner 10133b, a transmitter 10133c configured as an electronic device such as a wireless adapter connected to the air conditioner 10133b, a receiver 10133a configured as, for example, a smartphone, and a communication device configured as an access point or a router. 10133d and another electronic device 10133e configured as, for example, a wireless adapter, a wireless access point, or a router.

  FIG. 291 is a flowchart showing processing operations of the transmission / reception system in the present embodiment. Hereinafter, the air conditioner 10133b or the transmitter 10133c is referred to as an electronic device A, and the electronic device 10133e is referred to as an electronic device B.

  First, when the start button is pressed (step S10188), the electronic device A transmits information (individual ID, password, IP address, MAC address, encryption key, etc.) for connection to itself (step S10189). The connection is awaited (step S10190). The electronic device A may transmit these pieces of information directly or indirectly as described above.

  The receiver 10133a receives the information from the electronic device A (step S10181), and transmits the information to the electronic device B (step S10182). When electronic device B receives the information (step S10196), it connects to electronic device A according to the received information (step S10197). Then, the electronic device B determines whether or not the connection with the electronic device A is established (step S10198), and notifies the receiver 10133a of the success or failure (step S10199 or step S10200).

  If the electronic device A is connected to the electronic device B during a predetermined time (Y in step S10191), the electronic device A notifies the receiver 10133a of the connection success via the electronic device B (step S10192) and is not connected (step S10192). N in step S10191), the receiver 10133a is notified of the connection failure by visible light communication (step S10193). In addition, the electronic device A notifies the user of the success or failure of the connection through a screen display, a light emission state, sound, or the like. Accordingly, the electronic device A (transmitter 10133c) can be connected to the electronic device B (electronic device 10133e) without requiring complicated input by the user. Note that the air conditioner 10133b and the transmitter 10133c illustrated in FIG. 290 may be configured integrally, and similarly, the communication device 10133d and the electronic device 10133e may be configured integrally.

  Next, transmission of appropriate imaging information will be described.

  FIG. 292 is a diagram for describing an example of application of the transmission and reception system in this embodiment.

  This transmission / reception system includes a receiver 10135a configured as, for example, a digital still camera or a digital video camera, and a transmitter 10135b configured as, for example, illumination.

  FIG. 293 is a flowchart illustrating a processing operation of the transmission / reception system according to the present embodiment.

  First, the receiver 10135a sends an imaging information transmission command to the transmitter 10135b (step S10211). Next, the transmitter 10135b receives the imaging information transmission command, the imaging information transmission button is pressed, the imaging information transmission switch is turned on, or the power is turned on (in step S10221). Y), imaging information is transmitted (step S10222). The imaging information transmission command is a command for transmitting imaging information, and the imaging information indicates, for example, the color temperature of illumination, spectral distribution, illuminance, or light distribution. The transmitter 10135b may transmit the imaging information directly or indirectly as described above. When transmitting indirectly, the transmitter 10135b transmits an ID associated with the imaging information. The receiver 10135a that has received the ID downloads, for example, imaging information associated with the ID from a server or the like. At this time, the transmitter 10135b is a method for transmitting a transmission stop command to itself (frequency of radio waves, infrared rays, or sound waves for transmitting the transmission stop command, or an SSID, a password or an IP address for connection to self-confidence) ) May be sent.

  When receiving the imaging information (step S10212), the receiver 10135a transmits a transmission stop command to the transmitter 10135b (step S10213). Here, when the transmitter 10135b receives a transmission stop command from the receiver 10135a (step S10223), the transmitter 10135b stops transmission of imaging information and emits light uniformly (step S10224).

  Furthermore, the receiver 10135a sets the imaging parameter according to the imaging information received in step S10212 (step S10214), or notifies the user of the imaging information. The imaging parameter is, for example, white balance, exposure time, focal length, sensitivity, or scene mode. Thereby, it is possible to take an image with an optimum setting according to the illumination. Next, the receiver 10135a captures an image after the transmission of imaging information from the transmitter 10135b is stopped (Y in step S10215) (step S10216). Thereby, it is possible to perform imaging without changing the brightness of the subject due to signal transmission. Note that, after step S10216, the receiver 10135a may transmit a transmission start command for prompting the start of transmission of imaging information to the transmitter 10135b (step S10217).

  Next, the display of a charge state is demonstrated.

  FIG. 294 is a diagram for describing an example of application of a transmitter in this embodiment.

  For example, the transmitter 10137b configured as a charger includes a light emitting unit, and transmits a visible light signal indicating a charged state of the battery from the light emitting unit. Thus, the state of charge of the battery can be notified without an expensive display device. When a small LED is used as the light emitting unit, a visible light signal cannot be received unless the LED is imaged from nearby. In addition, in the transmitter 10137c having a protrusion near the LED, it is difficult to close-up the LED due to the protrusion. Therefore, the visible light signal from the transmitter 10137b having no protrusion near the LED can be received more easily than the visible light signal from the transmitter 10137c.

(Embodiment 14)
In this embodiment, each application example using a receiver such as a smartphone in each of the above embodiments and a transmitter that transmits information as a blinking pattern of an LED or an organic EL will be described.

  First, transmission in the demo mode and failure will be described.

  FIG. 295 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  When an error has occurred, the transmitter transmits a signal indicating that the error has occurred, or a signal corresponding to the error code. Can tell error details. The receiver can correct the error or appropriately report the error content to the service center by indicating an appropriate response according to the error content.

  If the transmitter is in demo mode, it will send a demo code. As a result, for example, when a demo of a transmitter that is a product is performed at a storefront, the store visitor can receive the demo code and acquire the product description associated with the demo code. Whether or not the demo mode is selected is determined by whether the transmitter operation setting is in the demo mode, the storefront CAS card is inserted, the CAS card is not inserted, or the recording medium is not inserted. Judging from the point.

  Next, signal transmission from the remote controller will be described.

  FIG. 296 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  For example, when a transmitter configured as an air conditioner remote controller receives main body information, the transmitter transmits the main body information, so that the receiver receives information on a distant main body from a nearby transmitter. can do. The receiver can also receive information from a main body that exists in a place where visible light communication is impossible, such as over a network.

  Next, a process of transmitting only when the place is bright is described.

  FIG. 297 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  The transmitter transmits if the ambient brightness is above a certain level, and stops transmitting if the brightness is below a certain level. Thereby, for example, a transmitter configured as an advertisement for a train can automatically stop its operation when the vehicle enters the garage, and battery consumption can be suppressed.

  Next, content distribution (association change / scheduling) in accordance with the display of the transmitter will be described.

  FIG. 298 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  The transmitter associates the content desired to be acquired by the receiver with the transmission ID in accordance with the display timing of the content to be displayed. Each time the display content is changed, the association change is registered with the server.

  When the display timing of the display content is known, the transmitter sets the server so that another content is delivered to the receiver in accordance with the change timing of the display content. When the request for the content associated with the transmission ID is received from the receiver, the server transmits the content according to the set schedule to the receiver.

  Thereby, for example, when a transmitter configured as digital signage changes display contents one after another, the receiver can acquire content that matches the content displayed by the transmitter.

  Next, content distribution (synchronization by time) according to the display of the transmitter will be described.

  FIG. 299 is a diagram illustrating an example of operation of a transmitter in this embodiment.

  In response to a content acquisition request associated with a predetermined ID, it is registered in advance in the server so that different content is passed according to the time.

  The transmitter synchronizes the time with the server, adjusts the timing so that a predetermined part is displayed at a predetermined time, and displays the content.

  Thereby, for example, when a transmitter configured as digital signage changes display contents one after another, the receiver can acquire content that matches the content displayed by the transmitter.

  Next, content distribution (transmission of display time) in accordance with the display of the transmitter will be described.

  FIG. 300 is a diagram illustrating an example of operation of a transmitter and a receiver in this embodiment.

  The transmitter transmits the display time of the content being displayed in addition to the ID of the transmitter. The content display time is information that can identify the currently displayed content, and can be expressed by, for example, an elapsed time from the start time of the content.

  The receiver acquires content associated with the received ID from the server, and displays the content in accordance with the received display time. Thereby, for example, when a transmitter configured as digital signage changes display contents one after another, the receiver can acquire content that matches the content displayed by the transmitter.

  Further, the receiver changes the content to be displayed as time elapses. As a result, even if a signal is not received again when the display content of the transmitter changes, the content that matches the display content is displayed.

  Next, data uploading according to the user's permission status will be described.

  FIG. 301 is a diagram illustrating an example of operation of a receiver in this embodiment.

  If the user has registered for an account, the receiver is authorized to access information such as the location of the receiver, phone number, ID, installed application, and user (Age, sex, occupation, preference, etc.) are transmitted to the server together with the received ID.

  If the account is not registered, if the user permits uploading of information as described above, it is similarly sent to the server. If not permitted, only the received ID is sent to the server. .

  As a result, the user can receive the content according to the situation at the time of reception and his / her personality, and the server can be used for data analysis by obtaining user information.

  Next, activation of the content reproduction application will be described.

  FIG. 302 is a diagram illustrating an example of operation of a receiver in this embodiment.

  The receiver acquires content associated with the received ID from the server. When the active application can handle (can display or play) the acquired content, the acquired content is displayed / reproduced by the active application. If it cannot be handled, it is confirmed whether or not an app that can be handled is installed in the receiver. If it is installed, the app is activated to display / play the acquired content. If it is not installed, it automatically installs, displays a message prompting installation, displays a download screen, and displays and plays the acquired content after installation.

  As a result, the acquired content can be appropriately handled (displayed / reproduced).

  Next, activation of the designated application will be described.

  FIG. 303 is a diagram illustrating an example of operation of a receiver in this embodiment.

  The receiver acquires the content associated with the received ID and information (application ID) for specifying the application to be activated from the server. When the running application is the designated application, the acquired content is displayed / reproduced. When the designated application is installed in the receiver, the designated application is activated to display / play the acquired content. If it is not installed, it automatically installs, displays a message prompting installation, displays a download screen, and displays and plays the acquired content after installation.

  The receiver may acquire only the application ID from the server and start the designated application.

  The receiver may perform specified settings. The receiver may set the designated parameter and activate the designated application.

  Next, selection between streaming reception and normal reception will be described.

  FIG. 304 is a diagram illustrating an example of operation of a receiver in this embodiment.

  The receiver determines that the signal is being streamed when the value of the predetermined address of the received data is a predetermined value or the received data includes a predetermined identifier, and receives the streaming data. Receive with. Otherwise, it is received by a normal receiving method.

  Thus, reception can be performed regardless of whether the signal is transmitted by the streaming distribution method or the normal distribution method.

  Next, private data will be described.

  FIG. 305 is a diagram illustrating an example of operation of a receiver in this embodiment.

  When the received ID value is within a predetermined range or includes a predetermined identifier, the receiver refers to the table in the application, and if the reception ID exists in the table, the receiver is designated in the table. Get the content. Otherwise, the content set as the reception ID is acquired from the server.

  As a result, content can be received without registering with the server. Further, since communication with the server is not performed, a quick response can be obtained.

  Next, the setting of the exposure time according to the frequency will be described.

  FIG. 306 is a diagram illustrating an example of operation of a receiver in this embodiment.

  The receiver detects the signal and recognizes the modulation frequency of the signal. The receiver sets the exposure time according to the period of the modulation frequency (modulation period). For example, the signal can be easily received by setting the exposure time to be approximately equal to the modulation period. Further, for example, by setting the exposure time to an integral multiple of the modulation period or a value close to it (approximately ± 30%), it is possible to easily receive a signal by convolutional decoding.

  Next, the optimum parameter setting of the transmitter will be described.

  FIG. 307 is a diagram illustrating an example of operation of a receiver in this embodiment.

  In addition to the data received from the transmitter, the receiver transmits current location information and information related to the user (address, sex, age, preference, etc.) to the server. The server transmits parameters for optimal operation of the transmitter to the receiver in accordance with the received information. The receiver sets the received parameter if it can be set in the transmitter. If it cannot be set, the parameter is displayed and the user is prompted to set the parameter.

  This allows, for example, operating the washing machine to optimize the water properties of the area where the transmitter is used, or operating the rice cooker to cook rice in a way that is optimal for the type of rice used by the user You can make it.

  Next, an identifier indicating the data structure will be described.

  FIG. 308 is a diagram for describing an example of the structure of transmission data in this embodiment.

  The transmitted information includes an identifier, and the receiver can know the configuration of the subsequent part by its value. For example, it is possible to specify the data length, the type and length of the error correction code, the data division point, and the like.

  Thereby, the transmitter can change the type and length of the data body and the error correction code according to the nature of the transmitter and the communication path. Also, the transmitter can cause the receiver to acquire an ID corresponding to the content ID by transmitting the content ID in addition to the ID of the transmitter.

  INDUSTRIAL APPLICABILITY The present invention can be used for information communication devices and the like, and in particular, portable terminals such as smartphones, tablets, mobile phones, smart watches, and head mounted displays, and home appliances such as air conditioners, lighting devices, rice cookers, TVs, recorders, and projectors. It can be used for an information communication device or the like used for a communication method.

A10 Information processing device A11 Frequency determination unit A12 Output unit A20 Receiver A21 Exposure time setting unit A22 Imaging unit A23 Decoding unit A30 Receiver A31 Multiple exposure time setting unit A32 Imaging unit A33 Decoding unit A40 Information processing device A41 Coding unit A42 Division Part A43 Output part

Claims (9)

A receiving program for receiving information from a light emitter,
An exposure time setting step for setting the exposure time of the image sensor using automatic exposure;
By causing the image sensor to take an image of the luminous body that changes in luminance for the set exposure time, a bright line image that is an image including a bright line corresponding to each of a plurality of exposure lines included in the image sensor is acquired. Bright line image acquisition step;
Causing the computer to execute an information acquisition step of acquiring information by decoding the plurality of bright line patterns included in the acquired bright line image;
In the exposure time setting step,
A receiving program for setting the sensitivity of the image sensor to a maximum value in a predetermined range for the image sensor, and setting the exposure time according to the sensitivity of the maximum value by the automatic exposure. In the exposure time setting step,
A value indicating the exposure correction level of the image sensor is set to a minimum value in a range set in advance for the image sensor, and the sensitivity corresponding to the maximum value and the exposure correction corresponding to the minimum value level are set. The receiving program according to claim 1, wherein the exposure time is set by the automatic exposure. In the exposure time setting step,
Identifying a brighter part than the other part in the first image obtained by imaging the subject including the light emitter by the image sensor;
Enlarging a part of the subject corresponding to the bright part by optical zoom,
By using the second image acquired by imaging the part of the enlarged subject by the image sensor for the input of the automatic exposure, the exposure time is set,
In the bright line image acquisition step,
The reception program according to claim 1, wherein the bright line image is acquired by causing the image sensor to capture an image of a part of the enlarged subject with the set exposure time. In the exposure time setting step,
Determining whether a central portion of a first image obtained by imaging by the image sensor of a subject including the light emitter is brighter than an average brightness of a plurality of positions in the first image;
When it is determined that the central portion is bright, a part of the subject corresponding to the central portion is enlarged by optical zoom,
By using the second image acquired by imaging the part of the enlarged subject by the image sensor for the input of the automatic exposure, the exposure time is set,
In the bright line image acquisition step,
The reception program according to claim 1, wherein the bright line image is acquired by causing the image sensor to capture an image of a part of the enlarged subject with the set exposure time. In the exposure time setting step,
Of the K (K is an integer of 3 or more) image sensors included in the image sensor, N (N is an integer of 2 or more) N elements that are uniformly distributed in the image sensor. In the first image acquired by imaging the subject including the luminous body only, a brighter part than the other part is identified,
By using, as an input for the automatic exposure, a second image acquired by imaging with only N dense image sensors corresponding to the bright part among the K image sensors included in the image sensor, Set the exposure time,
In the bright line image acquisition step,
The reception program according to claim 1, wherein the bright line image is acquired by causing only the dense N imaging elements included in the image sensor to capture an image with the set exposure time. In the exposure time setting step,
The image sensor sets a photometric position in an image acquired by imaging a subject including the light emitter, and sets the exposure time according to the brightness of the set photometric position by the automatic exposure. Item 6. The receiving program according to any one of Items 1 to 5. The reception program further includes:
Causing the computer to execute an imaging mode setting step of switching the imaging mode of the image sensor from a color imaging mode for acquiring a color image by shooting to a monochrome imaging mode for acquiring a monochrome image by shooting;
In the exposure time setting step,
The reception program according to any one of claims 1 to 6, wherein the exposure time is set by using an image acquired in the monochrome imaging mode for input of the automatic exposure. In the exposure time setting step,
Each time an image is acquired by imaging the illuminant by the image sensor, the exposure time of the image sensor is updated by using the acquired image for the input of the automatic exposure,
8. The exposure time is set by ending updating of the exposure time by the automatic exposure when a fluctuation range of the exposure time updated as needed falls below a predetermined range. The receiving program according to item 1. A receiving device for receiving information from a light emitter,
An exposure time setting unit for setting the exposure time of the image sensor using automatic exposure;
By causing the image sensor to take an image of the luminous body that changes in luminance for the set exposure time, a bright line image that is an image including a bright line corresponding to each of a plurality of exposure lines included in the image sensor is acquired. Bright line image acquisition unit,
An information acquisition unit that acquires information by decoding the plurality of bright line patterns included in the acquired bright line image;
The exposure time setting unit
A receiving device that sets the sensitivity of the image sensor to a maximum value in a predetermined range for the image sensor, and sets the exposure time according to the sensitivity of the maximum value by the automatic exposure.