WO2018088380A1 - Procédé de transmission, dispositif de transmission, et programme - Google Patents

Procédé de transmission, dispositif de transmission, et programme Download PDF

Info

Publication number
WO2018088380A1
WO2018088380A1 PCT/JP2017/040032 JP2017040032W WO2018088380A1 WO 2018088380 A1 WO2018088380 A1 WO 2018088380A1 JP 2017040032 W JP2017040032 W JP 2017040032W WO 2018088380 A1 WO2018088380 A1 WO 2018088380A1
Authority
WO
WIPO (PCT)
Prior art keywords
value
image
signal
receiver
mode
Prior art date
Application number
PCT/JP2017/040032
Other languages
English (en)
Japanese (ja)
Inventor
秀紀 青山
大嶋　光昭
Original Assignee
パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ filed Critical パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ
Priority to CN201780069560.4A priority Critical patent/CN110114988B/zh
Priority to JP2018550203A priority patent/JP7023239B2/ja
Publication of WO2018088380A1 publication Critical patent/WO2018088380A1/fr
Priority to US16/408,537 priority patent/US10819428B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
    • H04B10/11—Arrangements specific to free-space transmission, i.e. transmission through air or vacuum
    • H04B10/114—Indoor or close-range type systems
    • H04B10/116—Visible light communication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
    • H04B10/11—Arrangements specific to free-space transmission, i.e. transmission through air or vacuum
    • H04B10/114—Indoor or close-range type systems
    • H04B10/1141—One-way transmission
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
    • H04B10/11—Arrangements specific to free-space transmission, i.e. transmission through air or vacuum
    • H04B10/114—Indoor or close-range type systems
    • H04B10/1149—Arrangements for indoor wireless networking of information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content

Definitions

  • the present invention relates to a visible light signal transmission method, a transmission device, a program, and the like.
  • Patent Literature 1 in an optical space transmission device that transmits information to free space using light, limited transmission is performed by performing communication using a plurality of monochromatic light sources of illumination light. A technology for efficiently realizing communication between devices is described in the apparatus.
  • the conventional method is limited to a case where a device to be applied has a three-color light source such as illumination.
  • the present invention solves such a problem and provides a transmission method and the like that enables communication between various devices including devices other than lighting having a three-color light source.
  • a transmission method is a transmission method for transmitting a signal according to a change in luminance of a light source, the step of accepting a dimming degree designated for the light source as a designated dimming degree, and the designated dimming degree Is less than or equal to the first value, the signal encoded in the first mode is transmitted by a luminance change while causing the light source to emit light at the specified dimming level, and the specified dimming level is the first dimming level.
  • a transmission step of transmitting the signal encoded in the second mode by a luminance change while causing the light source to emit light at the specified dimming level, and the specified dimming level is
  • the value of the peak current of the light source for transmitting the signal encoded in the second mode by a change in luminance when it is greater than the first value and less than or equal to the second value is the specified dimming degree Is the first If it is smaller than the value of the peak current of the light source for transmitting the first said signal encoded in the mode of the luminance change.
  • the present invention it is possible to realize a transmission method that enables communication between devices including a device other than a lighting device having a three-color light source.
  • FIG. 1 is a diagram illustrating an example of an observation method of luminance of a light emitting unit in the first embodiment.
  • FIG. 2 is a diagram illustrating an example of a method of observing the luminance of the light emitting unit in the first embodiment.
  • FIG. 3 is a diagram illustrating an example of a method of observing the luminance of the light emitting unit in the first embodiment.
  • FIG. 4 is a diagram illustrating an example of a method of observing the luminance of the light emitting unit in the first embodiment.
  • FIG. 5A is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1.
  • FIG. 5B is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1.
  • FIG. 5A is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1.
  • FIG. 5B is a diagram illustrating an example of an observation method of lumina
  • FIG. 5C is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1.
  • FIG. 5D is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1.
  • FIG. 5E is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1.
  • FIG. 5F is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1.
  • FIG. 5G is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1.
  • FIG. 5H is a diagram illustrating an example of an observation method of luminance of a light emitting unit in Embodiment 1.
  • FIG. 6A is a flowchart of the information communication method in Embodiment 1.
  • FIG. 6B is a block diagram of the information communication apparatus according to Embodiment 1.
  • FIG. 7 is a diagram illustrating an example of a photographing operation of the receiver in the second embodiment.
  • FIG. 8 is a diagram illustrating another example of the photographing operation of the receiver in the second embodiment.
  • FIG. 9 is a diagram illustrating another example of the photographing operation of the receiver in the second embodiment.
  • FIG. 10 is a diagram illustrating an example of display operation of the receiver in Embodiment 2.
  • FIG. 11 is a diagram illustrating an example of display operation of the receiver in Embodiment 2.
  • FIG. 12 is a diagram illustrating an example of operation of a receiver in Embodiment 2.
  • FIG. 12 is a diagram illustrating an example of operation of a receiver in Embodiment 2.
  • FIG. 13 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 14 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 15 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 16 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 17 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 18 is a diagram illustrating an example of operations of the receiver, the transmitter, and the server in the second embodiment.
  • FIG. 19 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 20 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 20 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 21 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 22 is a diagram illustrating an example of operation of a transmitter in Embodiment 2.
  • FIG. 23 is a diagram illustrating another example of operation of a transmitter in Embodiment 2.
  • FIG. 24 is a diagram illustrating an example of application of a receiver in Embodiment 2.
  • FIG. 25 is a diagram illustrating another example of operation of a receiver in Embodiment 2.
  • FIG. 26 is a diagram illustrating an example of processing operations of the receiver, the transmitter, and the server in Embodiment 3.
  • FIG. 27 is a diagram illustrating an example of operations of a transmitter and a receiver in Embodiment 3.
  • FIG. 28 is a diagram illustrating an example of operations of a transmitter, a receiver, and a server in Embodiment 3.
  • FIG. 29 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 3.
  • FIG. 30 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 4.
  • FIG. 31 is a diagram illustrating an example of operations of a transmitter and a receiver in Embodiment 4.
  • FIG. 32 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 4.
  • FIG. 33 is a diagram illustrating an example of operations of a transmitter and a receiver in Embodiment 4.
  • FIG. 34 is a diagram illustrating an example of operations of a transmitter and a receiver in Embodiment 4.
  • FIG. 35 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 4.
  • FIG. 36 is a diagram illustrating an example of operations of a transmitter and a receiver in Embodiment 4.
  • FIG. 37 is a diagram for describing notification of visible light communication to a human in the fifth embodiment.
  • FIG. 38 is a diagram for explaining an application example to the route guidance in the fifth embodiment.
  • FIG. 39 is a diagram for explaining an application example to use log accumulation and analysis in the fifth embodiment.
  • FIG. 40 is a diagram for explaining an application example to screen sharing in the fifth embodiment.
  • FIG. 41 is a diagram illustrating an application example of the information communication method according to the fifth embodiment.
  • FIG. 42 is a diagram illustrating an example of application of the transmitter and the receiver in the sixth embodiment.
  • FIG. 43 is a diagram illustrating an example of application of the transmitter and the receiver in the sixth embodiment.
  • FIG. 44 is a diagram illustrating an example of a receiver in Embodiment 7.
  • FIG. 45 is a diagram illustrating an example of a reception system in the seventh embodiment.
  • FIG. 46 is a diagram illustrating an example of a signal transmission / reception system according to the seventh embodiment.
  • FIG. 47 is a flowchart showing a reception method in which interference is eliminated in the seventh embodiment.
  • FIG. 48 is a flowchart showing a method for estimating the orientation of a transmitter in the seventh embodiment.
  • FIG. 49 is a flowchart showing a reception start method according to the seventh embodiment.
  • FIG. 50 is a flowchart showing an ID generation method using information of another medium together in the seventh embodiment.
  • FIG. 51 is a flowchart showing a reception method selection method based on frequency separation in the seventh embodiment.
  • FIG. 52 is a flowchart showing a signal reception method when the exposure time is long in the seventh embodiment.
  • FIG. 53 is a diagram illustrating an example of a transmitter dimming (adjusting brightness) method in Embodiment 7.
  • FIG. 54 is a diagram illustrating an example of a method for configuring a dimming function of a transmitter in the seventh embodiment.
  • FIG. 55 is a diagram for explaining the EX zoom.
  • FIG. 56 is a diagram illustrating an example of a signal reception method in Embodiment 9.
  • FIG. 57 is a diagram illustrating an example of a signal reception method in Embodiment 9. FIG.
  • FIG. 58 is a diagram illustrating an example of a signal reception method in Embodiment 9.
  • FIG. 59 is a diagram illustrating an example of a screen display method of a receiver in Embodiment 9.
  • FIG. 60 is a diagram illustrating an example of a signal reception method in Embodiment 9.
  • FIG. 61 is a diagram illustrating an example of a signal reception method according to the ninth embodiment.
  • FIG. 62 is a flowchart illustrating an example of a signal reception method in the ninth embodiment.
  • FIG. 63 is a diagram illustrating an example of a signal reception method in the ninth embodiment.
  • FIG. 64 is a flowchart showing processing of the reception program in the ninth embodiment.
  • FIG. 65 is a block diagram of a receiving apparatus according to the ninth embodiment.
  • FIG. 66 is a diagram illustrating an example of display on the receiver when a visible light signal is received.
  • FIG. 67 is a diagram illustrating an example of display on the receiver when a visible light signal is received.
  • FIG. 68 is a diagram illustrating an example of the display of the acquired data image.
  • FIG. 69 is a diagram illustrating an example of an operation when saving or discarding acquired data.
  • FIG. 70 is a diagram illustrating a display example when browsing acquired data.
  • 71 is a diagram illustrating an example of a transmitter in Embodiment 9.
  • FIG. 72 is a diagram illustrating an example of a reception method in Embodiment 9.
  • FIG. 73 is a flowchart illustrating an example of a reception method in Embodiment 10.
  • FIG. 71 is a diagram illustrating an example of a transmitter in Embodiment 9.
  • FIG. 72 is a diagram illustrating an example of a reception method in Embodiment 9.
  • FIG. 73 is a flowchar
  • FIG. 74 is a flowchart illustrating an example of a reception method in Embodiment 10.
  • FIG. 75 is a flowchart illustrating an example of a reception method in Embodiment 10.
  • FIG. 76 is a diagram for describing a reception method in which the receiver according to Embodiment 10 uses an exposure time longer than the period of the modulation frequency (modulation period).
  • FIG. 77 is a diagram for describing a reception method in which the receiver according to Embodiment 10 uses an exposure time longer than the period of the modulation frequency (modulation period).
  • FIG. 78 is a diagram showing an efficient division number with respect to the size of transmission data in the tenth embodiment.
  • FIG. 79A is a diagram illustrating an example of a setting method in Embodiment 10.
  • FIG. 79A is a diagram illustrating an example of a setting method in Embodiment 10.
  • FIG. 79B is a diagram illustrating another example of the setting method according to the tenth embodiment.
  • FIG. 80 is a flowchart showing processing of the information processing program in the tenth embodiment.
  • FIG. 81 is a diagram for describing an application example of the transmission and reception system in the tenth embodiment.
  • FIG. 82 is a flowchart showing processing operations of the transmission / reception system in the tenth embodiment.
  • FIG. 83 is a diagram for describing an example of application of the transmission and reception system in the tenth embodiment.
  • FIG. 84 is a flowchart showing processing operations of the transmission / reception system in the tenth embodiment.
  • FIG. 85 is a diagram for describing an application example of the transmission and reception system in the tenth embodiment.
  • FIG. 86 is a flowchart showing processing operations of the transmission / reception system in the tenth embodiment.
  • FIG. 87 is a diagram for describing an example of application of a transmitter in Embodiment 10.
  • FIG. 88 is a diagram for describing an application example of the transmission and reception system in the eleventh embodiment.
  • FIG. 89 is a diagram for explaining an application example of the transmission and reception system in the eleventh embodiment.
  • 90 is a diagram for describing an example of application of a transmission and reception system in Embodiment 11.
  • FIG. FIG. 91 is a diagram for describing an application example of the transmission and reception system in the eleventh embodiment.
  • FIG. 92 is a diagram for describing an application example of the transmission and reception system in the eleventh embodiment.
  • FIG. 93 is a diagram for describing an application example of the transmission / reception system in Embodiment 11.
  • FIG. 94 is a diagram for describing an application example of the transmission and reception system in the eleventh embodiment.
  • FIG. 95 is a diagram for describing an application example of the transmission / reception system in Embodiment 11.
  • FIG. 96 is a diagram for describing an application example of the transmission / reception system in Embodiment 11.
  • FIG. FIG. 97 is a diagram for describing an application example of the transmission / reception system in Embodiment 11.
  • FIG. FIG. 98 is a diagram for describing an application example of the transmission / reception system in Embodiment 11.
  • FIG. 99 is a diagram for describing an application example of the transmission and reception system in Embodiment 11.
  • FIG. 100 is a diagram for describing an example of application of the transmission and reception system in Embodiment 11.
  • FIG. 101 is a diagram for explaining an application example of the transmission and reception system in the eleventh embodiment.
  • FIG. 102 is a diagram for describing operation of a receiver in Embodiment 12.
  • 103A is a diagram for describing another operation of the receiver in Embodiment 12.
  • FIG. FIG. 103B is a diagram illustrating an example of an indicator displayed by the output unit 1215 in the twelfth embodiment.
  • FIG. 103C is a diagram illustrating a display example of AR in the twelfth embodiment.
  • FIG. 104A is a diagram for describing an example of a transmitter in Embodiment 12.
  • FIG. FIG. 104B is a diagram for describing another example of the transmitter in Embodiment 12.
  • 105A is a diagram for describing an example of synchronous transmission by a plurality of transmitters in Embodiment 12.
  • FIG. 105B is a diagram for describing another example of synchronous transmission by a plurality of transmitters in Embodiment 12.
  • FIG. FIG. 106 is a diagram for describing another example of synchronous transmission by a plurality of transmitters in Embodiment 12.
  • FIG. 107 is a diagram for describing signal processing by a transmitter in Embodiment 12.
  • FIG. 108 is a flowchart illustrating an example of a reception method in Embodiment 12.
  • FIG. 110 is a flowchart illustrating another example of a reception method in Embodiment 12.
  • 111 is a diagram illustrating an example of a transmission signal in Embodiment 13.
  • FIG. 112 is a diagram illustrating another example of a transmission signal in Embodiment 13.
  • FIG. 113 is a diagram illustrating another example of a transmission signal in Embodiment 13.
  • 114A is a diagram for illustrating a transmitter in Embodiment 14.
  • FIG. FIG. 114B is a diagram showing each luminance change of RGB in the fourteenth embodiment.
  • FIG. 115 is a diagram illustrating afterglow characteristics of the green fluorescent component and the red fluorescent component in the fourteenth embodiment.
  • FIG. 110 is a flowchart illustrating another example of a reception method in Embodiment 12.
  • 111 is a diagram illustrating an example of a transmission signal in Embodiment 13.
  • FIG. 112 is a diagram illustrating another example of a transmission signal in Embodiment 13.
  • FIG. 116 is a diagram for describing a problem newly generated in order to suppress occurrence of a barcode reading error in the fourteenth embodiment.
  • 117 is a diagram for explaining downsampling performed by a receiver in Embodiment 14.
  • FIG. FIG. 118 is a flowchart illustrating a processing operation of the receiver in Embodiment 14.
  • FIG. 119 is a diagram illustrating processing operations of the reception device (imaging device) in Embodiment 15.
  • FIG. 120 is a diagram illustrating processing operation of a reception device (imaging device) in Embodiment 15.
  • FIG. 121 is a diagram illustrating processing operation of a reception device (imaging device) in Embodiment 15.
  • FIG. 122 is a diagram illustrating processing operation of the reception device (imaging device) in Embodiment 15.
  • FIG. 123 is a diagram illustrating an example of an application according to the sixteenth embodiment.
  • FIG. 124 is a diagram illustrating an example of an application according to the sixteenth embodiment.
  • FIG. 125 is a diagram illustrating an example of the transmission signal and an example of the audio synchronization method in Embodiment 16.
  • 126 is a diagram illustrating an example of a transmission signal in Embodiment 16.
  • FIG. 127 is a diagram illustrating an example of processing flow of a receiver in Embodiment 16.
  • FIG. 128 is a diagram illustrating an example of a user interface of the receiver in Embodiment 16.
  • 129 is a diagram illustrating an example of a process flow of a receiver in Embodiment 16.
  • FIG. 130 is a diagram illustrating another example of processing flow of a receiver in Embodiment 16.
  • FIG. 131A is a diagram for explaining a specific method of synchronized playback in the sixteenth embodiment.
  • FIG. 131B is a block diagram showing a configuration of a playback device (receiver) that performs synchronized playback in the sixteenth embodiment.
  • FIG. 131C is a flowchart illustrating a processing operation of a playback device (receiver) that performs synchronized playback in the sixteenth embodiment.
  • FIG. 132 is a diagram for describing preparation for synchronized playback in the sixteenth embodiment.
  • 133 is a diagram illustrating an example of application of a receiver in Embodiment 16.
  • FIG. 134A is a front view of a receiver held by a holder in Embodiment 16.
  • FIG. 134B is a rear view of a receiver held by a holder in Embodiment 16.
  • FIG. 135 is a diagram for describing a use case of a receiver held by a holder in Embodiment 16.
  • 136 is a flowchart illustrating processing operation of a receiver held by a holder in Embodiment 16.
  • FIG. FIG. 137 is a diagram illustrating an example of an image displayed by the receiver in Embodiment 16.
  • FIG. FIG. 138 is a diagram showing another example of the holder according to the sixteenth embodiment.
  • 139A is a diagram illustrating an example of a visible light signal in Embodiment 17.
  • FIG. 140 is a diagram illustrating a configuration of a visible light signal according to the seventeenth embodiment.
  • FIG. 141 is a diagram illustrating an example of bright line images obtained by imaging of the receiver in Embodiment 17.
  • FIG. 142 is a diagram illustrating another example of bright line images obtained by imaging by the receiver in Embodiment 17.
  • FIG. 143 is a diagram illustrating another example of the bright line image obtained by imaging by the receiver in Embodiment 17.
  • FIG. 144 is a diagram for describing adaptation of the receiver in Embodiment 17 to a camera system that performs HDR synthesis.
  • FIG. FIG. 145 is a diagram for explaining the processing operation of the visible light communication system in the seventeenth embodiment.
  • 146A is a diagram illustrating an example of vehicle-to-vehicle communication using visible light in Embodiment 17.
  • FIG. 146B is a diagram illustrating another example of vehicle-to-vehicle communication using visible light in Embodiment 17.
  • FIG. 147 is a diagram illustrating an example of a method for determining the positions of a plurality of LEDs in Embodiment 17.
  • FIG. FIG. 148 is a diagram illustrating an example of bright line images obtained by capturing an image of the vehicle in the seventeenth embodiment.
  • FIG. 149 is a diagram illustrating an example of application of the receiver and the transmitter in Embodiment 17.
  • FIG. 149 is a view of the automobile from the back.
  • FIG. 150 is a flowchart illustrating an example of processing operations of a receiver and a transmitter in Embodiment 17.
  • FIG. 151 is a diagram illustrating an example of application of the receiver and the transmitter in Embodiment 17.
  • FIG. 152 is a flowchart illustrating an example of processing operations of the receiver 7007a and the transmitter 7007b in Embodiment 17.
  • FIG. 153 is a diagram illustrating a configuration of a visible light communication system applied to the inside of a train in Embodiment 17.
  • FIG. 154 is a diagram illustrating a configuration of a visible light communication system applied to a facility such as an amusement park in Embodiment 17.
  • FIG. 155 is a diagram illustrating an example of a visible light communication system including a playground device and a smartphone according to Embodiment 17.
  • 156 is a diagram illustrating an example of a transmission signal in Embodiment 18.
  • FIG. 157 is a diagram illustrating an example of a transmission signal in Embodiment 18.
  • FIG. 158 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 159 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 160 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 161 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 162 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 163 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 164 is a diagram illustrating an example of a transmission and reception system in Embodiment 19.
  • FIG. 165 is a flowchart illustrating an example of processing of the transmission / reception system in the nineteenth embodiment.
  • FIG. 166 is a flowchart showing the operation of the server in the nineteenth embodiment.
  • FIG. 167 is a flowchart illustrating an example of operation of a receiver in Embodiment 19.
  • FIG. 168 is a flowchart illustrating a method of calculating the progress status in the simple mode according to the nineteenth embodiment.
  • FIG. 169 is a flowchart illustrating a method for calculating the progress in the maximum likelihood estimation mode according to the nineteenth embodiment.
  • FIG. 170 is a flowchart showing a display method in which the progress status does not decrease in the nineteenth embodiment.
  • FIG. 171 is a flowchart illustrating a progress status display method when there are a plurality of packet lengths according to the nineteenth embodiment.
  • 172 is a diagram illustrating an example of an operation state of a receiver in Embodiment 19.
  • FIG. 173 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 174 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 175 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 176 is a block diagram illustrating an example of a transmitter in Embodiment 19.
  • FIG. 177 is a timing chart when the LED display in Embodiment 19 is driven with the optical ID modulation signal of the present invention.
  • FIG. 178 is a timing chart when the LED display in Embodiment 19 is driven with the optical ID modulation signal of the present invention.
  • FIG. 179 is a timing chart when the LED display in Embodiment 19 is driven with the optical ID modulation signal of the present invention.
  • FIG. 179 is a timing chart when the LED display in Embodiment 19 is driven with the optical ID modulation signal of the present invention.
  • FIG. 180A is a flowchart illustrating a transmission method according to one embodiment of the present invention.
  • FIG. 180B is a block diagram illustrating a functional configuration of the transmission device according to one embodiment of the present invention.
  • FIG. 181 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 182 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 183 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 184 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 185 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 186 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 19 is a diagram illustrating an example of a transmission signal in Embodiment 19.
  • FIG. 187 is a diagram illustrating an example of a structure of a visible light signal in Embodiment 20.
  • FIG. 188 is a diagram illustrating an example of a detailed configuration of a visible light signal in Embodiment 20.
  • FIG. FIG. 189A is a diagram illustrating another example of a visible light signal in Embodiment 20.
  • FIG. 189B is a diagram illustrating another example of a visible light signal in Embodiment 20.
  • FIG. 189C is a diagram illustrating the signal length of a visible light signal in Embodiment 20.
  • FIG. FIG. 190 is a diagram illustrating a comparison result of luminance values between the visible light signal and the standard IEC visible light signal according to the twentieth embodiment.
  • FIG. 191 is a diagram illustrating a comparison result of the number of received packets and the reliability with respect to the angle of view between the visible light signal and the standard IEC visible light signal according to the twentieth embodiment.
  • FIG. 192 is a diagram illustrating comparison results of the number of received packets and reliability with respect to noise between the visible light signal and the standard IEC visible light signal according to the twentieth embodiment.
  • FIG. 193 is a diagram illustrating a comparison result of the number of received packets and the reliability with respect to the reception-side clock error between the visible light signal and the standard IEC visible light signal according to the twentieth embodiment.
  • FIG. 194 is a diagram illustrating a structure of a transmission target signal in the twentieth embodiment.
  • 195A is a diagram illustrating a visible light signal receiving method in Embodiment 20.
  • FIG. FIG. 195B is a diagram illustrating rearrangement of visible light signals in the twentieth embodiment.
  • 196 is a diagram illustrating another example of a visible light signal in Embodiment 20.
  • FIG. FIG. 197 is a diagram illustrating another example of a detailed configuration of a visible light signal in the twentieth embodiment.
  • FIG. 198 is a diagram illustrating another example of a detailed configuration of a visible light signal in the twentieth embodiment.
  • FIG. 199 is a diagram illustrating another example of a detailed configuration of a visible light signal in Embodiment 20.
  • FIG. 200 is a diagram illustrating another example of a detailed configuration of a visible light signal in Embodiment 20.
  • FIG. 201 is a diagram illustrating another example of a detailed configuration of a visible light signal according to the twentieth embodiment.
  • FIG. 202 is a diagram illustrating another example of a detailed configuration of a visible light signal in Embodiment 20.
  • FIG. 203 is a diagram for explaining a method of determining the values of x1 to x4 in FIG. 197.
  • FIG. 204 is a diagram for explaining a method of determining the values of x1 to x4 in FIG. 197.
  • FIG. 205 is a diagram for explaining a method of determining the values of x1 to x4 in FIG. 197.
  • FIG. 206 is a diagram for explaining a method of determining the values of x1 to x4 in FIG. FIG.
  • FIG. 207 is a diagram for explaining a method of determining the values of x1 to x4 in FIG. 197.
  • FIG. 208 is a diagram for explaining a method of determining the values of x1 to x4 in FIG. 197.
  • FIG. 209 is a diagram for explaining a method of determining the values of x1 to x4 in FIG.
  • FIG. 210 is a diagram for explaining a method of determining the values of x1 to x4 in FIG.
  • FIG. 211 is a diagram for explaining a method of determining the values of x1 to x4 in FIG. 197.
  • FIG. 212 is a diagram illustrating an example of a detailed configuration of a visible light signal according to the first modification of the twentieth embodiment.
  • FIG. 213 is a diagram illustrating another example of a visible light signal according to Modification 1 of Embodiment 20.
  • FIG. 214 is a diagram showing still another example of a visible light signal according to Modification 1 of Embodiment 20.
  • FIG. 215 is a diagram illustrating an example of packet modulation according to Modification 1 of Embodiment 20.
  • FIG. FIG. 216 is a diagram illustrating processing for dividing the original data into one according to the first modification of the twentieth embodiment.
  • FIG. 217 is a diagram illustrating a process of dividing the original data into two according to the first modification of the twentieth embodiment.
  • FIG. 218 is a diagram illustrating processing of dividing original data into three according to Modification 1 of Embodiment 20.
  • FIG. 219 is a diagram illustrating another example of the process of dividing the original data into three according to the first modification of the twentieth embodiment.
  • FIG. 220 is a diagram illustrating another example of the process of dividing the original data into three according to the first modification of the twentieth embodiment.
  • FIG. 221 is a diagram illustrating a process of dividing the original data into four according to the first modification of the twentieth embodiment.
  • FIG. 222 is a diagram showing processing for dividing original data into five parts according to Modification 1 of Embodiment 20.
  • 223 is a diagram illustrating processing of dividing original data into 6, 7, or 8 portions according to Modification Example 1 of Embodiment 20.
  • FIG. 224 is a diagram illustrating another example of the process of dividing the original data into 6, 7 or 8 according to the first modification of the twentieth embodiment.
  • FIG. 225 is a diagram illustrating a process of dividing the original data into nine according to the first modification of the twentieth embodiment.
  • FIG. 226 is a diagram illustrating processing of dividing original data into any number from 10 to 16 according to Modification 1 of Embodiment 20.
  • FIG. 227 is a diagram illustrating an example of a relationship among the number of original data divisions, a data size, and an error correction code according to the first modification of the twentieth embodiment.
  • FIG. 228 is a diagram illustrating another example of the relationship between the number of original data divisions, the data size, and the error correction code according to the first modification of the twentieth embodiment.
  • FIG. 229 is a diagram illustrating still another example of the relationship among the number of original data divisions, the data size, and the error correction code according to Modification 1 of Embodiment 20.
  • FIG. 230A is a flowchart illustrating a visible light signal generation method according to Embodiment 20.
  • FIG. FIG. 230B is a block diagram illustrating a configuration of the signal generation device according to Embodiment 20.
  • FIG. 231 is a diagram illustrating a method of receiving a high-frequency visible light signal in Embodiment 21.
  • 232A is a diagram illustrating another method of receiving a high-frequency visible light signal in Embodiment 21.
  • FIG. FIG. 232B is a diagram illustrating another method of receiving a high-frequency visible light signal in Embodiment 21.
  • FIG. 233 is a diagram illustrating a method of outputting a high-frequency signal in Embodiment 21.
  • FIG. FIG. 234 is a diagram for describing the autonomous flight apparatus according to the twenty-second embodiment.
  • FIG. 235 is a diagram illustrating an example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 236 is a diagram illustrating an example of a display system in Embodiment 23.
  • FIG. 237 is a diagram illustrating another example of the display system in Embodiment 23.
  • FIG. FIG. 238 is a diagram illustrating another example of the display system in Embodiment 23.
  • FIG. 239 is a flowchart illustrating an example of process operations of a receiver in Embodiment 23.
  • FIG. 240 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 241 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 242 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 243 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 244 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 245 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 246 is a flowchart illustrating another example of processing operations of a receiver in Embodiment 23.
  • FIG. 241 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 242 is a diagram illustrating another example in which the receiver in Embodiment
  • FIG. 247 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • 248 is a diagram illustrating a captured display image Ppre and a decoding image Pdec acquired by capturing by the receiver in Embodiment 23.
  • FIG. FIG. 249 is a diagram illustrating an example of a captured display image Ppre displayed on the receiver in Embodiment 23.
  • FIG. 250 is a flowchart illustrating another example of processing operations of a receiver in Embodiment 23.
  • FIG. 251 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 252 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 253 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 254 is a diagram illustrating another example in which the receiver in Embodiment 23 displays an AR image.
  • FIG. 255 is a diagram illustrating an example of recognition information according to the twenty-third embodiment.
  • FIG. 256 is a flowchart illustrating another example of processing operations of a receiver in Embodiment 23.
  • FIG. 257 is a diagram illustrating an example in which the receiver in Embodiment 23 identifies bright line pattern regions.
  • 258 is a diagram illustrating another example of a receiver in Embodiment 23.
  • FIG. 259 is a flowchart illustrating another example of processing operations of a receiver in Embodiment 23.
  • FIG. 260 is a diagram illustrating an example of a transmission system including a plurality of transmitters in Embodiment 23.
  • FIG. FIG. 261 is a diagram illustrating an example of a transmission system including a plurality of transmitters and receivers in Embodiment 23.
  • FIG. 262A is a flowchart illustrating an example of process operations of the receiver in Embodiment 23.
  • FIG. 262B is a flowchart illustrating an example of process operations of the receiver in Embodiment 23.
  • FIG. 263A is a flowchart illustrating a display method according to Embodiment 23.
  • FIG. 263B is a block diagram illustrating a structure of the display device in Embodiment 23.
  • FIG. 264 is a diagram illustrating an example in which the receiver in Modification 1 of Embodiment 23 displays an AR image.
  • FIG. 265 is a diagram illustrating another example in which the receiver 200 in the first modification of the twenty-third embodiment displays an AR image.
  • FIG. 266 is a diagram illustrating another example in which the receiver 200 in the first modification of the twenty-third embodiment displays an AR image.
  • FIG. 267 is a diagram illustrating another example in which the receiver 200 in the first modification of the twenty-third embodiment displays an AR image.
  • FIG. 268 is a diagram illustrating another example of the receiver 200 in the first modification of the twenty-third embodiment.
  • FIG. 269 is a diagram illustrating another example in which the receiver 200 in the first modification of the twenty-third embodiment displays an AR image.
  • FIG. 270 is a diagram illustrating another example in which the receiver 200 in the first modification of the twenty-third embodiment displays an AR image.
  • FIG. 271 is a flowchart illustrating an example of processing operations of the receiver 200 in the first modification of the twenty-third embodiment.
  • FIG. 272 is a diagram illustrating an example of a problem when an AR image assumed in the receiver in Embodiment 23 or the modification 1 thereof is displayed.
  • FIG. 273 is a diagram illustrating an example in which the receiver in Modification 2 of Embodiment 23 displays the AR image.
  • FIG. 274 is a flowchart illustrating an example of processing operations of a receiver in Modification 2 of Embodiment 23.
  • FIG. 275 is a diagram illustrating another example in which the receiver in the second modification of the twenty-third embodiment displays an AR image.
  • FIG. 276 is a flowchart illustrating another example of processing operations of a receiver in Modification 2 of Embodiment 23.
  • 277 is a diagram illustrating another example in which a receiver in Modification 2 of Embodiment 23 displays an AR image.
  • FIG. 278 is a diagram illustrating another example in which the receiver in the second modification of the twenty-third embodiment displays an AR image.
  • FIG. 279 is a diagram illustrating another example in which the receiver in Modification 2 of Embodiment 23 displays an AR image.
  • 280 is a diagram illustrating another example in which a receiver in Modification 2 of Embodiment 23 displays an AR image.
  • FIG. 281A is a flowchart illustrating a display method according to one embodiment of the present invention.
  • FIG. 281B is a block diagram illustrating a structure of a display device according to one embodiment of the present invention.
  • FIG. 282 is a diagram illustrating an example of expansion and movement of the AR image in the third modification of the twenty-third embodiment.
  • FIG. 283 is a diagram illustrating an example of expansion of an AR image in the third modification of the twenty-third embodiment.
  • FIG. 284 is a flowchart illustrating an example of processing operations regarding expansion and movement of an AR image by a receiver in Modification 3 of Embodiment 23.
  • FIG. 285 is a diagram illustrating an example of superimposition of AR images in the third modification of the twenty-third embodiment.
  • FIG. 286 is a diagram illustrating an example of superimposition of AR images in the third modification of the twenty-third embodiment.
  • FIG. 287 is a diagram illustrating an example of superimposition of AR images in the third modification of the twenty-third embodiment.
  • FIG. 288 is a diagram illustrating an example of superimposition of AR images in the third modification of the twenty-third embodiment.
  • FIG. 289A is a diagram illustrating an example of a captured display image obtained by imaging by the receiver in the third modification of the twenty-third embodiment.
  • FIG. 289B is a diagram illustrating an example of a menu screen displayed on the display of the receiver in Modification 3 of Embodiment 23.
  • FIG. 290 is a flowchart illustrating an example of processing operations of the receiver and the server in the third modification of the twenty-third embodiment.
  • FIG. 291 is a diagram for describing the volume of audio reproduced by the receiver in the third modification of the twenty-third embodiment.
  • FIG. 292 is a diagram illustrating a relationship between the distance from the receiver to the transmitter and the sound volume in the third modification of the twenty-third embodiment.
  • 293 is a diagram illustrating an example of superimposition of AR images by a receiver in Modification 3 of Embodiment 23.
  • FIG. 294 is a diagram illustrating an example of superimposition of AR images by a receiver in Modification 3 of Embodiment 23.
  • FIG. 295 is a diagram for describing an example of how to obtain a line scan time by a receiver in Modification 3 of Embodiment 23.
  • FIG. 296 is a diagram for describing an example of how to obtain a line scan time by a receiver in Modification 3 of Embodiment 23.
  • FIG. 297 is a flowchart illustrating an example of how to obtain a line scan time by a receiver in the third modification of the twenty-third embodiment.
  • 298 is a diagram illustrating an example of superimposition of AR images by a receiver in Modification 3 of Embodiment 23.
  • FIG. 299 is a diagram illustrating an example of superimposition of AR images by a receiver in Modification 3 of Embodiment 23.
  • FIG. 300 is a diagram illustrating an example of superimposition of AR images by a receiver in Modification 3 of Embodiment 23.
  • FIG. 301 is a diagram illustrating an example of a decoding image obtained according to the attitude of the receiver in Modification 3 of Embodiment 23.
  • FIG. 302 is a diagram illustrating another example of a decoding image acquired in accordance with the attitude of a receiver in Modification 3 of Embodiment 23.
  • FIG. FIG. 303 is a flowchart illustrating an example of processing operation of a receiver in Modification 3 of Embodiment 23.
  • 304 is a diagram illustrating an example of camera lens switching processing by a receiver in Modification 3 of Embodiment 23.
  • FIG. 301 is a diagram illustrating an example of a decoding image obtained according to the attitude of the receiver in Modification 3 of Embodiment 23.
  • FIG. 302 is a diagram illustrating another example of a decoding image acquired in accordance with the attitude of a receiver in Mod
  • FIG. 305 is a diagram illustrating an example of camera switching processing by a receiver in Modification 3 of Embodiment 23.
  • FIG. FIG. 306 is a flowchart illustrating an example of processing operations of the receiver and the server in Modification 3 of Embodiment 23.
  • 307 is a diagram illustrating an example of superimposition of AR images by a receiver in Modification 3 of Embodiment 23.
  • FIG. FIG. 308 is a sequence diagram illustrating processing operations of a system including a receiver, a microwave oven, a relay server, and an electronic payment server in Modification 3 of Embodiment 23.
  • FIG. 309 is a sequence diagram illustrating processing operations of a system including a POS terminal, a server, a receiver 200, and a microwave oven in Modification 3 of Embodiment 23.
  • FIG. 310 is a diagram illustrating an example of indoor use in Modification 3 of Embodiment 23.
  • FIG. 311 is a diagram illustrating an example of an augmented reality object display in the third modification of the twenty-third embodiment.
  • FIG. 312 is a diagram showing a configuration of a display system in Modification 4 of Embodiment 23.
  • FIG. 313 is a flowchart showing processing operations of the display system in Modification 4 of Embodiment 23.
  • FIG. 314 is a flowchart illustrating a recognition method according to an aspect of the present invention.
  • FIG. 310 is a diagram illustrating an example of indoor use in Modification 3 of Embodiment 23.
  • FIG. 311 is a diagram illustrating an example of an augmented reality object display in the third modification of the twenty-third
  • FIG. 315 is a diagram illustrating an example of operation modes of visible light signals according to the twenty-fourth embodiment.
  • FIG. 316 is a diagram illustrating an example of a PPDU format in mode 1 of the packet PWM according to the twenty-fourth embodiment.
  • FIG. 317 is a diagram illustrating an example of a PPDU format in mode 2 of the packet PWM according to the twenty-fourth embodiment.
  • FIG. 318 is a diagram illustrating an example of a PPDU format in mode 3 of the packet PWM according to the twenty-fourth embodiment.
  • FIG. 319 is a diagram illustrating an example of a pulse width pattern in each SHR of modes 1 to 3 of the packet PWM according to the twenty-fourth embodiment.
  • FIG. 316 is a diagram illustrating an example of a PPDU format in mode 1 of the packet PWM according to the twenty-fourth embodiment.
  • FIG. 317 is a diagram illustrating an example of a PPDU format in mode 2 of the packet PWM according to
  • FIG. 320 is a diagram showing an example of a PPDU format in mode 1 of the packet PPM according to the twenty-fourth embodiment.
  • FIG. 321 is a diagram illustrating an example of a PPDU format in mode 2 of the packet PPM according to the twenty-fourth embodiment.
  • FIG. 322 is a diagram illustrating an example of a PPDU format in mode 3 of the packet PPM according to the twenty-fourth embodiment.
  • FIG. 323 is a diagram illustrating an example of an interval pattern in each SHR of modes 1 to 3 of the packet PPM according to the twenty-fourth embodiment.
  • FIG. 324 is a diagram illustrating an example of 12-bit data included in the PHY payload according to the twenty-fourth embodiment.
  • FIG. 325 is a diagram illustrating processing of storing PHY frames in one packet according to Embodiment 24.
  • FIG. 326 is a diagram illustrating processing of dividing the PHY frame into 2 packets according to Embodiment 24.
  • FIG. 327 is a diagram illustrating processing of dividing a PHY frame into 3 packets according to Embodiment 24.
  • FIG. 328 is a diagram illustrating processing of dividing the PHY frame into 4 packets according to Embodiment 24.
  • FIG. 329 is a diagram illustrating processing of dividing the PHY frame into 5 packets according to Embodiment 24.
  • FIG. 331 is a diagram illustrating processing of dividing the PHY frame into 9 packets according to Embodiment 24.
  • FIG. 333A is a flowchart illustrating a visible light signal generation method according to Embodiment 24.
  • FIG. 333B is a block diagram showing a configuration of the signal generation apparatus according to Embodiment 24.
  • FIG. 334 is a diagram illustrating a format of an MPM MAC frame in the twenty-fifth embodiment.
  • FIG. 335 is a flowchart illustrating processing operations of the encoding device for generating an MPM MAC frame according to the twenty-fifth embodiment.
  • FIG. 336 is a flowchart illustrating processing operation of the decoding device for decoding the MAC frame of MPM in the twenty-fifth embodiment.
  • FIG. 337 is a diagram illustrating MAC PIB attributes according to the twenty-fifth embodiment.
  • FIG. 338 is a diagram for describing an MPM light control method according to the twenty-fifth embodiment.
  • FIG. 339 is a diagram illustrating attributes of a PHY PIB according to the twenty-fifth embodiment.
  • FIG. 340 is a diagram for describing MPM in the twenty-fifth embodiment.
  • FIG. 341 is a diagram illustrating PLCP header subfields according to Embodiment 25.
  • FIG. 342 is a diagram illustrating PLCP center subfields according to the twenty-fifth embodiment.
  • FIG. 343 is a diagram illustrating PLCP footer subfields according to the twenty-fifth embodiment.
  • FIG. 344 is a diagram illustrating a waveform in a PHY PWM mode in the MPM according to the twenty-fifth embodiment.
  • FIG. 345 is a diagram illustrating a PHY PPM mode waveform in the MPM according to the twenty-fifth embodiment.
  • FIG. 346 is a flowchart illustrating an example of the decoding method according to the twenty-fifth embodiment.
  • FIG. 347 is a flowchart illustrating an example of the coding method according to the twenty-fifth embodiment.
  • FIG. 348 is a diagram illustrating an example in which the receiver in Embodiment 26 displays an AR image.
  • FIG. 349 is a diagram illustrating an example of a captured display image on which an AR image is superimposed according to Embodiment 26.
  • FIG. FIG. 350 is a diagram illustrating another example in which the receiver in Embodiment 26 displays an AR image.
  • FIG. 351 is a flowchart illustrating operation of the receiver in Embodiment 26.
  • FIG. 352 is a diagram for describing an operation of a transmitter in Embodiment 26.
  • FIG. 353 is a diagram for describing another operation of the transmitter in Embodiment 26.
  • FIG. 354 is a diagram for describing another operation of the transmitter in Embodiment 26.
  • FIG. 355 is a diagram illustrating a comparative example for describing ease of reception of the optical ID in the twenty-sixth embodiment.
  • FIG. 356A is a flowchart illustrating operation of the transmitter in Embodiment 26.
  • FIG. 356B is a block diagram illustrating a configuration of a transmitter in Embodiment 26.
  • FIG. 357 is a diagram illustrating another example in which the receiver in Embodiment 26 displays an AR image.
  • 358 is a diagram for describing an operation of a transmitter in Embodiment 27.
  • FIG. 359A is a flowchart illustrating a transmission method according to Embodiment 27.
  • FIG. 359B is a block diagram illustrating a structure of a transmitter in Embodiment 27.
  • FIG. 360 is a diagram illustrating an example of a detailed configuration of a visible light signal in Embodiment 27.
  • FIG. 361 is a diagram illustrating another example of a detailed configuration of a visible light signal in Embodiment 27.
  • FIG. 362 is a diagram illustrating another example of a detailed configuration of a visible light signal in Embodiment 27.
  • FIG. 363 is a diagram illustrating another example of a detailed configuration of a visible light signal in Embodiment 27.
  • FIG. 364 is a diagram illustrating a relationship between the total sum of the variables y 0 to y 3 , the total time length, and the effective time length in the twenty-seventh embodiment.
  • FIG. 365A is a flowchart illustrating a transmission method according to Embodiment 27.
  • FIG. 365B is a block diagram illustrating a structure of a transmitter in Embodiment 27.
  • a transmission method is a transmission method for transmitting a signal according to a change in luminance of a light source, wherein a reception step of accepting a dimming degree designated for the light source as a designated dimming degree, and the designated dimming degree Is less than or equal to the first value, the signal encoded in the first mode is transmitted by a luminance change while causing the light source to emit light at the specified dimming level, and the specified dimming level is the first dimming level.
  • a transmission step of transmitting the signal encoded in the second mode by a luminance change while causing the light source to emit light at the specified dimming level, and the specified dimming level is
  • the value of the peak current of the light source for transmitting the signal encoded in the second mode by a change in luminance when it is greater than the first value and less than or equal to the second value is the specified dimming degree Is the first If it is smaller than the value of the peak current of the light source for transmitting the first said signal encoded in the mode of the luminance change.
  • the peak current value of the light source when the designated dimming degree is greater than the first value and equal to or less than the second value by switching the signal encoding mode is It becomes smaller than the value of the peak current of the light source when the luminous intensity is the first value. Therefore, as the designated dimming degree is increased, a large peak current can be suppressed from flowing to the light source. As a result, deterioration of the light source can be suppressed. Further, since deterioration of the light source can be suppressed, communication between various devices can be performed for a long time.
  • the signal encoded in the first mode is transmitted by a luminance change while causing the light source to emit light at the designated dimming degree.
  • the peak current value may be maintained constant with respect to the change in the designated dimming degree, and the third value may be smaller than the first value.
  • the specified dimming level is smaller than the third value, the specified dimming level is decreased by increasing the time during which the light source is turned off as the specified dimming level decreases.
  • the light source may be made to emit light at a luminous intensity, and the peak current value may be maintained at a constant value.
  • the value of the peak current is kept constant, so that the visible light signal (that is, the light ID) that is transmitted by the luminance change is easily received by the receiver. be able to.
  • the time for turning off the light source may be determined so that one cycle obtained by adding the time for transmitting the signal due to luminance change and the time for turning off the light source does not exceed 10 milliseconds.
  • the luminance change of the light source for transmitting the encoded signal may be perceived by the human eye as flickering. Therefore, in the present disclosure, since the time for turning off the light source is determined so that one period does not exceed 10 milliseconds, it is possible to suppress flickering from being recognized by a person.
  • the signal encoded in the first mode is transmitted by a luminance change while causing the light source to emit light at the designated dimming degree.
  • the peak current value is decreased to cause the light source to emit light at the specified dimming level, and the fourth value is smaller than the second value. Also good.
  • the light source can be appropriately emitted with the designated dimming degree.
  • the peak current value of the light source when the designated dimming level is the first value is the same as the peak current value of the light source when the designated dimming level is the maximum value. May be.
  • the maximum value of the specified dimming degree is 100%.
  • the duty ratio of the signal encoded in the second mode may be larger than the duty ratio of the signal encoded in the first mode.
  • the first mode is a mode in which the increase in peak current is increased even if the increase in dimming degree is small
  • the second mode is a mode in which the increase in peak current is suppressed even if the increase in dimming degree is large. Therefore, since the second mode suppresses a large peak current from flowing to the light source, deterioration of the light source can be suppressed. Further, since the first mode causes a large peak current to flow through the light source even when the dimming degree is small, the signal transmitted by the luminance change of the light source can be easily received by the receiver. Therefore, in the present disclosure, it is possible to achieve both suppression of deterioration of the light source and ease of signal reception.
  • the transmission of the signal due to a change in luminance of the light source may be stopped.
  • the brightness of the signal is set using a parameter value for causing the light source to emit light with a dimming degree greater than the specified dimming degree. You may send by change.
  • the pulse width of the current of the light source may be made larger than when the usage time is less than the predetermined time.
  • the pulse width of the current of the light source is increased, so that it is possible to prevent the signal transmitted due to the luminance change of the light source from becoming difficult to be received by the receiver.
  • the transmission method is a transmission method for transmitting a signal according to a change in luminance of a light source, and accepting a dimming degree designated for the light source as a designated dimming degree; A step of transmitting the signal encoded in the first mode or the second mode by a luminance change while causing the light source to emit light at a specified dimming level, and the encoded in the second mode
  • the duty ratio of the signal is larger than the duty ratio of the signal encoded in the first mode, and the designated dimming level is changed from a small value to a large value in the transmission step, the designated dimming level
  • the designated dimming level When is the first value, the mode used for encoding the signal is switched from the first mode to the second mode, and the designated dimming level is small from a large value
  • the designated dimming level is the second value
  • the mode used for encoding the signal is switched from the second mode to the first mode, and the second value is changed. Is smaller than the first value.
  • the designated dimming degree that is, the switching point
  • the duty ratio of the signal encoded in the second mode is larger than the duty ratio of the signal encoded in the first mode. Therefore, similarly to the transmission method according to one embodiment of the present invention, it is possible to suppress a large peak current from flowing to the light source as the designated dimming degree is increased.
  • deterioration of the light source can be suppressed. Further, since deterioration of the light source can be suppressed, communication between various devices can be performed for a long time. Further, when the designated dimming degree is small, the first mode with a small duty ratio is used. Therefore, the above-described peak current can be increased, and a signal that is easily received by the receiver can be transmitted as a visible light signal.
  • the encoded signal in the transmission step, is transmitted by a change in luminance when switching from the first mode to the second mode is performed.
  • Changing the peak current of the light source for changing from the first current value to a second current value smaller than the first current value, and switching from the second mode to the first mode.
  • the peak current is changed from a third current value to a fourth current value that is larger than the third current value, and the first current value is greater than the fourth current value.
  • the second current value is larger than the third current value.
  • a transmission method is a transmission method for transmitting a visible light signal according to a luminance change of a light emitter, and determining a pattern of the luminance change by modulating the signal; Transmitting the visible light signal by changing the luminance of red expressed by the light source included in the light emitter according to the determined pattern, wherein the visible light signal includes data, In the data, a first luminance value and a second luminance value smaller than the first luminance value appear on the time axis, and the first luminance value includes a preamble and a payload. And the length of time during which at least one of the second luminance values continues is not more than a first predetermined value, and in the preamble, that of the first and second luminance values. However, the first and second luminance values appear alternately along the time axis in the payload, and each of the first and second luminance values continues.
  • the time length to be performed is greater than the first predetermined value and is determined according to the signal and a predetermined method.
  • the visible light signal includes one payload having a waveform determined according to the signal to be modulated (that is, the L data portion or the R data portion), and includes two payloads.
  • the visible light signal that is, the packet of the visible light signal can be shortened. That is, a visible light signal can be transmitted in a short time, and communication between various devices can be performed in a short time. As a result, for example, even if the light emission period of red light expressed by the light source included in the light emitter is short, a packet of visible light signals can be transmitted during the light emission period.
  • the first luminance value having a first time length, the second luminance value having a second time length, the first luminance value having a third time length, and a fourth time length When the respective luminance values appear in the order of the second luminance values, and in the transmission step, the sum of the first time length and the third time length is smaller than a second predetermined value
  • the value of the current flowing through the light source is larger than when the sum of the first time length and the third time length is greater than the second predetermined value, and the second predetermined value is , Greater than the first predetermined value.
  • the first luminance value having the first time length D 0 the second luminance value having the second time length D 1 , and the first luminance value having the third time length D 2 are used.
  • the transmission step includes the data, the preamble, and the payload, Data, the preamble, and the payload may be transmitted in this order.
  • the fact that the packet of the visible light signal including the data (that is, invalid data) does not include the L data portion is indicated to the receiving apparatus that receives the packet by the data. I can inform you.
  • each of the first to fourth time lengths D 0 to D 3 (that is, the first to fourth time lengths D ′ 0 to D ′ 3 ) is set to W 0 or more.
  • a short waveform payload can be generated according to the signal.
  • the data, the preamble, and the payload are The payload, the preamble, and the data may be transmitted in this order.
  • the fact that the packet of the visible light signal including data (that is, invalid data) does not include the R data portion is indicated to the receiving device that receives the packet by the data. I can inform you.
  • the light emitter has a plurality of light sources including a red light source, a blue light source, and a green light source, and in the transmitting step, the visible light source is formed using only the red light source among the plurality of light sources.
  • An optical signal may be transmitted.
  • the light emitter can display an image using a red light source, a blue light source, and a green light source, and can transmit a visible light signal having a wavelength that can be easily received to the receiving device.
  • a recording medium such as an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM. You may implement
  • FIG. 1 shows an example in which imaging devices arranged in one row are exposed simultaneously, and imaging is performed by shifting the exposure start time in the order of closer rows.
  • the exposure line of the image sensor that is exposed simultaneously is referred to as an exposure line
  • the pixel line on the image corresponding to the image sensor is referred to as a bright line.
  • this image capturing method When this image capturing method is used to capture an image of a blinking light source on the entire surface of the image sensor, bright lines (light and dark lines of pixel values) along the exposure line appear on the captured image as shown in FIG. .
  • bright lines light and dark lines of pixel values
  • the imaging frame rate By recognizing the bright line pattern, it is possible to estimate the light source luminance change at a speed exceeding the imaging frame rate. Thereby, by transmitting a signal as a change in light source luminance, communication at a speed higher than the imaging frame rate can be performed.
  • LO lower luminance value
  • HI high
  • Low may be in a state where the light source is not shining, or may be shining weaker than high.
  • the imaging frame rate is 30 fps
  • a change in luminance with a period of 1.67 milliseconds can be recognized.
  • the exposure time is set shorter than 10 milliseconds, for example.
  • FIG. 2 shows a case where the exposure of the next exposure line is started after the exposure of one exposure line is completed.
  • the transmission speed is a maximum of flm bits per second.
  • the light emission time of the light emitting unit is controlled by a unit time shorter than the exposure time of each exposure line. More information can be transmitted.
  • information can be transmitted at a maximum rate of flElv bits per second.
  • the basic period of transmission can be recognized by causing the light emitting unit to emit light at a timing slightly different from the exposure timing of each exposure line.
  • FIG. 4 shows a case where the exposure of the next exposure line is started before the exposure of one exposure line is completed. That is, the exposure times of adjacent exposure lines are partially overlapped in time.
  • the S / N ratio can be improved.
  • the exposure time of adjacent exposure lines has a partial overlap in time, and a configuration in which some exposure lines have no partial overlap. It is also possible. By configuring a part of the exposure lines so as not to partially overlap in time, it is possible to suppress the generation of intermediate colors due to the overlap of exposure times on the imaging screen, and to detect bright lines more appropriately. .
  • the exposure time is calculated from the brightness of each exposure line, and the light emission state of the light emitting unit is recognized.
  • the brightness of each exposure line is determined by a binary value indicating whether the luminance is equal to or higher than a threshold value, in order to recognize the state where no light is emitted, the state where the light emitting unit does not emit light is indicated for each line. It must last longer than the exposure time.
  • FIG. 5A shows the influence of the difference in exposure time when the exposure start times of the exposure lines are equal.
  • 7500a is the case where the exposure end time of the previous exposure line is equal to the exposure start time of the next exposure line
  • 7500b is the case where the exposure time is longer than that.
  • the exposure time of adjacent exposure lines is partially overlapped in time, so that the exposure time can be increased. That is, the light incident on the image sensor increases and a bright image can be obtained.
  • the imaging sensitivity for capturing images with the same brightness can be suppressed to a low level, an image with less noise can be obtained, so that communication errors are suppressed.
  • FIG. 5B shows the influence of the difference in the exposure start time of each exposure line when the exposure times are equal.
  • 7501a is the case where the exposure end time of the previous exposure line is equal to the exposure start time of the next exposure line
  • 7501b is the case where the exposure of the next exposure line is started earlier than the end of exposure of the previous exposure line.
  • the sample interval ( difference in exposure start time) becomes dense, the change in the light source luminance can be estimated more accurately, the error rate can be reduced, and the change in the light source luminance in a shorter time is recognized. be able to.
  • the exposure time overlap it is possible to recognize blinking of the light source that is shorter than the exposure time by using the difference in exposure amount between adjacent exposure lines.
  • the exposure time satisfy the exposure time> (sample interval ⁇ pulse width).
  • the pulse width is a pulse width of light that is a period during which the luminance of the light source is High. Thereby, the High brightness can be detected appropriately.
  • the exposure time is set to be longer than that in the normal shooting mode.
  • the communication speed can be dramatically improved.
  • the exposure time needs to be set as exposure time ⁇ 1/8 ⁇ f. Blanking that occurs during shooting is at most half the size of one frame. That is, since the blanking time is less than half of the shooting time, the actual shooting time is 1 / 2f at the shortest time.
  • the exposure time since it is necessary to receive quaternary information within a time of 1 / 2f, at least the exposure time needs to be shorter than 1 / (2f ⁇ 4). Since the normal frame rate is 60 frames / second or less, it is possible to generate an appropriate bright line pattern in the image data and perform high-speed signal transmission by setting the exposure time to 1/480 seconds or less. Become.
  • FIG. 5C shows an advantage when the exposure times are short when the exposure times of the exposure lines do not overlap.
  • the exposure time is long, even if the light source has a binary luminance change as in 7502a, the captured image has an intermediate color portion as in 7502e, and it becomes difficult to recognize the luminance change of the light source.
  • the free time (predetermined waiting time) t D2 not predetermined exposure start exposure of the next exposure line, the luminance variation of the light source Can be easily recognized. That is, a more appropriate bright line pattern such as 7502f can be detected.
  • an exposure time t E can be realized to be smaller than the time difference t D of the exposure start time of each exposure line.
  • the exposure time is set shorter than the normal shooting mode until a predetermined idle time occurs. This can be realized. Further, even when the normal photographing mode is the case where the exposure end time of the previous exposure line and the exposure start time of the next exposure line are equal, by setting the exposure time short until a predetermined non-exposure time occurs, Can be realized.
  • the exposure time of adjacent exposure lines has a partial overlap in time, and a configuration in which some exposure lines have no partial overlap. It is also possible. Further, in all exposure lines, it is not necessary to provide a configuration in which an idle time (predetermined waiting time) in which a predetermined exposure is not performed is provided after the exposure of one exposure line is completed until the exposure of the next exposure line is started. It is also possible to have a configuration in which the lines partially overlap in time. With such a configuration, it is possible to take advantage of the advantages of each configuration.
  • the same readout method is used in the normal shooting mode in which shooting is performed at a normal frame rate (30 fps, 60 fps) and in the visible light communication mode in which shooting is performed with an exposure time of 1/480 second or less in which visible light communication is performed.
  • a signal may be read by a circuit.
  • FIG. 5D shows the relationship between the minimum change time t S of the light source luminance, the exposure time t E , the time difference t D of the exposure start time of each exposure line, and the captured image.
  • Figure 5E the transition and time t T of the light source luminance, which shows the relationship between the time difference t D of the exposure start time of each exposure line.
  • t D is larger than the t T, exposure lines to be neutral is reduced, it is easy to estimate the light source luminance.
  • the exposure line of the intermediate color is continuously 2 or less, which is desirable.
  • t T the light source is less than 1 microsecond in the case of LED, light source for an approximately 5 microseconds in the case of organic EL, a t D by 5 or more microseconds, to facilitate estimation of the light source luminance be able to.
  • Figure 5F shows a high frequency noise t HT of light source luminance, the relationship between the exposure time t E.
  • t E is larger than t HT , the captured image is less affected by high frequency noise, and light source luminance is easily estimated.
  • t E is an integral multiple of t HT , the influence of high frequency noise is eliminated, and the light source luminance is most easily estimated.
  • t E > t HT .
  • the main cause of high frequency noise derived from the switching power supply circuit since many of the t HT in the switching power supply for the lamp is less than 20 microseconds, by the t E and 20 micro-seconds or more, the estimation of the light source luminance It can be done easily.
  • Figure 5G is the case t HT is 20 microseconds, which is a graph showing the relationship between the size of the exposure time t E and the high frequency noise.
  • t E is the value becomes equal to the value when the amount of noise takes a maximum, 15 microseconds or more, or, 35 microseconds or more, or, It can be confirmed that the efficiency is good when it is set to 54 microseconds or more, or 74 microseconds or more. From the viewpoint of reducing high-frequency noise, it is desirable that t E be large. However, as described above, there is a property that light source luminance can be easily estimated in that the smaller the t E , the more difficult the intermediate color portion is generated.
  • t E when the light source luminance change period is 15 to 35 microseconds, t E is 15 microseconds or more, and when the light source luminance change period is 35 to 54 microseconds, t E is 35 microseconds or more.
  • t E is 54 microseconds or more when the cycle is 54 to 74 microseconds of change, t E when the period of the change in light source luminance is 74 microseconds or more may be set as 74 microseconds or more.
  • Figure 5H shows the relationship between the exposure time t E and the recognition success rate. Since the exposure time t E has a relative meaning with respect to the time when the luminance of the light source is constant, the value (relative exposure time) obtained by dividing the period t S where the luminance of the light source changes by the exposure time t E is taken as the horizontal axis. Yes. From the graph, it can be seen that if the recognition success rate is desired to be almost 100%, the relative exposure time should be 1.2 or less. For example, when the transmission signal is 1 kHz, the exposure time may be about 0.83 milliseconds or less.
  • the relative exposure time may be set to 1.25 or less, and when the recognition success rate is set to 80% or more, the relative exposure time may be set to 1.4 or less. Recognize. Also, the recognition success rate drops sharply when the relative exposure time is around 1.5, and becomes almost 0% at 1.6, so it can be seen that the relative exposure time should not be set to exceed 1.5. . It can also be seen that after the recognition rate becomes 0 at 7507c, it rises again at 7507d, 7507e, and 7507f.
  • the relative exposure time is 1.9 to 2.2, 2.4 to 2.6, and 2.8 to 3.0. Just do it.
  • these exposure times may be used as the intermediate mode.
  • FIG. 6A is a flowchart of the information communication method in the present embodiment.
  • the information communication method in the present embodiment is an information communication method for acquiring information from a subject, and includes steps SK91 to SK93.
  • a plurality of bright lines corresponding to a plurality of exposure lines included in the image sensor are generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject.
  • a first exposure time setting step SK91 for setting a first exposure time of the image sensor; and the image sensor shoots the subject whose luminance changes with the set first exposure time
  • First image acquisition step SK92 for acquiring a bright line image including a plurality of bright lines, and information acquisition for acquiring information by demodulating data specified by the patterns of the plurality of bright lines included in the acquired bright line image Step SK93, and in the first image acquisition step SK92, the plurality of exposure lines Re starts exposure at successively different times, and the exposure of the adjacent exposure line after a predetermined idle time from the end, the exposure is started adjacent to the exposure line.
  • FIG. 6B is a block diagram of the information communication apparatus according to the present embodiment.
  • the information communication device K90 in the present embodiment is an information communication device that acquires information from a subject, and includes constituent elements K91 to K93.
  • the information communication apparatus K90 causes a plurality of bright lines corresponding to a plurality of exposure lines included in the image sensor to be generated in response to a change in luminance of the subject in an image obtained by photographing the subject by an image sensor.
  • an exposure time setting unit K91 for setting an exposure time of the image sensor, and the image sensor for acquiring a bright line image including the plurality of bright lines by photographing the subject whose luminance changes with the set exposure time.
  • an information acquisition unit K93 that acquires information by demodulating data specified by the patterns of the plurality of bright lines included in the acquired bright line image, and the plurality of exposure lines. Each of these starts exposure at different times sequentially and is adjacent to the exposure line. From the exposure of the down is completed after a predetermined idle time has elapsed, exposure is started.
  • each of the plurality of exposure lines is exposed to the adjacent exposure line adjacent to the exposure line. Since exposure is started after a lapse of a predetermined idle time after the end, it is possible to easily recognize a change in luminance of the subject. As a result, information can be appropriately acquired from the subject.
  • each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component.
  • Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.
  • the program causes the computer to execute the information communication method shown by the flowchart of FIG. 6A.
  • shooting in the normal shooting mode or normal shooting mode is referred to as normal shooting
  • shooting in the visible light communication mode or visible light communication mode is referred to as visible light shooting (visible light communication).
  • shooting in an intermediate mode may be used, and an intermediate image may be used instead of a composite image described later.
  • FIG. 7 is a diagram illustrating an example of the photographing operation of the receiver in this embodiment.
  • the receiver 8000 switches the shooting mode to normal shooting, visible light communication, normal shooting, and so on. Then, the receiver 8000 generates a composite image in which the bright line pattern, the subject, and the surrounding area are clearly displayed by combining the normal captured image and the visible light communication image, and displays the composite image on the display. .
  • This composite image is an image generated by superimposing the bright line pattern of the visible light communication image on the portion where the signal in the normal captured image is transmitted. Further, the bright line pattern, the subject, and the surroundings displayed by the composite image are clear and have a sharpness sufficiently recognized by the user. By displaying such a composite image, the user can more clearly know from where or from where the signal is transmitted.
  • FIG. 8 is a diagram illustrating another example of the photographing operation of the receiver in this embodiment.
  • the receiver 8000 includes a camera Ca1 and a camera Ca2.
  • the camera Ca1 performs normal photographing
  • the camera Ca2 performs visible light photographing.
  • the camera Ca1 acquires the normal captured image as described above
  • the camera Ca2 acquires the visible light communication image as described above.
  • the receiver 8000 generates the above-described combined image by combining the normal captured image and the visible light communication image, and displays the combined image on the display.
  • FIG. 9 is a diagram illustrating another example of the photographing operation of the receiver in this embodiment.
  • the camera Ca1 switches the shooting mode to normal shooting, visible light communication, normal shooting, and so on.
  • the camera Ca2 continuously performs normal shooting.
  • the receiver 8000 receives from the normal shooting images acquired by these cameras using stereo vision (the principle of triangulation).
  • the distance from the machine 8000 to the subject (hereinafter referred to as subject distance) is estimated.
  • FIG. 10 is a diagram illustrating an example of the display operation of the receiver in this embodiment.
  • the receiver 8000 switches the photographing mode to visible light communication, normal photographing, visible light communication, and so on.
  • the receiver 8000 activates an application program when performing visible light communication for the first time.
  • the receiver 8000 estimates its own position based on the signal received by visible light communication.
  • the receiver 8000 displays AR (Augmented Reality) information on the normal shot image acquired by the normal shooting.
  • This AR information is acquired based on the position estimated as described above.
  • the receiver 8000 estimates the movement and direction change of the receiver 8000 based on the detection result of the 9-axis sensor and the motion detection of the normal captured image, and matches the estimated movement and direction change. To move the display position of the AR information.
  • the AR information can be made to follow the subject image of the normal captured image.
  • the receiver 8000 switches the shooting mode from the normal shooting to the visible light communication, the AR information is superimposed on the latest normal shooting image acquired at the time of the normal shooting immediately before the visible light communication.
  • the receiver 8000 displays a normal captured image on which the AR information is superimposed.
  • the receiver 8000 estimates the movement and direction change of the receiver 8000 on the basis of the detection result by the 9-axis sensor, and AR in accordance with the estimated movement and direction change.
  • Move information and normal captured images Thereby, AR information can be made to follow the subject image of the normal captured image in accordance with the movement of the receiver 8000 or the like in the case of visible light communication as in the case of normal imaging. Further, the normal image can be enlarged and reduced in accordance with the movement of the receiver 8000 or the like.
  • FIG. 11 is a diagram showing an example of the display operation of the receiver in this embodiment.
  • the receiver 8000 may display the composite image on which the bright line pattern is projected, as shown in FIG.
  • the receiver 8000 normally captures a signal explicit object that is an image having a predetermined color for notifying that a signal is transmitted instead of the bright line pattern.
  • a composite image may be generated by superimposing on the image, and the composite image may be displayed.
  • the receiver 8000 normally has a location where a signal is transmitted indicated by a dotted frame and an identifier (for example, ID: 101, ID: 102, etc.).
  • the captured image may be displayed as a composite image.
  • the receiver 8000 recognizes a signal that is an image having a predetermined color for notifying that a specific type of signal is transmitted instead of the bright line pattern.
  • a composite image may be generated by superimposing an object on a normal captured image, and the composite image may be displayed.
  • the color of the signal identification object differs depending on the type of signal output from the transmitter. For example, when the signal output from the transmitter is position information, a red signal identification object is superimposed, and when the signal output from the transmitter is a coupon, the green signal identification object is Superimposed.
  • FIG. 12 is a diagram illustrating an example of the operation of the receiver in this embodiment.
  • the receiver 8000 may display a normal captured image and output a sound for notifying the user that the transmitter has been found.
  • the receiver 8000 varies the type of output sound, the number of outputs, or the output time depending on the number of transmitters found, the type of received signal, or the type of information specified by the signal. It may be allowed.
  • FIG. 13 is a diagram illustrating another example of the operation of the receiver in this embodiment.
  • the receiver 8000 when the user touches the bright line pattern displayed in the composite image, the receiver 8000 generates an information notification image based on a signal transmitted from the subject corresponding to the touched bright line pattern, and the information notification Display an image.
  • This information notification image indicates, for example, a store coupon or a place.
  • the bright line pattern may be a signal explicit object, a signal identification object, a dotted line frame, or the like shown in FIG. The same applies to the bright line patterns described below.
  • FIG. 14 is a diagram illustrating another example of the operation of the receiver in this embodiment.
  • the receiver 8000 when the user touches the bright line pattern displayed in the composite image, the receiver 8000 generates an information notification image based on a signal transmitted from the subject corresponding to the touched bright line pattern, and the information notification Display an image.
  • the information notification image indicates the current location of the receiver 8000 by a map or the like.
  • FIG. 15 is a diagram illustrating another example of the operation of the receiver in this embodiment.
  • the receiver 8000 when the user performs a swipe on the receiver 8000 on which the composite image is displayed, the receiver 8000 performs normal shooting having a dotted frame and an identifier, similar to the normal shot image illustrated in FIG. An image is displayed and a list of information is displayed so as to follow the swipe operation. In this list, information specified by a signal transmitted from a location (transmitter) indicated by each identifier is shown.
  • the swipe may be, for example, an operation of moving a finger from outside the right side of the display in the receiver 8000.
  • the swipe may be an operation of moving a finger from the upper side, the lower side, or the left side of the display.
  • the receiver 8000 may display an information notification image (for example, an image showing a coupon) showing the information in more detail.
  • an information notification image for example, an image showing a coupon
  • FIG. 16 is a diagram illustrating another example of the operation of the receiver in this embodiment.
  • the receiver 8000 displays the information notification image superimposed on the composite image so as to follow the swipe operation.
  • This information notification image shows the subject distance with an arrow in an easy-to-understand manner for the user.
  • the swipe may be, for example, an operation of moving a finger from outside the lower side of the display in the receiver 8000.
  • the swipe may be an operation of moving a finger from the left side of the display, from the upper side, or from the right side.
  • FIG. 17 is a diagram illustrating another example of the operation of the receiver in this embodiment.
  • the receiver 8000 images a transmitter, which is a signage indicating a plurality of stores, as a subject, and displays a normal captured image acquired by the imaging.
  • the receiver 8000 when the user taps the signage image of one store included in the subject displayed in the normal captured image, the receiver 8000 generates an information notification image based on a signal transmitted from the signage of the store Then, the information notification image 8001 is displayed.
  • This information notification image 8001 is an image showing, for example, a vacant seat situation in a store.
  • FIG. 18 is a diagram illustrating an example of operations of the receiver, the transmitter, and the server in the present embodiment.
  • the transmitter 8012 configured as a television transmits a signal to the receiver 8011 by a luminance change.
  • This signal includes, for example, information for prompting the user to purchase content related to the program being viewed.
  • the receiver 8011 displays an information notification image that prompts the user to purchase content based on the signal.
  • the receiver 8011 receives information included in a SIM (Subscriber Identity Module) card inserted into the receiver 8011, user ID, terminal ID, credit card information, billing At least one of the information, the password, and the transmitter ID is transmitted to the server 8013.
  • SIM Subscriber Identity Module
  • the server 8013 manages a user ID and payment information in association with each user.
  • the server 8013 identifies the user ID based on the information transmitted from the receiver 8011, and confirms the payment information associated with the user ID. By this confirmation, the server 8013 determines whether or not to allow the user to purchase content. If the server 8013 determines to permit, the server 8013 transmits permission information to the receiver 8011. When receiving the permission information, the receiver 8011 transmits the permission information to the transmitter 8012. The transmitter 8012 that has received the permission information acquires and reproduces the content via a network, for example.
  • the transmitter 8012 may transmit information including the ID of the transmitter 8012 to the receiver 8011 by changing the luminance.
  • the receiver 8011 transmits the information to the server 8013.
  • the server 8013 obtains the information, the server 8013 can determine that, for example, a television program is being viewed by the transmitter 8012, and can perform a viewing rate survey of the television program.
  • the receiver 8011 includes the content operated by the user (voting or the like) in the above information and transmits it to the server 8013, so that the server 8013 can reflect the content in the television program. That is, a viewer participation type program can be realized. Further, when the receiver 8011 accepts writing by the user, the contents of the writing are included in the above-described information and transmitted to the server 8013 so that the server 8013 can write the writing to the TV program or a bulletin board on the network. Etc. can be reflected.
  • the server 8013 can charge for viewing of a television program by pay broadcasting or an on-demand program. Further, the server 8013 displays an advertisement on the receiver 8011, displays detailed information of a TV program displayed on the transmitter 8012, and displays a URL of a site indicating the detailed information. Can do. Further, the server 8013 obtains the number of times the advertisement is displayed by the receiver 8011 or the amount of the product purchased by the advertisement, and thereby charges the advertiser according to the number or amount. be able to. Such billing can be made even if the user who saw the advertisement does not purchase the product immediately.
  • the server 8013 acquires information indicating the manufacturer of the transmitter 8012 from the transmitter 8012 via the receiver 8011, the server 8013 provides a service (for example, a reward for sales of the above-described product to the manufacturer indicated by the information). Payment).
  • a service for example, a reward for sales of the above-described product to the manufacturer indicated by the information.
  • FIG. 19 is a diagram illustrating another example of the operation of the receiver in this embodiment.
  • the receiver 8030 is configured as a head mounted display including a camera, for example.
  • the receiver 8030 starts photographing in the visible light communication mode, that is, visible light communication when the start button is pressed.
  • the receiver 8030 notifies the user of information corresponding to the received signal. This notification is performed, for example, by outputting sound from a speaker provided in the receiver 8030 or by displaying an image.
  • the visible light communication is received by the receiver 8030 when an input of a voice instructing the start is received by the receiver 8030 or a signal instructing the start is received by wireless communication. It may be started when it is done.
  • visible light communication is started when the change width of the value obtained by the 9-axis sensor provided in the receiver 8030 exceeds a predetermined range or when a bright line pattern appears even in a normal photographed image. May be.
  • FIG. 20 is a diagram illustrating another example of the operation of the receiver in this embodiment.
  • the receiver 8030 displays the composite image 8034 as described above.
  • the user performs an operation of moving the fingertip so as to surround the bright line pattern in the composite image 8034.
  • the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.
  • FIG. 21 is a diagram illustrating another example of the operation of the receiver in this embodiment.
  • the receiver 8030 displays the composite image 8034 as described above.
  • the user performs an operation of placing the fingertip on the bright line pattern in the composite image 8034 for a predetermined time or more.
  • the receiver 8030 identifies the bright line pattern that is the target of the operation, and displays an information notification image 8032 based on a signal transmitted from a location corresponding to the bright line pattern.
  • FIG. 22 is a diagram illustrating an example of the operation of the transmitter according to the present embodiment.
  • the transmitter transmits the signal 1 and the signal 2 alternately at a predetermined cycle, for example. Transmission of the signal 1 and transmission of the signal 2 are performed by luminance changes such as blinking of visible light. Further, the luminance change pattern for transmitting the signal 1 and the luminance change pattern for transmitting the signal 2 are different from each other.
  • FIG. 23 is a diagram illustrating another example of the operation of the transmitter according to the present embodiment.
  • each block is arranged in the order of block 1, block 2, and block 3 in the first signal sequence, and each block is arranged in the order of block 3, block 1, and block 2 in the next signal sequence.
  • FIG. 24 is a diagram illustrating an application example of the receiver in this embodiment.
  • the receiver 7510a configured as a smartphone images the light source 7510b with a back camera (out camera) 7510c, receives a signal transmitted from the light source 7510b, and acquires the position and orientation of the light source 7510b from the received signal.
  • the receiver 7510a estimates the position and orientation of the receiver 7510a itself from how the light source 7510b is captured in the captured image and the sensor values of the 9-axis sensor provided in the receiver 7510a.
  • the receiver 7510a captures the user 7510e with a front camera (face camera, in-camera) 7510f, and estimates the position and orientation of the head of the 7510e and the line-of-sight direction (eyeball position and orientation) by image processing. .
  • the receiver 7510a transmits the estimation result to the server.
  • the receiver 7510a changes the behavior (display content and playback sound) according to the viewing direction of the user 7510e.
  • the imaging by the back camera 7510c and the imaging by the front camera 7510f may be performed simultaneously or alternately.
  • FIG. 25 is a diagram illustrating another example of the operation of the receiver in this embodiment.
  • the receiver displays the bright line pattern by the composite image or the intermediate image as described above. At this time, the receiver may not be able to receive a signal from the transmitter corresponding to the bright line pattern.
  • the receiver when the bright line pattern is selected by the user performing an operation (for example, tapping) on the bright line pattern, the receiver performs an optical zoom to enlarge the composite line image or the intermediate image in which the bright line pattern is enlarged. Display an image.
  • the receiver can appropriately receive a signal from the transmitter corresponding to the bright line pattern. That is, even if an image obtained by imaging is too small to acquire a signal, the signal can be appropriately received by performing optical zoom. Even when an image having a size capable of acquiring a signal is displayed, fast reception can be performed by performing optical zoom.
  • the information communication method is an information communication method for acquiring information from a subject, and an bright line corresponding to an exposure line included in the image sensor is included in an image obtained by photographing the subject with an image sensor.
  • a first exposure time setting step for setting an exposure time of the image sensor so as to occur in accordance with a change in luminance of the subject; and the image sensor photographs the subject whose luminance changes with the set exposure time.
  • the bright line image acquisition step of acquiring a bright line image that is an image including the bright line, and the spatial position of the portion where the bright line appears can be identified based on the bright line image, and the subject and the subject
  • An image display step for displaying a display image in which the surroundings of the subject are projected, and the image included in the acquired bright line image Including an information acquisition step of acquiring transmission information by demodulating the data identified by the pattern of lines.
  • a composite image or an intermediate image as shown in FIGS. 7, 8, and 11 is displayed as a display image.
  • the spatial position of the part where the bright line appears is identified by a bright line pattern, a signal explicit object, a signal identification object, a dotted line frame, or the like. Therefore, the user can easily find a subject that is transmitting a signal due to a change in luminance by viewing such a display image.
  • the information communication method further includes a second exposure time setting step for setting an exposure time longer than the exposure time, and the image sensor photographs the subject and the surroundings of the subject with the long exposure time.
  • a composite step of generating a composite image by superimposing the captured image, and the composite image may be displayed as the display image in the image display step.
  • the signal object is a bright line pattern, a signal explicit object, a signal identification object, a dotted line frame, or the like, and a composite image is displayed as a display image as shown in FIGS.
  • the user can more easily find the subject that is transmitting the signal due to the luminance change.
  • an exposure time is set to 1/3000 sec.
  • the bright line image acquisition step the bright line image in which the periphery of the subject is projected is acquired, and in the image display step.
  • the bright line image may be displayed as the display image.
  • the bright line image is acquired and displayed as an intermediate image. Therefore, it is not necessary to perform processing such as acquiring and synthesizing the normal captured image and the visible light communication image, and the processing can be simplified.
  • the image sensor includes a first image sensor and a second image sensor.
  • the first image sensor captures the normal captured image, and the bright line is acquired.
  • the bright line image may be acquired by capturing the second image sensor simultaneously with the capturing of the first image sensor.
  • a normal photographed image and a visible light communication image that is a bright line image are acquired by each camera. Therefore, compared with the case where a normal captured image and a visible light communication image are acquired with one camera, those images can be acquired earlier, and the processing can be speeded up.
  • the information communication method when the part where the bright line appears in the display image is designated by a user operation, the information communication method further includes the transmission information acquired from the pattern of the bright line of the designated part.
  • An information presentation step of presenting presentation information based on the information may be included. For example, the operation by the user is shown in association with a tap, swipe, an operation in which a fingertip is continuously applied to the part for a predetermined time, an operation in which a line of sight is directed to the part for a predetermined time, or the like.
  • the presentation information is displayed as an information notification image. Thereby, desired information can be presented to the user.
  • the image sensor may be provided in a head mounted display, and in the image display step, a projector mounted on the head mounted display may display the display image.
  • a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject.
  • the first exposure time setting step of setting the exposure time of the image sensor, and the image including the bright line by the image sensor photographing the subject whose luminance changes with the set exposure time.
  • the plurality of subjects are photographed during the period in which the image sensor is moved.
  • the information communication method By acquiring the bright line image including a plurality of parts where the bright lines appear, and in the information acquisition step, for each part, by demodulating data specified by the pattern of the bright lines of the parts, The information communication method further estimates the position of the image sensor based on the acquired positions of the plurality of subjects and the movement state of the image sensor. A position estimation step may be included.
  • a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject.
  • the first exposure time setting step of setting the exposure time of the image sensor, and the image including the bright line by the image sensor photographing the subject whose luminance changes with the set exposure time is performed.
  • the user can be authenticated and the convenience can be improved.
  • a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject.
  • the exposure time setting step for setting the exposure time of the image sensor, and the image sensor captures the bright line image including the bright line by photographing the subject whose luminance changes at the set exposure time.
  • the image acquisition step the image is reflected on the reflection surface.
  • the bright line image is acquired by photographing a plurality of the subjects, and the information acquisition step is performed.
  • the bright line is separated into bright lines corresponding to each of the plurality of subjects according to the intensity of the bright lines included in the bright line image, and each subject is specified by a bright line pattern corresponding to the subject.
  • Information may be acquired by demodulating the data.
  • a bright line corresponding to an exposure line included in the image sensor is generated in an image obtained by photographing the subject by an image sensor according to a change in luminance of the subject.
  • the exposure time setting step for setting the exposure time of the image sensor and the image sensor captures the bright line image including the bright line by photographing the subject whose luminance changes at the set exposure time.
  • the image acquisition step the image is reflected on the reflection surface.
  • the bright line image is acquired by photographing the subject, and the information communication method includes: , Based on the luminance distribution in the emission line image may include position estimation step for estimating the position of the object.
  • An information communication method for transmitting a signal according to a luminance change wherein a first determination step of determining a first pattern of luminance change by modulating a first signal to be transmitted, A second determination step of determining a second pattern of luminance change by modulating the signal of 2; and a luminance change according to the first pattern determined by the light emitter; A transmission step of transmitting the first and second signals by alternately performing a luminance change according to the second pattern.
  • the first signal and the second signal can be transmitted without delay.
  • the luminance change when the luminance change is switched between the luminance change according to the first pattern and the luminance change according to the second pattern, it may be switched with a buffer time.
  • An information communication method for transmitting a signal according to a luminance change wherein a determination step of determining a luminance change pattern by modulating a signal to be transmitted, and the light emitter changes in luminance according to the determined pattern And transmitting the signal to be transmitted, the signal comprising a plurality of large blocks, each of the plurality of large blocks including first data and a preamble for the first data. And a check signal for the first data, wherein the first data is composed of a plurality of small blocks, and the small blocks include second data, a preamble for the second data, and the first data. And a check signal for the second data may be included.
  • An information communication method for transmitting a signal by luminance change wherein a plurality of transmitters each modulate a signal to be transmitted to determine a luminance change pattern, and for each transmitter, And a transmitting step in which a light emitter provided in the transmitter changes the luminance according to the determined pattern and transmits the signal to be transmitted.
  • signals having different frequencies or protocols are transmitted. May be.
  • An information communication method for transmitting a signal by luminance change wherein a plurality of transmitters each modulate a signal to be transmitted to determine a luminance change pattern, and for each transmitter, A transmitter in which a light emitter provided in the transmitter transmits a signal to be transmitted by changing in luminance according to the determined pattern, wherein in the transmitting step, one of the plurality of transmitters One transmitter may receive a signal transmitted from the other transmitter and transmit another signal in a manner that does not interfere with the received signal.
  • Embodiment 3 each application example using a receiver such as a smartphone in Embodiment 1 or 2 and a transmitter that transmits information as a blinking pattern such as an LED or an organic EL will be described.
  • FIG. 26 is a diagram illustrating an example of processing operations of the receiver, the transmitter, and the server in the third embodiment.
  • the receiver 8142 configured as a smartphone acquires position information indicating its own position, and transmits the position information to the server 8141.
  • the receiver 8142 acquires position information when, for example, GPS is used or other signals are received.
  • the server 8141 transmits the ID list associated with the position indicated by the position information to the receiver 8142.
  • the ID list includes, for each ID such as “abcd”, the ID and information associated with the ID.
  • the receiver 8142 receives a signal from a transmitter 8143 configured as, for example, a lighting device. At this time, the receiver 8142 may receive only a part of the ID (for example, “b”) as the above signal. In this case, the receiver 8142 searches the ID list for an ID including a part of the ID. If the unique ID is not found, the receiver 8142 further receives a signal from the transmitter 8143 that includes another portion of that ID. As a result, the receiver 8142 acquires a larger part (for example, “bc”) of the ID. Then, the receiver 8142 searches the ID list again for an ID including a part of the ID (for example, “bc”).
  • a part of the ID for example, “b”
  • the receiver 8142 can specify all of the IDs even if only a part of the IDs can be acquired.
  • the receiver 8142 receives not only a part of the ID but also a check part such as CRC (Cyclic Redundancy Check).
  • FIG. 27 is a diagram illustrating an example of operations of the transmitter and the receiver in the third embodiment.
  • the transmitter 8165 configured as a television acquires an image and an ID (ID 1000) associated with the image from the control unit 8166.
  • the transmitter 8165 displays the image and transmits the ID (ID 1000) to the receiver 8167 by changing the luminance.
  • the receiver 8167 receives the ID (ID 1000) by imaging and displays information associated with the ID (ID 1000).
  • the control unit 8166 changes the image output to the transmitter 8165 to another image.
  • the control unit 8166 also changes the ID output to the transmitter 8165. That is, the control unit 8166 outputs the other ID (ID 1001) associated with the other image to the transmitter 8165 together with the other image.
  • the transmitter 8165 displays another image and transmits another ID (ID 1001) to the receiver 8167 by changing the luminance.
  • the receiver 8167 receives the other ID (ID 1001) by imaging, and displays information associated with the other ID (ID 1001).
  • FIG. 28 is a diagram illustrating an example of operations of the transmitter, the receiver, and the server in the third embodiment.
  • the transmitter 8185 configured as a smartphone displays information indicating “coupon 100 yen discount”, for example, by changing the luminance of the display 8185a excluding the barcode portion 8185b, that is, by visible light communication. Send.
  • the transmitter 8185 displays the barcode on the barcode portion 8185b without changing the luminance of the barcode portion 8185b.
  • This bar code indicates the same information as the information transmitted by the visible light communication described above.
  • the transmitter 8185 displays a character or a picture indicating information transmitted by visible light communication, for example, a character “coupon 100 yen discount” on a portion of the display 8185a excluding the barcode portion 8185b. By displaying such characters or pictures, the user of the transmitter 8185 can easily grasp what information is being transmitted.
  • the receiver 8186 acquires the information transmitted by visible light communication and the information indicated by the barcode by imaging, and transmits the information to the server 8187.
  • the server 8187 determines whether or not these pieces of information match or relate to each other. When it determines that these pieces of information match or relate to each other, the server 8187 executes processing according to the pieces of information. Alternatively, the server 8187 transmits the determination result to the receiver 8186, and causes the receiver 8186 to execute processing according to the information.
  • the transmitter 8185 may transmit a part of the information indicated by the barcode by visible light communication.
  • the barcode may indicate the URL of the server 8187.
  • the transmitter 8185 may acquire information associated with the ID by acquiring the ID as a receiver and transmitting the ID to the server 8187.
  • the information associated with this ID is the same as the information transmitted by the above visible light communication or the information indicated by the barcode.
  • the server 8187 may transmit an ID associated with information (visible light communication information or barcode information) transmitted from the transmitter 8185 via the receiver 8186 to the transmitter 8185.
  • FIG. 29 is a diagram illustrating an example of operations of the transmitter and the receiver in the third embodiment.
  • the receiver 8183 images a subject including a plurality of persons 8197 and street lamps 8195.
  • the streetlight 8195 includes a transmitter 8195a that transmits information according to a change in luminance.
  • the receiver 8183 acquires an image in which the image of the transmitter 8195a appears as the bright line pattern described above.
  • the receiver 8183 acquires the AR object 8196a associated with the ID indicated by the bright line pattern from, for example, a server. Then, the receiver 8183 superimposes the AR object 8196a on a normal captured image 8196 obtained by normal imaging, and displays a normal captured image 8196 on which the AR object 8196a is superimposed.
  • the information communication method is an information communication method for transmitting a signal according to a luminance change, wherein a determination step for determining a luminance change pattern by modulating a signal to be transmitted and a light emitter are determined.
  • the luminance change position that is the rising or falling position of the luminance in the time width is different for each of the different signals to be transmitted.
  • the integrated value of the luminance of the luminous body in the time width is the same according to the preset brightness As a value to determine the pattern of the luminance change.
  • the rising positions of the luminance are different from each other, and a predetermined time width is set.
  • the luminance change pattern is determined so that the integrated value of the luminance of the light emitter in (unit time width) becomes the same value according to a predetermined brightness (for example, 99% or 1%).
  • a predetermined brightness for example, 99% or 1%.
  • the brightness of the illuminant can be kept constant for each signal to be transmitted, flicker can be suppressed, and the receiver that images the illuminant is based on the luminance change position.
  • the brightness change pattern can be demodulated appropriately.
  • the luminance change pattern is a pattern in which one of two different luminance values (luminance H (High) or luminance L (Low)) appears at any position in the unit time width. The brightness of the body can be changed continuously.
  • the information communication method further includes an image display step of sequentially switching and displaying each of the plurality of images.
  • the displayed image is displayed each time the image is displayed in the image display step.
  • the luminance change pattern for the identification information is determined by modulating the identification information corresponding to the signal to be transmitted.
  • the transmission step the image is displayed every time the image is displayed in the image display step.
  • the identification information may be transmitted by the luminance change of the light emitter according to the luminance change pattern determined with respect to the identification information corresponding to the existing image.
  • each time an image is displayed identification information corresponding to the displayed image is transmitted. Therefore, the user receives the received information on the receiver based on the displayed image.
  • the identification information to be made can be easily selected.
  • the light emitter is further radiated according to a luminance change pattern determined for identification information corresponding to an image displayed in the past.
  • the identification information may be transmitted by changing.
  • the receiver cannot receive the identification signal transmitted before switching because the displayed image has been switched, the image displayed in the past together with the identification information corresponding to the currently displayed image is displayed. Since the identification information corresponding to is also transmitted, the identification information transmitted before switching can be properly received again by the receiver.
  • each time an image is displayed in the image display step the identification information corresponding to the displayed image and the time when the image is displayed are modulated as the signal to be transmitted.
  • a pattern of luminance change with respect to the identification information and the time is determined, and in the transmission step, each time an image is displayed in the image display step, the identification information and time corresponding to the displayed image are determined.
  • the identification information and the time are transmitted when the luminous body changes in luminance according to the luminance change pattern determined in the above, and the identification information and the time corresponding to an image displayed in the past are further determined.
  • the identification information and the time are transmitted when the luminous body changes in luminance according to a luminance change pattern. Good.
  • each time an image is displayed a plurality of ID time information (information consisting of identification information and time) is transmitted, and therefore the receiver transmits in the past from the received plurality of ID time information.
  • the identification signal that could not be received can be easily selected based on the time included in each of the ID time information.
  • each of the light emitters has a plurality of regions that emit light, light in regions adjacent to each other among the plurality of regions interfere with each other, and only one of the plurality of regions is determined.
  • the luminance change pattern in the transmission step, only the region arranged at the end of the plurality of regions may change in luminance according to the determined luminance change pattern.
  • the receiver can appropriately capture the luminance change pattern by photographing.
  • the transmission step includes: a region disposed at an end of the plurality of regions; The area adjacent to the area arranged at the end may change in luminance according to the determined luminance change pattern.
  • the luminance of the region arranged at the end (light emitting unit) and the region adjacent to the region arranged at the end (light emitting unit) change in luminance, compared to the case where the luminance of regions separated from each other changes. It is possible to keep a wide area in which the luminance continuously changes in brightness. As a result, the receiver can appropriately capture the luminance change pattern by photographing.
  • the information communication method is an information communication method for acquiring information from a subject, a location information transmission step for transmitting location information indicating a location of an image sensor used for photographing the subject, and the location information.
  • a list receiving step for receiving an ID list including a plurality of pieces of identification information associated with the position indicated by, and an image obtained by photographing the subject by the image sensor corresponding to an exposure line included in the image sensor
  • An exposure time setting step for setting an exposure time of the image sensor so that a bright line to be generated is generated according to a change in luminance of the subject, and the subject in which the image sensor changes in luminance is photographed at the set exposure time.
  • An image acquisition step of acquiring a bright line image including the bright line and An information acquisition step of acquiring information by demodulating data specified by the bright line pattern included in the bright line image, and a search step of searching the ID list for identification information including the acquired information. Including.
  • the method may further include a re-search step of searching the ID list for identification information including the acquired information and the new information.
  • the information communication method is an information communication method for acquiring information from a subject, and an bright line corresponding to an exposure line included in the image sensor is included in an image obtained by photographing the subject with an image sensor.
  • An exposure time setting step for setting an exposure time of the image sensor so as to occur according to a change in luminance of the subject, and the image sensor photographing the subject whose luminance changes with the set exposure time,
  • An image acquisition step of acquiring a bright line image including the bright line an information acquisition step of acquiring identification information by demodulating data specified by the pattern of the bright line included in the acquired bright line image, and
  • a transmission step for transmitting the identification information and position information indicating the position of the image sensor.
  • an error that receives error notification information for notifying an error when there is no acquired identification information in an ID list that includes a plurality of identification information associated with the position indicated by the position information.
  • the user of the receiver that has received the error notification information can change the information associated with the acquired identification information. It can be easily grasped that it cannot be obtained.
  • Embodiment 4 an application example using a receiver such as a smartphone in Embodiments 1 to 4 and a transmitter that transmits information as a blinking pattern such as an LED or an organic EL will be described.
  • FIG. 30 is a diagram illustrating an example of operations of the transmitter and the receiver in the fourth embodiment.
  • the transmitter includes an ID storage unit 8361, a random number generation unit 8362, an addition unit 8363, an encryption unit 8364, and a transmission unit 8365.
  • the ID storage unit 8361 stores the ID of the transmitter.
  • the random number generation unit 8362 generates different random numbers every certain time.
  • Adder 8363 combines the latest random number generated by random number generator 8362 with the ID stored in ID storage unit 8361, and outputs the result as an edit ID.
  • the encryption unit 8364 generates an encrypted edit ID by encrypting the edit ID.
  • the transmission unit 8365 transmits the encrypted edit ID to the receiver by changing the luminance.
  • the receiver includes a receiving unit 8366, a decoding unit 8367, and an ID acquisition unit 8368.
  • the receiving unit 8366 receives the encrypted edit ID from the transmitter by imaging the transmitter (visible light imaging).
  • the decryption unit 8367 restores the edit ID by decrypting the received encrypted edit ID.
  • the ID acquisition unit 8368 acquires the ID by extracting the ID from the restored editing ID.
  • the ID storage unit 8361 stores the ID “100”, and the random number generation unit 8362 generates the latest random number “817” (Example 1).
  • the adding unit 8363 generates and outputs the edit ID “100817” by combining the random number “817” with the ID “100”.
  • the encryption unit 8364 generates an encrypted edit ID “abced” by encrypting the edit ID “100817”.
  • the decryption unit 8367 of the receiver restores the edit ID “100817” by decrypting the encrypted edit ID “abced”.
  • the ID acquisition unit 8368 extracts the ID “100” from the restored editing ID “100817”. In other words, the ID acquisition unit 8368 acquires the ID “100” by deleting the last three digits of the edit ID.
  • the random number generation unit 8362 generates a new random number “619” (example 2).
  • the adding unit 8363 generates and outputs the edit ID “100619” by combining the random number “619” with the ID “100”.
  • the encryption unit 8364 generates an encrypted edit ID “diffia” by encrypting the edit ID “100619”.
  • the decryption unit 8367 of the transmitter restores the edit ID “100619” by decrypting the encrypted edit ID “diffia”.
  • the ID acquisition unit 8368 extracts the ID “100” from the restored editing ID “100619”. In other words, the ID acquisition unit 8368 acquires the ID “100” by deleting the last three digits of the edit ID.
  • the transmitter simply encrypts a combination of random numbers that change every fixed time without simply encrypting the ID, so the ID can be easily decrypted from the signal transmitted from the transmitter 8365. Can be prevented.
  • a simple encrypted ID is transmitted from the transmitter to the receiver several times, even if the ID is encrypted, if the ID is the same, the transmitter to the receiver. Since the transmitted signal is the same, the ID may be deciphered.
  • a random number that is changed at regular intervals is combined with an ID, and the ID that is combined with the random number is encrypted. Therefore, even when the same ID is transmitted to the receiver several times, the signals transmitted from the transmitter to the receiver can be made different if the transmission timings of these IDs are different. As a result, it is possible to prevent the ID from being easily decoded.
  • the receiver shown in FIG. 30 acquires the encryption edit ID
  • the receiver may transmit the encryption edit ID to the server and acquire the ID from the server.
  • FIG. 31 shows an example of a form of use of the present invention in a train platform.
  • the user holds the portable terminal over an electronic bulletin board or lighting, and obtains information displayed on the electronic bulletin board, train information of a station where the electronic bulletin board is installed, information on the premises of the station, or the like by visible light communication.
  • the information itself displayed on the electronic bulletin board may be transmitted to the portable terminal by visible light communication, or ID information corresponding to the electronic bulletin board is transmitted to the portable terminal, and the ID information acquired by the portable terminal
  • the information displayed on the electronic bulletin board may be acquired by inquiring the server.
  • the server transmits the content displayed on the electronic bulletin board to the mobile terminal based on the ID information.
  • the train ticket information stored in the memory of the mobile terminal is compared with the information displayed on the electronic bulletin board, and the ticket information corresponding to the user's ticket is displayed on the electronic bulletin board.
  • An arrow indicating the destination to the home where the user's scheduled train arrives is displayed on the display.
  • the route to the vehicle near the exit or the transfer route may be displayed. If a seat has been designated, the route to that seat may be displayed.
  • the arrow is displayed, it can be displayed more easily by displaying the arrow using the same color as the color of the train route in the map or the train guide information.
  • the user's reservation information (home number, vehicle number, departure time, seat number) can also be displayed. By displaying the user reservation information together, it is possible to prevent erroneous recognition.
  • the ticket information is stored in the server, query the server from the mobile terminal to obtain and compare the ticket information, or compare the ticket information with the information displayed on the electronic bulletin board on the server side.
  • the target route may be estimated from the history of the user performing a transfer search, and the route may be displayed. Further, not only the contents displayed on the electronic bulletin board but also the train information / premises information of the station where the electronic bulletin board is installed may be acquired and compared.
  • Information related to the user may be highlighted with respect to the display of the electronic bulletin board on the display, or may be rewritten and displayed.
  • an arrow for guiding to the boarding place on each route may be displayed.
  • an arrow for guiding to a store or restroom may be displayed on the display.
  • the user's behavior characteristics may be managed in advance by a server, and an arrow for guiding the user to a store / restroom may be displayed on the display when the user often stops at a store / restaurant in the station.
  • FIG. 32 shows an example in which coupon information acquired by visible light communication is displayed or a popup is displayed on the display of the mobile terminal when the user approaches the store.
  • a user acquires coupon information of a store from an electronic bulletin board etc. by visible light communication using a portable terminal.
  • the coupon information of the store or a pop-up is displayed.
  • Whether or not the user has entered the predetermined range from the store is determined using the GPS information of the mobile terminal and the store information included in the coupon information. Not only coupon information but also ticket information may be used. Since it automatically alerts when a store where coupons and tickets can be used approaches, the user can use coupons and tickets appropriately.
  • FIG. 33 shows an example in which a user acquires information from a home appliance by visible light communication using a mobile terminal.
  • ID information or information related to the home appliance is acquired from the home appliance by visible light communication
  • an application for operating the home appliance is automatically started.
  • FIG. 33 shows an example using a television. With such a configuration, it is possible to start an application for operating a home appliance simply by holding the portable terminal over the home appliance.
  • FIG. 34 shows an example of the configuration of the database held by the server that manages the ID transmitted by the transmitter.
  • the database has an ID-data table that holds data to be passed in response to an inquiry using the ID as a key, and an access log table that stores a record of the inquiry using the ID as a key.
  • the ID sent by the transmitter, the data passed in response to an inquiry using the ID as a key, the conditions for passing the data, the number of times the access was made using the ID as a key, and the data are passed with the conditions cleared Have the number of times.
  • the conditions for passing data include the date and time, the number of accesses, the number of successful accesses, the information of the terminal of the inquiry source (terminal model, the application that made the inquiry, the current location of the terminal, etc.) Age, gender, occupation, nationality, language, religion, etc.).
  • the log table clears the ID, requested user ID, time, other incidental information, whether or not the data is passed, and the passed data Record the contents of.
  • FIG. 35 is a diagram illustrating an example of operation of a transmitter and a receiver in Embodiment 4.
  • the receiver 8420a receives zone information from the base station 8420h, recognizes in which zone it is located, and selects a reception protocol.
  • the base station 8420h is configured as, for example, a mobile phone communication base station, a Wi-Fi access point, an IMES transmitter, a speaker, or a wireless transmitter (Bluetooth (registered trademark), ZigBee, specific low power wireless station, etc.).
  • the receiver 8420a may specify a zone based on position information obtained from GPS or the like. As an example, it is assumed that communication is performed at a signal frequency of 9.6 kHz in zone A, and communication is performed at a signal frequency of 15 kHz for ceiling lighting and signage of 4.8 kHz in zone B.
  • the receiver 8420a recognizes that the current location is zone A from the information of the base station 8420h, performs reception at a signal frequency of 9.6 kHz, and receives signals transmitted from the transmitters 8420b and 8420c.
  • the receiver 8420a recognizes that the current location is zone B from the information of the base station 8420i at the position 8420l, and is further trying to receive a signal from the ceiling lighting because the in-camera is directed upward. Is received at a signal frequency of 15 kHz, and signals transmitted by the transmitters 8420e and 8420f are received.
  • the receiver 8420a recognizes that the current location is the zone B from above the base station 8420i, and further estimates that it is trying to receive a signal transmitted by the signage from the movement of the out camera. Receive at a signal frequency of .8 kHz and receive the signal transmitted by transmitter 8420g. The receiver 8420a receives the signals of both the base station 8420h and the base station 8420i at the position 8420k, and cannot determine whether the current location is the zone A or the zone B. Therefore, the reception process is performed at both 9.6 kHz and 15 kHz. I do.
  • the part where the protocol differs depending on the zone is not limited to the frequency, but may be the server that inquires about the modulation method, signal format, and ID of the transmission signal.
  • the base stations 8420h and 8420i may transmit the protocol in the zone to the receiver, or may transmit only the ID indicating the zone, and the receiver may acquire the protocol information from the server using the zone ID as a key. Good.
  • the transmitters 8420b to 8420f receive the zone ID and protocol information transmitted by the base stations 8420h and 8420i, and determine the signal transmission protocol.
  • a transmitter 8420d capable of receiving signals transmitted by both base station 8420h and base station 8420i utilizes a base station zone protocol with stronger signal strength, or alternately uses both protocols.
  • FIG. 36 is a diagram illustrating an example of operation of a receiver and a transmitter in Embodiment 4.
  • the receiver 8421a recognizes the zone to which it belongs from the received signal.
  • the receiver 8421a provides services (coupon distribution, point assignment, route guidance, etc.) determined for each zone.
  • the receiver 8421a receives a signal transmitted from the left side of the transmitter 8421b and recognizes that it is in the zone A.
  • the transmitter 8421b may transmit different signals depending on the transmission direction.
  • the transmitter 8421b may transmit a signal such that a different signal is received according to the distance to the receiver by using a signal having a light emission pattern such as 2217a.
  • the receiver 8421a may recognize the positional relationship with the transmitter 8421b from the direction and size in which the transmitter 8421b is imaged, and may recognize the zone in which the receiver 8421a is located.
  • a part of the signal indicating that it is located in the same zone may be shared. For example, IDs representing zone A transmitted from the transmitter 8421b and the transmitter 8421c are common to the first half. Accordingly, the receiver 8421a can recognize the zone where the receiver 8421a is located only by receiving the first half of the signal.
  • the information communication method in the present embodiment is an information communication method for transmitting a signal by a luminance change, and a determination step for determining a plurality of luminance change patterns by modulating each of a plurality of transmission target signals;
  • Each of the plurality of light emitters changes a luminance according to any one of the determined plurality of luminance change patterns, thereby transmitting a transmission target signal corresponding to any one of the patterns.
  • a transmission step wherein two or more of the plurality of light emitters each have two types of brightness different from each other for each time unit predetermined for the light emitter.
  • the time unit predetermined for each of the two or more light emitters is set so that either one of the two lights is output. Differently, luminance changes at different frequencies to.
  • each of the two or more light emitters changes in luminance at a different frequency
  • a signal to be transmitted for example, the ID of the light emitter
  • each of the plurality of light emitters changes in luminance at any one of at least four frequencies, and two or more light emitters of the plurality of light emitters are:
  • the luminance may be changed at the same frequency.
  • the transmitting step when the plurality of light emitters are projected onto a light receiving surface of an image sensor for receiving the plurality of transmission target signals, all the light emitters adjacent to each other on the light receiving surface.
  • Each of the plurality of light emitters changes in luminance so that the frequency of the luminance change differs between them.
  • the number of frequency types is the number of illuminants. Even in the case where the number is smaller, the frequency of the luminance change can be surely made different among all the light emitters adjacent to each other on the light receiving surface of the image sensor based on the four-color problem or the four-color theorem. As a result, the receiver can easily distinguish and acquire each of the transmission target signals transmitted from the plurality of light emitters.
  • each of the plurality of light emitters may transmit the signal to be transmitted by changing in luminance at a frequency specified by a hash value of the signal to be transmitted.
  • each of the plurality of light emitters changes in luminance at a frequency specified by a hash value of a signal to be transmitted (for example, the ID of the light emitter)
  • a signal to be transmitted for example, the ID of the light emitter
  • the information communication method may further calculate a frequency corresponding to the transmission target signal as a first frequency from the transmission target signal stored in the signal storage unit according to a predetermined function.
  • a calculation step; a frequency determination step for determining whether or not the second frequency stored in the frequency storage unit matches the calculated first frequency; and the first frequency and the second frequency A frequency error notification step of notifying an error when it is determined that the frequency does not match, and a determination step when it is determined that the first frequency and the second frequency match.
  • a luminance change pattern is determined by modulating the transmission target signal stored in the signal storage unit, and in the transmission step, the plurality of light emitters are detected. Of any one of the light emitters according to the determined the pattern, by changing the luminance in the first frequency may transmit a signal of the transmission target stored in the signal storage unit.
  • the frequency stored in the frequency storage unit matches the frequency calculated from the signal to be transmitted stored in the signal storage unit (ID storage unit).
  • the information communication method further includes a check value calculation step of calculating a first check value from a transmission target signal stored in the signal storage unit according to a predetermined function, and a check value storage unit.
  • a check value determination step for determining whether or not the stored second check value matches the calculated first check value; and the first check value and the second check value.
  • a check value error notification step of notifying an error when it is determined that they do not match, and a determination step when it is determined that the first check value and the second check value match
  • a luminance change pattern is determined by modulating the signal to be transmitted stored in the signal storage unit.
  • the plurality of light emitters are determined. Any one of the emitters of blood, by changing luminance in accordance with the determined the pattern, may transmit a signal of the transmission target stored in the signal storage unit.
  • check value stored in the check value storage unit matches the check value calculated from the transmission target signal stored in the signal storage unit (ID storage unit), If it is determined that they do not coincide with each other, an error is notified, so that it is possible to easily detect abnormality of the signal transmission function by the light emitter.
  • the information communication method is an information communication method for acquiring information from a subject, and corresponds to a plurality of exposure lines included in the image sensor in an image obtained by photographing the subject by an image sensor.
  • An exposure time setting step for setting an exposure time of the image sensor so that a plurality of bright lines are generated according to a change in luminance of the subject, and the exposure time set for the subject whose luminance is changed by the image sensor.
  • a frequency specifying step of specifying the frequency of the luminance change of the subject For example, in the frequency specifying step, a plurality of header patterns, which are a plurality of predetermined patterns for indicating a header, are included in the plurality of bright line patterns, and the number of pixels between the plurality of header patterns is determined. Is determined as the frequency of luminance change of the subject.
  • the frequency of the luminance change of the subject is specified, when a plurality of subjects having different luminance change frequencies are photographed, information from these subjects can be easily distinguished and acquired.
  • the bright line image including a plurality of patterns each represented by a plurality of bright lines is acquired by photographing a plurality of subjects each changing in luminance, and the information acquisition step acquires the bright line image.
  • the plurality of patterns Information may be acquired from each of the patterns.
  • a plurality of bright line images are acquired by photographing the plurality of subjects a plurality of times at mutually different timings, and in the frequency specifying step, the bright line images are included in the bright line image.
  • a frequency for each of a plurality of patterns is specified, and in the information acquisition step, a plurality of patterns in which the same frequency is specified is searched from a plurality of previous bright line images, and the searched plurality of patterns are combined and combined.
  • the information may be obtained by demodulating data specified by the plurality of patterns.
  • a plurality of patterns in which the same frequency is specified are searched from a plurality of emission line images, the plurality of searched patterns are combined, and information is acquired from the combined patterns. Therefore, even when a plurality of subjects are moving, information from the plurality of subjects can be easily distinguished and acquired.
  • the information communication method may further include: identifying information on the subject included in the information acquired in the information acquisition step; and the frequency specification for a server in which frequencies are registered for each of the identification information.
  • related information associated with the identification information (ID) acquired based on the luminance change of the subject (transmitter) and the frequency of the luminance change is acquired. Therefore, by changing the frequency of the luminance change of the subject and updating the frequency registered in the server to the frequency after the change, the receiver that acquired the identification information before the frequency change acquires the related information from the server. Can be prevented. That is, by changing the frequency registered in the server in accordance with the change in the luminance change frequency of the subject, the receiver that has acquired the subject identification information in the past can acquire the related information from the server indefinitely. It can prevent becoming a state.
  • the information communication method is further acquired in the identification information acquisition step of acquiring the identification information of the subject by extracting a part from the information acquired in the information acquisition step, and in the information acquisition step.
  • a setting frequency specifying step of specifying a number indicated by the remaining part other than the part of the information as a setting frequency of the luminance change set for the subject may be included.
  • the information obtained from the plurality of bright line patterns can include the identification information of the subject and the set frequency of the luminance change set for the subject without depending on each other. Can increase the degree of freedom.
  • FIG. 37 is a diagram illustrating an example of operation of a transmitter in Embodiment 5.
  • the light emitting unit of the transmitter 8921a repeats blinking and visible light communication that are visible to humans, as shown in FIG. By performing blinking that is visible to humans, it is possible to inform humans that visible light communication is possible.
  • the user notices that visible light communication is possible because the transmitter 8921a is blinking, performs visible light communication with the receiver 8921b directed to the transmitter 8921a, and performs user registration of the transmitter 8921a.
  • the transmitter in the present embodiment alternately and repeatedly performs a step in which the light emitter transmits a signal due to a change in luminance and a step in which the light emitter blinks so as to be visually recognized by human eyes.
  • the transmitter may separately provide a visible light communication unit and a blinking unit (communication status display unit) as shown in FIG.
  • the light emitting unit is blinking while performing visible light communication. That is, for example, the transmitter repeatedly performs high luminance visible light communication with a brightness of 75% and low luminance visible light communication with a brightness of 1% alternately.
  • the operation shown in (c) of FIG. 37 is performed to alert the user without stopping the visible light communication. Can do.
  • FIG. 38 is a diagram illustrating an example of application of the transmission and reception system in the fifth embodiment.
  • the receiver 8955a receives, for example, the transmission ID of the transmitter 8955b configured as a guide plate, acquires the map data displayed on the guide plate from the server, and displays the map data.
  • the server may transmit an advertisement suitable for the user of the receiver 8955a, and the receiver 8955a may also display this advertisement information.
  • the receiver 8955a displays a route from the current location to a location designated by the user.
  • FIG. 39 is a diagram illustrating an example of application of the transmission / reception system in the fifth embodiment.
  • the receiver 8957a receives the ID transmitted from the transmitter 8957b configured as a signboard, for example, acquires coupon information from the server, and displays the coupon information.
  • the receiver 8957a stores subsequent user actions such as saving a coupon, moving to a store displayed on the coupon, shopping at the store, and leaving without saving the coupon. Save to 8957c.
  • the subsequent behavior of the user who has acquired information from the sign 8957b can be analyzed, and the advertising value of the sign 8957b can be estimated.
  • FIG. 40 is a diagram illustrating an example of application of the transmission and reception system in the fifth embodiment.
  • the transmitter 8960b configured as a projector or a display transmits information (SSID, password for wireless connection, IP address, password for operating the transmitter) for wireless connection to itself.
  • information SSID, password for wireless connection, IP address, password for operating the transmitter
  • an ID serving as a key for accessing these pieces of information is transmitted.
  • the receiver 8960a configured as a smartphone, a tablet, a laptop computer, or a camera receives the signal transmitted from the transmitter 8960b, acquires the information, and establishes a wireless connection with the transmitter 8960b.
  • This wireless connection may be connected via a router, or may be directly connected by Wi-Fi Direct, Bluetooth (registered trademark), Wireless Home Digital Interface, or the like.
  • the receiver 8960a transmits a screen displayed by the transmitter 8960b. Thereby, the image of the receiver can be easily displayed on the transmitter.
  • the transmitter 8960b When the transmitter 8960b is connected to the receiver 8960a, the transmitter 8960b notifies the receiver 8960a that a password is required in addition to the information transmitted by the transmitter in order to display the screen. If is not sent, the transmitted screen may not be displayed. At this time, the receiver 8960a displays a password input screen such as 8960d and allows the user to input the password.
  • the information communication method according to one or more aspects has been described based on the embodiment.
  • the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.
  • an information communication method according to an aspect of the present invention may be applied.
  • FIG. 41 is a diagram showing an example of application of the transmission / reception system in the fifth embodiment.
  • a camera configured as a receiver for visible light communication performs imaging in a normal imaging mode (Step 1).
  • the camera acquires an image file configured in a format such as EXIF (Exchangeable image file format).
  • the camera performs imaging in the visible light communication imaging mode (Step 2).
  • the camera acquires a signal (visible light communication information) transmitted by the visible light communication from the transmitter that is the subject (Step 3).
  • the camera obtains information corresponding to the key from the server by using the signal (reception information) as a key and accessing the server (Step 4).
  • the camera transmits a signal (visible light reception data) transmitted from the subject by visible light communication, information acquired from the server, and data indicating a position where the transmitter as the subject is projected in the image indicated by the image file. And the data indicating the time when the signal transmitted by the visible light communication is received (the time in the moving image) are stored as metadata in the above-described image file. Note that when a plurality of transmitters are projected as subjects in an image (image file) obtained by imaging, the camera stores, for each transmitter, some metadata corresponding to the transmitter. Save to file.
  • a display or projector configured as a transmitter for visible light communication displays an image indicated by the above-described image file
  • the display or projector transmits a signal corresponding to metadata included in the image file by visible light communication.
  • the display or the projector may transmit the metadata itself by visible light communication, or may transmit a signal associated with the transmitter displayed in the image as a key.
  • a portable terminal configured as a receiver for visible light communication receives a signal transmitted by visible light communication from the display or projector by capturing an image of the display or projector. If the received signal is the above-described key, the portable terminal uses the key to acquire the transmitter metadata associated with the key from the display, projector, or server. In addition, if the received signal is a signal (visible light reception data or visible light communication information) transmitted from an actual transmitter by visible light communication, the portable terminal receives the visible light from a display, a projector, or a server. Information corresponding to received light data or visible light communication information is acquired.
  • the information communication method in the present embodiment is an information communication method for acquiring information from a subject, and each exposure included in the image sensor is included in an image obtained by photographing the first subject that is the subject by an image sensor.
  • a first exposure time setting step for setting a first exposure time of the image sensor so that a plurality of bright lines corresponding to a line are generated according to a change in luminance of the first subject;
  • a first bright line image acquisition step of acquiring a first bright line image that is an image including the plurality of bright lines by photographing the first subject that changes with the set first exposure time;
  • the first transmission information is acquired by demodulating data specified by the pattern of the plurality of bright lines included in the acquired first bright line image.
  • An information obtaining step of, after said first transmission information is obtained by sending a control signal, and a door control steps to open the door against the opening and closing devices of the door.
  • the information communication method may further include a second image in which the image sensor includes a plurality of bright lines by photographing the second subject whose luminance changes with the set first exposure time.
  • Second transmission information is acquired by demodulating data specified by a pattern of the plurality of bright lines included in the acquired second bright line image and a second bright line image acquiring step of acquiring a bright line image A second information acquisition step; and an approach determination step of determining whether or not a receiving device including the image sensor is approaching the door based on the acquired first and second transmission information.
  • the control signal may be transmitted when it is determined that the receiving device is approaching the door.
  • the door can be opened only when the receiving device (receiver) approaches the door, that is, only at an appropriate timing.
  • a second exposure time setting step for setting a second exposure time longer than the first exposure time and the image sensor sets a third subject.
  • the charge is read after a predetermined time has elapsed from the time when the charge is read for the exposure line adjacent to the exposure line, and the first bright line image obtaining step is performed. Then, without using the optical black for the charge readout, the optical black in the image sensor is different from the optical black.
  • the charge is read after a time longer than the predetermined time from when the charge is read for the exposure line adjacent to the exposure line. Also good.
  • the readout (exposure) of charges from the optical black is not performed. Therefore, the readout (exposure) of charges from the effective pixel area, which is an area other than the optical black in the image sensor, is performed. This time can be lengthened. As a result, it is possible to increase the time for receiving a signal in the effective pixel region, and it is possible to acquire many signals.
  • the length in the direction perpendicular to each of the plurality of bright lines in the plurality of bright line patterns included in the first bright line image is less than a predetermined length.
  • the frame rate is reduced and the bright line is renewed.
  • An image is acquired as a third bright line image.
  • the length of the bright line pattern included in the third bright line image can be increased, and the transmitted signal can be acquired for one block.
  • the information communication method further includes a ratio setting step for setting a ratio between a vertical width and a horizontal width of an image obtained by the image sensor, and the first bright line image acquisition step includes the set ratio.
  • a clipping determination step for determining whether or not an end in a direction perpendicular to each exposure line in the image is clipped, and when it is determined that the end is clipped, the ratio set in the ratio setting step Changing the ratio to a non-clipping ratio which is a ratio at which the edge is not clipped, and the image sensor captures the first bright line with the non-clipping ratio by photographing the first subject whose luminance changes.
  • the ratio of the horizontal width to the vertical width of the effective pixel area of the image sensor is 4: 3
  • the ratio of the horizontal width to the vertical width of the image is set to 16: 9, and a bright line along the horizontal direction appears. That is, when the exposure line is along the horizontal direction, it is determined that the upper end and the lower end of the image are clipped. That is, it is determined that the end of the first bright line image is missing.
  • the ratio of the image is changed to 4: 3, which is a ratio that is not clipped.
  • the information communication method further includes a compression step of generating a compressed image by compressing the first bright line image in a direction parallel to each of the plurality of bright lines included in the first bright line image. And a compressed image transmission step of transmitting the compressed image.
  • the information communication method further determines that the receiving device including the image sensor has been moved in a predetermined manner, and a gesture determination step for determining whether or not the receiving device has been moved in a predetermined manner.
  • a gesture determination step for determining whether or not the receiving device has been moved in a predetermined manner.
  • an activation step of activating the image sensor may be included.
  • FIG. 42 is a diagram illustrating an application example of the transmitter and the receiver in the sixth embodiment.
  • the robot 8970 has, for example, a function as a self-propelled cleaner and a function as a receiver in each of the above embodiments.
  • the lighting devices 8971a and 8971b each have a function as a transmitter in each of the above embodiments.
  • the robot 8970 performs cleaning while moving in the room and photographs the lighting device 8971a that illuminates the room.
  • the lighting device 8971a transmits the ID of the lighting device 8971a by changing the luminance.
  • the robot 8970 receives the ID from the lighting device 8971a and estimates its own position (self-position) based on the ID as in the above embodiments. That is, the robot 8970 moves itself based on the detection result by the 9-axis sensor, the relative position of the lighting device 8971a reflected in the image obtained by photographing, and the absolute position of the lighting device 8971a specified by the ID. Is estimated.
  • the robot 8970 when the robot 8970 moves away from the lighting device 8971a by moving, the robot 8970 transmits a signal to turn off the lighting device 8971a (turn-off command). For example, when the robot 8970 leaves the lighting device 8971a by a predetermined distance, the robot 8970 transmits a turn-off command. Alternatively, the robot 8970 transmits a turn-off command to the lighting device 8971a when the lighting device 8971a does not appear in the image obtained by shooting or when another lighting device appears in the image. When the lighting device 8971a receives a turn-off command from the robot 8970, the lighting device 8971a turns off according to the turn-off command.
  • turn-off command For example, when the robot 8970 leaves the lighting device 8971a by a predetermined distance, the robot 8970 transmits a turn-off command. Alternatively, the robot 8970 transmits a turn-off command to the lighting device 8971a when the lighting device 8971
  • the robot 8970 detects that it has approached the lighting device 8971b based on the estimated self-position while moving and cleaning. That is, the robot 8970 holds information indicating the position of the lighting device 8971b, and when the distance between the self position and the position of the lighting device 8971b is equal to or less than a predetermined distance, the lighting device 8971b. Detecting that you are approaching. Then, the robot 8970 transmits a signal (lighting command) for instructing lighting to the lighting device 8971b. When the lighting device 8971b receives the lighting command, the lighting device 8971b lights up in accordance with the lighting command.
  • the robot 8970 can brighten only the surroundings while moving and can easily perform cleaning.
  • FIG. 43 is a diagram illustrating an application example of the transmitter and the receiver in the sixth embodiment.
  • the lighting device 8974 has a function as a transmitter in each of the above embodiments.
  • the lighting device 8974 illuminates a route bulletin board 8975 at a railway station, for example, while changing in luminance.
  • the receiver 8973 pointed to the route bulletin board 8975 by the user photographs the route bulletin board 8975.
  • the receiver 8973 acquires the ID of the route bulletin board 8975, and acquires detailed information about each route described in the route bulletin board 8975, which is information associated with the ID.
  • the receiver 8973 displays a guide image 8973a indicating the detailed information.
  • the guidance image 8973a indicates the distance to the route described on the route bulletin board 8975, the direction toward the route, and the time when the next train arrives on the route.
  • the receiver 8973 displays a supplementary guide image 8973b.
  • This supplementary guide image 8973b is, for example, a user's selection operation of any one of a railway timetable, information on another route different from the route indicated by the guide image 8973a, and detailed information on the station. It is an image for displaying accordingly.
  • FIG. 44 is a diagram illustrating an example of a receiver in Embodiment 7.
  • the receiver 9020a configured as a wristwatch includes a plurality of light receiving units.
  • the receiver 9020a is arranged in the vicinity of the character indicating 12 o'clock in the light receiving portion 9020b arranged at the upper end portion of the rotating shaft that supports the long hand and the short hand of the watch.
  • Light receiving portion 9020c The light receiving unit 9020b receives light traveling toward the light receiving unit 9020b along the direction of the rotation axis described above, and the light receiving unit 9020c transmits light traveling toward the light receiving unit 9020c along the direction connecting the rotation axis and the character indicating 12:00. Receive.
  • the light receiving unit 9020b can receive light from above.
  • the receiver 9020a can receive a signal from the ceiling lighting.
  • the light receiving unit 9020c can receive light from the front direction.
  • the receiver 9020a can receive a signal from a signage or the like at the front.
  • These light receiving units 9020b and 9020c have directivity so that signals can be received without interference even when there are a plurality of transmitters at close positions.
  • FIG. 45 is a diagram illustrating an example of a reception system in the seventh embodiment.
  • the receiver 9023b configured as a wristwatch is connected to the smartphone 9022a via wireless communication such as Bluetooth (registered trademark).
  • the receiver 9023b has a dial made up of a display such as a liquid crystal display, and can display information other than the time.
  • the smartphone 9022a recognizes the current location from the signal received by the receiver 9023b, and displays the route and distance to the destination on the display surface of the receiver 9023b.
  • FIG. 46 is a diagram illustrating an example of a signal transmission / reception system according to the seventh embodiment.
  • the signal transmission / reception system includes a smartphone (smartphone) that is a multi-function mobile phone, an LED light emitting device that is a lighting device, a home appliance such as a refrigerator, and a server.
  • the LED light emitter performs communication using BTLE (Bluetooth (registered trademark) Low Energy) and visible light communication using LED (Light-Emitting-Diode).
  • BTLE Bluetooth (registered trademark) Low Energy)
  • LED Light-Emitting-Diode
  • the LED light emitter controls a refrigerator or communicates with an air conditioner by BTLE.
  • the LED light emitter controls a power source of a microwave oven, an air purifier, a television (TV), or the like by visible light communication.
  • the television includes, for example, a solar power generation element, and uses the solar power generation element as an optical sensor. That is, when a signal is transmitted when the brightness of the LED light emitter changes, the television detects a change in the brightness of the LED light emitter based on a change in the power generated by the solar power generation element. Then, the television acquires the signal transmitted from the LED light emitter by demodulating the signal indicated by the detected luminance change.
  • the signal is a command indicating that the power is on
  • the television switches its main power source to ON
  • the signal is a command indicating that the power source is OFF
  • the television switches its main power source to OFF.
  • the server can communicate with the air conditioner via a router and a specific low power radio station (extra small). Furthermore, since the air conditioner can communicate with the LED light emitter via BTLE, the server can communicate with the LED light emitter. Therefore, the server can switch the power of the TV ON and OFF via the LED light emitter.
  • the smartphone can control the power supply of the TV via the server by communicating with the server via, for example, Wi-Fi (Wireless Fidelity).
  • the information communication method allows the mobile terminal (smart phone) to transmit a control signal (transmission data string or user command) by wireless communication (such as BTLE or Wi-Fi) different from visible light communication.
  • a visible light communication step in which the lighting device performs visible light communication by changing the luminance according to the control signal, and a control target device (such as a microwave oven). Includes an execution step of detecting a luminance change of the lighting device, acquiring a control signal by demodulating a signal specified by the detected luminance change, and executing a process according to the control signal.
  • the mobile terminal may be a wristwatch instead of a smartphone.
  • FIG. 47 is a flowchart showing a reception method in which interference is eliminated in the seventh embodiment.
  • step 9001a start is made in step 9001a, and it is checked whether there is a periodic change in the intensity of light received in step 9001b. If YES, the process proceeds to step 9001c. In the case of NO, the process proceeds to Step 9001d to receive a wide range of light by setting the lens of the light receiving unit to a wide angle, and returns to Step 9001b. In step 9001c, it is confirmed whether the signal can be received. If YES, the process proceeds to step 9001e, the signal is received, and the process ends in step 9001g. In the case of NO, the process proceeds to Step 9001f, the lens of the light receiving unit is telephoto, and light in a narrow range is received, and the process returns to Step 9001c.
  • This method makes it possible to receive signals from transmitters in a wide direction while eliminating signal interference from a plurality of transmitters.
  • FIG. 48 is a flowchart showing a method for estimating the orientation of a transmitter in the seventh embodiment.
  • step 9002a the lens of the light receiving unit is set to the maximum telephoto in step 9002b, and it is checked whether there is a periodic change in the intensity of light received in step 9002c. Proceed to 9002d. In the case of NO, the process proceeds to Step 9002e, where a wide range of light is received by setting the lens of the light receiving unit to a wide angle, and the process returns to Step 9002c.
  • step 9002d the signal is received, in step 9002f, the lens of the light receiving unit is set to the maximum telephoto position, the light receiving direction is changed along the boundary of the light receiving range, and the direction in which the light receiving intensity is maximized is detected. Estimating that it is in the direction, the process ends at step 9002d.
  • the maximum wide angle may be set first, and the telephoto may be gradually increased.
  • FIG. 49 is a flowchart showing a reception start method according to the seventh embodiment.
  • step 9003a start is performed in step 9003a, and in step 9003b, it is confirmed whether a signal from a base station such as Wi-Fi, Bluetooth (registered trademark), IMES or the like is received. If YES, the process proceeds to step 9003c. If NO, the process returns to step 9003b. In step 9003c, it is confirmed whether or not the base station is registered in the receiver or server as a trigger for starting reception. If YES, the process proceeds to step 9003d, starts receiving a signal, and ends in step 9003e. . If NO, the process returns to step 9003b.
  • a base station such as Wi-Fi, Bluetooth (registered trademark), IMES or the like
  • This method can start reception without the user performing a reception start operation. Further, power consumption can be suppressed as compared with the case where reception is always performed.
  • FIG. 50 is a flowchart showing an ID generation method using information of another medium together in the seventh embodiment.
  • step 9004a ID of carrier communication network, Wi-Fi, Bluetooth (registered trademark), etc. connected in step 9004b, or position information obtained from the ID, position information obtained from GPS, etc. Is transmitted to the upper bit ID index server.
  • step 9004c the upper bits of the visible light ID are received from the upper bit ID index server, and in step 9004d, the signal from the transmitter is received as the lower bits of the visible light ID.
  • step 9004e the upper and lower bits of the visible light ID are combined and transmitted to the ID resolution server, and the process ends in step 9004f.
  • This method makes it possible to obtain upper bits that are commonly used near the receiver and reduce the amount of data transmitted by the transmitter.
  • the receiving speed of the receiver can be increased.
  • the transmitter may transmit both the upper and lower bits.
  • the receiver using this method can synthesize the ID when the lower bit is received, and the receiver not using this method obtains the ID by receiving the entire ID from the transmitter. .
  • FIG. 51 is a flowchart showing a reception method selection method based on frequency separation in the seventh embodiment.
  • step 9005a start is performed in step 9005a, and the optical signal received in step 9005b is applied to a frequency filter circuit, or discrete Fourier series expansion is performed to perform frequency decomposition.
  • step 9005c it is confirmed whether or not a low frequency component exists. If YES, the process proceeds to step 9005d, a signal expressed in a low frequency region such as frequency modulation is decoded, and the process proceeds to step 9005e. If NO, the process proceeds to step 9005e. In step 9005e, it is confirmed whether or not the base station is registered in the receiver or server as a trigger for starting reception. If YES, the process proceeds to step 9005f, and a signal expressed in a high frequency region such as pulse position modulation. And the process proceeds to Step 9005g. If NO, the process advances to step 9005g. In step 9005g, signal reception is started, and in step 9005h, the process ends.
  • This method makes it possible to receive signals modulated by a plurality of modulation schemes.
  • FIG. 52 is a flowchart showing a signal reception method when the exposure time is long in the seventh embodiment.
  • step 9030a start in step 9030a, and if the sensitivity can be set in step 9030b, set the sensitivity to the maximum. If the exposure time can be set in step 9030c, the exposure time is set shorter than that in the normal shooting mode.
  • step 9030d two images are picked up and a difference in luminance is obtained. If the position or direction of the imaging unit changes during the imaging of two images, the change is canceled and an image as if it was captured from the same position and direction is generated to obtain the difference.
  • Step 9030e a value obtained by averaging the luminance values in the direction parallel to the difference image or the exposure line of the captured image is obtained.
  • step 9030f The averaged values in step 9030f are arranged in the direction perpendicular to the exposure line, and discrete Fourier transform is performed.
  • step 9030g it is recognized whether there is a peak near a predetermined frequency, and the process ends in step 9030h.
  • This method allows signals to be received even when the exposure time is long, such as when the exposure time cannot be set or when a normal image is captured simultaneously.
  • the exposure time is set automatically, when the camera is directed to a transmitter configured as illumination, the exposure time is set to about 1/60 second to 1/480 second by the automatic exposure correction function. If the exposure time cannot be set, a signal is received under this condition.
  • the illumination is blinked periodically, if the time of one cycle is about 1/16 or more of the exposure time, stripes can be visually recognized in the direction perpendicular to the exposure line. I was able to recognize it. At this time, since the brightness is too high in the portion where the illumination is reflected and it is difficult to confirm the stripe, it is preferable to obtain the signal period from the portion where the illumination light is reflected.
  • a method of periodically turning on / off the light emitting unit such as a frequency shift keying method or a frequency multiplexing modulation method
  • a frequency shift keying method or a frequency multiplexing modulation method even if the modulation frequency is the same as that of the pulse position modulation method, It is difficult for the viewer to see the flicker, and it is difficult for the flicker to appear in the video shot with the video camera. Therefore, a low frequency can be used as the modulation frequency. Since the temporal resolution of human vision is about 60 Hz, a frequency higher than this frequency can be used as the modulation frequency.
  • the modulation frequency is an integral multiple of the imaging frame rate of the receiver
  • the imaging frame rate of the receiver is usually 30 fps
  • reception is easy if the modulation frequency is set to a value other than an integral multiple of 30 Hz.
  • two disjoint modulation frequencies are assigned to the same signal, and the transmitter receives signals by alternately using the two modulation frequencies for transmission. The machine can easily restore the signal by receiving at least one signal.
  • FIG. 53 is a diagram illustrating an example of a transmitter dimming (adjusting brightness) method.
  • the average luminance changes and the brightness can be adjusted.
  • the period T 1 which repeats high and low brightness constant it is possible to keep the frequency peak constant.
  • transmission is performed while the time T1 between the first luminance change that becomes brighter than the average luminance and the second luminance change is kept constant.
  • the time for lighting brighter than the average brightness is shortened.
  • the illumination time is set longer than the average luminance.
  • 53 (b) and 53 (c) are dimmed darker than (a), and FIG. 53 (c) is dimmed the darkest. Thereby, dimming can be performed while transmitting signals having the same meaning.
  • the average brightness may be changed by changing the brightness value of the section with high brightness, the brightness of the section with low brightness, or both brightness values.
  • FIG. 54 is a diagram showing an example of a method for configuring the dimming function of the transmitter.
  • the dimming correction unit holds the correction value, and the dimming control unit controls the brightness of the light emitting unit according to the correction value.
  • the dimming control unit uses the changed dimming setting value and the correction value held in the dimming correction unit. In addition, the brightness of the light emitting unit is controlled.
  • the dimming control unit transmits the dimming setting value to another transmitter through the interlocking dimming unit.
  • the dimming control unit based on the dimming setting value and the correction value held in the dimming correction unit, To control the brightness.
  • a control method for controlling an information communication device that transmits a signal by changing luminance of a light emitter, and includes a plurality of different signals for a computer of the information communication device.
  • a control method may include a transmission step of transmitting a signal to be transmitted by changing the luminance of the light emitter so as to include only the pattern.
  • the determining step is configured such that the number of transmissions for transmitting one signal among a plurality of different signals is different from the number of transmissions for transmitting other signals within a predetermined time.
  • the number of transmissions may be determined.
  • the flickering at the time of transmission can be prevented because the number of transmissions for transmitting one signal is different from the number of transmissions for transmitting other signals.
  • the determining step may increase the number of transmissions of a signal corresponding to a high frequency within a predetermined time, compared to the number of transmissions of other signals.
  • the luminance change pattern may be a pattern in which the waveform of the luminance change over time is any one of a rectangular wave, a triangular wave, and a sawtooth wave.
  • the time during which the luminance of the illuminant is greater than a predetermined value at a time corresponding to a single frequency You may lengthen with respect to the case where the value of the average luminance of the said light-emitting body is made small.
  • the receiver can set the exposure time to a predetermined value by using an API for setting the exposure time (which stands for application programming interface and indicates a means for using the function of the OS).
  • the visible light signal can be received stably.
  • the receiver can set the sensitivity to a predetermined value by using an API for setting the sensitivity, and stably receives a visible light signal even when the brightness of the transmission signal is dark or bright. be able to.
  • FIG. 55 is a diagram for explaining the EX zoom.
  • the zoom that is, the method of obtaining a large image, includes an optical zoom that adjusts the focal length of the lens to change the size of the image captured on the image sensor, and a digital image that interpolates the image captured on the image sensor.
  • the EX zoom can be used when the number of image sensors included in the image sensor is larger than the resolution of the captured image.
  • 32 ⁇ 24 image sensors are arranged in a matrix. That is, 32 image sensors are arranged horizontally and 24 elements are arranged vertically.
  • the image sensor Only 16 ⁇ 12 image sensors (for example, image sensors indicated by black squares in the image sensor 1080a in FIG. 55A) arranged uniformly distributed throughout 10080a are used for imaging. That is, only an odd-numbered or even-numbered image sensor is used for imaging among a plurality of image sensors arranged in the vertical direction and the horizontal direction.
  • an image 10080b having a desired resolution is obtained.
  • a subject appears on the image sensor 1008a, in order to make it easy to understand the correspondence between each imaging element and an image obtained by imaging.
  • the receiver including the image sensor 10080a captures a wide range, and searches for a transmitter or receives information from many transmitters in the image sensor 10080a. An image is picked up using only a part of the image pickup elements arranged in a distributed manner.
  • the image sensor 10080a When the receiver performs the EX zoom, as shown in FIG. 55 (b), the image sensor 10080a has a part of the image pickup elements arranged locally densely (for example, FIG. 55 (b)). Only 16 ⁇ 12 image sensors indicated by black squares in the image sensor 1080a in FIG. As a result, a part of the image 10080b corresponding to a part of the imaging elements is zoomed, and an image 10080d is obtained.
  • an EX zoom it is possible to receive a visible light signal for a long time by capturing a large image of the transmitter, the reception speed is improved, and a visible light signal can be received from a distance.
  • the number of exposure lines that receive visible light signals cannot be increased, and the reception time of visible light signals does not increase. Therefore, it is better to use other zooms as much as possible.
  • the optical zoom requires a physical movement time of the lens and the image sensor.
  • the EX zoom is performed only by electronic setting change, there is an advantage that the time required for the zoom is short.
  • the priority order of each zoom is (1) EX zoom, (2) optical zoom, and (3) digital zoom.
  • the receiver may select and use any one or a plurality of zooms according to the priority order and the necessity of the zoom magnification.
  • image noise can be suppressed by using an unused imaging device.
  • the exposure time is set for each exposure line or each image sensor.
  • 56, 57, and 58 are diagrams illustrating an example of a signal reception method according to the ninth embodiment.
  • an exposure time is set for each exposure line. That is, a long exposure time for normal imaging is set for a predetermined exposure line (white exposure line in FIG. 56), and a visible light imaging for other exposure lines (black exposure line in FIG. 56).
  • a short exposure time is set. For example, a long exposure time and a short exposure time are alternately set for each exposure line arranged in the vertical direction.
  • the two exposure times may be alternately set for each line, may be set for every several lines, or different exposure times may be set for the upper part and the lower part of the image sensor 10010a.
  • a normal captured image 10010b and a bright line image showing a plurality of bright line patterns are obtained.
  • a visible light captured image 10010c is obtained.
  • the preview image is obtained by interpolating that portion. 10010d can be displayed.
  • information obtained by visible light communication can be superimposed on the preview image 10010d.
  • This information is information associated with a visible light signal obtained by decoding a plurality of bright line patterns included in the visible light captured image 10010c.
  • the receiver stores the normal captured image 10010b or an image obtained by performing interpolation on the normal captured image 10010b as a captured image, and stores the received visible light signal or information associated with the visible light signal. As additional information, it can also be added to the stored captured image.
  • an image sensor 10011a may be used instead of the image sensor 10010a.
  • the exposure time is set not for each exposure line but for each column (hereinafter, referred to as a vertical line) composed of a plurality of imaging elements arranged along a direction perpendicular to the exposure line. That is, a long exposure time for normal imaging is set for a predetermined vertical line (white vertical line in FIG. 57), and a visible light imaging for other vertical lines (black vertical line in FIG. 57). A short exposure time is set.
  • exposure is started at different timings for each exposure line, as in the image sensor 10010a.
  • the receiver obtains a normal captured image 10011b and a visible light captured image 10011c by imaging with the image sensor 10011a. Further, the receiver generates and displays a preview image 10011d based on the normal captured image 10011b and information associated with the visible light signal obtained from the visible light captured image 10011c.
  • this image sensor 10011a unlike the image sensor 10010a, all exposure lines can be used for visible light imaging. As a result, since the visible light captured image 10011c obtained by the image sensor 10011a includes more bright lines than the visible light captured image 10010c, the reception accuracy of the visible light signal can be increased.
  • an image sensor 10012a may be used instead of the image sensor 10010a.
  • the exposure time is set for each image sensor so that the same exposure time is not set continuously for each image sensor along the horizontal direction and the vertical direction. That is, the exposure time for each image sensor is such that a plurality of image sensors with a long exposure time set and a plurality of image sensors with a short exposure time are distributed like a grid or checkered pattern. Is set. Also in this case, similarly to the image sensor 10010a, the exposure is started at different timings for each exposure line, but the exposure time of each image sensor included in the exposure line is different for each exposure line.
  • the receiver obtains a normal captured image 10012b and a visible light captured image 10012c by imaging with the image sensor 10012a. Further, the receiver generates and displays a preview image 10012d based on the normal captured image 10012b and information associated with the visible light signal obtained from the visible light captured image 10012c.
  • the normal captured image 10012b obtained by the image sensor 10012a has data of a plurality of imaging elements arranged in a grid pattern or uniformly, interpolation or more accurately than the normal captured image 10010b and the normal captured image 10011b is performed. You can resize.
  • the visible light captured image 10012c is generated by imaging using all exposure lines of the image sensor 10012a. That is, in the image sensor 10012a, unlike the image sensor 10010a, all exposure lines can be used for visible light imaging. As a result, the visible light captured image 10012c obtained by the image sensor 10012a includes a larger number of bright lines than the visible light captured image 10010c, as in the visible light captured image 10011c. Can be done with precision.
  • FIG. 59 is a diagram illustrating an example of a receiver screen display method according to the ninth embodiment.
  • the receiver including the image sensor 10010a shown in FIG. 56 is set to an exposure time set for an odd-numbered exposure line (hereinafter referred to as an odd-numbered line) and an even-numbered exposure line (hereinafter referred to as an even-numbered line).
  • the exposure time is changed every predetermined time. For example, as shown in FIG. 59, at time t1, the receiver sets a long exposure time for each image sensor on the odd lines and sets a short exposure time on each image sensor on the even lines. Imaging is performed using the set exposure time. Further, at time t2, the receiver sets a short exposure time for each image sensor of the odd line, sets a long exposure time for each image sensor of the even line, and sets the set exposure time. The used imaging is performed. Then, at time t3, the receiver performs imaging using each exposure time set similarly to time t1, and uses each exposure time set similarly to time t2 at time t4. Take an image.
  • the receiver captures an image obtained from each of a plurality of odd lines by imaging (hereinafter referred to as an odd line image) and an image obtained from each of a plurality of even lines by imaging (hereinafter referred to as an even line image).
  • Image1 including the above.
  • the receiver since the exposure time is short in each of the plurality of even lines, the subject is not clearly displayed in each of the even line images. Therefore, the receiver generates a plurality of interpolated line images by interpolating pixel values for the plurality of odd line images. Then, the receiver displays a preview image including a plurality of interpolation line images instead of the plurality of even line images. That is, odd-numbered line images and interpolated line images are alternately arranged in the preview image.
  • the receiver acquires Image2 including a plurality of odd line images and even line images by imaging at time t2. At this time, since the exposure time is short in each of the plurality of odd lines, the subject is not clearly displayed in each of the odd line images. Therefore, the receiver displays a preview image including the odd line image of Image1 instead of the odd line image of Image2. That is, the odd line image of Image1 and the even line image of Image2 are alternately arranged in the preview image.
  • the receiver acquires Image3 including a plurality of odd line images and even line images by imaging.
  • the receiver displays a preview image including an even line image of Image2 instead of an even line image of Image3. That is, in the preview image, the even line image of Image2 and the odd line image of Image3 are alternately arranged.
  • the receiver acquires Image4 including a plurality of odd line images and even line images by imaging.
  • the receiver displays a preview image including the odd line image of Image3 instead of the odd line image of Image4. That is, the odd line image of Image3 and the even line image of Image4 are alternately arranged in the preview image.
  • the receiver performs so-called interlaced display, in which an image including even line images and odd line images obtained at different timings is displayed.
  • Such a receiver can display a fine preview image while performing visible light imaging.
  • the plurality of image sensors set with the same exposure time may be a plurality of image sensors arranged along the horizontal direction of the exposure line as in the image sensor 10010a, or the exposure line as in the image sensor 10011a.
  • a plurality of image sensors arranged along a direction perpendicular to the image sensor may be used, or a plurality of image sensors arranged according to a checkered pattern like the image sensor 10012a.
  • the receiver may save the preview image as imaging data.
  • FIG. 60 is a diagram illustrating an example of a signal reception method according to the ninth embodiment.
  • the ratio of the number of image sensors for which a long exposure time is set to the number of image sensors for which a short exposure time is set is 1: 1.
  • This ratio is a ratio between normal imaging and visible light imaging, and is hereinafter referred to as a spatial ratio.
  • the receiver may include an image sensor 10014a.
  • the number of image sensors with a short exposure time is larger than that of images with a long exposure time, and the spatial ratio is 1: N (N> 1).
  • the receiver may include an image sensor 10014c.
  • the image sensor with a short exposure time is smaller than the image sensor with a long exposure time, and the spatial ratio is N (N> 1): 1.
  • the receiver instead of the image sensors 10014a to 10014c, the receiver sets the exposure time for each of the above-described vertical lines, and the image sensors 10015a to 10015c have a spatial ratio of 1: N, 1: 1, or N: 1, respectively. Any of these may be provided.
  • the receiver may perform interlaced display as shown in FIG. 59 using the image sensors 10014a, 10014c, 10015a, and 10015c.
  • FIG. 61 is a diagram illustrating an example of a signal reception method according to the ninth embodiment.
  • the receiver may switch the imaging mode between the normal imaging mode and the visible light imaging mode for each frame, as shown in FIG.
  • the normal imaging mode is an imaging mode in which a long exposure time for normal imaging is set for all imaging elements of the image sensor of the receiver.
  • the visible light imaging mode is an imaging mode in which a short exposure time for visible light imaging is set for all imaging elements of the image sensor of the receiver. In this way, by switching between long and short exposure times, a preview image can be displayed by imaging with a long exposure time while receiving a visible light signal by imaging with a short exposure time.
  • the receiver ignores the image obtained by imaging with a short exposure time, and only determines the brightness of the image obtained by imaging with a long exposure time.
  • Automatic exposure may be performed with reference. Thereby, a long exposure time can be determined as an appropriate time.
  • the receiver may switch the imaging mode between the normal imaging mode and the visible light imaging mode for each set of a plurality of frames.
  • the exposure time or when it takes time to stabilize the exposure time, as shown in FIG. 61 (b)
  • Visible light imaging reception of visible light signals
  • normal imaging can be made compatible. Further, since the number of exposure time switching is reduced as the number of frames included in the set is increased, power consumption and heat generation in the receiver can be suppressed.
  • the number of at least one frame continuously generated by imaging with a long exposure time in the normal imaging mode and the at least one frame continuously generated by imaging with a short exposure time in the visible light imaging mode is not necessarily 1: 1. That is, in the cases shown in FIGS. 61A and 61B, the time ratio is 1: 1, but the time ratio may not be 1: 1.
  • the receiver may increase the number of frames in the visible light imaging mode as compared with the frame in the normal imaging mode, as illustrated in FIG. Thereby, the receiving speed of the visible light signal can be increased.
  • the frame rate of the preview image is equal to or higher than a predetermined rate, the difference in the preview image depending on the frame rate is not recognized by human eyes.
  • the imaging frame rate is sufficiently high, for example, when the frame rate is 120 fps, the receiver sets the visible light imaging mode for three consecutive frames, and then the visible light imaging for one frame. Set the mode. Thereby, the receiver can receive a visible light signal at high speed while displaying a preview image at a frame rate of 30 fps, which is sufficiently higher than the above-described predetermined rate. Further, since the number of times of switching is reduced, the effect described with reference to FIG.
  • the receiver may increase the number of frames in the normal imaging mode than that in the visible light imaging mode.
  • the preview image can be smoothly displayed by increasing the number of frames in the normal imaging mode, that is, the frames obtained by imaging with a long exposure time.
  • the number of times of receiving the visible light signal is reduced, there is a power saving effect. Further, since the number of times of switching is reduced, the effect described with reference to FIG.
  • the receiver first switches the imaging mode for each frame as in the case of (a) of FIG. 61, and then receives a visible light signal.
  • the number of frames in the normal imaging mode may be increased as in the case shown in FIG.
  • the search for a new visible light signal can be continued while the preview image is displayed smoothly. Further, since the number of times of switching is reduced, the effect described with reference to FIG.
  • FIG. 62 is a flowchart illustrating an example of a signal reception method according to the ninth embodiment.
  • the receiver starts visible light reception, which is a process of receiving a visible light signal (step S10017a), and sets the exposure time length setting ratio to a value designated by the user (step S10017b).
  • the exposure time length short / high setting ratio is at least one of the above-described space ratio and time ratio.
  • the user may specify only the space ratio, only the time ratio, or both the space ratio and the time ratio, or the receiver may automatically set regardless of the user's specification.
  • the receiver determines whether or not the reception performance is equal to or less than a predetermined value (step S10017c). If it is determined that the value is equal to or less than the predetermined value (Y in step S10017c), the receiver sets a high ratio of the short exposure time (step S10017d). Thereby, reception performance can be improved.
  • the ratio of the short exposure time is the ratio of the number of image sensors set with a short exposure time to the number of image sensors set with a long exposure time in the case of a spatial ratio. It is a ratio of the number of frames generated continuously in the visible light imaging mode to the number of frames generated continuously in the mode.
  • the receiver receives at least a part of the visible light signal and determines whether or not a priority is set for at least a part of the received visible light signal (hereinafter referred to as a received signal) (Ste S10017e).
  • a received signal an identifier indicating the priority is included in the received signal.
  • the receiver determines that the priority is set (Y in step S10017e)
  • the receiver sets the exposure time length / short ratio according to the priority (step S10017f). That is, if the priority is high, the receiver sets the ratio of the short exposure time high.
  • an emergency light configured as a transmitter emits an identifier indicating a high priority when the luminance changes. In this case, the receiver can increase the reception speed by increasing the ratio of the short exposure time, and can promptly display the evacuation route and the like.
  • the receiver determines whether or not reception of all visible light signals has been completed (step S10017g).
  • the receiver repeatedly executes the processing from step S10017c.
  • the receiver sets the ratio of the long exposure time to a high value and shifts to the power saving mode (step S10017h).
  • the ratio of the long exposure time is the ratio of the number of image sensors set with a long exposure time to the number of image sensors set with a short exposure time in the case of a spatial ratio. This is the ratio of the number of frames generated continuously in the normal imaging mode to the number of frames generated continuously in the imaging mode. As a result, the preview image can be displayed smoothly without receiving unnecessary visible light.
  • the receiver determines whether another visible light signal has been found (step S10017i).
  • the receiver determines whether another visible light signal has been found (Y in step S10017i).
  • FIG. 63 is a diagram illustrating an example of a signal reception method according to the ninth embodiment.
  • the receiver may set two or more exposure times for the image sensor. That is, as shown in FIG. 63A, each of the exposure lines included in the image sensor is continuously exposed for the longest exposure time among two or more set exposure times. For each exposure line, the receiver reads out the imaging data obtained by exposure of the exposure line when the above-described two or more set exposure times have elapsed. Here, the receiver does not reset the read image data until the longest exposure time has elapsed. Therefore, the receiver can obtain image data for a plurality of exposure times only by exposure with the longest exposure time by recording the accumulated value of the read image data. Note that the image sensor may or may not record the cumulative value of the imaging data. When the image sensor is not used, the components of the receiver that read data from the image sensor perform accumulation calculation, that is, recording the accumulated value of the imaging data.
  • the receiver captures visible light imaging data including a visible light signal generated by exposure with a short exposure time. Read out, followed by normal imaging data generated by exposure with a long exposure time.
  • visible light imaging that is imaging for receiving a visible light signal and normal imaging can be performed simultaneously, and normal imaging can be performed while receiving a visible light signal. Further, by using data of a plurality of exposure times, a signal frequency higher than the sampling theorem can be recognized, and a high-frequency signal or a high-density modulation signal can be received.
  • the receiver when outputting the imaging data, the receiver outputs a data string including the imaging data as an imaging data body, as shown in FIG. 63 (b). That is, the receiver has an imaging mode identifier indicating an imaging mode (visible light imaging or normal imaging), an imaging element identifier for specifying an imaging element or an exposure line to which the imaging element belongs, and an exposure number of an imaging data body.
  • the imaging mode identifier indicating an imaging mode (visible light imaging or normal imaging)
  • an imaging element identifier for specifying an imaging element or an exposure line to which the imaging element belongs
  • an exposure number of an imaging data body By adding additional information including an imaging data number indicating whether it is time-based imaging data and an imaging data length indicating the size of the imaging data body to the imaging data body, the above-described data string is generated and output. .
  • the respective imaging data is not always output in the order of exposure lines. Therefore, by adding the additional information shown in (b) of FIG. 63, it is possible to specify which exposure line the imaging data is.
  • FIG. 64 is a flowchart showing processing of the reception program in the ninth embodiment.
  • This reception program is a program that causes a computer provided in the receiver to execute the processes shown in FIGS. 56 to 63, for example.
  • this reception program is a reception program for receiving information from a light-emitting body that changes in luminance.
  • this reception program causes the computer to execute step SA31, step SA32, and step SA33.
  • step SA31 a first exposure time is set for some of the K image sensors (K is an integer of 4 or more) included in the image sensor, and the K image sensors are set.
  • a second exposure time shorter than the first exposure time is set for the remaining plurality of image sensors.
  • the image sensor captures the subject, which is a light-emitting body that changes in luminance, with the set first and second exposure times, and outputs from the plurality of image sensors having the first exposure time set.
  • step SA33 information is acquired by decoding a plurality of bright line patterns included in the acquired bright line image.
  • a normal image can be obtained by one imaging by the image sensor. And bright line images can be acquired. That is, it is possible to simultaneously capture a normal image and acquire information by visible light communication.
  • the first exposure time is set for a part of a plurality of image sensor rows in the L (L is an integer of 4 or more) image sensor rows included in the image sensor. Then, the second exposure time is set for the remaining plurality of image sensor rows in the L image sensor rows.
  • each of the L image sensor rows is composed of a plurality of image sensors arranged in a row included in the image sensor.
  • each of the L image sensor rows is an exposure line included in the image sensor as shown in FIG.
  • each of the L image pickup device arrays includes a plurality of image pickup devices arranged along a direction perpendicular to an exposure line included in the image sensor.
  • the same exposure time is used for each of the odd-numbered image sensor rows of the L image sensor rows included in the image sensor.
  • One of the first exposure time and the second exposure time is set, and the first exposure time and the second exposure time are the same exposure time for each of even-numbered image sensor rows of the L image sensor rows. You may set the other of time.
  • the exposure time setting step SA31, the image acquisition step SA32, and the information acquisition step SA33 are repeated, the repeated exposure time setting step SA31 is set for each of the odd-numbered image sensor rows in the previous exposure time setting step SA31.
  • the exposure time that has been set may be interchanged with the exposure time that has been set for each even-numbered imaging element array.
  • each time a normal image is acquired the plurality of image sensor rows used for the acquisition can be switched between an odd number of image sensor rows and an even number of image sensor rows.
  • each of the sequentially acquired normal images can be displayed by interlace.
  • a new normal image including an image by an odd-numbered plurality of imaging element arrays and an image by an even-numbered plurality of imaging element arrays is generated. be able to.
  • the first exposure time is set when the setting mode is switched between the normal priority mode and the visible light priority mode and can be switched to the normal priority mode.
  • the number of image sensors may be larger than the number of image sensors for which the second exposure time is set.
  • the number of image sensors for which the first exposure time is set may be smaller than the number of image sensors for which the second exposure time is set.
  • the setting mode when the setting mode is switched to the normal priority mode, the image quality of the normal image can be improved.
  • the setting mode is switched to the visible light priority mode, the reception efficiency of information from the light emitter is improved. can do.
  • a plurality of image sensors for which the first exposure time is set and a plurality of image sensors for which the second exposure time is set are in a checkered pattern (
  • the exposure time of the image sensor may be set for each image sensor included in the image sensor so as to be distributed like Checkered pattern.
  • the plurality of image pickup devices for which the first exposure time is set and the plurality of image pickup devices for which the second exposure time is set are uniformly distributed. Not normal images and bright line images can be acquired.
  • FIG. 65 is a block diagram of a receiving apparatus according to the ninth embodiment.
  • the receiving device A30 is the above-described receiver that executes the processing shown in FIGS. 56 to 63, for example.
  • the receiving device A30 is a receiving device that receives information from a light-emitting body that changes in luminance, and includes a multiple exposure time setting unit A31, an imaging unit A32, and a decoding unit A33.
  • the multiple exposure time setting unit A31 sets the first exposure time for some of the plurality of image pickup elements (K is an integer of 4 or more) included in the image sensor, and K pieces.
  • a second exposure time shorter than the first exposure time is set for the remaining plurality of image sensors.
  • the imaging unit A32 causes the image sensor to capture an image of a subject, which is a light-emitting body that changes in luminance, with the set first and second exposure times, so that a plurality of imaging elements with the first exposure time are set.
  • a bright image corresponding to each of the plurality of exposure lines included in the image sensor which is an image corresponding to the output from the plurality of imaging elements for which the second exposure time is set, while acquiring a normal image according to the output
  • the decoding unit A33 acquires information by decoding a plurality of bright line patterns included in the acquired bright line image.
  • 66 and 67 are diagrams showing an example of the display of the receiver when a visible light signal is received.
  • the receiver when the receiver captures an image of the transmitter 10020d, the receiver displays an image 10020a on which the transmitter 10020d is projected. Further, the receiver generates and displays an image 10020b by superimposing the object 10020e on the image 10020a.
  • the object 10020e is an image indicating that the image of the transmitter 10020d is present and that a visible light signal is received from the transmitter 10020d.
  • the object 10020e may be an image that varies depending on the reception state of the visible light signal (the state of reception, the state of searching for a transmitter, the degree of progress of reception, the reception speed, or the error rate).
  • the receiver changes the color of the object 1020e, the thickness of the line, the type of line (single line, double line, dotted line, or the like), or the interval between dotted lines. Thereby, a user can be made to recognize a receiving state.
  • the receiver generates and displays an image 10020c by superimposing an image indicating the content of the acquired data on the image 10020a as an acquired data image 10020f.
  • the acquired data is data associated with the received visible light signal or the ID indicated by the received visible light signal.
  • the receiver displays the acquired data image 10020f
  • the receiver displays the acquired data image 10020f like a balloon from the transmitter 10020d or near the transmitter 10020d.
  • the acquired data image 10020f is displayed.
  • the receiver may display the acquired data image 10020f so that the acquired data image 10020f gradually approaches the receiver side from the transmitter 10020d. Thereby, the user can recognize which transmitter the received data image 10020f is based on the visible light signal received from.
  • the receiver may display the acquired data image 10020f so that the acquired data image 10020f gradually emerges from the end of the receiver display. This makes it possible for the user to easily recognize that the visible light signal has been acquired at that time.
  • AR Augmented Reality
  • FIG. 68 is a diagram showing an example of display of the acquired data image 10020f.
  • the receiver moves the acquired data image 10020f in accordance with the movement of the transmitter image. This allows the user to recognize that the acquired data image 10020f corresponds to the transmitter.
  • the receiver may display the acquired data image 10020f in association with another image instead of the image of the transmitter. Thereby, AR display can be performed.
  • FIG. 69 is a diagram illustrating an example of an operation when saving or discarding acquired data.
  • the receiver when the user performs a swipe down on the acquired data image 10020f, the receiver stores the acquired data indicated by the acquired data image 10020f. .
  • the receiver places an acquired data image 10020f indicating the stored acquired data at the end of the acquired data image indicating one or more other already stored acquired data. This allows the user to recognize that the acquisition data indicated by the acquisition data image 10020f is the acquisition data stored last. For example, as shown in FIG. 69A, the receiver arranges the acquired data image 10020f in the forefront among the plurality of acquired data images.
  • the receiver when the user swipes the acquired data image 10020f to the right side, the receiver discards the acquired data indicated by the acquired data image 10020f.
  • the receiver may discard the acquired data indicated by the acquired data image 10020f when the image of the transmitter is framed out of the display by the user moving the receiver. Note that the same effect as described above can be obtained regardless of whether the swipe direction is up, down, left, or right.
  • the receiver may display the swipe direction corresponding to the save or discard. As a result, the user can recognize that the data can be saved or discarded by the operation.
  • FIG. 70 is a diagram showing a display example when browsing acquired data.
  • the receiver displays the acquired data images of a plurality of stored acquired data in a small manner overlapping the lower end of the display.
  • the receiver displays each of the plurality of acquired data images in a large size as shown in FIG.
  • the receiver When the user taps the acquired data image to be displayed in the state shown in (b) of FIG. 70, the receiver displays the tapped acquired data image in a larger size as shown in (c) of FIG. A lot of information is displayed in the acquired data image.
  • the receiver When the user taps the back surface display button 10024a, the receiver displays the back surface of the acquired data image, and displays other data related to the acquired data.
  • the receiver disables camera shake correction (off), or converts the captured image according to the correction direction and correction amount of camera shake correction, thereby acquiring the correct imaging direction and accurately performing self-position estimation.
  • the captured image is an image obtained by imaging by the imaging unit of the receiver.
  • Self-position estimation means that the receiver estimates its own position. In the self-position estimation, specifically, the receiver specifies the position of the transmitter based on the received visible light signal, and receives based on the size, position, or shape of the transmitter reflected in the captured image. Identify the relative positional relationship between the transmitter and transmitter. Then, the receiver estimates the position of the receiver based on the position of the transmitter and the relative positional relationship between the receiver and the transmitter.
  • the transmitter is out of frame with a slight blurring of the receiver. End up.
  • the receiver can continuously receive the signal by enabling the camera shake correction.
  • FIG. 71 is a diagram illustrating an example of a transmitter in the ninth embodiment.
  • the above-described transmitter includes a light emitting unit, and transmits a visible light signal by changing the luminance of the light emitting unit.
  • the receiver determines the relative positional relationship between the receiver and the transmitter based on the shape of the transmitter (specifically, the light emitting unit) in the captured image. Find the relative angle to the transmitter.
  • the transmitter includes a light-emitting unit 10090a having a rotationally symmetric shape, as described above, based on the shape of the transmitter in the captured image, The relative angle with the receiver cannot be determined accurately. Therefore, it is desirable that the transmitter includes a light emitting unit having a shape that is not rotationally symmetric.
  • the receiver can obtain
  • the transmitter may include a light emitting unit 10090b that is not in a completely rotationally symmetric shape.
  • the shape of the light emitting unit 10090b is symmetric with respect to 90 ° rotation, but is not completely rotationally symmetric.
  • the receiver obtains a rough angle with the azimuth sensor, and further uses the shape of the transmitter in the captured image to uniquely limit the relative angle between the receiver and the transmitter. And accurate self-position estimation can be performed.
  • the transmitter may include a light emitting unit 10090c shown in FIG.
  • the shape of the light emitting unit 10090c is basically a rotationally symmetric shape. However, since a light guide plate or the like is provided in a part of the light emitting unit 10090c, the shape of the light emitting unit 10090c is not rotationally symmetric.
  • the transmitter may include a light emitting unit 10090d shown in FIG.
  • Each of the light emitting units 10090d includes rotationally symmetric illumination.
  • the overall shape of the light emitting unit 10090d configured by combining them is not a rotationally symmetric shape. Therefore, the receiver can perform accurate self-position estimation by imaging the transmitter.
  • the transmitter may include a light emitting unit 10090e and an object 10090f shown in FIG.
  • the object 10090f is an object (for example, a fire alarm or a pipe) configured so that the positional relationship with the light emitting unit 10090e does not change. Since the shape of the combination of the light emitting unit 10090e and the object 10090f is not a rotationally symmetric shape, the receiver can accurately perform self-position estimation by imaging the light emitting unit 10090e and the object 10090f.
  • the receiver can perform self-position estimation from the position and shape of the transmitter in the captured image. As a result, the receiver can estimate the moving direction and distance of the receiver being imaged. Further, the receiver can perform more accurate self-position estimation by performing triangulation using a plurality of frames or images. By integrating estimation results using a plurality of images and estimation results using a plurality of different combinations of images, the receiver can perform self-position estimation more accurately. At this time, the receiver can perform self-position estimation more accurately by emphasizing and summing up the results estimated from recent captured images.
  • FIG. 72 is a diagram illustrating an example of a reception method according to the ninth embodiment.
  • the horizontal axis of the graph shown in FIG. 72 indicates time, and the vertical axis indicates the position of each exposure line in the image sensor. Furthermore, a solid line arrow in the graph indicates a time (exposure timing) at which exposure of each exposure line in the image sensor is started.
  • the receiver reads the horizontal optical black signal in the image sensor as shown in FIG. 72A, but skips the horizontal optical black signal as shown in FIG. 72B. May be. Thereby, a continuous visible light signal can be received.
  • Horizontal optical black is optical black in the horizontal direction on the exposure line.
  • the vertical optical black is a portion of the optical black other than the horizontal optical black.
  • the black level can be adjusted using the optical black at the start of the visible light imaging similarly to the normal imaging.
  • the receiver can perform continuous reception and black level adjustment by performing black level adjustment using only vertical optical black.
  • the receiver may adjust the black level using horizontal optical black every predetermined time. In the case where the normal imaging and the visible light imaging are alternately performed, the receiver skips the horizontal optical black signal when continuously performing the visible light imaging, and reads the horizontal optical black signal otherwise. Then, the receiver can adjust the black level while continuously receiving the visible light signal by adjusting the black level based on the read signal.
  • the receiver may adjust the black level by setting the darkest part of the visible light captured image as black.
  • the transmitter may add a transmitter identifier indicating the type of transmitter to the visible light signal for transmission.
  • the receiver can perform a receiving operation according to the type of the transmitter.
  • the transmitter displays a content ID indicating which content is currently displayed in addition to the transmitter ID for performing individual identification of the transmitter as a visible light signal.
  • the receiver can display information according to the content currently displayed by the transmitter by separately handling these IDs based on the transmitter identifier. For example, when the transmitter identifier indicates digital signage or an emergency light, the receiver can reduce reception errors by imaging with increased sensitivity.
  • FIG. 73 is a flowchart showing an example of the reception method in the present embodiment.
  • the receiver receives the packet (step S10101) and performs error correction (step S10102). Then, the receiver determines whether or not a packet having the same address as that of the received packet has already been received (step S10103). If it is determined that the data is received (Y in step S10103), the receiver compares the data. That is, the receiver determines whether the data parts are equal (step S10104). Here, when it is determined that they are not equal (N in step S10104), the receiver further determines whether the difference in the plurality of data parts is equal to or greater than a predetermined number, specifically, the number of different bits or the luminance. It is determined whether or not the number of slots having different states is equal to or greater than a predetermined number (step S10105).
  • the receiver discards the packet that has already been received (step S10106). Thereby, when a packet is started to be received from another transmitter, interference with a packet received from a previous transmitter can be avoided.
  • the receiver sets the data of the data part with the most packets having the same data part as the data of the address (step S10107). Alternatively, the receiver takes the most equal bit as the value of that bit at that address. Alternatively, the receiver sets the luminance state having the largest number of equal luminance states as the luminance state of the slot at the address, and demodulates the data at the address.
  • the receiver first acquires a first packet including a data portion and an address portion from a plurality of bright line patterns.
  • the receiver has at least one second packet which is a packet including the same address part as the address part of the first packet among at least one packet already acquired before the first packet. It is determined whether or not there is a packet.
  • the receiver determines whether the data parts of the at least one second packet and the first packet are all equal. judge.
  • the receiver in each of the at least one second packet, out of the parts included in the data part of the second packet, the first packet It is determined whether the number of parts different from each part included in the data part is greater than or equal to a predetermined number.
  • the receiver includes the at least one second packet. Is discarded.
  • the receiver may include the first packet and the at least one second packet.
  • a plurality of packets having the largest number of packets having the same data part are specified.
  • the receiver then decodes at least a part of the visible light identifier (ID) by decoding the data part included in each of the plurality of packets as a data part corresponding to the address part included in the first packet. get.
  • ID visible light identifier
  • a plurality of packets having the same address part are received, even if the data part of those packets is different, the appropriate data part can be decoded, and at least a part of the visible light identifier is You can get it correctly. That is, a plurality of packets having the same address part transmitted from the same transmitter basically have the same data part. However, when the receiver switches the transmitter that is the transmission source of the packet, the receiver may receive a plurality of packets having different data parts even though they have the same address part. In such a case, in the present embodiment, as in step S10106 in FIG. 73, the already received packet (second packet) is discarded, and the data portion of the latest packet (first packet).
  • FIG. 74 is a flowchart showing an example of the reception method in this embodiment.
  • the receiver receives the packet (step S10111) and performs error correction of the address part (step S10112). At this time, the receiver does not demodulate the data part and holds the pixel value obtained by imaging as it is. Then, the receiver determines whether or not there are a predetermined number or more of packets having the same address among the plurality of packets that have already been received (step S10113). If it is determined that the packet exists (Y in step S10113), the receiver performs demodulation processing by combining pixel values of portions corresponding to data portions of a plurality of packets having the same address (step S10114).
  • the first packet including the data portion and the address portion is acquired from the plurality of bright line patterns. Then, among at least one packet that has already been acquired before the first packet, there are a predetermined number or more of second packets that include the same address part as the address part of the first packet. It is determined whether or not. If it is determined that there are more than the predetermined number of second packets, the pixel values of the partial areas of the bright line image corresponding to the respective data portions of the second packet more than the predetermined number, The pixel values of a part of the bright line image corresponding to the data portion of the packet are matched. That is, the pixel values are added. By the addition, a composite pixel value is calculated, and at least a part of the visible light identifier (ID) is obtained by decoding the data portion including the composite pixel value.
  • ID visible light identifier
  • the portion to be demodulated as described above includes a larger amount of data (number of samples) than the data portion of a single packet. Thereby, a data part can be demodulated more correctly. Further, by increasing the number of samples, a signal modulated with a higher modulation frequency can be demodulated.
  • the data part and its error correction code part are modulated at a higher frequency than the header part, the address part, and the error correction code part of the address part.
  • FIG. 75 is a flowchart showing an example of a reception method in the present embodiment.
  • the receiver receives the packet (step S10121), and determines whether or not a packet in which all the bits of the data part are 0 (hereinafter referred to as a 0 termination packet) is received (step S10122).
  • a 0 termination packet a packet in which all the bits of the data part are 0
  • the receiver determines whether or not all packets having addresses below the zero-termination packet address are available, that is, It is determined whether or not it has been received (step S10123).
  • the address is set to a value that increases in accordance with the order of transmission of each packet generated by dividing the transmitted data.
  • the receiver determines that the address of the 0-termination packet is the last address of the packet transmitted from the transmitter. Then, the receiver restores the data by connecting the data of the packets of each address up to the 0 terminal packet (step S10124). Further, the receiver performs an error check on the restored data (step S10125). This makes it possible to send and receive variable-length address data even when the data to be transmitted is not divided into several parts, that is, when the address is not fixed length but variable length. More IDs can be transmitted and received with high efficiency.
  • the receiver acquires a plurality of packets each including a data portion and an address portion from a plurality of bright line patterns. Then, the receiver determines whether or not there is a 0-termination packet that is a packet in which all bits included in the data part indicate 0 among the plurality of acquired packets. If it is determined that there is a zero-termination packet, the receiver, among a plurality of packets, N packets (N is an integer equal to or greater than one) including an address part associated with the address part of the zero-termination packet. It is determined whether or not all related packets are present.
  • the receiver acquires a visible light identifier (ID) by arranging and decoding the data portions of the N related packets.
  • ID visible light identifier
  • the address part associated with the address part of the 0-termination packet is an address part that indicates an address that is smaller than the address shown in the address part of the 0-termination packet and is 0 or more.
  • 76 and 77 are diagrams for explaining a reception method in which the receiver according to the present embodiment uses an exposure time longer than the period of the modulation frequency (modulation period).
  • the visible light signal may not be received correctly.
  • the modulation period is the time of one slot described above. That is, in such a case, there are few exposure lines (exposure lines indicated by black in FIG. 76) reflecting the luminance state of a certain slot. As a result, it is difficult to estimate the luminance of the transmitter when a lot of noise is accidentally included in the pixel values of these exposure lines.
  • the visible light signal can be correctly received. That is, in such a case, since there are many exposure lines reflecting the brightness of a certain slot, the brightness of the transmitter can be estimated from the pixel values of many exposure lines, and it is resistant to noise.
  • the luminance change that is, the change in the pixel value of each exposure line
  • the luminance change received by the receiver can sufficiently follow the luminance change used for transmission.
  • the luminance change received by the receiver cannot follow the luminance change used for transmission at all. That is, the longer the exposure time, the higher the noise resistance because the luminance can be estimated from many exposure lines. However, the longer the exposure time, the lower the identification margin or the smaller the identification margin. With these balances, the noise resistance can be maximized by setting the exposure time to about 2 to 5 times the modulation period.
  • FIG. 78 is a diagram showing an efficient number of divisions with respect to the size of transmission data.
  • the transmitter transmits data due to a change in luminance
  • transmission data data to be transmitted
  • the data size of the packet is large.
  • the transmission data is divided into a plurality of partial data and the partial data is included in each packet, the data size of each packet is reduced.
  • the receiver receives the packet by imaging.
  • the larger the data size of the packet the more difficult it is for the receiver to receive the packet by one imaging, and it is necessary to repeat imaging.
  • FIGS. 78 (a) and 78 (b) it is desirable for the transmitter to increase the number of divisions of the transmission data as the data size of the transmission data increases. However, if the number of divisions is too large, the transmission data cannot be restored unless all of the partial data is received.
  • the data size of the address (address size) is variable, and the data size of the transmission data is 2-16 bits, 16-24 bits, 24-64 bits, 66- In the case of 78 bits, 78-128 bits, 128 bits or more, send data into 1-2 pieces, 2-4 pieces, 4 pieces, 4-6 pieces, 6-8 pieces, 7 pieces or more, respectively. Is divided, transmission data can be efficiently transmitted by a visible light signal. Further, as shown in FIG.
  • the data size of the address (address size) is fixed to 4 bits, and the data size of the transmission data is 2-8 bits, 8-16 bits, 16-30 bits, For 30-64 bits, 66-80 bits, 80-96 bits, 96-132 bits, 132 bits or more, 1-2, 2-3, 2-4, 4-5,
  • the transmission data can be efficiently transmitted by a visible light signal.
  • the transmitter sequentially changes the luminance based on each packet including each of a plurality of partial data. For example, the transmitter changes the luminance based on the packets in the order of the addresses of the packets. Further, the transmitter may perform the luminance change based on the plurality of partial data again in an order different from the address order. Thereby, each partial data can be reliably received by the receiver.
  • FIG. 79A is a diagram showing an example of a setting method in the present embodiment.
  • the receiver acquires a notification operation identifier for identifying the notification operation and a priority of the notification operation identifier (specifically, an identifier indicating the priority) from a server near the receiver.
  • a notification operation when each packet including each of a plurality of partial data is transmitted due to a change in luminance and received by the receiver, reception is performed to notify the user of the receiver that the packets have been received.
  • the operation of the machine For example, the operation is sounding, vibration, or screen display.
  • the receiver receives a packetized visible light signal, that is, each packet including each of a plurality of partial data (step S10132).
  • the receiver acquires the notification operation identifier and the priority of the notification operation identifier (specifically, an identifier indicating the priority) included in the visible light signal (step S10133).
  • the receiver sets the current notification operation setting of the receiver, that is, the notification operation identifier preset in the receiver and the priority of the notification operation identifier (specifically, an identifier indicating the priority). ) Is read out (step S10134).
  • the notification operation identifier set in advance in the receiver is set by a user operation, for example.
  • the receiver selects an identifier having the highest priority among the preset notification operation identifiers and the notification operation identifiers acquired in steps S10131 and S10133 (step S10135).
  • the receiver resets the selected notification operation identifier to itself, thereby performing the operation indicated by the selected notification operation identifier, and notifies the user of reception of the visible light signal (step S10136).
  • the receiver may select a notification operation identifier having a higher priority from the two notification operation identifiers without performing any one of steps S10131 and S10133.
  • the priority of notification action identifiers transmitted from servers installed in theaters or museums, or the priority of notification action identifiers included in visible light signals transmitted within those facilities are set high. Also good. Thereby, it is possible to prevent the sound for the reception notification from being sounded in the facility regardless of the setting of the user. In other facilities, by setting the priority of the notification operation identifier low, the receiver can notify the reception by the operation according to the user's setting.
  • FIG. 79B is a diagram showing another example of the setting method in the present embodiment.
  • the receiver acquires a notification operation identifier for identifying the notification operation and a priority of the notification operation identifier (specifically, an identifier indicating the priority) from a server near the receiver. (Step S10141).
  • the receiver receives a packetized visible light signal, that is, each packet including each of a plurality of partial data (step S10142).
  • the receiver acquires the notification operation identifier and the priority of the notification operation identifier (specifically, an identifier indicating the priority) included in the visible light signal (step S10143).
  • the receiver sets the current notification operation setting of the receiver, that is, the notification operation identifier preset in the receiver and the priority of the notification operation identifier (specifically, an identifier indicating the priority). ) Is read (step S10144).
  • the receiver includes an operation notification identifier indicating an operation for prohibiting the generation of the notification sound in the notification operation identifier set in advance and the notification operation identifier acquired in each of steps S10141 and S10143. It is determined whether or not it is (step S10145). Here, if it is determined that it is included (Y in step S10145), the receiver sounds a notification sound for notifying completion of reception (step S10146). On the other hand, if it is determined that it is not included (N in step S10145), the receiver notifies the user of the completion of reception by, for example, vibration (step S10147).
  • the receiver does not perform either one of steps S10141 and S10143, and determines whether or not the two notification operation identifiers include an operation notification identifier indicating an operation that prohibits the generation of the notification sound. May be.
  • the receiver may perform self-position estimation based on an image obtained by imaging, and notify the user of reception by an operation associated with the estimated position or a facility at the position.
  • FIG. 80 is a flowchart showing processing of the information processing program in the tenth embodiment.
  • This information processing program is a program for changing the luminance of the light emitter of the transmitter described above according to the number of divisions shown in FIG.
  • this information processing program is an information processing program for causing a computer to process information to be transmitted in order to transmit the information to be transmitted by a change in luminance.
  • the information processing program includes an encoding step SA41 that generates an encoded signal by encoding information to be transmitted, and the number of bits of the generated encoded signal is within a range of 24 to 64 bits.
  • the computer executes a division step SA42 for dividing the encoded signal into four partial signals and an output step SA43 for sequentially outputting the four partial signals. These partial signals are output as packets.
  • the information processing program may cause the computer to specify the number of bits of the encoded signal and determine the number of partial signals based on the specified number of bits. In this case, the information processing program causes the computer to generate the determined number of partial signals by dividing the encoded signal.
  • the encoded signal is divided into four partial signals and output.
  • the four partial signals are transmitted as visible light signals and received by the receiver.
  • the larger the number of bits of the output signal the more difficult it is for the receiver to properly receive the signal by imaging, and the reception efficiency decreases. Therefore, it is desirable to divide the signal into signals having a small number of bits, that is, small signals.
  • the receiver cannot receive the original signal unless each small signal is individually received, resulting in a decrease in reception efficiency.
  • the encoded signal when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and sequentially output to indicate the information to be transmitted.
  • the encoded signal can be transmitted as a visible light signal with the best reception efficiency. As a result, communication between various devices can be enabled.
  • four partial signals may be output according to the first order, and further, the four partial signals may be output again according to a second order different from the first order.
  • the four partial signals are repeatedly output in a different order. Therefore, when each output signal is transmitted to the receiver as a visible light signal, the reception efficiency of the four partial signals is increased. It can be further increased. That is, even if the four partial signals are repeatedly output in the same order, the same partial signal may not be received by the receiver. However, by changing the order, the occurrence of such a case can be suppressed.
  • a notification operation identifier may be further attached to the four partial signals.
  • the notification operation identifier is used to identify the operation of the receiver notifying the user of the receiver that the four partial signals are received when the four partial signals are transmitted by the luminance change and received by the receiver. Identifier.
  • the receiver when the notification operation identifier is transmitted as a visible light signal and received by the receiver, the receiver receives the four partial signals to the user according to the operation identified by the notification operation identifier. You can be notified. That is, the notification operation by the receiver can be set on the side that transmits the information to be transmitted.
  • a priority identifier for identifying the priority of the notification operation identifier may be further output in association with the four partial signals.
  • the receiver handles the notification operation identifier according to the priority identified by the priority identifier. be able to. That is, when the receiver acquires another notification operation identifier, the receiver is identified by the notification operation identified by the notification operation identifier transmitted as the visible light signal and the other notification operation identifier.
  • One of the notification operations can be selected based on the priority.
  • An information processing program is an information processing program that causes a computer to process information on a transmission target in order to transmit the information on the transmission target with a luminance change, and encodes the information on the transmission target
  • An encoding step for generating an encoded signal by dividing the encoded signal into four partial signals when the number of bits of the generated encoded signal is in a range of 24 to 64 bits; And causing the computer to execute an output step of sequentially outputting the four partial signals.
  • the encoded signal when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and output.
  • the four partial signals are transmitted as visible light signals and received by the receiver.
  • the larger the number of bits of the output signal the more difficult it is for the receiver to properly receive the signal by imaging, and the reception efficiency decreases. Therefore, it is desirable to divide the signal into signals having a small number of bits, that is, small signals.
  • the receiver cannot receive the original signal unless each small signal is individually received, resulting in a decrease in reception efficiency.
  • the encoded signal when the number of bits of the encoded signal is in the range of 24 to 64 bits, the encoded signal is divided into four partial signals and sequentially output to indicate the information to be transmitted.
  • the encoded signal can be transmitted as a visible light signal with the best reception efficiency. As a result, communication between various devices can be enabled.
  • the four partial signals may be output according to a first order, and the four partial signals may be output again according to a second order different from the first order.
  • the four partial signals are repeatedly output in a different order. Therefore, when each output signal is transmitted to the receiver as a visible light signal, the reception efficiency of the four partial signals is increased. It can be further increased. That is, even if the four partial signals are repeatedly output in the same order, the same partial signal may not be received by the receiver. However, by changing the order, the occurrence of such a case can be suppressed.
  • the four partial signals are further output with a notification operation identifier attached thereto, and the notification operation identifier is transmitted when the four partial signals are transmitted by a luminance change and received by a receiver. Furthermore, an identifier for identifying the operation of the receiver that notifies the user of the receiver that the four partial signals have been received may be used.
  • the receiver when the notification operation identifier is transmitted as a visible light signal and received by the receiver, the receiver receives the four partial signals to the user according to the operation identified by the notification operation identifier. You can be notified. That is, the notification operation by the receiver can be set on the side that transmits the information to be transmitted.
  • a priority identifier for identifying a priority of the notification operation identifier may be output along with the four partial signals.
  • the receiver handles the notification operation identifier according to the priority identified by the priority identifier. be able to. That is, when the receiver acquires another notification operation identifier, the receiver is identified by the notification operation identified by the notification operation identifier transmitted as the visible light signal and the other notification operation identifier.
  • One of the notification operations can be selected based on the priority.
  • FIG. 81 is a diagram for explaining an application example of the transmission / reception system in the present embodiment.
  • This transmission / reception system includes a transmitter 10131b configured as an electronic device such as a washing machine, a receiver 10131a configured as a smartphone, and a communication device 10131c configured as an access point or a router.
  • FIG. 82 is a flowchart showing the processing operation of the transmission / reception system in the present embodiment.
  • the transmitter 10131b transmits Wi-Fi, Bluetooth (registered trademark) information such as an SSID, a password, an IP address, a MAC address, or an encryption key to connect to the transmitter 10131b. ) Or Ethernet (registered trademark) or the like (step S10166), and waits for connection.
  • the transmitter 10131b may transmit these pieces of information directly or indirectly.
  • the transmitter 10131b transmits an ID associated with the information. For example, the receiver 10131a that has received the ID downloads information associated with the ID from a server or the like.
  • the receiver 10131a receives the information (step S10151), connects to the transmitter 10131b, and connects to the communication device 10131c configured as an access point or router (SSID, password, IP address, MAC address, Alternatively, the encryption key or the like is transmitted to the transmitter 10131b (step S10152).
  • the receiver 10131a registers information (MAC address, IP address, encryption key, etc.) for the transmitter 10131b to connect to the communication device 10131c in the communication device 10131c, and makes the communication device 10131c wait for connection. Further, the receiver 10131a notifies the transmitter 10131b that preparation for connection from the transmitter 10131b to the communication device 10131c is completed (step S10153).
  • the transmitter 10131b disconnects from the receiver 10131a (step S10168) and connects to the communication device 10131c (step S10169). If the connection is successful (Y in step S10170), the transmitter 10131b notifies the receiver 10131a of the connection success via the communication device 10131c, and notifies the user of the connection success with a screen display, LED status, voice, or the like. (Step S10171). If the connection fails (N in step S10170), the transmitter 10131b notifies the receiver 10131a of the connection failure through visible light communication, and notifies the user in the same way as when it is successful (step S10172). The connection success may be notified by visible light communication.
  • the receiver 10131a connects to the communication device 10131c (step S10154), and if there is no notification of a connection success or failure (N in step S10155 and N in step S10156), the transmitter 10131b can be accessed via the communication device 10131c. It is confirmed whether or not (step S10157). If not (N in step S10157), the receiver 10131a determines whether or not the connection to the transmitter 10131b using the information received from the transmitter 10131b has been made a predetermined number of times or more (step S10158). If it is determined that the predetermined number of times has not been performed (N in step S10158), the receiver 10131a repeats the process from step S10152.
  • the receiver 10131a notifies the user of the processing failure (step S10159). If the receiver 10131a determines in step S10156 that there has been a notification of a successful connection (Y in step S10156), the receiver 10131a notifies the user of the processing success (step S10160). That is, the receiver 10131a notifies the user whether or not the transmitter 10131b has been able to connect to the communication device 10131c by screen display, voice, or the like. Accordingly, the transmitter 10131b can be connected to the communication device 10131c without requiring complicated input by the user.
  • FIG. 83 is a diagram for explaining an application example of the transmission / reception system in the present embodiment.
  • This transmission / reception system includes an air conditioner 10133b, a transmitter 10133c configured as an electronic device such as a wireless adapter connected to the air conditioner 10133b, a receiver 10133a configured as, for example, a smartphone, and a communication device configured as an access point or a router. 10133d and another electronic device 10133e configured as, for example, a wireless adapter, a wireless access point, or a router.
  • FIG. 84 is a flowchart showing the processing operation of the transmission / reception system in the present embodiment.
  • the air conditioner 10133b or the transmitter 10133c is referred to as an electronic device A
  • the electronic device 10133e is referred to as an electronic device B.
  • step S10188 when the start button is pressed (step S10188), the electronic device A transmits information (individual ID, password, IP address, MAC address, encryption key, etc.) for connection to itself (step S10189). The connection is awaited (step S10190). The electronic device A may transmit these pieces of information directly or indirectly as described above.
  • the receiver 10133a receives the information from the electronic device A (step S10181) and transmits the information to the electronic device B (step S10182).
  • electronic device B receives the information (step S10196), it connects to electronic device A according to the received information (step S10197). Then, the electronic device B determines whether or not the connection with the electronic device A is established (step S10198), and notifies the receiver 10133a of the success or failure (step S10199 or step S10200).
  • the electronic device A If the electronic device A is connected to the electronic device B during a predetermined time (Y in step S10191), the electronic device A notifies the receiver 10133a of the connection success via the electronic device B (step S10192) and is not connected (step S10192). N in step S10191), the receiver 10133a is notified of the connection failure by visible light communication (step S10193). In addition, the electronic device A notifies the user of the success or failure of the connection through a screen display, a light emission state, sound, or the like. Accordingly, the electronic device A (transmitter 10133c) can be connected to the electronic device B (electronic device 10133e) without requiring complicated input by the user. Note that the air conditioner 10133b and the transmitter 10133c illustrated in FIG. 83 may be configured integrally, and similarly, the communication device 10133d and the electronic device 10133e may be configured integrally.
  • FIG. 85 is a diagram for explaining an application example of the transmission / reception system according to the present embodiment.
  • This transmission / reception system includes a receiver 10135a configured as, for example, a digital still camera or a digital video camera, and a transmitter 10135b configured as, for example, illumination.
  • FIG. 86 is a flowchart showing the processing operation of the transmission / reception system in the present embodiment.
  • the receiver 10135a sends an imaging information transmission command to the transmitter 10135b (step S10211).
  • the transmitter 10135b receives the imaging information transmission command, the imaging information transmission button is pressed, the imaging information transmission switch is turned on, or the power is turned on (in step S10221). Y), imaging information is transmitted (step S10222).
  • the imaging information transmission command is a command for transmitting imaging information, and the imaging information indicates, for example, the color temperature of illumination, spectral distribution, illuminance, or light distribution.
  • the transmitter 10135b may transmit the imaging information directly or indirectly as described above. When transmitting indirectly, the transmitter 10135b transmits an ID associated with the imaging information.
  • the receiver 10135a that has received the ID downloads, for example, imaging information associated with the ID from a server or the like.
  • the transmitter 10135b is a method for transmitting a transmission stop command to itself (frequency of radio waves, infrared rays, or sound waves for transmitting the transmission stop command, or an SSID, a password or an IP address for connection to self-confidence) ) May be sent.
  • the receiver 10135a When receiving the imaging information (step S10212), the receiver 10135a transmits a transmission stop command to the transmitter 10135b (step S10213).
  • the transmitter 10135b receives a transmission stop command from the receiver 10135a (step S10223), the transmitter 10135b stops transmission of imaging information and emits light uniformly (step S10224).
  • the receiver 10135a sets the imaging parameter according to the imaging information received in step S10212 (step S10214), or notifies the user of the imaging information.
  • the imaging parameter is, for example, white balance, exposure time, focal length, sensitivity, or scene mode. Thereby, it is possible to take an image with an optimum setting according to the illumination.
  • the receiver 10135a captures an image after the transmission of imaging information from the transmitter 10135b is stopped (Y in step S10215) (step S10216). Thereby, it is possible to perform imaging without changing the brightness of the subject due to signal transmission.
  • the receiver 10135a may transmit a transmission start command for prompting the start of transmission of imaging information to the transmitter 10135b (step S10217).
  • FIG. 87 is a diagram for describing an application example of the transmitter in this embodiment.
  • the transmitter 10137b configured as a charger includes a light emitting unit, and transmits a visible light signal indicating a charged state of the battery from the light emitting unit.
  • the state of charge of the battery can be notified without an expensive display device.
  • a small LED is used as the light emitting unit, a visible light signal cannot be received unless the LED is imaged from nearby.
  • the transmitter 10137c having a protrusion near the LED it is difficult to close-up the LED due to the protrusion. Therefore, the visible light signal from the transmitter 10137b having no protrusion near the LED can be received more easily than the visible light signal from the transmitter 10137c.
  • FIG. 88 is a diagram for explaining an example of the operation of the transmitter in this embodiment.
  • the transmitter transmits a signal indicating that the error has occurred, or a signal corresponding to the error code. Can tell error details.
  • the receiver can correct the error or appropriately report the error content to the service center by indicating an appropriate response according to the error content.
  • the transmitter When the transmitter is in demo mode, it will send a demo code.
  • a demo of a transmitter that is a product is performed at a storefront, the store visitor can receive the demo code and acquire the product description associated with the demo code.
  • Whether or not the demo mode is selected is determined by whether the transmitter operation setting is in the demo mode, the storefront CAS card is inserted, the CAS card is not inserted, or the recording medium is not inserted. Judging from the point.
  • FIG. 89 is a diagram for explaining an example of the operation of the transmitter in this embodiment.
  • a transmitter configured as an air conditioner remote controller receives main body information
  • the transmitter transmits the main body information, so that the receiver receives information on a distant main body from a nearby transmitter. can do.
  • the receiver can also receive information from a main body that exists in a place where visible light communication is impossible, such as over a network.
  • FIG. 90 is a diagram for explaining an example of the operation of the transmitter in this embodiment.
  • the transmitter transmits if the ambient brightness is above a certain level, and stops transmitting if it is below a certain level.
  • a transmitter configured as an advertisement for a train can automatically stop its operation when the vehicle enters the garage, and battery consumption can be suppressed.
  • FIG. 91 is a diagram for explaining an example of the operation of the transmitter in this embodiment.
  • the transmitter associates the content that the receiver wants to acquire with the transmission ID in accordance with the display timing of the content to be displayed. Each time the display content is changed, the association change is registered with the server.
  • the transmitter sets the server so that another content is delivered to the receiver in accordance with the change timing of the display content.
  • the server transmits the content according to the set schedule to the receiver.
  • the receiver can acquire content that matches the content displayed by the transmitter.
  • FIG. 92 is a diagram for explaining an example of the operation of the transmitter in this embodiment.
  • the transmitter synchronizes the time with the server, adjusts the timing so that a predetermined part is displayed at a predetermined time, and displays the content.
  • the receiver can acquire content that matches the content displayed by the transmitter.
  • FIG. 93 is a diagram for explaining an example of operations of the transmitter and the receiver in this embodiment.
  • the transmitter transmits the display time of the content being displayed in addition to the transmitter ID.
  • the content display time is information that can identify the currently displayed content, and can be expressed by, for example, an elapsed time from the start time of the content.
  • the receiver acquires the content associated with the received ID from the server and displays the content according to the received display time. Thereby, for example, when a transmitter configured as digital signage changes display contents one after another, the receiver can acquire content that matches the content displayed by the transmitter.
  • the receiver changes the content to be displayed over time. As a result, even if a signal is not received again when the display content of the transmitter changes, the content that matches the display content is displayed.
  • FIG. 94 is a diagram for explaining an example of the operation of the receiver in this embodiment.
  • the receiver is authorized to access information such as the location of the receiver, phone number, ID, installed application, and user (Age, sex, occupation, preference, etc.) are transmitted to the server together with the received ID.
  • the account is not registered, if the user permits uploading of information as described above, it is similarly sent to the server. If not permitted, only the received ID is sent to the server. .
  • FIG. 95 is a diagram for explaining an example of operation of the receiver in this embodiment.
  • the receiver acquires content associated with the received ID from the server.
  • the active application can handle (can display or play) the acquired content
  • the acquired content is displayed / reproduced by the active application. If it cannot be handled, it is confirmed whether or not an app that can be handled is installed in the receiver. If it is installed, the app is activated to display / play the acquired content. If it is not installed, it automatically installs, displays a message prompting installation, displays a download screen, and displays and plays the acquired content after installation.
  • the acquired content can be handled appropriately (display / playback etc.).
  • FIG. 96 is a diagram for explaining an example of operation of the receiver in this embodiment.
  • the receiver acquires the content associated with the received ID and information (application ID) for specifying the application to be activated from the server.
  • application ID information for specifying the application to be activated from the server.
  • the running application is the designated application
  • the acquired content is displayed / reproduced.
  • the designated application is installed in the receiver, the designated application is activated to display / play the acquired content. If it is not installed, it automatically installs, displays a message prompting installation, displays a download screen, and displays and plays the acquired content after installation.
  • the receiver may acquire only the application ID from the server and start the designated application.
  • the receiver may perform specified settings.
  • the receiver may set the designated parameter and activate the designated application.
  • FIG. 97 is a diagram for explaining an example of operation of the receiver in this embodiment.
  • the receiver determines that the signal is being streamed when the value of the predetermined address of the received data is a predetermined value or the received data includes a predetermined identifier, and receives the streaming data. Receive with. Otherwise, it is received by a normal receiving method.
  • FIG. 98 is a diagram for explaining an example of operation of the receiver in this embodiment.
  • the receiver refers to the table in the application, and if the reception ID exists in the table, the receiver is designated in the table. Get the content. Otherwise, the content set as the reception ID is acquired from the server.
  • FIG. 99 is a diagram for explaining an example of operation of the receiver in this embodiment.
  • the receiver detects the signal and recognizes the modulation frequency of the signal.
  • the receiver sets the exposure time according to the period of the modulation frequency (modulation period). For example, the signal can be easily received by setting the exposure time to be approximately equal to the modulation period. Further, for example, by setting the exposure time to an integral multiple of the modulation period or a value close to it (approximately ⁇ 30%), it is possible to easily receive a signal by convolutional decoding.
  • FIG. 100 is a diagram for explaining an example of operation of the receiver in this embodiment.
  • the receiver transmits current location information and information related to the user (address, gender, age, preference, etc.) to the server.
  • the server transmits parameters for optimal operation of the transmitter to the receiver in accordance with the received information.
  • the receiver sets the received parameter if it can be set in the transmitter. If it cannot be set, the parameter is displayed and the user is prompted to set the parameter.
  • FIG. 101 is a diagram for explaining an example of a configuration of transmission data in the present embodiment.
  • the information to be transmitted includes an identifier, and the receiver can know the configuration of the subsequent part by its value. For example, it is possible to specify the data length, the type and length of the error correction code, the data division point, and the like.
  • the transmitter can change the type and length of the data body and error correction code according to the nature of the transmitter and the communication path. Also, the transmitter can cause the receiver to acquire an ID corresponding to the content ID by transmitting the content ID in addition to the ID of the transmitter.
  • FIG. 102 is a diagram for explaining the operation of the receiver in this embodiment.
  • the receiver 1210a in the present embodiment switches the shutter speed between high speed and low speed, for example, in units of frames when performing continuous shooting by the image sensor. Furthermore, the receiver 1210a switches the process for the frame to a barcode recognition process and a visible light recognition process based on the frame obtained by the shooting.
  • the barcode recognition process is a process for decoding a barcode reflected in a frame obtained by a low shutter speed.
  • the visible light recognition process is a process of decoding the above-described bright line pattern reflected in a frame obtained with a high shutter speed.
  • Such a receiver 1210a includes a video input unit 1211, a barcode / visible light identification unit 1212, a barcode recognition unit 1212a, a visible light recognition unit 1212b, and an output unit 1213.
  • the video input unit 1211 includes an image sensor, and switches the shutter speed for shooting by the image sensor. That is, the video input unit 1211 switches the shutter speed alternately between low speed and high speed, for example, in units of frames. More specifically, the video input unit 1211 switches the shutter speed to high speed for odd-numbered frames and switches the shutter speed to low speed for even-numbered frames. Shooting at a low shutter speed is shooting in the above-described normal shooting mode, and shooting at a high shutter speed is shooting in the above-described visible light communication mode. That is, when the shutter speed is low, the exposure time of each exposure line included in the image sensor is long, and a normal captured image on which the subject is projected is obtained as a frame. Further, when the shutter speed is high, the exposure time of each exposure line included in the image sensor is short, and a visible light communication image in which the above-described bright line is projected is obtained as a frame.
  • the barcode / visible light identifying unit 1212 switches processing for the image by determining whether a barcode appears or whether a bright line appears in the image obtained by the video input unit 1211. For example, if a barcode appears in a frame obtained by shooting at a low shutter speed, the barcode / visible light identification unit 1212 causes the barcode recognition unit 1212a to perform processing on the image. On the other hand, if a bright line appears in an image obtained by shooting at a high shutter speed, the barcode / visible light identifying unit 1212 causes the visible light recognizing unit 1212b to execute processing on the image.
  • the barcode recognition unit 1212a decodes a barcode appearing in a frame obtained by shooting at a low shutter speed.
  • the barcode recognition unit 1212a acquires barcode data (for example, a barcode identifier) by decoding, and outputs the barcode identifier to the output unit 1213.
  • the barcode may be a one-dimensional code or a two-dimensional code (for example, a QR code (registered trademark)).
  • the visible light recognizing unit 1212b decodes the bright line pattern appearing in the frame obtained by photographing at a high shutter speed.
  • the visible light recognizing unit 1212b obtains visible light data (for example, a visible light identifier) by the decoding, and outputs the visible light identifier to the output unit 1213.
  • the visible light data is the above-described visible light signal.
  • the output unit 1213 displays only frames obtained by shooting at a low shutter speed. Therefore, when the subject imaged by the video input unit 1211 is a barcode, the output unit 1213 displays the barcode.
  • the output unit 1213 displays an image of the digital signage without displaying the bright line pattern.
  • the output part 1213 acquires a barcode identifier, it acquires the information matched with the barcode identifier from a server etc., for example, and displays the information.
  • the output unit 1213 acquires a visible light identifier
  • the output unit 1213 acquires information associated with the visible light identifier from, for example, a server and displays the information.
  • the receiver 1210a as a terminal device includes an image sensor, and the image sensor is switched while alternately switching the shutter speed of the image sensor between a first speed and a second speed higher than the first speed. Perform continuous shooting with.
  • the receiver 1210a obtains an image showing the barcode by photographing when the shutter speed is the first speed, A barcode identifier is obtained by decoding the barcode reflected in the image.
  • the receiver 1210a when the subject to be photographed by the image sensor is a light source (for example, digital signage), the receiver 1210a includes a plurality of images included in the image sensor by photographing when the shutter speed is the second speed.
  • a bright line image that is an image including a bright line corresponding to each of the exposure lines is acquired. Then, the receiver 1210a acquires a visible light signal as a visible light identifier by decoding a plurality of bright line patterns included in the acquired bright line image. Further, the receiver 1210a displays an image obtained by photographing when the shutter speed is the first speed.
  • the barcode is decoded and the visible light signal can be received by switching between the barcode recognition process and the visible light recognition process. Furthermore, power consumption can be suppressed by switching.
  • the receiver in this embodiment may perform image recognition processing simultaneously with visible light processing instead of barcode recognition processing.
  • FIG. 103A is a diagram for explaining another operation of the receiver in this embodiment.
  • the receiver 1210b in the present embodiment switches the shutter speed between high speed and low speed, for example, in units of frames when performing continuous shooting by the image sensor. Further, the receiver 1210b simultaneously performs the image recognition process and the above-described visible light recognition process on the image (frame) obtained by the photographing.
  • the image recognition process is a process for recognizing a subject appearing in a frame obtained with a low shutter speed.
  • Such a receiver 1210b includes a video input unit 1211, an image recognition unit 1212c, a visible light recognition unit 1212b, and an output unit 1215.
  • the video input unit 1211 includes an image sensor, and switches the shutter speed for shooting by the image sensor. That is, the video input unit 1211 switches the shutter speed alternately between a low speed and a high speed, for example, in frame units. More specifically, the video input unit 1211 switches the shutter speed to high speed for odd-numbered frames and switches the shutter speed to low speed for even-numbered frames. Shooting at a low shutter speed is shooting in the above-described normal shooting mode, and shooting at a high shutter speed is shooting in the above-described visible light communication mode. That is, when the shutter speed is low, the exposure time of each exposure line included in the image sensor is long, and a normal captured image on which the subject is projected is obtained as a frame. Further, when the shutter speed is high, the exposure time of each exposure line included in the image sensor is short, and a visible light communication image in which the above-described bright line is projected is obtained as a frame.
  • the image recognition unit 1212c recognizes a subject appearing in a frame obtained by shooting at a low shutter speed and specifies the position of the subject in the frame. As a result of recognition, the image recognition unit 1212c determines whether or not the subject is an AR (Augmented Reality) target (hereinafter referred to as an AR target). When the image recognition unit 1212c determines that the subject is an AR object, the image recognition unit 1212c generates image recognition data that is data for displaying information about the subject (for example, the position of the subject and the AR marker). The AR marker is output to the output unit 1215.
  • an AR target Augmented Reality
  • the output unit 1215 displays only frames obtained by shooting at a low shutter speed, as with the output unit 1213 described above. Therefore, when the subject photographed by the video input unit 1211 is digital signage that transmits a visible light signal, the output unit 1213 displays the image of the digital signage without displaying the bright line pattern. Further, when acquiring the image recognition data from the image recognition unit 1212c, the output unit 1215 superimposes a white frame-shaped indicator surrounding the subject on the frame based on the position of the subject in the frame indicated by the image recognition data. .
  • FIG. 103B is a diagram illustrating an example of an indicator displayed by the output unit 1215.
  • the output unit 1215 superimposes a white frame-shaped indicator 1215b surrounding a subject image 1215a configured as digital signage on a frame, for example. That is, the output unit 1215 displays the indicator 1215b indicating the subject whose image has been recognized. Furthermore, when the output unit 1215 acquires the visible light identifier from the visible light recognition unit 1212b, the output unit 1215 changes the color of the indicator 1215b from white to red, for example.
  • FIG. 103C is a diagram showing a display example of AR.
  • the output unit 1215 further acquires information related to the subject associated with the visible light identifier as related information from, for example, a server.
  • the output unit 1215 describes related information in the AR marker 1215c indicated by the image recognition data, and displays the AR marker 1215c in which the related information is described in association with the subject image 1215a in the frame.
  • an AR using visible light communication can be realized by simultaneously performing an image recognition process and a visible light recognition process.
  • the receiver 1210a illustrated in FIG. 103A may also display the indicator 1215b illustrated in FIG. 103B, similarly to the receiver 1210b.
  • the receiver 1210a displays a white frame-shaped indicator 1215b surrounding the barcode. Then, when the barcode is decoded, the receiver 1210a changes the color of the indicator 1215b from white to red.
  • the receiver 1210a identifies a part in the low-speed frame corresponding to the part where the bright line pattern is located. For example, when the digital signage is transmitting a visible light signal, an image of the digital signage in the low-speed frame is specified. Note that the low speed frame is a frame obtained by shooting at a low shutter speed. Then, the receiver 1210a displays a white frame-shaped indicator 1215b surrounding a specified portion (for example, the above-mentioned digital signage image) in the low-speed frame so as to be superimposed on the low-speed frame. Then, when the bright line pattern is decoded, the receiver 1210a changes the color of the indicator 1215b from white to red.
  • FIG. 104A is a diagram for describing an example of a transmitter in this embodiment.
  • the transmitter 1220a in the present embodiment transmits a visible light signal in synchronization with the transmitter 1230. That is, the transmitter 1220a transmits the same visible light signal as the visible light signal at the timing when the transmitter 1230 transmits the visible light signal.
  • the transmitter 1230 includes a light emitting unit 1231, and transmits a visible light signal when the luminance of the light emitting unit 1231 changes.
  • Such a transmitter 1220 a includes a light receiving unit 1221, a signal analyzing unit 1222, a transmission clock adjusting unit 1223 a, and a light emitting unit 1224.
  • the light emitting unit 1224 transmits a visible light signal that is the same as the visible light signal transmitted from the transmitter 1230 by changing the luminance.
  • the light receiving unit 1221 receives a visible light signal from the transmitter 1230 by receiving visible light from the transmitter 1230.
  • the signal analysis unit 1222 analyzes the visible light signal received by the light receiving unit 1221 and transmits the analysis result to the transmission clock adjustment unit 1223a.
  • the transmission clock adjustment unit 1223a adjusts the timing of the visible light signal transmitted from the light emitting unit 1224 based on the analysis result.
  • the transmission clock adjustment unit 1223a uses the light emitting unit 1224 so that the timing at which the visible light signal is transmitted from the light emitting unit 1231 of the transmitter 1230 matches the timing at which the visible light signal is transmitted from the light emitting unit 1224. Adjust the brightness change timing.
  • the waveform of the visible light signal transmitted by the transmitter 1220a and the waveform of the visible light signal transmitted by the transmitter 1230 can be matched in timing.
  • FIG. 104B is a diagram for describing another example of the transmitter in this embodiment.
  • the transmitter 1220b in the present embodiment transmits a visible light signal in synchronization with the transmitter 1230, similarly to the transmitter 1220a. That is, the transmitter 1200b transmits the same visible light signal as the visible light signal at the timing when the transmitter 1230 transmits the visible light signal.
  • Such a transmitter 1220b includes a first light receiving unit 1221a, a second light receiving unit 1221b, a comparison unit 1225, a transmission clock adjusting unit 1223b, and a light emitting unit 1224.
  • the first light receiving unit 1221a receives the visible light from the transmitter 1230 by receiving the visible light from the transmitter 1230, similarly to the light receiving unit 1221.
  • the second light receiving unit 1221 b receives visible light from the light emitting unit 1224.
  • the comparison unit 1225 compares the first timing at which visible light is received by the first light receiving unit 1221a and the second timing at which visible light is received by the second light receiving unit 1221b. Then, the comparison unit 1225 outputs the difference (that is, the delay time) between the first timing and the second timing to the transmission clock adjustment unit 1223b.
  • the transmission clock adjusting unit 1223b adjusts the timing of the visible light signal transmitted from the light emitting unit 1224 so that the delay time is shortened.
  • the waveform of the visible light signal transmitted by the transmitter 1220b and the waveform of the visible light signal transmitted by the transmitter 1230 can be matched more accurately in terms of timing.
  • two transmitters transmit the same visible light signal, but different visible light signals may be transmitted. That is, when the two transmitters transmit the same visible light signal, they are transmitted in synchronization as described above. When two transmitters transmit different visible light signals, only one of the two transmitters transmits a visible light signal, while the other transmitter is uniformly lit or extinguished. To do. Thereafter, one transmitter is uniformly lit or extinguished, while only the other transmitter transmits a visible light signal. Two transmitters may transmit different visible light signals simultaneously.
  • FIG. 105A is a diagram for describing an example of synchronous transmission by a plurality of transmitters in the present embodiment.
  • the plurality of transmitters 1220 in the present embodiment are arranged, for example, in a line as shown in FIG. 105A. Note that these transmitters 1220 have the same configuration as the transmitter 1220a shown in FIG. 104A or the transmitter 1220b shown in FIG. 104B. Each of the plurality of transmitters 1220 transmits a visible light signal in synchronization with one of the transmitters 1220 on both sides.
  • FIG. 105B is a diagram for describing an example of synchronous transmission by a plurality of transmitters in the present embodiment.
  • One transmitter 1220 among the plurality of transmitters 1220 in this embodiment is a reference for synchronizing the visible light signal, and the remaining plurality of transmitters 1220 are visible light signals so as to match the reference. Send.
  • FIG. 106 is a diagram for describing another example of synchronous transmission by a plurality of transmitters in the present embodiment.
  • Each of the plurality of transmitters 1240 in the present embodiment receives a synchronization signal and transmits a visible light signal in accordance with the synchronization signal. Thereby, a visible light signal is transmitted from each of the plurality of transmitters 1240 in synchronization.
  • each of the plurality of transmitters 1240 includes a control unit 1241, a synchronization control unit 1242, a photocoupler 1243, an LED drive circuit 1244, an LED 1245, and a photodiode 1246.
  • the control unit 1241 receives the synchronization signal and outputs the synchronization signal to the synchronization control unit 1242.
  • the LED 1245 is a light source that emits visible light, and blinks (that is, changes in luminance) in accordance with control by the LED drive circuit 1244. Thus, a visible light signal is transmitted from the LED 1245 to the outside of the transmitter 1240.
  • the photocoupler 1243 transmits a signal between the synchronization control unit 1242 and the LED drive circuit 1244 while being electrically insulated. Specifically, the photocoupler 1243 transmits a transmission start signal described later transmitted from the synchronization control unit 1242 to the LED drive circuit 1244.
  • the LED drive circuit 1244 When the LED drive circuit 1244 receives the transmission start signal from the synchronization control unit 1242 via the photocoupler 1243, the LED drive circuit 1244 causes the LED 1245 to start transmitting the visible light signal at the timing when the transmission start signal is received.
  • the photodiode 1246 detects visible light emitted from the LED 1245, and outputs a detection signal indicating that the visible light has been detected to the synchronization control unit 1242.
  • the synchronization control unit 1242 When receiving the synchronization signal from the control unit 1241, the synchronization control unit 1242 transmits a transmission start signal to the LED drive circuit 1244 via the photocoupler 1243. By transmitting this transmission start signal, transmission of the visible light signal is started.
  • the synchronization control unit 1242 receives the detection signal from the photodiode 1246 by transmitting the visible light signal, the synchronization control unit 1242 is a delay that is a difference between the timing at which the detection signal is received and the timing at which the synchronization signal is received from the control unit 1241. Calculate time.
  • the synchronization control unit 1242 When the synchronization control unit 1242 receives the next synchronization signal from the control unit 1241, the synchronization control unit 1242 adjusts the timing for transmitting the next transmission start signal based on the calculated delay time. That is, the synchronization control unit 1242 adjusts the timing of transmitting the next transmission start signal so that the delay time for the next synchronization signal becomes a predetermined set delay time. Thus, the synchronization control unit 1242 transmits the next transmission start signal at the adjusted timing.
  • FIG. 107 is a diagram for explaining signal processing in the transmitter 1240.
  • the synchronization control unit 1242 When receiving the synchronization signal, the synchronization control unit 1242 generates a delay time setting signal that generates a delay time setting pulse at a predetermined timing. Note that receiving the synchronization signal specifically means receiving a synchronization pulse. That is, the synchronization control unit 1242 generates the delay time setting signal so that the delay time setting pulse rises at the timing when the set delay time has passed since the falling edge of the synchronization pulse.
  • the synchronization control unit 1242 transmits a transmission start signal to the LED drive circuit 1244 via the photocoupler 1243 at a timing delayed by the correction value N obtained last time from the falling edge of the synchronization pulse.
  • a visible light signal is transmitted from the LED 1245 by the LED drive circuit 1244.
  • the synchronization control unit 1242 receives the detection signal from the photodiode 1246 at a timing delayed by the sum of the intrinsic delay time and the correction value N from the falling edge of the synchronization pulse. That is, transmission of a visible light signal is started from that timing.
  • this timing is referred to as transmission start timing.
  • the above-described intrinsic delay time is a delay time caused by a circuit such as the photocoupler 1243, and is a delay time that occurs even when the synchronization control unit 1242 receives the synchronization signal and immediately transmits the transmission start signal. .
  • the correction correction value N can be a negative value as well as a positive value.
  • each of the plurality of transmitters 1240 transmits the visible light signal after the set delay time has elapsed after receiving the synchronization signal (synchronization pulse). It can. That is, even if there is a variation in the inherent delay time caused by a circuit such as the photocoupler 1243 in each of the plurality of transmitters 1240, the visible light from each of the plurality of transmitters 1240 is not affected by the variation.
  • the transmission of the optical signal can be accurately synchronized.
  • the LED drive circuit consumes a large amount of power, and is electrically insulated from the control circuit that handles the synchronization signal using a photocoupler or the like. Therefore, when such a photocoupler is used, it is difficult to synchronize the transmission of visible light signals from a plurality of transmitters due to the variation in the inherent delay time described above.
  • the light emission timing of the LED 1245 is detected by the photodiode 1246, the delay time from the synchronization signal is detected by the synchronization control unit 1242, and the delay time is set in advance. The time is adjusted to be the time (the set delay time described above).
  • a visible light signal (for example, visible light ID) is synchronized with high accuracy from the plurality of LED lighting. Can be sent.
  • the LED illumination may be turned on or off outside the visible light signal transmission period.
  • the first falling edge of the visible light signal may be detected.
  • the first rising edge of the visible light signal may be detected.
  • the transmitter 1240 transmits a visible light signal every time a synchronization signal is received.
  • the transmitter 1240 may transmit a visible light signal without receiving the synchronization signal. That is, if the transmitter 1240 transmits a visible light signal once in response to reception of the synchronization signal, the transmitter 1240 may sequentially transmit the visible light signal without receiving the synchronization signal. Specifically, the transmitter 1240 may sequentially transmit the visible light signal 2 to several thousand times with respect to one reception of the synchronization signal.
  • the transmitter 1240 may transmit a visible light signal corresponding to the synchronization signal at a rate of once every 100 milliseconds or once every few seconds.
  • the transmitter 1240 may transmit a visible light signal corresponding to the synchronization signal at a period of 60 Hz or more. Thereby, blinking is performed at high speed, and the blinking is difficult to be visually recognized by a person. As a result, the occurrence of flicker can be suppressed.
  • the transmitter 1240 may transmit a visible light signal corresponding to the synchronization signal at a sufficiently long cycle such as once every few minutes.
  • a sufficiently long cycle such as once every few minutes.
  • FIG. 108 is a flowchart illustrating an example of a reception method in this embodiment.
  • FIG. 109 is an explanatory diagram for describing an example of a reception method in this embodiment.
  • the receiver calculates the average value of the pixel values of a plurality of pixels arranged in a direction parallel to the exposure line (step S1211). If the pixel values of N pixels are averaged according to the central limit theorem, the expected value of the noise amount becomes N minus 1 ⁇ 2 power, and the SN ratio is improved.
  • the receiver leaves only the part where the pixel value changes in the vertical direction in all the colors, and removes the change in the pixel value where the pixel value changes differently (step S1212).
  • the transmission signal visible light signal
  • the luminance of the illumination of the transmitter or the backlight of the display changes.
  • the pixel values change in the same direction for all the colors.
  • the pixel values change differently for each color. In these portions, the pixel value fluctuates due to reception noise or a picture of the display or signage. Therefore, by removing these fluctuations, the SN ratio can be improved.
  • the receiver obtains a luminance value (step S1213). Since the luminance is not easily changed by color, the influence of the display or signage picture can be eliminated, and the SN ratio can be improved.
  • the receiver applies a low-pass filter to the luminance value (step S1214).
  • the S / N ratio can be improved by using a low-pass filter that cuts a high frequency region. Since there are many signal components up to the frequency up to the reciprocal of the exposure time, the effect of improving the S / N ratio can be increased by blocking the higher frequency. When the frequency component included in the signal is finite, the S / N ratio can be improved by blocking a frequency higher than that frequency.
  • a filter (such as a Butterworth filter) that does not include a frequency vibration component is suitable for the low-pass filter.
  • FIG. 110 is a flowchart illustrating another example of the reception method in this embodiment.
  • the reception method when the exposure time is longer than the transmission cycle will be described with reference to FIG.
  • the exposure time is an integral multiple of the transmission period
  • reception can be performed with the highest accuracy. Even if it is not an integral multiple, it can be received within a range of (N ⁇ 0.33) times (N is an integer).
  • the receiver sets the transmission / reception offset to 0 (step S1221).
  • the transmission / reception offset is a value for correcting a difference between the transmission timing and the reception timing. Since this deviation is unknown, the receiver gradually changes the value that is a candidate for the transmission / reception offset, and adopts the most suitable value as the transmission / reception offset.
  • the receiver determines whether or not the transmission / reception offset is less than the transmission cycle (step S1222).
  • the reception value for example, pixel value
  • the receiver determines in step S1222 that it is less than the transmission cycle (Y in step S1222)
  • the reception value for example, pixel value
  • the receiver obtains a difference between the received values obtained for each transmission cycle (step S1224).
  • the receiver adds a predetermined value to the transmission / reception offset (step S1226), and repeatedly executes the processing from step S1222. If the receiver determines in step S1222 that it is not less than the transmission cycle (N in step S1222), the receiver specifies the highest likelihood among the likelihoods of the received signals calculated for each transmission / reception offset. Then, the receiver determines whether or not the highest likelihood is greater than or equal to a predetermined value (step S1227). If it is determined that the value is equal to or greater than the predetermined value (Y in step S1227), the receiver uses the received signal with the highest likelihood as the final estimation result.
  • the receiver uses, as a received signal candidate, a received signal having a likelihood equal to or higher than a value obtained by subtracting a predetermined value from the highest likelihood (step S1228). On the other hand, if it is determined in step S1227 that the highest likelihood is less than the predetermined value (N in step S1227), the receiver discards the estimation result (step S1229).
  • the likelihood decreases. Therefore, when the likelihood is low, the reliability of the received signal can be improved by discarding the estimation result.
  • the maximum likelihood decoding outputs a valid signal as an estimation result. However, since the likelihood also decreases in this case, this problem can be avoided by discarding the estimation result when the likelihood is low.
  • Multi-value amplitude pulse signal 111, 112, and 113 are diagrams illustrating an example of a transmission signal in the present embodiment.
  • a transmission packet is configured using the patterns (a) and (b).
  • Data can be delimited by using a pattern with a specific length as a header of the entire packet and using a pattern with a different length as a separator.
  • signal detection can be facilitated by including this pattern in the middle.
  • the receiver can connect and decode the data. This also makes it possible to make the packet length variable by adjusting the number of separators.
  • the length of the entire packet may be expressed by the length of the packet header pattern.
  • the receiver can synthesize the partially received data by using the separator as a packet header and the length of the separator as the data address.
  • the transmitter repeatedly transmits the packet configured as described above.
  • the contents of packets 1 to 4 in (c) of FIG. 113 may all be the same, or may be combined as different data on the receiving side.
  • FIG. 114A is a diagram for describing a transmitter according to the present embodiment.
  • the transmitter in this embodiment is configured as a backlight of a liquid crystal display, for example, and includes a blue LED 2303 and a phosphor 2310 including a green fluorescent component 2304 and a red fluorescent component 2305.
  • Blue LED 2303 emits blue (B) light.
  • the phosphor 2310 emits yellow (Y) light when receiving blue light emitted from the blue LED 2303 as excitation light. That is, the phosphor 2310 emits yellow light.
  • the phosphor 2130 includes a green fluorescent component 2304 and a red fluorescent component 2305, yellow light is emitted by the emission of these fluorescent components.
  • the green fluorescent component 2304 emits green light when receiving blue light emitted from the blue LED 2303 as excitation light. That is, the green fluorescent component 2304 emits green (G) light.
  • the red fluorescent component 2305 emits red light when receiving blue light emitted from the blue LED 2303 as excitation light. That is, the red fluorescent component 2305 emits red (R) light.
  • This transmitter transmits a visible light signal of white light by changing the luminance of the blue LED 2303 as in the above embodiments. At this time, a visible light signal having a predetermined carrier frequency is output as the luminance of white light changes.
  • the barcode reader irradiates the barcode with the red laser beam, and reads the barcode based on the luminance change of the red laser beam reflected from the barcode.
  • the barcode reading frequency of the red laser light may coincide with or approximate the carrier frequency of the visible light signal output from a general transmitter currently in practical use. Therefore, in such a case, when the barcode reader attempts to read a barcode illuminated with white light, which is a visible light signal from the general transmitter, the luminance of the red light contained in the white light Depending on the change, the reading may fail. In other words, a barcode reading error occurs due to interference between the carrier frequency of the visible light signal (particularly red light) and the barcode reading frequency.
  • the red fluorescent component 2305 in the present embodiment changes in luminance at a frequency sufficiently lower than the luminance change frequency of the blue LED 2303 and the green fluorescent component 2304.
  • the red fluorescent component 2305 smoothes the frequency of the red luminance change included in the visible light signal.
  • FIG. 114B is a diagram showing luminance changes of RGB.
  • Blue light from the blue LED 2303 is included in the visible light signal and output as shown in FIG. 114B (a).
  • the green fluorescent component 2304 emits green light when receiving blue light from the blue LED 2303.
  • the duration of afterglow in the green fluorescent component 2304 is short. Therefore, when the blue LED 2303 changes in luminance, the green fluorescent component 2304 emits green light whose luminance changes at substantially the same frequency as the luminance change frequency of the blue LED 2303 (that is, the visible light signal carrier frequency). .
  • the red fluorescent component 2305 emits red light when receiving blue light from the blue LED 2303 as shown in (c) of FIG. 114B.
  • the duration of the afterglow in the red fluorescent component 2305 is long. Therefore, when the blue LED 2303 changes in luminance, the red fluorescent component 2305 emits red light whose luminance changes at a frequency lower than the frequency of luminance change of the blue LED 2303 (that is, the carrier frequency of the visible light signal). .
  • FIG. 115 is a diagram showing the afterglow characteristics of the green fluorescent component 2304 and the red fluorescent component 2305 in the present embodiment.
  • the intensity becomes smaller than the intensity I0.
  • the frequency f exceeds the threshold value fb, it gradually decreases.
  • the carrier frequency f1 is a carrier frequency of luminance change by the blue LED 2303 provided in the transmitter. Further, the above-described intensity I1 is 1/3 of the intensity I0 or ⁇ 10 dB of the intensity I0. For example, the carrier frequency f1 is 10 kHz or 5 to 100 kHz.
  • the transmitter according to the present embodiment is a transmitter that transmits a visible light signal, and receives a blue LED that emits blue light whose luminance changes as light included in the visible light signal, and the blue light.
  • a green fluorescent component that emits green light as light included in the visible light signal
  • a red fluorescent component that emits red light as light included in the visible light signal by receiving the blue light.
  • the duration of afterglow in the red fluorescent component is longer than the duration of afterglow in the green fluorescent component.
  • the green fluorescent component and the red fluorescent component may be included in a single phosphor that receives yellow light and emits yellow light as light included in the visible light signal.
  • the green fluorescent component may be included in a green phosphor
  • the red fluorescent component may be included in a red phosphor that is separate from the green phosphor.
  • the persistence time of the afterglow in the red fluorescent component is long, it is possible to change the luminance of the red light at a frequency lower than the frequency in the luminance change of the blue and green light. Therefore, even if the frequency of the luminance change of blue and green light included in the visible light signal of white light is the same as or close to the barcode reading frequency of the red laser light, it is included in the visible light signal of white light.
  • the frequency of the red light can be significantly different from the barcode reading frequency. As a result, the occurrence of barcode reading errors can be suppressed.
  • the red fluorescent component may emit red light whose luminance changes at a frequency lower than the frequency of luminance change of the light emitted from the blue LED.
  • the red fluorescent component may include a red fluorescent material that emits red light by receiving blue light and a low-pass filter that transmits only light in a predetermined frequency band.
  • the low-pass filter transmits only light in a low frequency band out of blue light emitted from the blue LED and applies the light to the red fluorescent material.
  • the red fluorescent material may have the same afterglow characteristics as the green fluorescent component.
  • the low-pass filter transmits only light in a low frequency band out of red light emitted from the red fluorescent material when blue light emitted from the blue LED hits the red fluorescent material. Let Even when such a low-pass filter is used, the occurrence of barcode reading errors can be suppressed as described above.
  • the red fluorescent component may be made of a fluorescent material having a predetermined afterglow characteristic.
  • the carrier frequency f1 may be approximately 10 kHz.
  • the carrier frequency used for transmitting visible light signals which is currently in practical use, is 9.6 kHz, it is effective to generate bar code reading errors in this practical transmission of visible light signals. Can be suppressed.
  • the carrier frequency f1 may be approximately 5 to 100 kHz.
  • carrier frequencies such as 20 kHz, 40 kHz, 80 kHz, and 100 kHz will be used in future visible light communication due to the advancement of image sensors (imaging devices) of receivers that receive visible light signals. Therefore, by setting the above-mentioned carrier frequency f1 to approximately 5 to 100 kHz, it is possible to effectively suppress the occurrence of barcode reading errors in future visible light communication.
  • the green fluorescent component and the red fluorescent component are contained in a single phosphor, or each of these two fluorescent components is contained in a separate phosphor.
  • the above effects can be achieved. That is, even when a single phosphor is used, the afterglow characteristics, that is, the frequency characteristics, of the red light and the green light emitted from the phosphor are different. Therefore, the above-mentioned effects can also be achieved by using a single phosphor that is inferior in afterglow characteristics or frequency characteristics in red light and superior in afterglow characteristics or frequency characteristics in green light.
  • poor afterglow characteristics or frequency characteristics means that the duration of afterglow is long or the intensity of light in a high frequency band is weak, and that the afterglow characteristics or frequency characteristics are superior means that afterglow characteristics The duration of the light is short, or the light intensity in the high frequency band is high.
  • the occurrence of barcode reading errors is suppressed by smoothing the frequency of the red luminance change included in the visible light signal, but the carrier frequency of the visible light signal is reduced. By making it high, the occurrence of the reading error may be suppressed.
  • FIG. 116 is a diagram for explaining a problem newly generated in order to suppress occurrence of a barcode reading error.
  • the reading frequency of the red laser light used for reading the barcode is also about 10 to 20 kHz. A barcode reading error occurs.
  • the carrier frequency fc of the visible light signal from about 10 kHz to, for example, 40 kHz, it is possible to suppress the occurrence of barcode reading errors.
  • the sampling frequency fs for the receiver to sample the visible light signal by photographing needs to be 80 kHz or more.
  • the receiver in this embodiment performs downsampling.
  • FIG. 117 is a diagram for explaining the downsampling performed by the receiver in this embodiment.
  • the transmitter 2301 in the present embodiment is configured as, for example, a liquid crystal display, digital signage, or a lighting device.
  • the transmitter 2301 then outputs a frequency-modulated visible light signal.
  • the transmitter 2301 switches the carrier frequency fc of the visible light signal to, for example, 40 kHz and 45 kHz.
  • the receiver 2302 in this embodiment photographs the transmitter 2301 at a frame rate of 30 fps, for example.
  • the receiver 2302 performs imaging with a short exposure time so that bright lines are generated in each image (specifically, each frame) obtained by imaging.
  • the receiver 2302 in this embodiment estimates the carrier frequency fc of the visible light signal by observing and analyzing the aliasing.
  • FIG. 118 is a flowchart showing a processing operation of the receiver 2302 in this embodiment.
  • the receiver 2302 observes and analyzes an alias generated in the frame obtained by the downsampling (step S2311).
  • the receiver 2302 specifies the frequency of the alias as, for example, 5.1 kHz or 5.5 kHz.
  • the receiver 2302 estimates the carrier frequency fc of the visible light signal based on the identified alias frequency (step S2311). That is, the receiver 2302 restores the original frequency from the alias. Thereby, the receiver 2302 estimates the carrier frequency fc of the visible light signal as 40 kHz or 45 kHz, for example.
  • the receiver 2302 in this embodiment can appropriately receive a visible light signal having a high carrier frequency by performing downsampling and restoration of a frequency based on an alias.
  • the carrier frequency of the visible light signal can be increased from 30 kHz to 60 kHz from the frequency (about 10 kHz) currently in practical use.
  • the carrier frequency of the visible light signal and the barcode reading frequency (10 to 20 kHz) can be greatly different, and interference between the frequencies can be suppressed.
  • the occurrence of barcode reading errors can be suppressed.
  • Such a receiving method in the present embodiment is a receiving method for acquiring information from a subject, and corresponds to a plurality of exposure lines included in the image sensor in a frame obtained by photographing the subject by an image sensor.
  • An exposure time setting step for setting an exposure time of the image sensor so that a plurality of bright lines are generated according to a change in luminance of the subject, and exposure of each of the plurality of exposure lines included in the image sensor at different times sequentially
  • the image sensor causes the image sensor to shoot the subject whose luminance changes at a predetermined frame rate and at the set exposure time, and for each frame obtained by the shooting.
  • Specified by the plurality of bright line patterns included in the frame Including an information acquisition step of acquiring information by demodulating the over data.
  • each of the plurality of exposure lines sequentially repeats starting exposure at different times, so that the sampling frequency is lower than the carrier frequency of the visible light signal transmitted by the luminance change of the subject.
  • the visible light signal is down-sampled, and in the information acquisition step, the frequency of the alias specified by the pattern of the plurality of bright lines included in the frame is specified and specified for each frame obtained by the imaging.
  • the information is obtained by estimating the frequency of the visible light signal from the frequency of the alias and demodulating the estimated frequency of the visible light signal.
  • a visible light signal having a high carrier frequency can be appropriately received by performing downsampling and frequency restoration based on alias.
  • a visible light signal having a carrier frequency higher than 30 kHz may be downsampled. Accordingly, interference between the carrier frequency of the visible light signal and the barcode reading frequency (10 to 20 kHz) can be avoided, and barcode reading errors can be more effectively suppressed.
  • FIG. 119 is a diagram illustrating processing operations of the reception device (imaging device). Specifically, FIG. 119 is a diagram for describing an example of switching processing between the normal imaging mode and the macro imaging mode when receiving visible light communication.
  • the reception device 1610 receives visible light emitted from a transmission device including a plurality of light sources (four light sources in FIG. 119).
  • the receiving device 1610 when the receiving device 1610 transitions to a mode for performing visible light communication, the receiving device 1610 activates the imaging unit in the normal imaging mode (S1601). Note that the receiving device 1610 displays a frame 1611 for imaging a light source on the screen when the mode is changed to a mode for performing visible light communication.
  • the receiving device 1610 switches the imaging mode of the imaging unit to the macro imaging mode (S1602).
  • the timing of switching from step S1601 to step S1602 may not be after a predetermined time from step S1601, but may be when the receiving device 1610 determines that the light source is captured within the frame 1611.
  • the user can easily place the light source in the frame 1611 because the light source can be stored in the frame 1611 with a clear image in the normal imaging mode before the image is blurred in the macro imaging mode. Can fit in.
  • the receiving device 1610 determines whether or not a signal from the light source has been received (S1603). If it is determined that the signal from the light source is received (Yes in S1603), the process returns to the normal imaging mode in Step S1601, and if it is determined that the signal from the light source is not received (No in S1603), the macro imaging in Step 1602 is performed. Continue mode. In the case of Yes in step S1603, processing based on the received signal (for example, processing for displaying an image indicated by the received signal) may be performed.
  • the user can take an image in a blurred state by switching from the normal imaging mode to the macro imaging mode by touching the display unit of the light source 1611 of the smartphone with a finger.
  • the image captured in the macro imaging mode includes more bright areas than the image captured in the normal imaging mode.
  • stripe-like images are separated as shown in the left diagram of FIG.
  • the problem that it cannot be received as a signal can be demodulated as a continuous reception signal for forming a continuous stripe as shown in the right figure. Since a long code can be received at once, the response time is shortened. As shown in FIG.
  • FIG. 120 is a diagram illustrating processing operations of the receiving device (imaging device). Specifically, FIG. 120 is a diagram for describing another example of the switching process between the normal imaging mode and the macro imaging mode when receiving visible light communication.
  • the receiving device 1620 receives visible light emitted from a transmitting device including a plurality of light sources (four light sources in FIG. 120).
  • the imaging device 1620 transitions to a mode for performing visible light communication
  • the imaging device is activated in the normal imaging mode, and an image 1623 having a wider range than the image 1622 displayed on the screen of the receiving device 1620 is captured.
  • the image data indicating the captured image 1623 and the posture information indicating the posture of the receiving device 1620 detected by the gyro sensor, the geomagnetic sensor, and the acceleration sensor of the receiving device 1620 when the image 1623 is captured are stored in the memory. (S1611).
  • the captured image 1623 is an image having a wide range by a predetermined width in the vertical direction and the horizontal direction with reference to the image 1622 displayed on the screen of the reception device 1620.
  • the receiving apparatus 1620 displays a frame 1621 for imaging a light source on the screen.
  • the receiving device 1620 switches the imaging mode of the imaging unit to the macro imaging mode (S1612). Note that the timing of switching from step S1611 to step S1612 is not after a predetermined time from step S1611, but when the image 1623 is captured and it is determined that the image data indicating the captured image 1623 is held in the memory. Good. At this time, the receiving device 1620 displays an image 1624 having a size corresponding to the screen size of the receiving device 1620 among the images 1623 based on the image data held in the memory.
  • an image 1624 displayed on the receiving device 1620 at this time is a part of the image 1623, and the posture of the receiving device 1620 indicated by the posture information acquired in step S1611 (indicated by a white broken line). Position) and the current posture of the receiving device 1620.
  • This is an image of an area predicted to be captured by the current receiving device 1620. That is, the image 1624 is a partial image of the image 1623 and is an image of an area corresponding to the imaging target of the image 1625 actually captured in the macro imaging mode. That is, in step S1612, the posture (imaging direction) changed from the time of step S1611 is acquired, and the imaging target that is presumed to be currently imaged from the acquired current posture (imaging direction) is identified and imaged in advance.
  • An image 1624 corresponding to the current posture (imaging direction) is specified from the image 1623, and processing for displaying the image 1624 is performed. For this reason, as shown by an image 1623 in FIG. 120, when the receiving device 1620 moves from the position indicated by the white broken line in the direction of the white arrow, the receiving device 1620 cuts out the image 1623 according to the amount of movement. An area 1624 can be determined, and an image 1624 that is the image 1623 in the determined area can be displayed.
  • the reception device 1620 does not display the image 1625 captured in the macro imaging mode even when capturing in the macro imaging mode, and starts from the image 1623 captured in the clearer normal imaging mode.
  • An image 1624 cut out in accordance with the current posture of the receiving device 1620 can be displayed.
  • the stored normal plane image is displayed on the display unit, and the user takes a picture using a smartphone. In this case, camera shake occurs, and the direction of the actual captured image and the direction of the still image displayed from the memory shifts, and a problem that the user cannot adjust the direction to the target light source is expected to occur.
  • the camera shake is detected by the image swing detection means or the swing gyro detection means, and the target image in the still image is shifted in a predetermined direction.
  • the user can see the deviation from the direction.
  • This display makes it possible for the user to point the camera at the target light source, so that multiple divided light sources can be photographed while displaying normal images, and signals are received continuously. can do.
  • the normal image is displayed, the light source divided into a plurality of parts can be received.
  • the receiving device 1620 determines whether or not a signal from the light source has been received (S1613). If it is determined that the signal from the light source has been received (Yes in S1613), the process returns to the normal imaging mode in Step S1611. If it is determined that the signal from the light source has not been received (No in S1613), the macro imaging in Step 1612 is performed. Continue mode. In the case of Yes in step S1613, processing based on the received signal (for example, processing for displaying an image indicated by the received signal) may be performed.
  • the receiving device 1620 can pick up an image including a brighter region in the macro image pickup mode. For this reason, in the macro imaging mode, the number of exposure lines that can generate bright lines for the subject can be increased.
  • 121 is a diagram showing processing operations of the receiving device (imaging device).
  • the transmission device 1630 is a display device such as a television, for example, and transmits different transmission IDs by visible light communication at a predetermined time interval ⁇ 1630. Specifically, at times t1631, t1632, t1633, and t1634, ID1631, ID1632, ID1633, and ID1634, which are transmission IDs associated with the data corresponding to the displayed images 1631, 1632, 1633, and 1634, respectively, are transmitted. To do. That is, ID 1631 to ID 1634 are transmitted one after another at a predetermined time interval ⁇ t 1630 from the transmission device 1630.
  • the receiving device 1640 requests the data associated with each transmission ID to the server 1650 based on the transmission ID received by visible light communication, receives the data from the server, and displays an image corresponding to the data. Specifically, images 1641, 1642, 1643, and 1644 respectively corresponding to ID1631, ID1632, ID1633, and ID1634 are displayed at times t1631, t1632, t1633, and t1634, respectively.
  • the receiving apparatus 1640 may acquire ID information indicating a transmission ID scheduled to be transmitted from the transmitting apparatus 1630 from the server 1650 at subsequent times t1632 to t1634.
  • the receiving device 1640 uses the acquired ID information, so that the data associated with ID 1632 to ID 1634 at times t1632 to t1634 can be stored in the server 1650 without receiving a transmission ID from the transmitting device 1630 each time.
  • the received data can be displayed at times t1632 to t1634.
  • the receiving device 1640 may request data corresponding to the ID 1631 at time t1631 without acquiring information indicating the transmission ID scheduled to be transmitted from the transmitting device 1630 from time t1632 to t1634 from the server 1650.
  • the data associated with the transmission ID corresponding to the subsequent times t1632 to t1634 may be received from the server 1650, and the received data may be displayed at each time t1632 to t1634. That is, when server 1650 receives a request for data associated with ID 1631 transmitted at time t1631 from reception device 1640, server 1650 receives data associated with a transmission ID corresponding to subsequent times t1632 to t1634.
  • the server 1650 holds association information in which each time t1631 to 1634 and data associated with the transmission ID corresponding to each time t1631 to 1634 are associated, and based on the association information
  • the predetermined data associated with the predetermined time is transmitted at the predetermined time.
  • receiving apparatus 1640 can obtain transmission ID 1631 by visible light communication at time t1631, data corresponding to each time t1632 to t1634 from server 1650 can be obtained at subsequent times t1632 to t1634 without performing visible light communication. Can be received. For this reason, the user does not need to keep the receiving device 1640 directed at the transmitting device 1630 in order to acquire the transmission ID by visible light communication, and can easily display the data acquired from the server 1650 on the receiving device 1640. In this case, when the receiving device 1640 obtains data corresponding to the ID from the server every time, the time delay from the server occurs and the response time becomes longer.
  • the receiving device 1640 displays images 1641, 1642, 1643, and 1644 corresponding to the transmission IDs ID1631, ID1632, ID1633, and ID1634, respectively, at times t1631, t1632, t1633, and t1634. Displayed respectively.
  • the reception device 1640 may present not only the image but also other information at each time. That is, at time t1631, the receiving device 1640 displays the image 1641 corresponding to the ID 1631 and outputs sound or sound corresponding to the ID 1631. At this time, the receiving device 1640 may further display, for example, a purchase site for the product displayed in the image. Such sound output and purchase site display are performed in the same manner at times t1632, t1633, and t1634 other than time t1631.
  • an image with a normal image quality is displayed with a normal shutter speed and a normal focus for the left eye.
  • the right-eye camera uses a shutter that is faster than the left eye and / or is set to a focal point or macro at a short distance to obtain the stripe-like bright line of the present invention and demodulate the data.
  • an image having normal image quality is displayed on the display unit, and the optical communication data of a plurality of light sources divided in distance can be received by the right eye camera.
  • FIG. 123 is a diagram illustrating an example of an application according to the sixteenth embodiment.
  • a receiver 1800a configured as a smartphone receives a signal (visible light signal) transmitted from a transmitter 1800b configured as, for example, a street digital signage. That is, the receiver 1800a receives the timing of image reproduction by the transmitter 1800b. The receiver 1800a reproduces sound at the same timing as the image reproduction. In other words, the receiver 1800a performs synchronized reproduction of the sound so that the image and sound reproduced by the transmitter 1800b are synchronized. Note that the receiver 1800a may reproduce the same image as the image (reproduced image) reproduced by the transmitter 1800b or a related image related to the reproduced image together with the sound. Further, the receiver 1800a may cause a device connected to the receiver 1800a to reproduce sound and the like. Further, after receiving the visible light signal, the receiver 1800a may download content such as sound or related images associated with the visible light signal from the server. The receiver 1800a performs synchronous reproduction after the download.
  • the user can select the sound that matches the display of the transmitter 1800b. Can hear. Further, even when there is a distance that takes time to reach the voice, it is possible to listen to the voice that matches the display.
  • FIG. 124 is a diagram illustrating an example of an application according to the sixteenth embodiment.
  • Each of the receiver 1800a and the receiver 1800c obtains and reproduces audio corresponding to a video such as a movie displayed on the transmitter 1800d from the server, in the language set in the receiver.
  • the transmitter 1800d transmits a visible light signal indicating an ID for identifying the displayed video to the receiver.
  • the receiver transmits a request signal including the ID indicated in the visible light signal and the language set in the receiver to the server.
  • the receiver acquires the audio corresponding to the request signal from the server and reproduces it. Thereby, the user can enjoy the work displayed on the transmitter 1800d in the language set by the user.
  • 125 and 126 are diagrams showing an example of a transmission signal and an example of a voice synchronization method in the sixteenth embodiment.
  • Different data are associated with a time every fixed time (N seconds).
  • These data may be, for example, an ID for identifying time, may be time, or may be audio data (for example, 64 Kbps data).
  • the following description is based on the assumption that the data is an ID. Different IDs may have different additional information parts attached to the ID.
  • the packets that make up the ID are different. Therefore, it is desirable that IDs are not continuous.
  • the transmitter 1800d transmits the ID in accordance with the reproduction time of the displayed image, for example.
  • the receiver can recognize the reproduction time (synchronization time) of the image of the transmitter 1800d by detecting the timing when the ID is changed.
  • the synchronization time can be recognized by the following method.
  • N By setting N to 0.5 seconds or less, it can be synchronized accurately.
  • FIG. 126 is a diagram illustrating an example of a transmission signal in the sixteenth embodiment.
  • a time packet is a packet that holds the time of transmission.
  • the time packet is divided into a time packet 1 representing a fine time and a time packet 2 representing a rough time.
  • time packet 2 indicates the hour and minute of the time
  • time packet 1 indicates only the second of the time.
  • a packet indicating the time may be divided into three or more time packets. Since the coarse time is less necessary, the receiver can recognize the synchronization time quickly and accurately by transmitting more fine time packets than coarse time packets.
  • the visible light signal includes the second information (hour packet 2) indicating the hour and minute of the time, and the first information (time packet 1) indicating the second of the time.
  • the time when the visible light signal is transmitted from the transmitter 1800d is indicated.
  • the receiver 1800a receives the second information and receives the first information more times than the number of times of receiving the second information.
  • FIG. 127 is a diagram illustrating an example of a process flow of the receiver 1800a according to the sixteenth embodiment.
  • a processing delay time is designated for the receiver 1800a (step S1801). This may be stored in the processing program or specified by the user. When the user performs correction, it is possible to realize more accurate synchronization according to the individual receiver. This processing delay time can be synchronized more accurately by changing it depending on the receiver model, the temperature of the receiver, and the CPU usage rate.
  • the receiver 1800a determines whether or not a time packet has been received or whether or not an ID associated for voice synchronization has been received (step S1802).
  • the receiver 1800a determines whether it has been received (Y in step S1802), it further determines whether there is an image waiting for processing (step S1804). If it is determined that there is an image waiting for processing (Y in step S1804), the receiver 1800a discards the image waiting for processing or delays processing of the image waiting for processing to receive from the latest acquired image. Is performed (step S1805). Thereby, it is possible to avoid an unexpected delay due to the amount of waiting for processing.
  • the receiver 1800a measures the position in the image where the visible light signal (specifically the bright line) is located (step S1806). In other words, by measuring the position in the direction perpendicular to the exposure line from the first exposure line in the image sensor, the time difference (delay time in the image) from the image acquisition start time to the signal reception time is obtained. Can be calculated.
  • the receiver 1800a can accurately perform synchronized reproduction by reproducing the sound or moving image at the time obtained by adding the processing delay time and the in-image delay time to the recognized synchronization time (step S1807).
  • step S1802 if it is determined in step S1802 that the receiver 1800a has not received the time packet or the voice synchronization ID, the receiver 1800a receives a signal from the image obtained by imaging (step S1803).
  • 128 is a diagram illustrating an example of a user interface of the receiver 1800a in Embodiment 16.
  • the user can adjust the processing delay time described above by pressing any of the buttons Bt1 to Bt4 displayed on the receiver 1800a as shown in FIG. 128 (a).
  • the processing delay time may be set by a swipe operation as shown in FIG. 128 (b). Thereby, synchronous reproduction can be performed more accurately based on the user's sense.
  • FIG. 129 is a diagram illustrating an example of a process flow of the receiver 1800a according to the sixteenth embodiment.
  • the earphone-only playback shown by this processing flow enables audio playback without disturbing the surroundings.
  • the receiver 1800a checks whether or not the setting limited to the earphone is performed (step S1811).
  • the setting limited to the earphone is performed, for example, the setting limited to the earphone is set in the receiver 1800a.
  • settings that are limited to earphones are made in the received signal (visible light signal). Alternatively, it is recorded in the server or the receiver 1800a in association with the received signal that it is limited to the earphone.
  • step S1813 it is determined whether or not the earphone is connected to the receiver 1800a.
  • the receiver 1800a When the receiver 1800a confirms that the earphone is not limited (N in Step S1811) or determines that the earphone is connected (Y in Step S1813), the receiver 1800a reproduces the sound (Step S1812). When playing back audio, the receiver 1800a adjusts the volume so that the volume is within the set range. This setting range is set similarly to the setting limited to the earphone.
  • the receiver 1800a determines that the earphone is not connected (N in step S1813)
  • the receiver 1800a performs a notification prompting the user to connect the earphone (step S1814).
  • This notification is performed by, for example, screen display, audio output, or vibration.
  • the receiver 1800a prepares an interface for forced reproduction and determines whether or not the user has performed an operation of forced reproduction (Ste S1815). If it is determined that the forced playback operation has been performed (Y in step S1815), the receiver 1800a plays back the audio even when the earphone is not connected (step S1812).
  • the receiver 1800a retains the audio data received in advance and the analyzed synchronization time so that the earphone is connected. Quickly synchronize audio playback.
  • FIG. 130 is a diagram illustrating another example of the process flow of the receiver 1800a according to the sixteenth embodiment.
  • the receiver 1800a receives an ID from the transmitter 1800d (step S1821). That is, the receiver 1800a receives a visible light signal indicating the ID of the transmitter 1800d or the ID of the content displayed on the transmitter 1800d.
  • the receiver 1800a downloads information (content) associated with the received ID from the server (step S1822). Alternatively, the receiver 1800a reads out the information from the data holding unit in the receiver 1800a. Hereinafter, this information is referred to as related information.
  • the receiver 1800a determines whether or not the synchronous reproduction flag included in the related information indicates ON (step S1823). If it is determined that the synchronous reproduction flag does not indicate ON (N in step S1823), the receiver 1800a outputs the content indicated by the related information (step S1824). That is, when the content is an image, the receiver 1800a displays an image, and when the content is audio, the receiver 1800a outputs audio.
  • receiver 1800a determines that the synchronous reproduction flag indicates ON (Y in step S1823), is the time adjustment mode included in the related information set to the transmitter reference mode? Then, it is determined whether or not the absolute time mode is set (step S1825). If it is determined that the absolute time mode is set, the receiver 1800a determines whether or not the last time adjustment has been performed within a certain time from the current time (step S1826). The time adjustment at this time is processing for obtaining time information by a predetermined method and using the time information to adjust the time of a clock provided in the receiver 1800a to the absolute time of the reference clock.
  • the predetermined method is, for example, a method using a GPS (Global Positioning System) radio wave or an NTP (Network Time Protocol) radio wave. Note that the current time described above may be a time when the receiver 1800a, which is a terminal device, receives a visible light signal.
  • the receiver 1800a determines that the last time adjustment has been performed within a certain time (Y in step S1826), the receiver 1800a outputs the related information based on the time of the clock of the receiver 1800a, and is displayed on the transmitter 1800d. Content and related information are synchronized (step S1827).
  • the content indicated by the related information is, for example, a moving image
  • the receiver 1800a displays the moving image so as to be synchronized with the content displayed on the transmitter 1800d.
  • the content indicated by the related information is, for example, audio
  • the receiver 1800a outputs the audio so as to be synchronized with the content displayed on the transmitter 1800d.
  • the related information indicates sound
  • the related information includes each frame constituting the sound, and these frames are time stamped.
  • the receiver 1800a outputs a sound synchronized with the content of the transmitter 1800d by playing back a frame with a type stamp corresponding to the time of its own clock.
  • the receiver 1800a determines that the last time adjustment has not been performed within a certain time (N in step S1826), the receiver 1800a attempts to obtain the time information by a predetermined method, and whether or not the time information has been obtained. Is determined (step S1828). If it is determined that the time information has been obtained (Y in step S1828), the receiver 1800a updates the time of the clock of the receiver 1800a using the time information (step S1829). Then, the receiver 1800a executes the process of step S1827 described above.
  • step S1825 If it is determined in step S1825 that the time adjustment mode is the transmitter reference mode, or if it is determined in step S1828 that time information could not be obtained (N in step S1828), the receiver 1800a
  • the time information is acquired from the transmitter 1800d (step S1830). That is, the receiver 1800a acquires time information that is a synchronization signal from the transmitter 1800d through visible light communication.
  • the synchronization signals are time packet 1 and time packet 2 shown in FIG.
  • the receiver 1800a acquires time information from the transmitter 1800d by radio waves such as Bluetooth (registered trademark) or Wi-Fi. Then, the receiver 1800a executes the processes of steps S1829 and S1827 described above.
  • processing is performed for synchronization between the clock of the terminal device that is the receiver 1800a and the reference clock by GPS radio waves or NTP radio waves.
  • the time of the terminal device, the time of the terminal device, and the time of the transmitter according to the time indicated by the visible light signal transmitted from the transmitter 1800d. Synchronize between. Accordingly, the terminal device can reproduce the content (moving image or sound) at the timing synchronized with the transmitter-side content reproduced by the transmitter 1800d.
  • FIG. 131A is a diagram for explaining a specific method of synchronized playback in the sixteenth embodiment. As a method of synchronous reproduction, there are methods a to e shown in FIG. 131A.
  • the transmitter 1800d outputs a visible light signal indicating the content ID and the content playback time by changing the luminance of the display, as in the above embodiments.
  • the content playback time is the playback time of data that is part of the content that is being played back by the transmitter 1800d when the content ID is transmitted from the transmitter 1800d.
  • the data is a picture or a sequence constituting the moving image if the content is a moving image, or a frame constituting the sound if the content is sound.
  • the playback time indicates, for example, the playback time from the beginning of the content as the time. If the content is a moving image, the playback time is included in the content as a PTS (Presentation Time Stamp). That is, the content includes the reproduction time (display time) of the data for each data constituting the content.
  • PTS Presentation Time Stamp
  • the receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the content ID indicated by the visible light signal to the server 1800f. The server 1800f receives the request signal, and transmits the content associated with the content ID included in the request signal to the receiver 1800a.
  • the receiver 1800a When the receiver 1800a receives the content, the receiver 1800a plays the content from the time of (content playback time + elapsed time since ID reception).
  • the elapsed time from the reception of the ID is an elapsed time from when the content ID is received by the receiver 1800a.
  • the transmitter 1800d outputs a visible light signal indicating the content ID and the content playback time by changing the luminance of the display, as in the above embodiments.
  • the receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the content ID indicated by the visible light signal and the content playback time to the server 1800f.
  • the server 1800f receives the request signal, and transmits only a part of the content after the content playback time to the receiver 1800a among the content associated with the content ID included in the request signal.
  • the receiver 1800a When the receiver 1800a receives the part of the content, the receiver 1800a reproduces the part of the content from the time point (elapsed time since the ID reception).
  • the transmitter 1800d outputs a visible light signal indicating the transmitter ID and the content reproduction time by changing the luminance of the display, as in the above embodiments.
  • the transmitter ID is information for identifying the transmitter.
  • the receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the transmitter ID indicated by the visible light signal to the server 1800f.
  • the server 1800f holds, for each transmitter ID, a reproduction schedule that is a timetable of content reproduced by the transmitter with the transmitter ID. Further, the server 1800f includes a clock. When such a server 1800f receives the request signal, the content associated with the transmitter ID included in the request signal and the clock time (server time) of the server 1800f is the content being played back. Identify from the playback schedule. Then, the server 1800f transmits the content to the receiver 1800a.
  • the receiver 1800a When the receiver 1800a receives the content, the receiver 1800a plays the content from the time of (content playback time + elapsed time since ID reception).
  • the transmitter 1800d outputs a visible light signal indicating the transmitter ID and the transmitter time by changing the luminance of the display as in the above embodiments.
  • the transmitter time is a time indicated by a clock provided in the transmitter 1800d.
  • the receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the transmitter ID indicated by the visible light signal and the transmitter time to the server 1800f.
  • the server 1800f holds the above reproduction schedule.
  • the server 1800f receives the request signal
  • the server 1800f identifies the content associated with the transmitter ID and the transmitter time included in the request signal as the content being reproduced from the reproduction schedule.
  • the server 1800f specifies the content playback time from the transmitter time. That is, the server 1800f finds the playback start time of the specified content from the playback schedule, and specifies the time between the transmitter time and the playback start time as the content playback time. Then, the server 1800f transmits the content and the content playback time to the receiver 1800a.
  • the receiver 1800a Upon receiving the content and the content playback time, the receiver 1800a plays the content from the time of (content playback time + elapsed time since reception of ID).
  • the visible light signal indicates the time when the visible light signal is transmitted from the transmitter 1800d. Therefore, the receiver 1800a, which is a terminal device, can receive content associated with the time (transmitter time) at which the visible light signal is transmitted from the transmitter 1800d. For example, if the transmitter time is 5:43, content played back at 5:43 can be received.
  • the server 1800f has a plurality of contents each associated with a time.
  • the content associated with the time indicated by the visible light signal may not exist in the server 1800f.
  • the receiver 1800a as the terminal device is closest to the time indicated by the visible light signal and is associated with the time after the time indicated by the visible light signal among the plurality of contents. Content may be received. Thereby, even if the content associated with the time indicated by the visible light signal does not exist in the server 1800f, it is possible to receive appropriate content from among the plurality of contents in the server 1800f.
  • the reproduction method includes a signal receiving step of receiving a visible light signal by a sensor of the receiver 1800a (terminal device) from a transmitter 1800d that transmits a visible light signal according to a luminance change of the light source, and a receiver 1800a transmits a request signal for requesting the content associated with the visible light signal to the server 1800f, the receiver 1800a receives the content from the server 1800f, and reproduces the content.
  • the visible light signal indicates a transmitter ID and a transmitter time.
  • the transmitter ID is ID information.
  • the transmitter time is the time indicated by the clock of the transmitter 1800d, and the time when the visible light signal is transmitted from the transmitter 1800d.
  • the receiver 1800a receives the content associated with the transmitter ID and the transmitter time indicated by the visible light signal. As a result, the receiver 1800a can reproduce appropriate content with respect to the transmitter ID and the transmitter time.
  • the transmitter 1800d outputs a visible light signal indicating the transmitter ID by changing the luminance of the display as in the above embodiments.
  • the receiver 1800a receives the visible light signal by photographing the transmitter 1800d as in the above embodiments. Then, the receiver 1800a transmits a request signal including the transmitter ID indicated by the visible light signal to the server 1800f.
  • the server 1800f holds the above-described reproduction schedule and further includes a clock.
  • the server 1800f receives the request signal
  • the server 1800f identifies the content associated with the transmitter ID and the server time included in the request signal from the reproduction schedule as content being reproduced.
  • the server time is the time indicated by the clock of the server 1800f.
  • the server 1800f finds the reproduction start time of the specified content from the reproduction schedule table. Then, the server 1800f transmits the content and the content reproduction start time to the receiver 1800a.
  • the receiver 1800a When the receiver 1800a receives the content and the content playback start time, the receiver 1800a plays the content from the time of (receiver time-content playback start time).
  • the receiver time is a time indicated by a clock provided in the receiver 1800a.
  • the reproduction method includes a signal receiving step of receiving a visible light signal by a sensor of the receiver 1800a (terminal device) from a transmitter 1800d that transmits a visible light signal due to a luminance change of the light source;
  • the transmitting step of transmitting a request signal for requesting the content associated with the visible light signal from the receiver 1800a to the server 1800f, and the receiver 1800a include each time and data reproduced at each time
  • the transmitter 1800d if content related to the content (transmitter-side content) is reproduced, the receiver 1800a can reproduce the content in synchronization with the transmitter-side content appropriately. .
  • the server 1800f may transmit only a part of the content after the content playback time to the receiver 1800a.
  • the receiver 1800a transmits a request signal to the server 1800f and receives necessary data from the server 1800f.
  • the data in the server 1800f is transmitted in advance without performing such transmission / reception. You may keep it.
  • FIG. 131B is a block diagram showing a configuration of a playback apparatus that performs synchronized playback by the method e described above.
  • the playback device B10 is a receiver 1800a or a terminal device that performs synchronous playback by the method e described above, and includes a sensor B11, a request signal transmission unit B12, a content reception unit B13, a clock B14, and a playback unit B15. I have.
  • Sensor B11 is, for example, an image sensor, and receives the visible light signal from a transmitter 1800d that transmits a visible light signal according to a change in luminance of the light source.
  • the request signal transmission unit B12 transmits a request signal for requesting content associated with the visible light signal to the server 1800f.
  • the content receiving unit B13 receives content including each time and data reproduced at each time from the server 1800f.
  • the reproduction unit B15 reproduces data corresponding to the time of the clock B14 in the content.
  • FIG. 131C is a flowchart showing the processing operation of the terminal device that performs synchronous reproduction by the method e described above.
  • the playback device B10 is a receiver 1800a or a terminal device that performs synchronized playback by the method e described above, and executes each process of steps SB11 to SB15.
  • step SB11 the visible light signal is received from the transmitter 1800d that transmits the visible light signal according to the luminance change of the light source.
  • step SB12 a request signal for requesting content associated with the visible light signal is transmitted to server 1800f.
  • step SB13 content including each time and data reproduced at each time is received from server 1800f.
  • step SB15 data corresponding to the time of the clock B14 is reproduced from the content.
  • the data in the content can be appropriately played back at the correct time indicated by the content without being played back at the wrong time.
  • each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component.
  • Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.
  • the software that realizes the playback apparatus B10 and the like of the present embodiment is a program that causes a computer to execute each step included in the flowchart shown in FIG. 131C.
  • FIG. 132 is a diagram for explaining preparations for synchronized playback in the sixteenth embodiment.
  • the receiver 1800a adjusts the time of the clock provided in the receiver 1800a to the time of the reference clock in order to perform synchronized playback. For this time adjustment, the receiver 1800a performs the following processes (1) to (5).
  • the receiver 1800a receives a signal.
  • This signal may be a visible light signal transmitted by a change in luminance of the display of the transmitter 1800d, or a radio wave signal based on Wi-Fi or Bluetooth (registered trademark) from a wireless device.
  • the receiver 1800a acquires position information indicating the position of the receiver 1800a by, for example, GPS instead of receiving such a signal. Then, the receiver 1800a recognizes that the receiver 1800a has entered a predetermined place or building based on the position information.
  • the receiver 1800a When the receiver 1800a receives the above signal or recognizes that it has entered a predetermined location, it receives a request signal for requesting data (related information) associated with the signal or location. It transmits to the server (visible light ID resolution server) 1800f.
  • the server visible light ID resolution server
  • the server 1800f transmits the above-described data and a time adjustment request for causing the receiver 1800a to adjust the time to the receiver 1800a.
  • the receiver 1800a When receiving the data and the time adjustment request, the receiver 1800a transmits the time adjustment request to the GPS time server, the NTP server, or the base station of the telecommunications carrier (carrier).
  • the server or the base station Upon receiving the time adjustment request, the server or the base station transmits time data (time information) indicating the current time (reference clock time or absolute time) to the receiver 1800a.
  • time data time information
  • the receiver 1800a adjusts the time by adjusting the time of the clock provided to the receiver 1800a to the current time indicated by the time data.
  • a GPS (Global Positioning System) radio wave or an NTP (Network Time Protocol) radio wave is used between the clock provided in the receiver 1800a (terminal device) and the reference clock. Synchronized. Therefore, the receiver 1800a can reproduce the data corresponding to the time at an appropriate time according to the reference clock.
  • FIG. 133 is a diagram illustrating an example of application of the receiver 1800a according to the sixteenth embodiment.
  • the receiver 1800a is configured as a smartphone as described above, and is used by being held by a holder 1810 formed of, for example, a translucent resin or glass member.
  • the holder 1810 includes a back plate portion 1810a and a locking portion 1810b provided upright on the back plate portion 1810a.
  • the receiver 1800a is inserted between the back plate portion 1810a and the locking portion 1810b so as to be along the back plate portion 1810a.
  • FIG. 134A is a front view of receiver 1800a held by holder 1810 in Embodiment 16.
  • FIG. 134A is a front view of receiver 1800a held by holder 1810 in Embodiment 16.
  • the receiver 1800a is held by the holder 1810 in the inserted state as described above.
  • the locking portion 1810b locks with the lower portion of the receiver 1800a and sandwiches the lower portion with the back plate portion 1810a.
  • the back surface of the receiver 1800a faces the back plate portion 1810a, and the display 1801 of the receiver 1800a is exposed.
  • FIG. 134B is a rear view of receiver 1800a held by holder 1810 in the sixteenth embodiment.
  • a through hole 1811 is formed in the back plate portion 1810a, and a variable filter 1812 is attached in the vicinity of the through hole 1811.
  • camera 1802 of receiver 1800a is exposed through back hole 1811 from back plate portion 1810a.
  • the flashlight 1803 of the receiver 1800a faces the variable filter 1812.
  • the variable filter 1812 is formed in a disk shape, for example, and has three color filters (a red filter, a yellow filter, and a green filter) each having a fan shape and the same size.
  • the variable filter 1812 is attached to the back plate portion 1810a so as to be rotatable about the center of the variable filter 1812.
  • the red filter is a filter having red translucency
  • the yellow filter is a filter having yellow translucency
  • the green filter is a filter having green translucency.
  • variable filter 1812 is rotated, and, for example, the red filter is disposed at a position facing the flashlight 1803a.
  • the light emitted from the flashlight 1803a is diffused inside the holder 1810 as red light by passing through the red filter.
  • substantially the entire holder 1810 emits red light.
  • variable filter 1812 is rotated and, for example, the yellow filter is disposed at a position facing the flashlight 1803a.
  • the light emitted from the flashlight 1803a is diffused inside the holder 1810 as yellow light by passing through the yellow filter.
  • substantially the entire holder 1810 emits yellow light.
  • variable filter 1812 is rotated so that, for example, the green filter is disposed at a position facing the flashlight 1803a.
  • the light emitted from the flashlight 1803a is diffused inside the holder 1810 as green light by passing through the green filter.
  • substantially the entire holder 1810 emits green light.
  • the holder 1810 lights in red, yellow or green like a penlight.
  • FIG. 135 is a diagram for describing a use case of the receiver 1800a held by the holder 1810 in the sixteenth embodiment.
  • a receiver with a holder that is a receiver 1800a held by a holder 1810 is used in an amusement park or the like. That is, the plurality of receivers with holders that are directed to the float moving in the amusement park blink in synchronization with the music flowing from the float.
  • the float is configured as a transmitter in each of the above embodiments, and transmits a visible light signal by a change in luminance of a light source attached to the float.
  • the float transmits a visible light signal indicating the ID of the float.
  • the receiver with a holder receives the visible light signal, ie, ID, by imaging
  • the receiver 1800a that has received the ID acquires a program associated with the ID from, for example, a server.
  • This program includes instructions for turning on the flashlight 1803 of the receiver 1800a at each predetermined time. Each predetermined time is set in accordance with the music flowing from the float (so as to be synchronized). Then, the receiver 1800a blinks the flashlight 1803a according to the program.
  • each receiver 1800a that has received the ID repeats lighting at the same timing according to the music flowing from the float of the ID.
  • each receiver 1800a blinks the flashlight 1803 in accordance with a set color filter (hereinafter referred to as a setting filter).
  • the setting filter is a color filter that faces the flashlight 1803 of the receiver 1800a.
  • Each receiver 1800a recognizes the current setting filter based on an operation by the user. Alternatively, each receiver 1800a recognizes the current setting filter based on the color of an image obtained by photographing with the camera 1802.
  • the receiver 1800a held in the holder 1810 is synchronized with the float music and the receiver 1800a held in the other holder 1810 in the same manner as the synchronous playback shown in FIGS. 123 to 129 described above. Then, the flashlight 1803, that is, the holder 1810 is blinked.
  • FIG. 136 is a flowchart showing the processing operation of the receiver 1800a held by the holder 1810 in the sixteenth embodiment.
  • the receiver 1800a receives the float ID indicated by the visible light signal from the float (step S1831). Next, the receiver 1800a acquires a program associated with the ID from the server (step S1832). Next, the receiver 1800a executes the program to turn on the flashlight 1803 at each predetermined time according to the setting filter (step S1833).
  • the receiver 1800a may cause the display 1801 to display an image corresponding to the received ID or the acquired program.
  • FIG. 137 is a diagram illustrating an example of an image displayed by the receiver 1800a according to the sixteenth embodiment.
  • the receiver 1800a when the receiver 1800a receives an ID from a Santa Claus float, the receiver 1800a displays a Santa Claus image as shown in FIG. Further, as illustrated in FIG. 137 (b), the receiver 1800a may change the background color of the Santa Claus image to the color of the setting filter simultaneously with the lighting of the flashlight 1803. For example, when the color of the setting filter is red, the holder 1810 is lit red by turning on the flashlight 1803, and at the same time, a Santa Claus image having a red background color is displayed on the display 1801. That is, the blinking of the holder 1810 and the display on the display 1801 are synchronized.
  • FIG. 138 is a diagram showing another example of the holder according to the sixteenth embodiment.
  • the holder 1820 is configured in the same manner as the holder 1810 described above, but does not include the through hole 1811 and the variable filter 1812.
  • a holder 1820 holds the receiver 1800a in a state where the display 1801 of the receiver 1800a is directed to the back plate portion 1820a.
  • the receiver 1800a causes the display 1801 to emit light instead of the flashlight 1803.
  • light from the display 1801 is diffused over substantially the entire holder 1820. Therefore, when the receiver 1800a causes the display 1801 to emit light with red light according to the above-described program, the holder 1820 is lit red. Similarly, when the receiver 1800a causes the display 1801 to emit light with yellow light according to the above-described program, the holder 1820 is lit in yellow.
  • the holder 1820 lights up in green. If such a holder 1820 is used, the setting of the variable filter 1812 can be omitted.
  • FIG. 17 (Embodiment 17) (Visible light signal) 139A to 139D are diagrams illustrating an example of a visible light signal in Embodiment 17.
  • FIG. 17 (Embodiment 17) (Visible light signal) 139A to 139D are diagrams illustrating an example of a visible light signal in Embodiment 17.
  • the transmitter generates a 4PPM visible light signal and changes the luminance in accordance with the visible light signal, for example, as shown in FIG. 139A.
  • the transmitter allocates 4 slots to one signal unit, and generates a visible light signal composed of a plurality of signal units.
  • the signal unit indicates High (H) or Low (L) for each slot.
  • the transmitter emits light brightly in the H slot and emits light darkly or extinguishes in the L slot.
  • one slot is a period corresponding to a time of 1/9600 seconds.
  • the transmitter may generate a visible light signal in which the number of slots allocated to one signal unit is variable.
  • the signal unit includes a signal indicating H in one or more consecutive slots and a signal indicating L in one slot following the H signal. Since the number of slots of H is variable, the total number of slots in the signal unit is variable.
  • the transmitter generates a visible light signal including these signal units in the order of a signal unit of 3 slots, a signal unit of 4 slots, and a signal unit of 6 slots. Also in this case, the transmitter emits light brightly in the H slot and emits light darkly or extinguishes in the L slot.
  • the transmitter may assign an arbitrary period (signal unit period) to one signal unit without assigning a plurality of slots to one signal unit.
  • the signal unit period includes an H period and an L period following the H period.
  • the period of H is adjusted according to the signal before modulation.
  • the period L may be fixed and may be a period corresponding to the slot.
  • the H period and the L period are, for example, periods of 100 ⁇ s or more. For example, as shown in FIG.
  • the transmitter transmits a visible light signal including signal units in the order of a signal unit having a signal unit period of 210 ⁇ s, a signal unit having a signal unit period of 220 ⁇ s, and a signal unit having a signal unit period of 230 ⁇ s.
  • the transmitter emits light brightly during the H period and emits light darkly or extinguishes during the L period.
  • the transmitter may generate a signal indicating L and H alternately as a visible light signal.
  • the L period and the H period in the visible light signal are adjusted according to the signals before modulation.
  • the transmitter indicates H for a period of 100 ⁇ s, then indicates L for a period of 120 ⁇ s, then indicates H for a period of 110 ⁇ s, and further indicates L for a period of 200 ⁇ s.
  • a visible light signal is transmitted.
  • the transmitter emits light brightly during the H period and emits light darkly or extinguishes during the L period.
  • FIG. 140 is a diagram showing a configuration of a visible light signal in the seventeenth embodiment.
  • the visible light signal includes, for example, a signal 1, a brightness adjustment signal corresponding to the signal 1, a signal 2, and a brightness adjustment signal corresponding to the signal 2.
  • the transmitter When the transmitter generates the signal 1 and the signal 2 by modulating the signals before modulation, the transmitter generates a brightness adjustment signal for the signals and generates the above-described visible light signal.
  • the brightness adjustment signal corresponding to signal 1 is a signal that compensates for increase / decrease in brightness due to a luminance change according to signal 1.
  • the brightness adjustment signal corresponding to the signal 2 is a signal that compensates for increase / decrease in brightness due to a luminance change according to the signal 2.
  • the brightness B1 is expressed by the luminance change according to the signal 1 and the brightness adjustment signal of the signal 1
  • Brightness B2 is expressed.
  • the transmitter in the present embodiment generates the brightness adjustment signals of signal 1 and signal 2 as part of the visible light signal so that the brightness B1 and brightness B2 are equal. Thereby, the brightness is kept constant and flicker can be suppressed.
  • the transmitter 1 when the transmitter 1 generates the signal 1, the transmitter 1 generates the signal 1 including the data 1, the preamble (header) following the data 1, and the data 1 following the preamble.
  • the preamble is a signal corresponding to data 1 arranged before and after the preamble.
  • this preamble is a signal serving as an identifier for reading data 1.
  • FIG. 141 is a diagram illustrating an example of bright line images obtained by imaging of the receiver in Embodiment 17.
  • the receiver captures a bright line image including a visible light signal transmitted from the transmitter as a bright line pattern by capturing an image of the transmitter that changes in luminance. With such imaging, a visible light signal is received by the receiver.
  • the receiver uses N exposure lines included in the image sensor to capture an image at time t1, so that each line includes a region a and a region b where a bright line pattern appears. Get an image. Regions a and b are regions in which bright line patterns appear when the luminance of the transmitter, which is the subject, changes.
  • the receiver demodulates the visible light signal from the bright line pattern of the region a and the region b. However, if the receiver determines that the demodulated visible light signal alone is not sufficient, only M (M ⁇ N) consecutive exposure lines corresponding to the area a are used among the N exposure lines. The image is taken at time t2. Thereby, the receiver acquires a bright line image including only the region a out of the regions a and b. The receiver repeatedly performs such imaging at times t3 to t5. As a result, a visible light signal having a sufficient amount of data from the subject corresponding to the region a can be received at high speed.
  • the receiver captures an image at time t6 using only L (L ⁇ N) consecutive exposure lines corresponding to the region b among the N exposure lines. Thereby, the receiver acquires a bright line image including only the region b out of the regions a and b.
  • the receiver repeatedly performs such imaging at times t7 to t9. As a result, a visible light signal having a sufficient amount of data from the subject corresponding to the region b can be received at high speed.
  • the receiver may acquire a bright line image including only the region a by performing the same imaging at the times t2 to t5 at the times t10 and t11. Further, the receiver may acquire a bright line image including only the region b by performing imaging similar to that at times t6 to t9 at times t12 and t13.
  • the receiver when the receiver determines that the visible light signal is insufficient, the receiver performs continuous shooting of the bright line image including only the region a from time t2 to t5. If bright lines appear in the image obtained by the above, the above-described continuous shooting may be performed. Similarly, when the receiver determines that the visible light signal is insufficient, the receiver performs continuous shooting of the bright line image including only the region b from time t6 to time t9, which is obtained by imaging at time t1. If bright lines appear in the image, the above-described continuous shooting may be performed. The receiver may alternately perform acquisition of a bright line image including only the region a and acquisition of a bright line image including only the region b.
  • the M consecutive exposure lines corresponding to the area a are exposure lines that contribute to the generation of the area a
  • the L consecutive exposure lines corresponding to the area b are the generation of the area b. Is an exposure line that contributes to
  • FIG. 142 is a diagram illustrating another example of the bright line image obtained by imaging of the receiver in the seventeenth embodiment.
  • the receiver captures an image at time t1 using N exposure lines included in the image sensor, so that each line includes a region a and a region b where a bright line pattern appears.
  • each line includes a region a and a region b where a bright line pattern appears.
  • Each of the areas a and b is an area where a bright line pattern appears when the luminance of the transmitter, which is a subject, changes as described above.
  • each of the region a and the region b has a region that overlaps with each other along the direction of the bright line or the exposure line (hereinafter referred to as an overlapping region).
  • the receiver determines that the visible light signal demodulated from the bright line pattern of the area a and the area b is insufficient, P (P ⁇ N) corresponding to the overlapping area among the N exposure lines.
  • An image is taken at time t2 using only the continuous exposure lines.
  • the receiver acquires a bright line image including only the overlapping regions of the region a and the region b.
  • the receiver repeatedly performs such imaging at times t3 and t4.
  • a visible light signal having a sufficient amount of data from the subject corresponding to each of the region a and the region b can be received substantially simultaneously and at high speed.
  • FIG. 143 is a diagram illustrating another example of the bright line image obtained by imaging by the receiver in the seventeenth embodiment.
  • the receiver uses the N exposure lines included in the image sensor to capture an image at time t ⁇ b> 1, thereby clearly displaying a portion a in which the bright line pattern appears unclearly.
  • a bright line image including an area including the appearing portion b is acquired. Similar to the above, this region is a region where a bright line pattern appears when the luminance of the transmitter that is the subject changes.
  • the receiver may perform continuous shooting of the bright line image including only the part a after the continuous shooting of the bright line image including only the part b.
  • the receiver assigns an order to each region, and according to the order, only that region is included.
  • the order may be an order corresponding to the magnitude of the signal (area or area size) or an order corresponding to the clarity of the bright line.
  • the order may be an order corresponding to the color of light from the subject corresponding to these areas. For example, the first continuous shooting is performed on a region corresponding to red light, and the next continuous shooting is performed on a region corresponding to white light. Further, only continuous shooting of the area with respect to red light may be performed.
  • (HDR synthesis) 144 is a diagram for describing adaptation of the receiver in Embodiment 17 to a camera system that performs HDR synthesis.
  • FIG. 1 A diagram for describing adaptation of the receiver in Embodiment 17 to a camera system that performs HDR synthesis.
  • the vehicle is equipped with a camera system to prevent collisions.
  • This camera system performs HDR (High Dynamic Range) composition using an image obtained by imaging by a camera. By this HDR synthesis, an image with a wide dynamic range of luminance can be obtained.
  • the camera system recognizes surrounding vehicles, obstacles, or people based on this wide dynamic range image.
  • the camera system has a normal setting mode and a communication setting mode as setting modes.
  • the setting mode is the normal setting mode, for example, as shown in FIG. 144
  • the camera system performs four times of imaging at the same 1/100 second shutter speed and at different sensitivities at times t1 to t4. .
  • the camera system performs HDR synthesis using the four images obtained by the four imaging operations.
  • the camera system may not perform HDR synthesis.
  • the camera system performs imaging three times with a shutter speed of 1/10000 seconds and different sensitivities from time t10 to t12.
  • the camera system recognizes a surrounding vehicle, an obstacle, a person, or the like from an image obtained by the first one of the four images. Further, the camera system receives a visible light signal by the last three imagings among the above four imagings, and demodulates the bright line pattern appearing in the image obtained by the imaging.
  • imaging is performed with different sensitivities at times t10 to t12.
  • imaging may be performed with the same sensitivity.
  • Such a camera system can perform HDR synthesis and can also receive a visible light signal.
  • FIG. 145 is a diagram for explaining the processing operation of the visible light communication system in the seventeenth embodiment.
  • This visible light communication system includes, for example, a transmitter arranged at a cash register, a smartphone as a receiver, and a server. Note that the communication between the smartphone and the server and the communication between the transmitter and the server are each performed via a secure communication line. Communication between the transmitter and the smartphone is performed by visible light communication.
  • the visible light communication system according to the present embodiment ensures security by determining whether a visible light signal from a transmitter is accurately received by a smartphone.
  • the transmitter transmits, for example, a visible light signal indicating the value “100” to the smartphone by changing the luminance at time t1.
  • the smartphone receives the visible light signal at time t2
  • the smartphone transmits a radio signal indicating the value “100” to the server.
  • the server receives the radio signal from the smartphone at time t3.
  • the server performs a process for determining whether or not the value “100” indicated by the radio signal is the value of the visible light signal received by the smartphone from the transmitter. That is, the server transmits, for example, a radio signal indicating a value “200” to the transmitter.
  • the transmitter that has received the radio wave signal transmits a visible light signal indicating the value “200” to the smartphone by changing the luminance at time t4.
  • the smartphone When the smartphone receives the visible light signal at time t5, the smartphone transmits a radio signal indicating the value “200” to the server.
  • the server receives the radio signal from the smartphone at time t6.
  • the server determines whether or not the value indicated by the received radio signal is the same as the value indicated by the radio signal transmitted at time t3. If they are the same, the server determines that the value “100” indicated by the visible light signal received at time t3 is the value of the visible light signal transmitted from the transmitter to the smartphone and received. On the other hand, if not the same, the server determines that the value “100” indicated by the visible light signal received at time t3 is suspicious as the value of the visible light signal transmitted from the transmitter to the smartphone and received.
  • communication using a radio wave signal is performed between the smartphone, the server, and the transmitter.
  • communication using an optical signal other than a visible light signal or communication using an electrical signal may be performed.
  • the visible light signal transmitted from the transmitter to the smartphone indicates, for example, a charging value, a coupon value, a monster value, or a bingo value.
  • (Vehicle related) 146A is a diagram illustrating an example of vehicle-to-vehicle communication using visible light in Embodiment 17.
  • FIG. 1 is a diagram illustrating an example of vehicle-to-vehicle communication using visible light in Embodiment 17.
  • the head vehicle recognizes that there is an accident in the direction of travel by a sensor (camera etc.) mounted on the vehicle.
  • the leading vehicle transmits a visible light signal by changing the brightness of the tail lamp.
  • the leading vehicle transmits a visible light signal that prompts the subsequent vehicle to decelerate.
  • the succeeding vehicle receives the visible light signal by imaging with a camera mounted on the vehicle, the following vehicle decelerates according to the visible light signal and further transmits a visible light signal that prompts the subsequent vehicle to decelerate. To do.
  • the visible light signal that prompts deceleration is sequentially transmitted from the head to a plurality of vehicles traveling in a line, and the vehicle that receives the visible light signal decelerates. Since the transmission of the visible light signal to each vehicle is performed quickly, the plurality of vehicles can be decelerated in the same manner at substantially the same time. Therefore, it is possible to reduce traffic congestion due to accidents.
  • FIG. 146B is a diagram illustrating another example of vehicle-to-vehicle communication using visible light according to the seventeenth embodiment.
  • the front vehicle may transmit a visible light signal indicating a message (for example, “thank you”) to the subsequent vehicle by changing the brightness of the tail lamp.
  • This message is generated, for example, by a user operation on a smartphone.
  • a smart phone transmits the signal which shows the message to the above-mentioned previous vehicle.
  • the preceding vehicle can transmit a visible light signal indicating the message to the subsequent vehicle.
  • FIG. 147 is a diagram illustrating an example of a method for determining the positions of a plurality of LEDs in the seventeenth embodiment.
  • a vehicle headlight has a plurality of LEDs (Light Emitting Diodes).
  • the transmitter of this vehicle transmits a visible light signal from each LED by individually changing the brightness of each of the plurality of LEDs of the headlight.
  • Other vehicle receivers receive visible light signals from their LEDs by imaging the vehicle with the headlights.
  • the receiver determines the position of each of the plurality of LEDs from the image obtained by the imaging in order to recognize which LED the received visible light signal is transmitted from. .
  • the receiver uses an acceleration sensor attached to the same vehicle as the receiver, and uses a plurality of gravitational directions (for example, a downward arrow in FIG. 147) indicated by the acceleration sensor as a reference. Determine the position of each LED.
  • an LED is used as an example of a light-emitting body that changes in luminance, but a light-emitting body other than the LED may be used.
  • FIG. 148 is a diagram illustrating an example of a bright line image obtained by imaging a vehicle in the seventeenth embodiment.
  • a receiver mounted on a traveling vehicle acquires a bright line image shown in FIG. 148 by imaging a subsequent vehicle (following vehicle).
  • the transmitter mounted on the following vehicle transmits a visible light signal to the preceding vehicle by changing the brightness of the two headlights of the vehicle.
  • a camera for imaging the rear is attached to the rear part of the front vehicle or the side mirror.
  • the receiver acquires a bright line image by imaging with the camera of the following vehicle as a subject, and demodulates a bright line pattern (visible light signal) included in the bright line image. Thereby, the visible light signal transmitted from the transmitter of the following vehicle is received by the receiver of the preceding vehicle.
  • the receiver acquires the ID of the vehicle having the headlight, the speed of the vehicle, and the type of the vehicle from each of the visible light signals transmitted and demodulated from the two headlights. If the IDs of the two visible light signals are the same, the receiver determines that the two visible light signals are signals transmitted from the same vehicle. And a receiver specifies the length (distance between lights) between the two headlights which the vehicle has from the model of the vehicle. Further, the receiver measures a distance L1 between two regions where the bright line pattern appears, which is included in the bright line image. Then, the receiver calculates the distance (inter-vehicle distance) from the vehicle on which the receiver is mounted to the following vehicle by triangulation using the distance L1 and the inter-light distance. The receiver determines the risk of collision based on the inter-vehicle distance and the vehicle speed acquired from the visible light signal, and notifies the vehicle driver of a warning corresponding to the determination result. Thereby, the collision of a vehicle can be avoided.
  • the receiver specifies the inter-light distance from the vehicle type included in the visible light signal, but may specify the inter-light distance from information other than the vehicle type.
  • the receiver issues a warning when it is determined that there is a risk of a collision.
  • the receiver may output a control signal for causing the vehicle to perform an operation to avoid the risk. Good.
  • the control signal is a signal for accelerating the vehicle or a signal for causing the vehicle to change lanes.
  • the camera images the following vehicle, but may image the oncoming vehicle.
  • the receiver determines from the image obtained by imaging by the camera that fog is in the vicinity of the receiver (that is, the vehicle equipped with the receiver)
  • the receiver enters a mode for receiving the visible light signal as described above. May be.
  • the receiver of the vehicle can identify the position and speed of the oncoming vehicle by receiving the visible light signal transmitted from the headlight of the oncoming vehicle, even if fog is in the vicinity.
  • FIG. 149 is a diagram illustrating an example of application of the receiver and the transmitter in the seventeenth embodiment.
  • FIG. 149 is a view of the automobile from the back.
  • a transmitter (car) 7006a having two tail lamps (light emitting unit or light) of a car transmits identification information (ID) of the transmitter 7006a to a receiver configured as a smartphone, for example.
  • ID identification information
  • the receiver acquires information associated with the ID from the server.
  • the information includes the ID of the car or transmitter, the distance between the light emitting parts, the size of the light emitting part, the size of the car, the shape of the car, the weight of the car, the car number, the appearance in front, or the danger. This is information indicating the presence or absence of.
  • the receiver may acquire these pieces of information directly from the transmitter 7006a.
  • FIG. 150 is a flowchart illustrating an example of processing operations of the receiver and the transmitter 7006a in the seventeenth embodiment.
  • the ID of the transmitter 7006a and the information passed to the receiver that has received the ID are associated with each other and stored in the server (Step 7106a).
  • the information to be passed to the receiver includes identification of the size of the light emitting unit that becomes the transmitter 7006a, the distance between the light emitting units, the shape of the object having the transmitter 7006a as a component, the weight, the body number, etc. Information such as numbers, places that are difficult to observe from the receiver, and presence or absence of danger may be included.
  • the transmitter 7006a transmits the ID (Step 7106b).
  • the transmission content may include the URL of the server and information stored in the server.
  • the receiver receives information such as the transmitted ID (Step 7106c).
  • the receiver acquires information associated with the received ID from the server (Step 7106d).
  • the receiver displays the received information and the information acquired from the server (Step 7106e).
  • the receiver and the light emitting unit can be triangulated from the size information of the light emitting unit and the appearance size of the imaged light emitting unit, or from the distance information between the light emitting units and the distance between the imaged light emitting units. Is calculated (Step 7106f).
  • the receiver issues a warning of danger based on information such as the state of the place that is difficult to observe from the receiver and the presence or absence of danger (Step 7106g).
  • FIG. 151 is a diagram illustrating an example of application of the receiver and the transmitter in the seventeenth embodiment.
  • a transmitter (car) 7007b having two tail lamps (light emitting unit or light) of a car transmits information of the transmitter 7007b to a receiver 7007a configured as a transmission / reception device of a parking lot, for example.
  • the information of the transmitter 7007b indicates identification information (ID) of the transmitter 7007b, a car number, a car size, a car shape, or a car weight.
  • ID identification information
  • the receiver 7007a transmits information indicating whether parking is possible, billing information, or a parking position. Note that the receiver 7007a may receive the ID and acquire information other than the ID from the server.
  • FIG. 152 is a flowchart illustrating an example of processing operations of the receiver 7007a and the transmitter 7007b according to the seventeenth embodiment.
  • the transmitter 7007b includes an in-vehicle transmitter and an in-vehicle receiver in order to perform not only transmission but also reception.
  • the ID of the transmitter 7007b and the information passed to the receiver 7007a that has received the ID are associated with each other and stored in the server (parking lot management server) (Step 7107a).
  • Information to be passed to the receiver 7007a includes an identification number such as the shape, weight, and body number of the object having the transmitter 7007b as a component, the identification number of the user of the transmitter 7007b, and information for payment. May be included.
  • the transmitter 7007b transmits the ID (Step 7107b).
  • the contents of transmission may include the URL of the server and information stored in the server.
  • the parking lot receiver 7007a (parking lot transmission / reception device) transmits the received information to a server (parking lot management server) that manages the parking lot (Step 7107c).
  • the parking lot management server acquires information associated with the ID using the ID of the transmitter 7007b as a key (Step 7107d).
  • the parking lot management server investigates the parking lot availability (Step 7107e).
  • the parking lot receiver 7007a (parking lot transmission / reception device) transmits whether or not parking is possible, parking position information, or the address of a server holding these pieces of information (Step 7107f). Or a parking lot management server transmits such information to another server.
  • the transmitter (on-vehicle receiver) 7007b receives the information transmitted above (Step 7107g). Or an in-vehicle system acquires these information from another server.
  • the parking lot management server controls the parking lot so as to facilitate parking (Step 7107h). For example, control of a multistory parking lot is performed.
  • the transmission / reception device of the parking lot transmits the ID (Step 7107i).
  • the in-vehicle receiver (transmitter 7007b) makes an inquiry to the parking lot management server based on the user information of the in-vehicle receiver and the received ID (Step 7107j).
  • the parking lot management server charges according to the parking time (7107k).
  • the parking lot management server controls the parking lot so that the parked vehicle can be easily accessed (Step 7107m). For example, control of a multistory parking lot is performed.
  • the in-vehicle receiver (transmitter 7007b) displays a map to the parking position and performs navigation from the current location (Step 7107n).
  • FIG. 153 is a diagram illustrating a configuration of a visible light communication system applied to the inside of a train in Embodiment 17.
  • the visible light communication system includes, for example, a plurality of lighting devices 1905 arranged in a train, a smartphone 1906 held by a user, a server 1904, and a camera 1903 arranged in the train.

Abstract

L'invention concerne un procédé de transmission comprenant : une étape (S551) consistant à recevoir, en tant que degré d'ajustement de lumière désigné, un degré d'ajustement de lumière désigné par rapport à une source de lumière ; et une étape (S552) dans laquelle, lorsque le degré d'ajustement de lumière désigné n'est pas supérieur à une première valeur, une source de lumière émet de la lumière au degré d'ajustement de lumière désigné tandis qu'un signal codé dans un premier mode est transmis via des changements de luminance, et lorsque le degré d'ajustement de lumière désigné est supérieur à la première valeur, la source de lumière émet de la lumière au degré d'ajustement de lumière désigné tandis qu'un signal codé dans un second mode est transmis via des changements de luminance. La valeur de courant de crête de la source de lumière lorsque le degré d'ajustement de lumière désigné est supérieur à la première valeur mais pas plus d'une seconde valeur est inférieure à la valeur de courant de crête de la source de lumière lorsque la valeur d'ajustement de lumière désignée est égale à la première valeur.
PCT/JP2017/040032 2016-11-10 2017-11-07 Procédé de transmission, dispositif de transmission, et programme WO2018088380A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201780069560.4A CN110114988B (zh) 2016-11-10 2017-11-07 发送方法、发送装置及记录介质
JP2018550203A JP7023239B2 (ja) 2016-11-10 2017-11-07 送信方法、送信装置、およびプログラム
US16/408,537 US10819428B2 (en) 2016-11-10 2019-05-10 Transmitting method, transmitting apparatus, and program

Applications Claiming Priority (20)

Application Number Priority Date Filing Date Title
JP2016220024 2016-11-10
JP2016-220024 2016-11-10
US201662434644P 2016-12-15 2016-12-15
JP2016-243825 2016-12-15
JP2016243825 2016-12-15
US62/434644 2016-12-15
US201762446632P 2017-01-16 2017-01-16
US62/446632 2017-01-16
US201762457382P 2017-02-10 2017-02-10
US62/457382 2017-02-10
US201762466534P 2017-03-03 2017-03-03
US62/466534 2017-03-03
US201762467376P 2017-03-06 2017-03-06
US62/467376 2017-03-06
JP2017-080595 2017-04-14
JP2017080595 2017-04-14
JP2017080664 2017-04-14
JP2017-080664 2017-04-14
US201762558629P 2017-09-14 2017-09-14
US62/558629 2017-09-14

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/408,537 Continuation US10819428B2 (en) 2016-11-10 2019-05-10 Transmitting method, transmitting apparatus, and program

Publications (1)

Publication Number Publication Date
WO2018088380A1 true WO2018088380A1 (fr) 2018-05-17

Family

ID=62110315

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/040032 WO2018088380A1 (fr) 2016-11-10 2017-11-07 Procédé de transmission, dispositif de transmission, et programme

Country Status (3)

Country Link
JP (1) JP7023239B2 (fr)
TW (1) TWI736702B (fr)
WO (1) WO2018088380A1 (fr)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3393132A4 (fr) * 2015-12-17 2019-01-16 Panasonic Intellectual Property Corporation of America Procédé d'affichage et dispositif d'affichage
US10263701B2 (en) 2015-11-12 2019-04-16 Panasonic Intellectual Property Corporation Of America Display method, non-transitory recording medium, and display device
US10389446B2 (en) 2014-11-14 2019-08-20 Panasonic Intellectual Property Corporation Of America Reproduction method for reproducing contents
US10819428B2 (en) 2016-11-10 2020-10-27 Panasonic Intellectual Property Corporation Of America Transmitting method, transmitting apparatus, and program
CN112383366A (zh) * 2020-11-12 2021-02-19 广州通导信息技术服务有限公司 一种数字荧光频谱的频谱监测方法、装置及存储介质
CN113268400A (zh) * 2021-04-27 2021-08-17 新华三信息技术有限公司 一种指示灯同步闪烁方法、装置以及服务器
WO2021248341A1 (fr) * 2020-06-10 2021-12-16 京东方科技集团股份有限公司 Appareil, système et procédé de communication optique
TWI769471B (zh) * 2020-07-02 2022-07-01 黑快馬股份有限公司 自動攝錄影跟拍系統及自動攝錄影跟拍方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230142456A1 (en) * 2016-10-21 2023-05-11 Panasonic Intellectual Property Corporation Of America Transmission device, reception device, communication system, transmission method, reception method, and communication method
TWI706385B (zh) * 2019-10-21 2020-10-01 大陸商南京深視光點科技有限公司 車用光訊號傳遞接收系統及其實施方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09312612A (ja) * 1996-05-24 1997-12-02 Sharp Corp 光通信用発光回路
JP2008206087A (ja) * 2007-02-22 2008-09-04 Matsushita Electric Works Ltd 可視光通信システム
JP2010056644A (ja) * 2008-08-26 2010-03-11 Panasonic Electric Works Co Ltd 可視光通信システム
JP2011198524A (ja) * 2010-03-17 2011-10-06 Mitsubishi Electric Lighting Corp 点灯装置
JP2015173508A (ja) * 2009-09-18 2015-10-01 インターデイジタル パテント ホールディングス インコーポレイテッド 可視光通信（ｖｌｃ）のためのレート制御を有する調光方法および装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8446481B1 (en) * 2012-09-11 2013-05-21 Google Inc. Interleaved capture for high dynamic range image acquisition and synthesis
CN105874728B (zh) * 2012-12-27 2019-04-05 松下电器（美国）知识产权公司 信息通信方法及信息通信装置
WO2015075937A1 (fr) * 2013-11-22 2015-05-28 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Programme de traitement d'informations, programme de réception, et dispositif de traitement d'informations
JP2015184778A (ja) * 2014-03-20 2015-10-22 コニカミノルタ株式会社 拡張現実表示システム、拡張現実情報生成装置、拡張現実表示装置、サーバー、拡張現実情報生成プログラム、拡張現実表示プログラムおよび拡張現実情報のデータ構造

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09312612A (ja) * 1996-05-24 1997-12-02 Sharp Corp 光通信用発光回路
JP2008206087A (ja) * 2007-02-22 2008-09-04 Matsushita Electric Works Ltd 可視光通信システム
JP2010056644A (ja) * 2008-08-26 2010-03-11 Panasonic Electric Works Co Ltd 可視光通信システム
JP2015173508A (ja) * 2009-09-18 2015-10-01 インターデイジタル パテント ホールディングス インコーポレイテッド 可視光通信（ｖｌｃ）のためのレート制御を有する調光方法および装置
JP2011198524A (ja) * 2010-03-17 2011-10-06 Mitsubishi Electric Lighting Corp 点灯装置

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10389446B2 (en) 2014-11-14 2019-08-20 Panasonic Intellectual Property Corporation Of America Reproduction method for reproducing contents
US10263701B2 (en) 2015-11-12 2019-04-16 Panasonic Intellectual Property Corporation Of America Display method, non-transitory recording medium, and display device
US10951309B2 (en) 2015-11-12 2021-03-16 Panasonic Intellectual Property Corporation Of America Display method, non-transitory recording medium, and display device
EP3393132A4 (fr) * 2015-12-17 2019-01-16 Panasonic Intellectual Property Corporation of America Procédé d'affichage et dispositif d'affichage
US10504584B2 (en) 2015-12-17 2019-12-10 Panasonic Intellectual Property Corporation Of America Display method and display device
US10819428B2 (en) 2016-11-10 2020-10-27 Panasonic Intellectual Property Corporation Of America Transmitting method, transmitting apparatus, and program
WO2021248341A1 (fr) * 2020-06-10 2021-12-16 京东方科技集团股份有限公司 Appareil, système et procédé de communication optique
TWI769471B (zh) * 2020-07-02 2022-07-01 黑快馬股份有限公司 自動攝錄影跟拍系統及自動攝錄影跟拍方法
CN112383366A (zh) * 2020-11-12 2021-02-19 广州通导信息技术服务有限公司 一种数字荧光频谱的频谱监测方法、装置及存储介质
CN113268400A (zh) * 2021-04-27 2021-08-17 新华三信息技术有限公司 一种指示灯同步闪烁方法、装置以及服务器
CN113268400B (zh) * 2021-04-27 2022-07-12 新华三信息技术有限公司 一种指示灯同步闪烁方法、装置以及服务器

Also Published As

Publication number Publication date
JPWO2018088380A1 (ja) 2019-10-03
JP7023239B2 (ja) 2022-02-21
TW201830892A (zh) 2018-08-16
TWI736702B (zh) 2021-08-21

Similar Documents

Publication Publication Date Title
JP6876617B2 (ja) 表示方法および表示装置
US10521668B2 (en) Display method and display apparatus
JP6876615B2 (ja) 表示方法、プログラムおよび表示装置
US10530486B2 (en) Transmitting method, transmitting apparatus, and program
US10819428B2 (en) Transmitting method, transmitting apparatus, and program
JP7023239B2 (ja) 送信方法、送信装置、およびプログラム
JP6122233B1 (ja) 可視光信号の生成方法、信号生成装置およびプログラム
JP6842413B2 (ja) 信号復号方法、信号復号装置およびプログラム
JP7134094B2 (ja) 送信方法、送信装置、およびプログラム
WO2016136256A1 (fr) Procédé de génération de signal, dispositif de génération de signal et programme
WO2016098355A1 (fr) Procédé de transmission, dispositif de transmission et programme
WO2018110373A1 (fr) Procédé d'émission, dispositif d'émission et programme

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17868678

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2018550203

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17868678

Country of ref document: EP

Kind code of ref document: A1