JP6847411B2 - Information transmitters, information receivers, information transmission systems and programs, positioning systems, lighting fixtures and lighting systems - Google Patents

Description

The present invention relates to an information transmission system for transmitting information using, for example, visible light communication, and also relates to an information transmission device, an information receiving device and a program, a positioning system, a lighting fixture, and a lighting system for the information transmission system.

Visible light communication has been tried for a long time, but its development research became active after 2010 when tablet terminals and smartphones became widespread.

Casio's Picapicamera pioneered commercialization in Japan (see, for example, Patent Documents 1 and 2). The light emitting source is a built-in microcomputer board, which is received by the smartphone together with the application software. The light source and the video frame of the camera are not always in sync. The transmission speed is about 10 bps or less.

Panasonic is trying a visible light communication technology that utilizes the characteristics of the rolling shutter operation of a CMOS video camera widely used in smartphones (see, for example, Patent Document 3). Normally, a signal source of a point light source is used in visible light communication, but a feature is that a light emitter of a surface light source is used as a light source because it is used in combination with a rolling shutter operation. In addition, in order to receive the high-speed brightness change of the light source, the electronic shutter opening of the camera (time ratio of how many seconds the light receiving element is performing the light receiving operation in the video frame of 1/60 second) is counted. It is also a feature that it is set to a very short percentage. Due to this effect, it claims that even a commercially available camera can transmit about 10 kbps.

The camera operation with a short shutter opening used in this method is effective for receiving the high-speed modulation signal of the light source, but it is difficult to use with a dark light source because it increases noise and lowers the sensitivity. Alternatively, it has drawbacks such as incompatibility with general photography. Moreover, since it is designed on the premise of the rolling shutter function, the shape of the light source is limited to the surface light source. This narrows the usage pattern when compared with the fact that the signal source of a point light source can be easily set up, or that a large number of point light sources can be easily arranged and multiplexed to improve the communication speed.

Intel Corporation of the United States has announced a technology that uses a signal source of a point light source, uses a camera of 30 fps, and enables transmission of 15 bps with a signal from a light source synchronized with the camera (see, for example, Non-Patent Document 1). .. This was proposed with the intention of standardizing including visible light communication in the category of the international standard of IEEE802.15 (Near Field Communication). Since a point light source is used, it is possible to multiplex using multiple point light sources for high-speed communication. Since this method also receives high-speed signal changes, the shutter opening of the video is set as small as about 10% or less.

Japanese Unexamined Patent Publication No. 2013-09072 Japanese Unexamined Patent Publication No. 2013-090774 International Application Publication No. 2013/175803 Japanese Patent No. 4041899 Japanese Unexamined Patent Publication No. 2014-155213 Japanese Patent No. 5213045 Japanese Unexamined Patent Publication No. 2016-085208

Rick Roberts et al., "Project: IEEE P802.15 Working Group for Wireless Personal Area Networks (WPANs)", Submitted on January 8, 2016, Internet, [Search April 4, 2016], <URL> https: //mentor.ieee.org/802.15/dcn/16/15-16-0006-00-007a-intel-occ-proposal.pdf Z. Zhang, "Flexible Camera Calibration by Viewing a Plane from Unknown Orientations," Proceedings of ICCV 1999, Kerkyra, Greece, pp.666-673, 1999.

However, the visible light communication according to the prior art has a problem that the communication speed is still extremely low.

An object of the present invention is to solve the above problems, to provide an information transmission system capable of performing visible light communication at a higher speed than that of the prior art, and to provide an information transmission device and information for the above information transmission system. The purpose is to provide a receiving device and a program.

Another object of the present invention is to provide a positioning system using the information transmission system, a lighting fixture using the positioning system, and a lighting system.

The information transmitting device according to the first invention is modulated by an information receiving device having a moving image camera that receives a visible light signal and a modulated signal having a basic frequency of a waveform that is an integral fraction of the frame frequency of the moving image camera. An information transmission device for an information transmission system including an information transmission device having a light source for transmitting a visible light signal.
The information transmission device performs orthogonal amplitude modulation using a virtual sine wave obtained by Fourier transforming m frame output signals output from the moving image camera according to an input digital data signal, thereby performing the modulation signal. It is characterized by being provided with a modulation means for generating the above.

The information receiving device according to the second invention is modulated by an information receiving device having a moving image camera that receives a visible light signal and a modulated signal having a basic frequency of a waveform that is an integral fraction of the frame frequency of the moving image camera. An information receiving device for an information transmission system including an information transmitting device having a light source for transmitting a visible light signal.
Said visible light signal being received by imaging by the video camera, and further comprising a demodulation means for demodulating the digital data signals to m frames output signal output from the video camera converts Fourier To do.

The information transmission system according to the third invention is
With the above information transmitter
It is characterized by being provided with the above information receiving device.

The program executed by the computer according to the fourth invention is an information receiving device having a moving image camera that receives a visible light signal, and a modulation having a basic frequency of a waveform that is an integral fraction of the frame frequency of the moving image camera. A program for an information receiver for an information transmission system including an information transmitter having a light source for transmitting a signal-modulated visible light signal.
It said visible light signal being received by imaging by the video camera, characterized in that it comprises a step of demodulating the digital data signals to m frames output signal output from the video camera converts Fourier.

The positioning system according to the fifth invention is
A receiving device having a moving image camera for receiving a visible light signal and a receiving means for receiving an acoustic signal,
A positioning system including a light source for transmitting a visible light signal modulated by a modulation signal by a modulation method based on the frame frequency of the moving image camera, and a transmission device having a transmission means for transmitting the acoustic signal.
The transmitting device generates and transmits an optical signal and an acoustic signal modulated according to a predetermined synchronization signal.
The receiving device receives the optical signal and the acoustic signal, and receives the optical signal and the acoustic signal.
The above receiving device
The synchronization timing of the received acoustic signal is generated based on the phase difference between the frame strobe signal from the moving image camera and the frame strobe signal and the signal in the visible light signal, and the synchronization timing and the reception are received. The propagation time of the acoustic signal is calculated from the time difference from the reception timing of the acoustic signal, and the distance between the transmitting device and the receiving device is calculated by multiplying the propagation time by the propagation speed of the acoustic signal. It is characterized by having a measurement processing unit.

In the above positioning system
The transmission device is a plurality of speakers provided at different positions, and transmits a plurality of acoustic signals having different frequencies, or a plurality of acoustic signals having the same frequency and sequentially transmitted at a predetermined time difference. Equipped with multiple speakers
The measurement processing unit calculates each distance between the plurality of speakers and the receiving device based on the plurality of acoustic signals, and positions the receiving device based on the calculated distances. It is a feature.

In addition, in the above positioning system,
The transmitting device is the information transmitting device.
The receiving device is the information receiving device.

The luminaire according to the fifth invention is provided with a transmission device for the positioning system and is characterized in that it illuminates.

The lighting system according to the sixth invention is
Equipped with a transmitter for the above positioning system
With the above light source for lighting,
It is characterized by having the plurality of speakers and providing means for reflecting or diffusing visible light from the light source to the receiving device.

In the positioning system according to the seventh invention, a plurality of the information transmitting devices are installed at known positions, known unique information is transmitted, the unique information is received by the information receiving device, and the information is transmitted from the received contents. It is characterized in that the existing position of the information receiving device is determined by optically knowing the apparent geometrical arrangement of each of the information transmitting devices while identifying the device.

Therefore, according to the present invention, it is possible to provide an information transmission system or the like capable of performing visible light communication at a higher speed than that of the prior art.

It is a figure which shows the structural example of the packet used in the information transmission system which concerns on Embodiment 1 of this invention. It is a waveform diagram which shows the waveform example of the modulation signal b (t) used in the information transmission system which concerns on Embodiment 1. FIG. It is a waveform diagram which shows the waveform example of the modulation signal b (t) of the rectangular wave used in the information transmission system which concerns on Embodiment 1. FIG. It is a waveform diagram which shows the waveform example of the time waveform B (t) and the virtual sine wave c (t) used in the information transmission system which concerns on Embodiment 1. FIG. It is a timing chart which shows the relationship between the modulation signal b (t) at the time of a perfect synchronization operation, and the timing of a shutter operation of a moving image camera in the information transmission system which concerns on Embodiment 1. FIG. It is a timing chart which shows the relationship between the modulation signal b (t) at the time of a partial synchronization operation, and the timing of a shutter operation of a moving image camera in the information transmission system which concerns on Embodiment 1. FIG. It is a block diagram which shows the structural example of the information transmission system 300 which concerns on Embodiment 1. FIG. It is a block diagram which shows the structural example of the information transmission apparatus 100 of FIG. 4A. It is a block diagram which shows the structural example of the information receiving apparatus 200 of FIG. 4A. It is a block diagram which shows the structural example of the transmission signal processing circuit 110A which concerns on Example 1. FIG. It is a block diagram which shows the structural example of the transmission signal processing circuit 110B which concerns on Example 2. FIG. It is a flowchart which shows the information receiving process executed by the information receiving apparatus 200 of FIG. 4C. It is a perspective view which shows the 3D constellation of the modulation signal (m = 3) QAM-modulated with 8 bits of a horizontal plane and 3 bits of a vertical axis by the modulation circuit 12 of FIG. 4B. It is a block diagram which shows the structural example of the transmission signal processing circuit 110C which concerns on Example 3. FIG. It is a block diagram which shows the structure of the positioning system which concerns on Embodiment 2 of this invention. It is a block diagram which shows the structure of the measurement controller 502 of FIG. It is a timing chart of each signal which shows the operation of the measurement controller 502 of FIG. It is a block diagram which shows the structure of the smart terminal device 503 under measurement of FIG. It is a timing chart of each signal which shows the operation of the smart terminal device 503 under measurement of FIG. FIG. 3 is a waveform diagram showing a time waveform of pixel values indicating the brightness of the modulation light source 501 measured by the CMOS video camera 504 of the smart terminal device 503 to be measured in FIG. 13. It is a graph which shows the synchronization accuracy at the time of performing the synchronization communication in the information transmission system which concerns on Embodiment 1. FIG. It is a block diagram which shows the structural example of the information transmission system which concerns on Embodiment 3 of this invention. It is a timing chart which shows the operation of the transmission device 600A of the information transmission system of FIG. It is a timing chart which shows the operation of the receiving device 700B of the information transmission system of FIG. It is a block diagram which shows the structural example of the transmission device 600A of FIG. It is a block diagram which shows the structural example of the receiving apparatus 700B of FIG. It is a perspective view which shows the configuration example of the direct luminaire 901 suspended from the ceiling 900 which is the configuration example of the transmission device 600A of FIG. It is a perspective view which shows the configuration example of the indirect lighting system 902 which is the configuration example of the transmission device 600A of FIG. It is a block diagram which shows the structural example of the information transmission system 800 which concerns on Embodiment 4 of this invention. It is a front photograph image which shows the arrangement example of the LED14 array of FIG. FIG. 2 is a plan view showing a plurality of measurement points P1 to P9 and a camera posture in an experimental example of the information transmission system 800 of FIG. 22. It is a table which shows the average error value and standard deviation of the estimation of the three-dimensional position and the posture at each measurement point P1 to P9 which are the experimental results of Embodiment 4. It is a graph which shows the cumulative distribution function of 3D position estimation error which is the experimental result of Embodiment 4. It is a graph which shows the coding error rate at different measurement points P1 to P9 which is the experimental result of Embodiment 4.

Hereinafter, each embodiment of the present invention will be described with reference to the drawings. In each of the following embodiments, the same reference numerals are given to the same components.

Embodiment 1.
Outline of Embodiment 1 according to the present invention.
This is an invention of an optical communication technology represented by visible light communication using an LED (Light Emitting Diode) and a moving image camera. A video camera built into a portable information terminal such as a smartphone is used for signal detection to realize information transmission at a speed exceeding the shooting frame rate. LEDs are widely used for indoor and outdoor lighting, traffic lights, automobile lights, etc., and can apply high-speed and precise light intensity modulation. Portable information terminals with built-in video cameras are also widespread. If both of them transmit optical information mainly in visible light, it will be possible to apply communication with features not found in radio wave wireless communication, which is premised on the presence of reality, as a communication method only within the line-of-sight distance.

In such an application, there is an example in which communication of several megabits / second or more is realized by using a dedicated light receiving element or a camera having a high frame rate for high-speed communication. However, the frame rate of a commercially available video camera built into a commercially available mobile device such as a smartphone is about 60 frames / second (60 fps), and when it is used, only communication of about 10 bits / second (10 bps) is practical. There wasn't. Therefore, the use of visible light communication was very limited.

On the other hand, such video cameras are slow for communication but are widely used. Therefore, if a commercially available video camera can perform communication at a bit rate that matches or exceeds the frame rate, it will be widely used in civilian life. It is expected that visible light communication will be used. Specifically, it includes light-emitting posters, digital signage (bulletin boards), information transmission from traffic lights, and vehicle-to-vehicle communication using automobile tail lamps.

The first embodiment according to the present invention relates to visible light communication using a low-speed commercially available moving image camera. For example, a 60 fps camera enables communication of 100 bps or more per LED light source. By adopting a highly efficient demodulation algorithm, by arranging a plurality of light sources, for example, a signal source cluster of 100 LEDs can be formed and communication of more than 10 kbps can be performed by software demodulation of a smartphone.

The shooting parameters of the moving image camera include an electronic shutter, which is related to the photointegration time for each shooting frame. When this shutter opening is made small (when the photointegration time for each frame is set short), a fast-moving subject can be clearly photographed, and this is used for special photography. There has been a technology that utilizes this characteristic in visible light communication, sets the shutter opening to 10% or less when receiving communication, and detects high-speed optical signal changes to increase the communication speed.

However, if the shutter opening degree is reduced, noise increases in the image and the obtained image becomes dark. Therefore, in general photography, the shutter opening is usually used at a large value close to 100%. When the shutter opening is made small for communication, there is a drawback that it is incompatible with the camera setting conditions for general photography. In the first embodiment of the present invention, the detection algorithm is adapted for each shutter opening degree to correct the information. For this reason, high-speed communication can be performed even with a shutter opening of 100%, which is a condition for normal shooting, there is little noise, and the sensitivity is high. Visible light can be used even if a light source partially reflected in a general shot image is used. It has the feature of being able to communicate.

Embodiment 1 of the present invention is similar to Intel's method in the sense that it uses a point light source synchronized with the frame rate of a moving image camera. However, the electronic shutter opening degree (hereinafter referred to as shutter opening degree) η of the moving image camera is set to close to 100% (preferably 0.5 ≦ η ≦ 0.99, more preferably 0.9 ≦ η ≦ 0. 99) The feature is that it can be carried out even in the same state as the normal shooting operation. By increasing the shutter opening η, the signal-to-noise ratio of the light receiving element of the moving image camera can be improved. In addition, since visible light communication can be used with the same moving image camera settings in the normal shooting state, it is possible to take a wider range of usage opportunities. Since a known method receives a high-speed signal (high-speed modulation), a particularly small shutter opening (for example, 0.02 to 0.03) is set. On the other hand, in the first embodiment of the present invention, it is possible to receive a high-speed signal while maintaining a large shutter opening degree by predicting waveform deformation due to the shutter opening degree and collating the code.

If you use it with a small shutter opening, the camera will only cut and receive a part of the signal and discard a lot of it. The transmission signal of the communication method premised on a small shutter opening repeats the same content and is transmitted redundantly so that any part may be received. In the method according to the first embodiment of the present invention in which the shutter opening degree is increased, there are few redundant parts in the signal, the communication speed can be efficiently increased, and various additional functions such as flicker reduction can be incorporated in the modulated signal. it can.

Description of the method of Embodiment 1 of the present invention.
Let Tp (seconds) be the shooting time of one frame of the moving image camera. The reciprocal is the video frame rate fp, and the relationship is fp = 1 / Tp. Many commercially available digital video cameras or digital video cameras of smartphones use fp = 60 (60 frames / second).

Light is emitted from the LED with temporal brightness modulation (amplitude modulation, quadrature amplitude modulation including phase modulation (hereinafter referred to as QAM modulation), DC component bias modulation), and information is transmitted by receiving the light emission signal with a moving image camera. To do. Here, the unit of information transmitted from the LED is called a symbol. One symbol corresponds to a plurality of information bits of about 10 bits. The symbol generation rate fq is synchronized with the video frame rate, and the symbol generation cycle Tq is set to m times the video frame cycle Tp. That is, it is expressed by the following equation.

Tq = mTp

Here, m is an integer of 1 or more, that is, a natural number. In terms of frequency, the relationship is fq = fp / m. Information can be transmitted as an arbitrary integer of 1 or more, but in order to better exert the effect of the first embodiment of the present invention, it is preferable to select m ≧ 3. In particular, when m = 3, various characteristic properties according to the first embodiment of the present invention can be obtained, and this operation will be described in detail later. The principle of is described below.

Transmission waveform.
The transmitter uses an information transmitter to drive the LED with a modulated waveform with Tq = mTp as the basic time unit to emit light. Here, the modulated signal is b (t). Since the modulated signal b (t) has Tq as the basic time unit, it can be expanded into a series having _{a complex Fourier coefficient β k, assuming that it is repeated in this cycle.} k is a subscript of an integer and takes a value of 0, ± 1, ± 2, .... The modulated signal b (t) can be converted into a complex Fourier coefficient having fq = 1 / Tq as the fundamental frequency by the following equation.

Characteristics of received signal.
The receiver uses an information receiving device to receive this using a moving image camera with a frame rate of fp = 1 / Tp (however, the shutter opening degree η). The light receiving element of the moving image camera does not take in light over the entire Tp, and the shutter opening degree η (0 <η <1; however, in the first embodiment, preferably 0.5 ≦ η ≦ 0) in a predetermined range. The signal from the optical conversion cell is integrated for the time that can be expressed as ηTp according to .99). Since light reception, analog / digital conversion (hereinafter referred to as AD conversion), and data transmission are performed alternately, the shutter opening η cannot be set to 1, but the maximum value can be set up to about 0.99 (99%). When the shutter opening η of the moving image camera is set to a value close to 1, the noise of the image sensor can be reduced and the sensitivity is also improved. Therefore, in general shooting, the shutter opening η is operated at a value close to that value. On the other hand, if you want to clearly shoot a subject moving at high speed, you should shorten the shutter opening η. Therefore, in many moving image cameras, the shutter opening η can be set to various values according to the shooting conditions, and is used at an appropriate value each time the shooting is performed.

The received signal undergoes the following two types of modifications, assuming that the intensity change due to the distance and the optical system, or the superposition of noise due to the ambient light and the light receiving element is ignored.

(1) Since the light receiving element integrates the light of the light source in the time of 0 ≦ t ≦ ηTp, the received waveform is changed as compared with the transmitted waveform.
(2) The integration result is AD-converted for each Tp and output as a discrete signal. The reciprocal fp of the video frame period Tp can be regarded as the sampling frequency of the signal change, which is at most m times the fundamental frequency of the signal source. Therefore, the component of the modulated signal exceeding the Nyquist frequency, that is, the component of the harmonic β _k of the modulated waveform in which k ≧ m / 2, becomes aliasing noise as a result of sampling, and changes the detection signal.

The first embodiment according to the present invention is characterized in that the influences of both are precisely tracked and the signal is detected while correcting the influences thereof. Let's take a closer look at both corrections.

Correction of signal changes due to integration operation.
The light receiving element of the moving image camera can be regarded as integrating the waveform b (t + δTq) of the signal source with a time width of 0 ≦ t ≦ ηTp. Cutting and integrating the signal by the above time width is nothing but mathematically convolving a rectangular window having a time width of ηTp by multiplying the signal waveform. Here, 0 ≦ δ <1 is the start time difference between the transmission signal and the received moving image camera operation, that is, the operation phase difference, and the case where the moving image camera is slightly delayed is assumed to be a positive value. If some kind of synchronization mechanism is placed between the light source and the operation of the moving image camera, δ = 0 can be set, but if not, there is a non-zero δ.

Since the light source operates at a cycle of m times that of the camera frame, information on the brightness of the light source for m frames of _{data signals c 1} , c ₂ , ... _{cm can be obtained from the camera light receiving element.} .. This data signal c ₁ , c ₂ , ..., C _m is Fourier transformed with Tq = mTp as the basic period, and its coefficient B _k is expressed by the following equation, ignoring the modification due to aliasing noise described later. To.

ともとの波形のフーリエ級数β_ｋの乗算となること、及び信号源の遅延δＴｐは遅延因子

の乗算であることから理解できる。光源と動画カメラの間に同期化機構を導入し、δ＝０とすれば、この式は次式と簡略化できる。 In the above equation, the integral operation of the image sensor of the moving image camera is the convolution of a rectangular window with a time width of ηTp, which is the Fourier transform of the time window in the Fourier transform equation.

The original waveform is _{multiplied by the Fourier series β k} , and the signal source delay δTp is a delay factor.

It can be understood from the fact that it is a multiplication of. If a synchronization mechanism is introduced between the light source and the moving image camera and δ = 0, this equation can be simplified to the following equation.

The problem of the operating phase of the light source and the moving image camera is the presence or absence of the above multiplication factor, and there is no big difference in the mathematical formulas. Therefore, for the sake of simplicity in the following discussion, it is assumed that δ = 0 holds. Consideration in other cases will be described in detail later.

The integrated time waveform B (t) can be reconstructed from the coefficient B _{k by the inverse Fourier transform of the following equation.}

_{The brightness (data signal intensity) c 1} , c ₂ , ..., C _m of the light source recorded in the shooting frame of the moving image camera over m frames is expressed by the following equation.

c _i = B ((i-1) Tp); i = 1, 2, ..., m

If the waveform b (t) of the optical modulation signal can be completely restored from the time waveform B (t), all the information of the original signal can be obtained. However, equation (4) includes a factor of sinc (πηk / m), and this function decreases the value as the absolute value of k increases. Alternatively, even if ηk / m is an integer value or a value in the vicinity thereof, it is zero or a small value. At that time, a part of the original signal information is lost, and it becomes difficult or impossible to restore the modulated signal b (t). Therefore, the correction referred to here is not necessarily a complete restoration of the original signal, but it is possible to know exactly what kind of modification the signal is subjected to by the integration operation of the shutter opening η by the equations (2) to (3). At the same time, the purpose is to provide a communication system that utilizes it in the decoding process.

Aliasing correction.
The modulation waveform of the light source having Tq = mTp as the basic period is converted into m output values _{of data signals c 1} , c ₂ , ..., C _{m by the moving image camera.} If the video camera operation is regarded as signal sampling with a sample rate of fp = 1 / Tp, _{the sequence of data signals c 1} , c ₂ ... Is a time waveform B modified by integrating the modulation signal b (t) which is a real signal ( t) is further sampled by the video frame of the moving image camera. Since the video frame rate assumed here is very close to the modulation fundamental frequency of the light source, the frame output is strongly affected by aliasing noise.

Here, the time waveform B (t) is a repeating waveform with fq = 1 / Tq as the fundamental frequency and has a line spectrum. Since some of the wave-tuning components that make up the time waveform B (t) are outside the Nyquist frequency ± fp / 2 of the video frame rate, and the fundamental frequency is selected as an integral fraction of the frame rate, the Nyquist frequency is selected. The wave tuning component outside the frequency is folded back by sampling, and the folded line spectrum all overlaps with the line spectrum within the range of Nyquist frequency ± fp / 2, and is converted into a new finite number (at most m) of line spectrum series. Will be done.

Just as the conversion of the modulated signal b (t) to the time waveform B (t) by the integration operation is irreversible, the information modification by this aliasing noise is also a lossy process, and is accompanied by the lack of tuning information. Therefore, as pointed out in the above-mentioned "Correction of signal change by integration operation", the intention is to correctly understand the nature of the remaining information and use it for information transmission communication.

Here, a virtual sine wave called c (t) is introduced. It is a function whose frequency components are all within the range of Nyquist frequency −fp / 2 ≦ f ≦ fp / 2 and satisfy the relation of the following equation.

c _i = B ((i-1) Tp) = c ((i-1) Tp)

The virtual sine wave c (t) is different from this because the time waveform B (t) generally has a frequency component that exceeds the Nyquist frequency of the frame rate. The relationship between the time waveform B (t) and the virtual sine wave c (t) will be considered below.

First, if the relationship of aliasing noise is organized, it is understood that the correspondence is as follows.

Dividing the (positive and negative) integer k by m gives the quotient q and the remainder r of 0, 1, 2, ... m-1. Even for a negative integer k, the remainder is taken positively, and k = qm + r, where the relational expression of the remainder r ≧ 0 is satisfied. Then, the finite number of spectral values s _r (r = 0, 1, 2, ..., M-1) considering the influence of aliasing noise are expressed by the following equations.

Here, the relationship between the spectrum value s _{r and the complex Fourier coefficient C k} of the spectrum of the virtual sine wave c (t) will be further described.

(A) When m is an odd number: The spectrum value s ₀ of r = 0 is C ₀ (direct current component). _{Further, the spectral values s r} from 1 to r <m / 2 correspond to _{Cr. Further, the spectral values s r} above m / 2 and up to m-1 correspond to the negative frequency components of the complex Fourier coefficient _Crm.

(B) When m is an even number: _{The correspondence between the spectrum value s r} and the complex Fourier coefficient C _k is the same as in the case of an odd number, but in addition, a spectrum value s _r of r = m / 2 occurs. This is _{a component of C m / 2} and C − _{m / 2} , and since it matches the Nyquist frequency, it has the same spectral value.

Extraction _{of Fourier coefficient C k} from the observation series of the frame.
The Fourier coefficient C _k is a method of obtaining the Fourier transform of the emission waveform from the characteristics of the imaging device and the aliasing noise of the sampling, as well as the observed values of the brightness of the light source observed in m consecutive moving image frames c ₁ , c ₂ , ... also it is obtained by the following equation from _{c m.}

However, the subscript k is a positive / negative integer including zero, and satisfies −m / 2 ≦ k ≦ m / 2. This is an operation known as the discrete Fourier transform of m points (hereinafter referred to as DFT), and can be expressed as the following equation using the symbol.

C _k = DFT _m [c (t)]

Here, the virtual sine wave c (t) is a time waveform in which all the frequency spectra are within ± fp / 2, and the m point of t = 0, Tp, 2Tp, ..., (M-1) Tp. in, consistent with the virtual sine wave value c _i. That is, the virtual sine wave value c _i = c ((i-1) Tp). It is taught by the sampling theorem that such a virtual sine wave c (t) is uniquely determined when any c ₁ , c ₂ , ..., C _{m is specified.}

The modulated waveform b (t) sent by the information transmitting device is modified by integration and is affected by sampling of video frames, so that it does not reach the information receiving device in that form. However, assuming that the information transmitting device has knowledge about the shutter opening η used by the information receiving device, how it is transmitted to the information receiving device as _{virtual sine wave values c 1} , c ₂ , ..., _cm. You can expect it. In particular, the virtual sine wave _{c (t) reconstructed from the virtual sine wave values c 1} , c ₂ , ..., C _m is a non-existent waveform, but as described below, a finite number of sine waves are superimposed. (Duplicate).

Considering that the information transmitting device according to the first embodiment does not send out b (t) but "modulates a virtual sine wave c (t) with transmission information and sends it out", it is completely converted into an information receiving device. reach. This is the gist of Embodiment 1 of the present invention. Since c (t) has a sinusoidal property, it is called a virtual sinusoidal wave. In addition, since this can be regarded as a normal sine wave carrier wave and modulated, it will be referred to as a "virtual sine carrier" for that purpose.

Virtual sine wave.
_{Using m Fourier coefficients C k} in the range of −m / 2 ≦ k ≦ m / 2, c (t) can be obtained by the relational expression of the following equation.

Equation (7) is an operation known as the inverse discrete Fourier transform of point m, and can be expressed as the following equation.

c (t) = IDFT _m [C _k ]

Real observation sequence (virtual sine wave _{value) c 1, c 2, ...} , the Fourier coefficients _{C k} constructed from _{c m} is a complex number, is always _C -i _{= C} ^{i *} relationship. However, * represents the complex conjugate. Due to its nature, the inverse discrete Fourier transform may be expressed as in the following equation, using only Fourier coefficients with non-negative subscripts.

(A) When m is odd:

(B) When m is an even number:

However, here, [m / 2] is an integer that does not exceed m / 2, and | C _k | and ∠C _k are the absolute value and the argument of the complex number C _{k, respectively.} C _{m / 2, which exists when C 0} and m are even numbers, is always a real number. The virtual sine wave c (t) of a real signal can be uniquely decomposed into the sum of the DC component, [m / 2] sine waves, and the Nyquist frequency component.

Modulation and demodulation of virtual sine waves.
The virtual sine wave c (t) can be theoretically obtained from the modulation signal b (t) of the light source as "waveform change due to integration" and "result of aliasing noise" received by the image sensor of the moving image camera. Further, the modulated waveform can be actually obtained from m consecutive frame outputs (virtual sine wave values) c ₁ , c ₂ , ..., C _m of the image sensor by DFT (Discrete Fourier Transformation) calculation. If the influence of noise etc. is ignored, the two will match.

A virtual sine wave can be considered as a sine carrier wave having properties such as amplitude and phase. Modulation for a light source can be thought of as amplitude or phase modulation for a virtual sine wave (rather than a real waveform). Then, it (modulation of amplitude and phase with respect to a virtual sine wave) can be demodulated from the Fourier coefficient by performing a DFT calculation on the _{received frame sequences c 1} , c ₂ , ..., C _m.

There are m types of amplitude and phase components of the spectrum constituting the virtual sine wave c (t) including the amplitudes of the DC component and the Nyquist frequency component, and they are independent as orthogonal axes. In communication using light, information can be simultaneously placed on these m types of axes by modulation.

Next, a specific configuration method according to the first embodiment of the present invention will be described below.

Three-cycle method.
A specific method for performing visible light communication according to the first embodiment of the present invention will be described. The transmission of digital information is performed by the transmission of symbols, and one symbol corresponds to about 10 bits of digital information. Information transmission is performed by packets, which are a collection of symbols.

FIG. 1 is a diagram showing a configuration example of a packet used in the information transmission system according to the first embodiment of the present invention. In FIG. 1, one packet is composed of a preamble 51 of 0 to several tens of symbols, data 52 of arbitrary length, and an optional frame check sequence (hereinafter referred to as FCS) 53.

The preamble 51 is a predetermined series of symbols, and in many cases, the same symbol is repeatedly transmitted. It is added to correct the reception sensitivity of the information receiving device and to synchronize the light emission of the light source of the information transmitting device with the information receiving device. In addition to the signal from the signal source, the video camera also receives the background light, but its intensity can also be confirmed and corrected by the preamble. Since a certain amount of symbols of the data 52 are stored in the buffer memory of the information receiving device and synchronization can be performed by analyzing the known symbols there, the preamble can be omitted when such a method is used. The data 52 is composed of symbols of arbitrary length and bears substantial transmission data. FCS53 is additional information for error detection / correction, for example, and is added only when it is necessary.

The modulated signal b (t) from the LED light source is generated according to the packet structure of FIG.

FIG. 2A is a waveform diagram showing a waveform example of the modulated signal b (t) used in the information transmission system according to the first embodiment. An example of the modulated signal b (t) is shown in FIG. 2A. The symbol of the preamble 51 is generated while changing the DC component, thereby establishing the symbol synchronization required for communication between the information transmitting device and the information receiving device. Also, while checking the level of the background light, the sensitivity can be corrected between the information transmission / reception devices. _{The frame output signals c 1} , c ₂ , and c ₃ output from the video camera on the receiving side after receiving the preamble are those for which background light level correction and reception sensitivity correction are performed based on the information obtained by the preamble. And.

Here, it is assumed that the frame period of the moving image camera and the basic timing period of light emission are synchronized with a ratio of 1: m. m may be any integer of 1 or more, but at m = 1, only the real DC component can be transmitted, and at m = 2, only the Nyquist frequency component (also a real number) can be transmitted in addition to the DC, both of which are Fourier. Since the coefficient is a real number, phase modulation cannot be applied. Therefore, in order to fully exert the effect of the present invention, it is preferable to select m ≧ 3 so as to include a virtual sine wave that enables phase amplitude modulation, and the simplest case is shown below as an example of m = 3. Since the symbol is transmitted in units of 3 frames, this is called the "3-cycle method".

FIG. 3A is a timing chart showing the relationship between the modulation signal b (t) during the complete synchronous operation and the timing of the shutter operation of the moving image camera in the information transmission system according to the first embodiment. Further, FIG. 3B is a timing chart showing the relationship between the modulation signal b (t) during the partial synchronization operation and the timing of the shutter operation of the moving image camera in the information transmission system according to the first embodiment. Here, the following two cases are assumed for the synchronous operation of the light emission and the moving image camera.
(Fig. 3A) Full synchronization operation;
(Fig. 3B) Partial synchronization operation.

The perfect synchronization operation is a case where the switching timing of the light emission and the shutter timing of the moving image frame match including the phase. This synchronization can be achieved by using the information frame preamble to detect the phase difference between the light emission and the shutter and thereby adjusting the shutter timing, but a video camera with a shutter release time adjustment mechanism must be used. ..

In the partial synchronization operation, the light emission cycle and the camera frame period are synchronized at 1: m, but the operation involves only the phase difference δ. If the light emission cycle and the camera frame cycle are the same, but the light emission is recorded by a moving image camera that does not have a shutter release time adjustment mechanism, the recording is performed by partial synchronization operation. In the case of the partial synchronization operation, some of the generally recorded frames are photographed at the symbol switching timing. It becomes a broken frame for the symbol recording, making it difficult to decrypt. The countermeasures will be described in detail later. Hereinafter, for the sake of simplicity, the case of complete synchronization operation will be described.

Even if the timing deviation between the information transmitting device and the information receiving device can be confirmed and corrected at a certain point in the complete synchronization operation or the partial synchronization operation, it is long if there is a slight offset (deviation) between the operating clock frequencies of both. Offsets accumulate in the meantime, and the amount of timing shift further changes. However, in the visible light communication of the present invention, since a communication packet having a short duration of about several seconds is assumed, the accumulation of offset is slight, and in such a case, a synchronization mechanism for correcting it is not used. It's okay.

It is assumed that a commercially available moving image camera with fp = 60 (60 frames / sec) captures the emission of a rectangular wave of fq = fp / m = 60/3 = 20 Hz. This square wave is composed of odd-order harmonics and direct current with 20 Hz as the fundamental frequency.

FIG. 2B is a waveform diagram showing a waveform example of a rectangular wave modulated signal b (t) used in the information transmission system according to the first embodiment. Further, FIG. 2C is a waveform diagram showing waveform examples of the time waveform B (t) and the virtual sine wave c (t) used in the information transmission system according to the first embodiment.

When the square wave b (t) of FIG. 2B is photographed, the time waveform B (t) of the recorded signal change is as shown in FIG. 2C due to the influence of the integration operation of the image sensor of the moving image camera (FIG. 2C). Broken line). Here, the shutter opening η = 1 is assumed. It shows one cycle of the time waveform B (t). It is a 20 Hz repetitive waveform corresponding to three video frames. The actual frame output is obtained by sampling three points of this waveform, B (0), B (Tp), and B (2Tp), which are hereinafter referred to as c ₁ , c ₂ , and c ₃ , respectively.

In FIG. 2C, the time waveform B (t) affected by the integration and the virtual sine wave values c ₁ , c ₂ , and c ₃ sampled for each video frame are shown, and the dotted line in FIG. 2C is virtual. Represents a virtual sine wave c (t) passing through the sine wave values c ₁ , c ₂ , and c _3. In FIG. 2C, it is assumed that there is no time delay from the light source of the information transmitting device to the shooting operation of the moving image camera, that is, it is shown as “perfect synchronization operation” by δ = 0. When the "partial synchronization operation" has a delay time of δ ≠ 0, the time waveform B (t) of FIG. 2C may be shifted to the left by the phase difference δ (see FIG. 3B).

It can be considered that the moving image camera samples the time waveform B (t) having a basic period of 20 Hz at a sample frequency of 60 Hz. The Nyquist frequency for sampling is 60/2 = 30 Hz. _{Therefore, the data signals (virtual sine wave values) c 1} , c ₂ , and c ₃ of the frame output signal of the frame output signal output from the moving image camera are not the time waveform B (t), but their spectra are 0 (DC). ), 20 Hz, and -20 Hz, and can be represented as a data signal c (t) having _{spectral values s 0} , s ₁ , and s _{2, which are frequency components.} Therefore, the time waveform B (t) can be obtained as the following equation from the _{Fourier transform Fourier coefficient B k.} Here, q is an integer.

Alternatively, these spectral values s ₀ , s ₁ , and s ₂ can be obtained from the data signals (virtual sine wave values) c ₁ , c ₂ , and c ₃ , which are the frame output signals of the moving image camera, by using the following equations.

であり、１，ω，ω^２は３次の円周等分方程式ｚ^３＝１の３根である。ω^２＝ω^＊（複素共役）なので、情報受信装置における実数の観測値ｃ_１，ｃ_２，ｃ_３について、次式が成り立つ。 here,

, 1, ω, and ω ² are the three roots of the cubic circumferential equation z ^{3 = 1.} Since ω ² = ω ^* (complex conjugate), the following equation holds for the real observation values c ₁ , c ₂ , and c _{3 in the information receiving device.}

s ₂ = s ₁ ^*

In the information receiving device, the virtual sine wave c (t) can be obtained by inverse discrete Fourier transforming _{the real observation values s 0} , s ₁ , and s _{2 into a time waveform.} Although the virtual sine wave (dotted line) is also displayed in FIG. 2C, it coincides with the time waveform B (t) at at least three points of time 0, Tp, and 2 Tp. However, in the rectangular wave used for transmission here, there are three more coincidence points due to the symmetry of the waveform. _{The spectral values or virtual sine wave c (t) theoretically obtained} from the complex Fourier coefficient B k and obtained from the data signal output from the moving image camera are the same except for the influence of noise and the like. principle over amplitude and phase modulation to a square wave of the light source by which the by read as amplitude and phase modulation to a virtual sine wave (to s ₁ without s ₂ component), it is possible to perform information transmission interposed virtual sine wave.

First, an information transmission system for information communication using the above three-cycle method will be described below.

FIG. 4A is a block diagram showing a configuration example of the information transmission system 300 according to the first embodiment. Further, FIG. 4B is a block diagram showing a configuration example of the information transmission device 100 of FIG. 4A. Further, FIG. 4C is a block diagram showing a configuration example of the information receiving device 200 of FIG. 4A.

In FIG. 4A, the information transmission system according to the first embodiment includes an information transmission device 100 and, for example, a smartphone 400 including an information reception device 200. The information transmission device 100 includes a transmission signal processing circuit 110, a drive circuit 13, and an LED 14. The transmission signal processing circuit 110 generates a modulation signal b (t) based on the digital signal of the transmission information data, and outputs the modulation signal b (t) to the drive circuit 13. The drive circuit 13 amplifies the modulation signal b (t) and applies it to the LED 14 as a drive signal to cause the LED 14 to emit light according to the modulation signal b (t). The emitted visible light signal is captured by the moving image camera 21 of the smartphone 400.

The smartphone 400 includes a moving image camera 21, a reception signal processing circuit 210, and a display 410. The moving image camera 21 receives the visible light signal, takes an image, and outputs the frame output signals c ₁ , c ₂ , and c ₃ to the reception signal processing circuit 210. The reception signal processing circuit 210 demodulates the digital signal of the original transmission information data by performing the above-mentioned demodulation processing on the _{input frame output signals c 1} , c ₂ , and c _{3, and outputs the digital signal to the display 410 for display.} To do.

The information transmission device 100 of FIG. 4B includes a transmission signal processing circuit 110, a drive circuit 13, and an LED 14. Here, the transmission signal processing circuit 110 includes a bit dividing circuit 11 and a modulation circuit 12. The bit division circuit 11 extracts the following three bit data groups from the bit string of the digital data signal, which is the input transmission information data, and uses them as one symbol.
(1) Bit data for DC component bias modulation (for example, 3 bits);
(2) Bit data for amplitude modulation (for example, 4 bits);
(3) Bit data for phase modulation (for example, 4 bits).

By repeating this operation, the transmission bit string is converted into a bit string constituting the symbol, that is, a symbol string. The end of the input bit string is also completed as a symbol string by supplementing with an appropriate blank bit if necessary.

The modulation circuit 12 performs DC component bias modulation, amplitude modulation, and phase modulation (the latter two modulations are collectively referred to as QAM modulation) according to the bit data of the input symbol based on the above three-cycle method. The signal b (t) is generated and output to the drive circuit 13. The bit-dividing circuit 11 is a symbol encoder, and uniquely characterizes the symbol to be transmitted by the set of {DC bias value, amplitude value, phase value} of its output. The drive circuit 13 amplifies the modulation signal b (t) and applies it to the LED 14 as a drive signal to cause the LED 14 to emit light according to the modulation signal b (t), and the emitted visible light signal is a moving image of the smartphone 400. It radiates to the camera 21.

The information receiving device 200 of FIG. 4C includes a moving image camera 21, a DFT calculation circuit 22, a signal separation circuit 23, a codeword ROM 24, a shutter opening degree correction circuit 25, and a collation circuit 26. The moving image camera 21 receives the incident visible light signal and performs imaging processing to generate a frame output signal and a shutter opening signal indicating the shutter opening, respectively, and generates a DFT calculation circuit 22 and a shutter opening correction circuit, respectively. Output to 25 and codeword ROM 24. The DFT calculation circuit 22 performs a DFT calculation on the input frame output signals c ₁ , c ₂ , and c ₃ , converts the DFT calculation into a spectrum, and calculates the spectrum values s ₀ , s ₁ , s ₂ , and the signal separation circuit 23. Output to. The signal separation circuit 23 converts the input spectrum values s ₀ , s ₁ , s ₂ into the following data values and outputs them to the collation circuit 26.
_{(1) Fourier coefficient C 0} corresponding to the DC component bias modulation value;
(2) Absolute value of Fourier coefficient corresponding to the amplitude modulation of the 20 Hz carrier | C ₁ |;
(3) Phase value of Fourier coefficient corresponding to phase modulation of 20 Hz carrier ∠C ₁ .

_{The codeword ROM 24 corresponds to virtual sine wave values c 1} , c ₂ , and c ₃ based on the modulation signals b (t) of all 2048 kinds of symbols corresponding to, for example, 11-bit data, which can be transmitted by the information transmission device 100. The Fourier coefficients and the like C ₀ , | C ₁ |, and ∠C ₁ are stored in advance, and the data of these reference signals are read out from the signal separation circuit 23 at the timing of signal separation completion (details will be described later). Is output to the shutter opening degree correction circuit 25 in response to. The shutter opening correction circuit 25 corresponds to the currently set shutter opening η, and has equations (2) to _{C1 for Fourier coefficients and the like stored in the codeword ROM 24, such as C 0} , | C ₁ |, and ∠C _1. It is corrected based on the equation (3), and the corrected Fourier coefficients and the like are output to the collation circuit 26 as _{C 0} ', | C ₁ |', and ∠C _{1', respectively.} The collation circuit 26 collates the Fourier coefficients C ₀ , | C ₁ |, ∠C ₁ of the received visible light signal with the corrected Fourier coefficients C ₀ ', | C ₁ |', ∠C ₁ '. The symbol is specified by, and is output as demodulated digital data by the reverse operation of the encoding operation performed by the bit dividing circuit 11 of the information transmission device 100.

It should be noted that the processing of the received signal processing circuit 210 (including the first embodiment and the modification described later) of FIG. 4C is realized as a program executed by a CPU or a computer of an electronic device such as a smartphone, and the program is executed. You may.

In the experiments of the present inventors, the information transmission device 100 bit-decomposes the transmission symbol _{and allocates σ 1} = 8 bits (256 sections) _{to the AC phase amplitude and σ 0} = 3 bits (8 levels) to the DC level. And send. The modulated waveform may be any waveform having a known spectrum with a fundamental period of 20 Hz, but a square wave is used as an example. The camera frame frequency of the information receiving device 200 is 60 fps, and the shutter opening degree is fixed at η = 0.5 for shooting. A spectrum is obtained from three consecutive frames by performing a discrete Fourier transform with Eq. (7). The spectrum of the square wave is known, and the spectrum change when it is integrated with the shutter opening η = 0.5 can be found from equations (2) to (3). , Information can be compounded.

FIG. 5 is a block diagram showing a configuration example of the transmission signal processing circuit 110A according to the first embodiment. In FIGS. 4B and 4C, the waveform of b (t) was generated in the time domain corresponding to the transmission bit, but if it is performed in the frequency domain, it will be as shown in FIG.

In FIG. 5, the transmission signal processing circuit 110A includes a DC component bias modulator 31, a QAM modulator 32, an adder 33, an inverse Fourier transformer 34, and a low-pass filter 35. The bit division circuit 11 separates the input digital data signal into bit data (σ ₀ bit) for DC component bias modulation and bit data (σ ₁ bit) for QAM modulation, respectively. Output to the DC component bias modulator 31 and the QAM modulator 32. The DC component bias modulator 31 _{reads the input bit data (σ 0} bit) as the Fourier coefficient β ₀ corresponding to the bias modulation and outputs the input bit data to the adder 33. The QAM modulator 32 performs QAM modulation using a carrier having a frequency fp / 3 according to the input bit data (σ _{1 bit), and outputs the modulated signal to the adder 33. Here, for QAM modulation, for example, a complex Fourier coefficient sequence β k} corresponding to a square wave having a fundamental frequency of 20 Hz and an amplitude of 1 is prepared as a waveform (the maximum order of tuning is a range in which a square wave waveform can be sufficiently expressed. The absolute value is increased or decreased by the amplitude modulation, and the deviation angle is increased or decreased according to the tuning order by the phase modulation. The adder 33 adds the two input signals and outputs them to the inverse Fourier transformer 34. The inverse Fourier transformer 34 performs an inverse discrete Fourier transform on the input tuning signal to convert it into a discrete time domain signal, and the converted signal is the maximum tuning of _{the Fourier coefficient β k handled by the modulation.} A continuous modulated signal b (t) is obtained by passing a low-pass filter 35 that passes through a component and blocks other frequencies.

FIG. 6 is a block diagram showing a configuration example of the transmission signal processing circuit 110B according to the second embodiment. Since the types of information symbols are finite, as shown in FIG. 6, the waveform of the modulation signal b (t) to be generated by the inverse Fourier transformer 34 of the modulation circuit 12A is calculated in advance and stored in the modulation waveform ROM 41. At the time of transmission, a similar waveform can be created by reproducing it by DA conversion.

In FIG. 6, the modulation circuit 12B includes a clock generator 40, a modulation waveform ROM 41, a DA converter 42, and a low-pass filter 35. The bit dividing circuit 11 divides the input digital data signal into a predetermined bit length corresponding to a predetermined symbol, and outputs the input digital data signal to the modulated waveform ROM 41 as a base address for reading the ROM in the form of parallel data. The modulated waveform ROM 41 increments the address in response to the clock from the clock generator 40, reads out the modulated waveform value (digital value) as a time-series digital data signal for Tq hours, and uses the DA converter 42 and the low-pass filter. A modulation signal b (t) to be passed through 35 is generated.

FIG. 7 is a flowchart showing an information receiving process executed by the information receiving device 200 of FIG. 4C. In FIG. 7, the background DC bias voltage is detected in step S1, and it is determined whether or not the signal detection preamble is detected in step S2. If YES, the process proceeds to step S3, while if NO, step S1 Return to. In step S3, during the partial synchronization operation, the phase difference δ is detected based on the preamble, the data signal received in step S4 is decoded, and optional error detection / correction processing is performed based on the FCS received in step S5. .. Further, the decoded data is output in step S6. Here, through steps S2 to S5, the shutter opening degree correction is performed based on the equations (2) to (3) while referring to the shutter opening degree η used for reception.

The extraction of the phase difference δ from the preamble in step S3 of the information reception process of FIG. 7 will be described in detail. In the preamble, basically the same symbol is repeatedly transmitted. When the preamble from the information transmission device 100 is received with the phase difference δ, the frame output signals c ₁ , c ₂ , and c _{3 of} each of the three frames from the moving image camera 21 also repeat the same contents. If the reception processing it on the basis of the information reception process in FIG. 7, the angle ∠C1 output from the signal separation circuit 23 of the received signal processing circuit 210 is information of an angle ∠C ₁ of known preamble symbols angle ∠ P ₁ is moved by 2πδ. Therefore, the phase difference δ can be obtained by the following equation.

δ = (∠C _{1 −} ∠P ₁ ) / 2π

If δ <0 in this equation, it is corrected to the range of 0 ≦ δ <1 by δ ′ = δ + 1. δ corresponds to the timing difference between the information transmitting device 100 and the information receiving device 200, and by correcting it, synchronous communication becomes possible even in the partial synchronous operation. Of the output signals from the signal separation circuit 23 of the reception signal processing circuit 210, the information transmission device 100 is obtained by correcting the output angle ∠C1 for each symbol to ∠C1-2πδ and then performing the collation processing with the collation circuit 26. The timing deviation δ between the signal receiving device 200 and the information receiving device 200 can be corrected, and reception can be performed while maintaining timing synchronization. If the image sensor of the moving image camera 21 has a mechanism that can correct the shutter release time with an external signal, the phase difference δ detected in this way is fed back to the moving image camera 21, and the data part of the packet is completely synchronized. Can be received.

FIG. 8 is a perspective view showing a three-dimensional constellation of a modulated signal (m = 3) QAM-modulated with 8 bits in the horizontal plane and 3 bits in the vertical axis by the modulation circuit 12 of FIG. 4B. That is, FIG. 8 shows the result of an experimental transmission / reception experiment of visible light communication in this system as a symbol constellation, and completes the 256QAM signal performed on a 20 Hz virtual sine carrier. Can be decrypted. Further, since the modulation of 3 bits and 8 levels is also applied in the axial direction of the direct current component, 256QAM is laminated in 8 layers to form a m = 3D constellation. In total, it can be seen that the transmission of 11 bits per symbol can be executed without error. Since one symbol is transmitted in 1/20 second and 2048 values, that is, 11 bits are transmitted per symbol, the transmission speed is 20 × 11 = 220 bps. This is much higher performance than previously reported.

Communication by the general m-period method.
In the 4-period method with m = 4, that is, even numbers, a spectrum component corresponding to 2fq is added, but the Nyquist frequency becomes a spectrum having only a real number like direct current, and the constellation can be arranged in a four-dimensional space. Further, in the general m-period method, the information bit of the symbol can be arranged in the m-dimensional space.

It is the principle of OFDM (Orthogonal Frequency Division Multiplexing) performed in wireless communication to put information on the phase and amplitude of AC carriers, but in that case, the number of axes that can be used independently was twice the number of carriers. In optical communication, in addition to that, information can be carried on the DC axis and the Nyquist frequency axis (which occurs when m is an even number). In the visible light communication of the fully synchronous operation described as an example, information can be put on the entire spectrum of the m-dimensional by the communication using a virtual sine wave, and the most efficient communication can be performed.

However, in a "partially synchronized operation" communication system, the use of the Nyquist frequency axis requires caution. Similar to the DC axis, the Nyquist frequency axis can take real values and amplitude-modulate its carriers. However, if there is a phase difference δ between the light emission timing and the shutter timing between the transmitter and receiver, the Nyquist frequency carrier signal is affected by the phase difference δ, and the value is increased or decreased. This is in contrast to the information carried on the DC carrier, which is not affected by the phase difference δ. Depending on the value of the phase difference δ, it may be difficult to detect the Nyquist frequency carrier. Therefore, it is not a good idea to actively use the Nyquist frequency carrier for information transmission in a system with partial synchronization operation. However, the property of the Nyquist frequency carrier to react to the phase difference δ is useful for timing synchronization. If it is placed in the packet preamble, the phase difference δ can be easily detected by examining the signal level of the decoded Nyquist frequency carrier in S3 of the flowchart of FIG. 7.

FIG. 9 is a block diagram showing a configuration example of the transmission signal processing circuit 110C according to the third embodiment. FIG. 9 is a diagram for explaining modulation waveform generation when a large number of carriers are used, and is a configuration diagram when waveform synthesis is performed in the frequency domain using the inverse Fourier transform. In FIG. 9, the transmission signal processing circuit 110C includes a bit dividing circuit 11 and a modulation circuit 12C. Here, the modulation circuit 12C includes a DC component bias modulator 31, a QAM modulator 32-1 to 32-2 ((m-1) / 2), an amplitude modulator 32- (m / 2), and an adder. It is configured to include 33, an inverse Fourier transformer 34, and a low-pass filter 35. Here, the amplitude modulator 32- (m / 2) is provided when m is an even number.

In FIG. 9, the bit dividing circuit 11 divides the input digital data signal into bits to allocate information bit data to the phase, amplitude, and direct current of the frequency spectrum to be used, and if necessary, to the Nyquist frequency axis. The number of bits is σ _0, σ _1, σ _2, ..., Σ _{m / 2} , and information of σ (= σ ₀ + σ ₁ + σ ₂ + ... + σ _{m / 2} ) bits per symbol is transmitted. The DC component bias modulator 31 _{performs bias modulation according to the input bit data (σ 0} bit) and outputs it to the adder 33. The QAM modulator 32-1 performs QAM modulation using a carrier having a frequency of fp / m according to the input bit data (σ ₁ bit) and adds a modulated signal in the same manner as the operation of the QAM modulator 32 in FIG. Output to the device 33. The QAM modulator 32-2 performs QAM modulation according to the input bit data (σ ₂ bits) using a carrier having a frequency of 2 fp / m in the same manner as the operation of the QAM modulator 32 in FIG. Output to the adder 33. The QAM modulators 32-3 to ((m-1) / 2) perform QAM modulation in the same manner and output the modulated signal to the adder 33. The amplitude modulator 32- (m / 2) provided when m is an even number performs QAM modulation using a carrier with a frequency (m / 2) fp / m according to the _{input bit data (σ m / 2 bits).} The modulation signal is output to the adder 33.

In FIG. 9, the modulated signal in the frequency domain obtained by the modulator 31 to 32-2 (m / 2) is converted into time and space by the inverse Fourier transform, and the modulated signal b is interpolated by the low-pass filter 35. (T) is obtained, and the LED 14 is modulated and transmitted. If the number of types of code symbols is finite, the calculation may be performed in advance, sampling at a sufficiently high sampling frequency, and stored in the modulated waveform ROM. In that case, this information transmission device reads the time waveform of the signal from the ROM, performs DA conversion, and transmits the signal. Although the types of spectra to be collated with the information receiving device increase, basically the same operation as in FIG. 2 in the case of m = 3 may be performed.

Insertion of guard frame (3 + 1 cycle method).
The explanation of the operation so far has been performed by "perfect synchronization operation" in which the change of the light source and the shutter release time of the moving image camera match. However, most commercially available video cameras do not have a fine adjustment function for the shutter release time, and it is difficult to match the operating phases of the two, so "partial synchronous operation" must be taken into consideration. For example, when a signal having a length of 3 frames is received in a partial synchronization operation of m = 3, the symbol code is switched in the middle of the 3rd frame of shooting due to the influence of the phase difference δ ≠ 0 between the light emission and the shutter. The third frame captures a composite of two types of symbols, that is, it becomes a damaged frame in which signal switching is captured, and normal decoding cannot be performed using this.

In order to avoid this, the signal corresponding to the 4th frame is transmitted in spite of m = 3. The redundant fourth frame will be called a guard frame. The same waveform as that transmitted in the time interval of the first frame is repeatedly transmitted to the guard frame as the modulation signal b (t). In the 4th frame of the partial synchronization operation, the symbol switching is still photographed, resulting in a damaged frame. But even if it is discarded, the recipient gets a normal 3 frames. In this way, it is possible to avoid the damaged frame of the code change and always select and decode the normal 3 frames. This is named "3 + 1 cycle method".

The guard frame is redundant for information transmission, and by inserting the guard frame, the information transmission speed 220 bps in the embodiment of the three-cycle method is reduced to 165 bps, which is 3/4 of the information transmission speed.

The guard frame is named after the guard interval because it has the same function as the guard interval inserted in order to avoid the code switching part being broken due to the superposition of multipaths in the wireless OFDM communication system. By inserting a guard frame, the 3-cycle method is modified to the 3 + 1 cycle method. The guard frame insertion is a new invention, which is an appropriate method for using the light source and the camera in "partially synchronized operation".

A supplementary note on packet preamble when using guard frames. Since the preamble is usually a continuous transmission of the same symbol, the symbol is not switched in the preamble, and the damaged frame is not received. Therefore, the guard frame insertion of the 3 + 1 cycle method may be performed only for the packet data 52 and FCS53 (FIG. 1), and the preamble may be sent as the 3-cycle method. The timing phase difference δ between the information transmitting device and the information receiving device can be detected by the above-mentioned method (FIG. 7). Alternatively, the 3 + 1 cycle method can be applied to the preamble. In this case, only the preamble portion that transmits continuous symbols can be treated as a 4-cycle waveform, and since it has an even-numbered cycle, the Nyquist frequency carrier is detected. Therefore, it is possible to detect the timing phase difference based on the above-mentioned Nyquist frequency carrier (FIG. 7).

Reduction of flicker.
Visible light communication is performed by changing the intensity of the light source, but in some cases, the change in intensity can be seen with the naked eye. When performing visible light communication according to the present invention with digital signage or the like, the signal light source area of communication may be small and flicker does not matter so much, but visible light communication is performed by applying such modulation to the main lighting of the room. And those who are in the field will experience a very unpleasant flicker. The virtual sine wave property can be used for flicker-reduced modulation.

Incorporating the results of the flicker test of the fatigue test, what can be seen with the naked eye as flicker is a frequency component of about 50 Hz or less. Virtual sine wave communication is the conversion of the spectrum of a modulated light source to a low frequency due to the effect of folding noise. Therefore, even if the modulated light does not contain a physical low frequency component and the flicker cannot be confirmed with the naked eye, the modulated signal can be synthesized so that an information carrier wave including a direct current can be obtained as a virtual sine wave. Visible light communication using a camera with a large shutter opening, high sensitivity, and low noise while reducing flicker is also a unique effect brought about by the first embodiment of the present invention.

In communication with a frame rate of 60 fps and m = 3, a spectral component _{of a DC component Fourier coefficient C 0} and an AC frequency 20 Hz Fourier coefficient C _{1 is used as a virtual sine wave for communication.} If these are transmitted directly, they will all be visible to the naked eye as flicker. However, since the communication of the first embodiment according to the present invention is based on the premise that the high frequency component is frequency-converted by sampling the video frame, the transmission is physically performed by a high frequency carrier so that the flicker is not recognized. The reception can be performed by _{the Fourier coefficient C 0} of the DC component converted by the aliasing noise and the Fourier coefficient C _{1 of the AC frequency 20 Hz.}

It is assumed that the spectral values used in the information transmitter are, for example, ζ ₀ (direct current), ζ ₁ (120 Hz), and ζ ₂ (140 Hz). Since the spectrum values ζ ₀ and ζ ₁ are integral multiples of the frame frequency of 60 Hz, they are both detected as direct current by the information transmitter. Since the frequency of the spectrum value ζ ₂ is a frequency of more than 20 Hz when divided by 60, it is detected as a 20 Hz spectrum having a _{Fourier coefficient C 1 for the information receiving device.}

Information transmission treats the spectral values ζ ₁ and ζ ₂ as direct current and 20 Hz carriers, and modulates them with information bits. However, since the light emission of the LED cannot be a negative value, a constant bias value ζ ₀ is added to all codewords so that the amplitude value does not become negative. Since this bias value ζ ₀ is a constant value in the entire transmission process, it does not become flicker. Further, since the spectral values ζ ₁ and ζ ₂ are frequencies that exceed the flicker detection limit of a normal person, they do not become flicker. Therefore, if the codeword designed in this way is used, communication with reduced flicker can be realized.

Although the smartphone 400 has been described in the above-described first embodiment, the present invention is not limited to this, and the present invention may be applied to an electronic device such as a personal computer instead of the smartphone 400.

Embodiment 2.
Regarding the invention according to the second embodiment, a part of the configuration of the second embodiment is disclosed here for use in the third and fourth embodiments (see, for example, Patent Document 7). In the second embodiment, a synchronization timing detection system and the like capable of detecting the synchronization timing with high accuracy in a short time as compared with the prior art while using a commercially available general CMOS camera will be disclosed. First, the problems and objectives to be solved by the second embodiment of the present invention will be described below.

By measuring the propagation delay time between the mobile terminal and the beacon whose position is known using radio waves or sound waves, the distance can be known by assuming that the propagation speed is known, and the coordinate values of the mobile terminal can be determined by multipoint surveying. The use of GNSS (Global Navigation Satellite System: satellite positioning system represented by GPS) that uses microwaves outdoors and uses satellites as beacons is widespread, but sound waves are used indoors where the radio waves do not reach. The system is often used. As sound waves, the use of inaudible ultrasonic waves and high-frequency audible sounds that do not cause discomfort to users has been proposed.

In recent years, with the generalization of smart terminals (multifunctional terminals) such as smartphones and tablet terminals, there are many requests to accurately measure the position indoors and provide location-dependent services such as guidance in buildings. Since these smart terminals are equipped with microphones in the audible range, using high-frequency audible sound waves of about 15 kHz to 22 kHz can be used without any additional equipment and without bothering the user with the measurement sound. Positioning by multi-point survey can be performed.

There are two types of multipoint surveying principles based on propagation delay: TDoA (Time Difference of Arrival) and ToA (Time of Arrival). TDoA measurement is a method adopted when time synchronization is not established between the beacon system and the smart terminal to be measured. For 3D measurement, in addition to the 3D seat (x, y, z), time t Is also treated as an unknown number. Therefore, at least four beacons are used corresponding to the total number of unknowns. Further, ToA measurement is a method adopted when the time synchronization between the two is established. All system elements have a common clock, and smart terminals know when sound waves are emitted from the beacon. Therefore, the propagation time can be directly extracted from the time of arrival of the sound wave, and the three-dimensional measurement requires only three beacons having the same number as the unknowns (x, y, z).

The superiority or inferiority of TDoA and ToA in terms of measurement performance is not simply the number of required beacons. From the measurement principle, TDoA finds the position of the smart terminal as the intersection of hyperbolas with a fixed beacon as the focal point. On the other hand, in ToA, it is calculated as the intersection of circles centered on the fixed beacon. Due to the difference in geometrical properties between the two, it is known that the measurement accuracy of TDoA measurement deteriorates in the direction of the measurement signal source (distance direction). It is a car navigation system that uses GNSS that performs TDoA measurement, and it is well known that longitude and latitude information is relatively good, but altitude information is accompanied by a large error.

In the inventor's preliminary experiment, in the positioning of the smart terminal using high-frequency audible sound of about 17 kHz, the measurement accuracy was about 2.5 cm in the lateral direction (azimus direction) of the beacon at a distance of several meters from the beacon. In the distance direction (range direction) with respect to the beacon, it was about 20 cm, which was about 10 times worse. If the time synchronization is established between the smart terminal and the beacon, it is considered that the measurement by ToA can be performed, and the measurement can be performed in the range direction with the same accuracy as the azimuth direction under the same conditions.

In order to carry out ToA measurement, a time synchronization function between the smart terminal and the beacon system is prepared, but if the time synchronization accuracy is poor, the position measurement accuracy is rather deteriorated due to it. If the accuracy target of the positioning system is set to a few centimeters and the synchronization accuracy is suppressed to a few millimeters, which does not have a large effect on it, the time synchronization function is within 10 microseconds, considering the speed of sound in the air. Time synchronization must be done with accuracy.

The inventors have once proposed an NTP method using a time synchronization protocol between PCs as a positioning synchronization system that satisfies the accuracy requirement (see Patent Document 4). However, in the NTP method, since the clock frequency of the mobile terminal is estimated by communication and the processing is performed to match it with the system side, it is necessary to perform synchronous communication for several minutes to several tens of minutes until synchronization is established. In addition, since software processing accompanied by interrupts is performed, the time delay of information processing is indefinite, which deteriorates the accuracy. In order to reduce it, statistical averaging by communication for a certain period of time was required. For these reasons, it was difficult to use it when there was a request to start measurement immediately on the spot, such as a smart terminal.

FIG. 10 is a block diagram showing a configuration of a positioning system according to a second embodiment of the present invention. In FIG. 10, the positioning system according to the second embodiment includes a transmitting device 600 and a receiving device 700. Here, the transmission device 600 includes a modulation light source 501, for example, three speakers S501, S502, and S503 separated from each other by a predetermined distance (at substantially the same position) in the vicinity of the modulation light source 501, and a modulation light source. It is configured to include 501 and a measurement controller 502 that controls the operation of the speakers S501 to S503. The receiving device 700 includes a CMOS video camera having a rolling shutter effect (not necessarily a CMOS camera as long as it is a video camera having a rolling shutter effect) 504 and a smart terminal device 503 to be measured (for example, a microphone 505). , Is a smartphone). The positioning system according to the second embodiment receives the modulated light from the modulation light source 501 and the acoustic signals from the speakers S501 to S503, receives them by the smart terminal device 503 to be measured, detects the synchronization timing, and then detects the synchronization timing. The feature is that the distance from the speakers S501 to S503 to the smart terminal device 503 to be measured is measured, and the position of the smart terminal device 503 to be measured is determined based on the distance. That is, the positioning system according to the second embodiment includes a synchronous timing detection system and a distance measuring system.

In the second embodiment of the present invention, in order to avoid the problem in the TDoA method, the system is generated by generating modulated light from the modulated light source 501 and observing it with the CMOS video camera 504 of the smart terminal device 503 to be measured. It synchronizes the time of. The smart terminal device 503 to be measured includes a general CMOS video camera 504 for shooting a moving image, and it can be expected that the CMOS video camera 504 is used to synchronize the time by an optical method. Further, since the CMOS video camera 504 can be used at the same time as the acoustic measurement, it is suitable for performing positioning based on the ToA measurement without an additional mechanism. On the other hand, since the image sensor of the CMOS video camera 504 records a moving image at a speed of 60 frames per second or the like, simply using flash-like light emission obtains only a time resolution of about 16 milliseconds, which is a frame period. The synchronization accuracy of about 10 microseconds required for positioning could not be achieved.

In the second embodiment, the rolling shutter distortion of the image sensor is used in order to obtain a time resolution exceeding the frame period of the image sensor. In recent years, CMOS image sensors have been widely adopted as image sensors for consumer video cameras including smart terminals. This is because it is advantageous in terms of sensitivity characteristics, resolution, cost, etc., but it is known that a phenomenon called rolling shutter distortion occurs when an object moving at high speed is photographed (see, for example, Patent Document 5). This is because the CMOS image sensor drives the photodiode for each horizontal line, and the drive timing is slightly shifted between the upper and lower lines. Therefore, when a quadrangular horizontally moving object is photographed, the time shifts for each line. It is a phenomenon that is recorded in and is photographed in a distorted shape in a parallelogram. The second embodiment is characterized in that the characteristics peculiar to the CMOS image sensor are used in reverse, and an illumination light whose intensity is modulated with a specific waveform is used to obtain a highly accurate time motivation.

An image (shading) recorded on the screen when the modulation light source is recorded by a rolling shutter type CMOS image sensor having a frame rate f _p (frame period T _p = 1 / f _{p) and the number of lines N will be described.} In the description, for the sake of simplicity, a case where a surface light source that emits light uniformly in a wide area is modulated at a predetermined timing and recorded in the entire recording area of the image sensor will be described.

It is assumed that the light source periodically emits light at a period of m times the frame rate of the image sensor, and its time waveform is b (t). The complex Fourier coefficient is β _k . That is, the following equation is obtained. However, m is an integer of 2 or more.

Here, the periodicity of b (t) is assumed, but it is possible to deal with a wide range of b (t) by increasing m.

When this light source is photographed by the above image sensor, there is no difference in shade in each line of the image of the image sensor in the horizontal direction, and there is a change in shade caused by the rolling shutter effect only in the vertical direction, assuming that the light source is sufficiently large. .. From the assumption of the periodicity of the modulation light source, since the recorded image repeats m frames in a cycle, the images for m frames are connected vertically and considered as if it is an image sensor of M = mN line. The brightness of the μ line of the recorded image of the M line is written as Cμ.

If N is sufficiently large, and as a result M is sufficiently large, as in the image sensors currently on the market, the vertical direction of the pixels can be expressed as continuous with the parameter x = (μ-1) / M. Well, then C (x), which is a continuation of Cμ, _{can be expanded to the complex Fourier series γ k} , that is, the following equation is obtained.

Here, a is a coefficient expressing the sensitivity of the camera or the image sensor. Further, C (x) records the change in brightness of the modulation light source b (t) via the rolling shutter effect, and the relationship between the two is obtained by the complex Fourier coefficient.

Using this relationship, the feature portion of the waveform of the modulation light source b (t) can be extracted as the corresponding feature portion of the shade Cμ recorded in the camera, and the specific waveform transmission timing of the modulation light source is recorded in the image sensor. It can be extracted from shades. This is the principle of a timing detection system that uses a modulation light source and an image sensor that has a rolling shutter effect.

The case where a sufficiently large surface light source is used as a modulation light source has been described above, but even when a light source having a limited region or a surface light source is composed of a plurality of point light sources, a mask corresponding to the spatial distribution of Cμ is applied. Similarly, the relationship of the complex Fourier coefficient is established, and the timing detection of the method according to the second embodiment can be executed. The modulation pattern of the light source may be as long as the spectrum is known, and the transmitter and receiver may be designed integrally or independently. The modulation light source may be external to the system.

In the second embodiment, the phase-locked loop (PLL) or the phase comparison / seek operation associated therewith is not used, and therefore the usage of the modulated light is also different from such a system. The invention of Patent Document 6 aims at continuous synchronization of the clock of the entire system, but the second embodiment aims at taking a single timing of sound wave generation for positioning, and the meaning of synchronization is different. In the invention of Patent Document 4, it is not possible to know the specific timing of sound wave generation only from the synchronous signal waveform. Further, in Patent Document 6, since PLL is used as a synchronization method, a low-pass filter (LPF) is provided in the loop to prolong the settling time for establishing synchronization. In the second embodiment, the PLL is not used, and instead, the rolling shutter mechanism of the image sensor of the CMOS video camera is subjected to time decomposition, so that quick and accurate synchronization can be established even if a general-purpose low-speed image sensor is used. In the case of PLL, even if the synchronization is maintained for a long time, there is still a residual phase error corresponding to the loop gain. Further, if the loop gain is increased in an attempt to reduce the error, it becomes difficult to secure the loop stability. In the second embodiment, the synchronization error can be theoretically reduced as much as possible by statistical processing by extending the observation time. Further, in the invention of Patent Document 6, a special imaging mechanism capable of changing the frame rate for continuous synchronization is assumed, but such a mechanism is not necessary in the second embodiment.

Next, the configuration and operation when the system according to the second embodiment is used as an acoustic positioning system will be described in detail below.

In FIG. 10, the measurement controller 502 of the transmission device 500 also controls the operation of the modulation light source 501 and, for example, three speakers S501 to S503, based on the timing by a specific clock. Here, the speakers S501 to S503 are arranged at substantially the same position as the modulation light source 501 and separated from each other by a predetermined distance. The smart terminal device 503 to be measured is a mobile terminal device that seeks its coordinate position by the positioning system, and is usually assumed to be carried by a user and brought to the vicinity of a transmission system. There may be more than one at the same time. The smart terminal device 503 to be measured includes a CMOS video camera 504 and a microphone 505 that perform a rolling shutter operation, and also has a signal processing function for positioning processing.

The smart terminal device 503 to be measured can know the timing at which a plurality of acoustic signals are transmitted from the speakers S501 to S503 (timing synchronization) by observing the modulated light with the video camera 504 by the algorithm described below. Further, the microphone 505 knows the time when the plurality of propagating acoustic signals are received, and obtains the propagation time of each acoustic signal from the time difference. By multiplying this by the propagation speed of the acoustic signal (sound wave), the distance between the speakers S501 to S503 and the smart terminal device 503 to be measured is obtained (measurement processing unit 508 in FIG. 13). By measuring each distance from the three speakers S501 to S503 whose coordinate positions are known, it is possible to solve the equations of the three unknowns (x, y, z), and the three dimensions of the smart terminal device 503 to be measured. The position can be obtained (measurement processing unit 508 in FIG. 13).

By measuring each distance from two speakers whose coordinate positions are known, it is possible to solve the equation of two unknowns (x, y) and obtain the two-dimensional position of the smart terminal device 503 to be measured. You may.

Further, by performing timing synchronization with the modulated light, positioning based on the arrival time (ToA) becomes possible. If the number of speakers is four, positioning can be performed by the arrival time difference (TDoA) without synchronizing the timing of the system, but it is necessary to install surplus speakers S501 to S503, and in general, TDoA positioning is performed on the sound source. It has a drawback that the measurement accuracy deteriorates in the distance direction of. The positioning system according to the second embodiment avoids its drawbacks by introducing a timing synchronization function of a modulation light source.

FIG. 11 is a block diagram showing the configuration of the measurement controller 502 of FIG. In FIG. 11, the measurement controller 502 includes a timing control circuit 515, a light emission drive signal derivative ROM 510, three acoustic signal generation ROMs 511, 512, 513, and four DA conversion / drivers 520, 521, 522, 523. Is configured with. The measurement controller 502 controls the operation of the modulation light source 501 in which a plurality of LEDs are arranged in a grid shape, for example, and the three speakers S501 to S503. The modulation pattern of the light emission drive signal of the modulation light source 501 is stored in the light emission drive signal generation ROM 510, and each waveform of the acoustic signal is stored in the acoustic signal generation ROMs 511 to 513, based on the clock from the timing control circuit 515. A light emission drive signal and an acoustic signal are generated by being periodically read from each ROM 510 to 513. The light emission drive signal from the light emission drive signal generation ROM 510 has, for example, a pulse shape, is DA converted and amplified by the DA conversion / driver 520, and then is input to the modulation light source 501 to drive the modulation light source 501. As a result, the modulated light is emitted from the modulated light source 501. Further, each acoustic signal from the acoustic signal generation ROMs 511 to 513 has, for example, a predetermined time period and each frequency (in the FDM system described later, three different frequencies f1, f2, f3; in the TDM system, for example, they are the same. It has a frequency f0), and after DA conversion and amplification by the DA conversion / drivers 521 to 523, it is input to the speakers S501 to S503 and drives the speakers S501 to S503. As a result, a predetermined acoustic signal is radiated from the speakers S501 to S503.

FIG. 12 is a timing chart of each signal showing the operation of the measurement controller 502 of FIG. FIG. 12 shows an example of the radiation timing of the modulated light and the acoustic signal. In this example, since the modulated light is simple, only two states of ON and OFF are used, and the modulated light is driven by a rectangular wave. t1, t2, ..., T8 are basic timings, and one set of the basic timings t1 to t8 is repeatedly generated, and a predetermined period Tp (meaning the time period between each basic timing) from each basic timing t1 to t8. In the second embodiment, the period of the light emission drive signal is set to 2 Tp for m = 2 (see FIGS. 12 and 14), which is set to be equal to the video frame rate of the smart terminal device 503 to be measured. Since the general frame rate of the video camera 504 is 59.94 Hz (period Tp = 16.683 ms), for example, it is set as such. Further, each acoustic signal is generated in accordance with the timing of the modulated light, for example, at the rising edge of the basic timing t1. In the smart terminal device 503 to be measured, it is necessary to discriminate each acoustic signal from the speakers S501 to S503 having different frequencies by using, for example, a bandpass filter (BPF). For example, in the FDM (frequency division multiplexing) method, different frequencies are used. The acoustic signals of f1, f2, and f3 are transmitted from the speakers S501 to S503.

In the modified example, the speakers S501 to S503 may be driven by the TDM (time division multiplexing) method in accordance with the rise of the basic timings t1, t3, t5 and the like by using an acoustic signal having the same frequency f0.

By blinking the modulated light a plurality of times or by repeating the generation of an acoustic signal, it is possible to improve the synchronization timing detection accuracy and the distance detection accuracy by statistical processing in the smart terminal device 503 to be measured. .. Therefore, in this example, the operations of the basic timings t1 to t8 are repeated at a predetermined cycle. The blinking pattern is changed at the basic timings t7 and t8 in order to transmit the repeating frame structure (sound generation timing) to the smart terminal device 503, but if it is not necessary, this change may not be made. Good.

13 is a block diagram showing the configuration of the smart terminal device 503 to be measured in FIG. 10, and FIG. 14 is a timing chart of each signal showing the operation of the smart terminal device 503 to be measured in FIG. In FIG. 13, the measured smart terminal device 503 includes a CMOS video camera 504, a microphone 505, an AD converter 506, a synchronous timing extraction processing unit 507, a measurement processing unit 508, and a display 509. To. The synchronous timing extraction processing unit 507 and the measurement processing unit 508 are composed of, for example, an application program executed by the CPU (computer control device) of the smart terminal device 503 to be measured.

In FIG. 13, the CMOS video camera 504 receives the modulated light from the modulated light source 501, and based on the received modulated light, the video stream signal obtained by scanning the video frame with N lines of scanning lines and the repeat timing of the line scan ( A frame strobe signal F indicating (vertical synchronization signal) is generated, the former video stream signal is output to the synchronization timing extraction processing unit 507, and the latter frame strobe signal F is output to the measurement processing unit 508. As shown in FIG. 14, it is assumed that the rising tf of the frame strobe signal F coincides with the metering start timing tv1 of the scanning line 1. By analyzing the video stream signal, the synchronous timing extraction processing unit 507 can obtain a scanning line k having a metering start timing tvk that coincides with the rising timings of the basic timings t1, t2, ..., T8, and the metering start thereof. The timing ts (= tvk) is transmitted to the measurement processing unit 508.

Each acoustic signal from the speakers S501 to S503 is received by the microphone 505 and converted into an electric signal, then AD-converted by the AD converter 506, and then the audio stream signal after the AD conversion is input to the measurement processing unit 508. To. Here, the measurement processing unit 508 can detect the synchronization timing by knowing that the signal was sent at the timing ts delayed by the time period S from the rising timing tf of the frame strobe F, and each of the detection timing ts and the microphone 505 has been detected. The propagation delay time of the acoustic signal can be calculated by comparing the acoustic signals (arriving later than the timing ts).

In FIG. 14, which shows the principle of the synchronous timing extraction process, it is assumed that the CMOS image sensor of the CMOS video camera 504 performs a rolling shutter operation in which the metering start timing of each scanning line is deviated for each scanning line. The brightness (pixel value) of the modulation light source 501 captured by the image pickup devices of lines 1 to N is determined by the on / off timing phase of the modulation light source 501 of the image pickup device. If the photometric start timing tvk of the kth scanning line completely matches the start timing ts of the blinking phase of the modulation light source 501, the brightness of the modulated light observed in this line is the brightest (on-timing). , Or it becomes dark (off timing). As described above, the measurement processing unit 508 measures the distance between each speaker S501 to S503 and the smart terminal device 503 to be measured for each acoustic signal, thereby measuring the equation of three unknowns (x, y, z). Can be solved, and the three-dimensional position of the smart terminal device 503 to be measured can be obtained. The obtained three-dimensional position is displayed on the display 509 as position information.

FIG. 15 is a waveform diagram showing a time waveform of pixel values indicating the brightness of the modulation light source 501 measured by the CMOS video camera 504 of the smart terminal device 503 to be measured in FIG. As shown in FIG. 15, the brightness of the modulation light source 501 captured in the video frame has the shape of a triangular wave having the k-th line as the apex according to the line position.

For the sake of intuitive understanding, it was explained that a square wave is used as the modulation light source and that it is recorded as a triangular wave in the image pickup device. However, the same result can be obtained by calculating with the relation of the complex Fourier series by the equation (14). Obtained, and by utilizing it, a wider range of modulation waveforms with feature points can be adopted as the modulation light source.

Returning to the explanation using the square wave, by using the rising timing as a feature point, it corresponds to the folding timing of the triangular wave, that is, the starting timing of the kth line. It is the timing obtained by adding the delay time S to tvk to the rising timing tf of the frame strobe signal F, and S can be calculated by (tvk−tf) and is expressed by the following equation based on the frame rate Tp of the video.

S = ((k-1) / N) Tp

Here, N is the maximum number of lines of the video stream signal. Although k is an integer value, by measuring with a high SN ratio or improving the effective SN ratio by statistical processing, the light and dark triangular waves by discrete observation for each line are interpolated with a continuous straight line. , K can also be obtained as a real value having a decimal part. This contributes to the improvement of measurement accuracy.

Depending on the operation mode of the CMOS video camera 504, as shown in FIG. 15, a dead time D may occur between the metering start time of the line N and the metering start time of the line 1 in the metering operation of the frame scan. Since the dead time D is known at that time as well, the modified equation of the following equation may be used.

S = ((k-1) / N) (Tp-D)

The position of the line k can be obtained as an intersection of the straight lines corresponding to the triangular waves fitted to the output image data from the CMOS video camera 504. Alternatively, the triangular wave can be multiplied while shifting in the time direction to perform correlation processing, and can be obtained as the peak of the correlation value.

According to the positioning system according to the second embodiment configured as described above, the synchronization timing can be detected in a short time and with high accuracy as compared with the conventional technique. Further, by using the synchronous timing detection system, it is possible to provide a distance measuring system capable of measuring a distance in a short time and with high accuracy as compared with the conventional technique. Further, by using the above-mentioned ranging system, it is possible to provide a positioning system capable of positioning in a short time and with high accuracy as compared with the conventional technique.

In the second embodiment, the synchronous timing extraction processing unit 507 detects the rising timing or rising timing of the optical signal from the video stream signal F and outputs the detection timing, and the measurement processing unit 508 sets the output timing. By comparing with each line of the video stream signal F, the time of the above detection timing based on the frame strobe signal F is detected as the synchronization timing of the received acoustic signal. However, the present invention is not limited to this, and the measurement processing unit 508 compares the output timing with each line of the video stream signal F without transmitting and receiving an acoustic signal, thereby using the frame strobe signal F as a reference. The time of the detection timing may be detected as a predetermined synchronization timing such as a synchronization timing for shutter synchronization by transmitting and receiving an optical signal between two cameras, for example.

As described above, the disclosure of the invention according to the second embodiment is performed in order to use a part of the configuration of the second embodiment in the third and fourth embodiments (see, for example, Patent Document 7).

Embodiment 3.
Next, the information transmission system of the first embodiment according to the present invention is used as a part of the acoustic positioning system, and the acoustic positioning signal is received by the microphone while performing visible light communication with the moving image camera 21 such as a smartphone, and the accuracy is high. The third embodiment that obtains the terminal position information at the same time will be described.

In the partial synchronization operation, the communication method according to the first embodiment of the present invention detects the time difference between the information transmitting device and the information receiving device, and corrects the time difference so that the time difference becomes 0, thereby performing communication by synchronizing the time. I do. This enables information transmission, but in the state where the communication is established, highly accurate time synchronization is obtained between the transmission / reception devices.

On the other hand, there is an acoustic positioning application that measures the position of a smartphone by generating sound from a plurality of speakers whose positions are known, receiving the sound with a microphone such as a smartphone, and measuring the propagation delay of the sound. At that time, if the absolute time of sound generation is known, in other words, if the time is synchronized between the transmitting and receiving devices, positioning can be performed with a smaller number of speakers than in the case where it is not, and generally high-precision positioning can be performed. It can be carried out. Recent portable terminals such as smartphones are equipped with a video camera and a microphone, and the microphone is also operated at the same time when shooting a video, so a measurement method that uses both acoustic and optical methods can be rationally implemented. As an invention for improving acoustic measurement accuracy by using an optical synchronization method in combination, there is an invention according to the second embodiment (see, for example, Patent Document 7).

In the invention according to the second embodiment, the time-synchronized light emission is an embodiment only for that purpose, and there is no function of transmitting arbitrary information. Further, the mode of the modulation light source 501 is only a surface light source, and the point light source (LED14) as in the first embodiment cannot be used. However, according to the invention according to the first embodiment, since the time synchronization between the two is also obtained at the time when the information transmission between the transmitters and receivers is established, the acoustic positioning is simultaneously performed while communicating, and the second embodiment is performed. Highly accurate positioning according to the invention can be achieved. Further, it is permissible to use not only a surface light source but also a point light source as the modulation light source.

The visible light communication according to the second embodiment is targeted for a limited space in which the light source can be seen, and the acoustic positioning is also targeted for a limited space through which the sound is transmitted, which is similar to the first embodiment. .. If both can be done at the same time, complex, fine-grained location-dependent services can be executed.

FIG. 16 is a graph showing the synchronization accuracy when synchronous communication is performed in the information transmission system according to the first embodiment. In FIG. 16, each measurement point includes a laboratory room, a room with fluorescent lamp illumination, and the like, and as is clear from FIG. 16, when communication is performed by the communication method according to the first embodiment, the measurement points include the laboratory room and the fluorescent lamp illumination. Even in the room, the synchronization accuracy within ± 50 μs is obtained. This means that the origin of distance measurement due to propagation delay could be transmitted with an accuracy of within ± 15 mm when converted by the propagation speed of spatial acoustics, and it is possible to provide a reference point with sufficient accuracy in many position-dependent services. it can.

FIG. 17 is a block diagram showing a configuration example of the information transmission system according to the third embodiment of the present invention. The information transmission system according to the third embodiment is an information transmission system in which the information transmission system according to the first embodiment and the positioning system according to the second embodiment are combined, and as shown in FIG. 17, the embodiment of FIG. It is characterized in that the following points are different from the information transmission system according to 2.
(1) Instead of the transmission device 600 of FIG. 10, a transmission device 600A having the visible light communication acoustic positioning controller 502A of FIG. 19 is provided. Here, instead of the modulation light source 501 of FIG. 19, for example, a modulation light source 501A which is a point light source is provided.
(2) Instead of the receiving device 700 of FIG. 10, the receiving device 700B of FIG. 20 is provided. Here, instead of the CMOS video camera 504 of FIG. 10, the moving image camera 21 of FIG. 4A is provided. In the second embodiment, it was effective only in the CMOS sensor video camera 504 having the rolling shutter effect, but in the third embodiment, all the moving image cameras 12 can be used regardless of the type of camera.
This constitutes an information transmission system that simultaneously executes visible light communication and acoustic positioning of the synchronous communication method.

In FIG. 17, for example, a modulation light source 501A which is a point light source is provided in the central portion of the transmission device 600A, and visible light communication of a synchronous communication method is performed using the modulation light source 501A. Three positioning speakers S501, S502, and S503 are provided around the modulation light source 501A for the purpose of three-dimensional positioning, and the positions of the speakers S501 to S503 are known, or the coordinate information thereof is communicated by visible light. May be transmitted by. Here, the modulation light source 501A and the speakers S501 to S503 are controlled by the visible light communication acoustic positioning controller 502A.

FIG. 18A is a timing chart showing the operation of the transmission device 600A of the information transmission system of FIG. 17, and FIG. 18B is a timing chart showing the operation of the reception device 700B of the information transmission system of FIG.

As shown in FIG. 18A, the transmission device 600A transmits the visible light communication packet composed of the preamble, data, and FCS according to the first embodiment to the reception device 700B in synchronization with the preamble which is a synchronization signal, and also transmits the transmission device 600A. The visible light communication acoustic positioning controller 502A drives the speakers S501 to S503 as in the second embodiment, and transmits an acoustic signal for positioning to the receiving device 700B. In FIG. 18A, for simplification, the acoustic signal is transmitted at the same time as the packet preamble. The acoustic signals from the speakers S501 to S503 use carriers having different frequencies f1 to f3, and are identified by a frequency division multiplexing method at the time of reception. The transmission of the acoustic signal from the speakers S501 to S503 may be executed with a known time offset for the visible light communication packet, and at that time, another time offset is set for each of the speakers S501 to S503. Therefore, the sound waves of the acoustic signals from the speakers S501 to S503 may be identified by time division multiplexing.

When the frequency division multiplexing method is used, the acoustic signals from the speakers S501 to S503 are received by the microphone 505 of the receiving device 700B as shown in FIG. 18B. That is, since the optical signal of the visible light communication is propagating at 3 × 10 8 ^{m /} s, but may be considered to reach without delay, to propagate at the speed of the acoustic signal is approximately 340m / s, with a delay time corresponding Is received. When the delay time from the transmission time of the acoustic signal from each speaker S501 to S503 confirmed by the optical signal is (d ₁ , d ₂ , d ₃ ), the propagation speed of the acoustic signal is c, and each speaker S501 to S503 The distance from is (L ₁ , L ₂ , L ₃ ) = c (d ₁ , d ₂ , d ₃ ). If the position vector of the speaker S501~S503 and _{r 1, r 2, r 3} , the position vector R of the receiving apparatus 700B is theoretically a solution of the equation of the following equation.

| R-r ₁ | = d ₁
｜ R-r ₂ ｜ ＝ d ₂
｜ R-r ₃ ｜ ＝ d ₃

However, since the measurement distance generally includes an error, the position of the receiving device 700B is determined by the relational expression | R-r ₁ | = d ₁ + ε _{1 of the following equation.}
| R-r ₂ | = d ₂ + ε ₂
| R-r ₃ | = d ₃ + ε ₃
(15)
In, it is obtained as R that minimizes the error ε ² = ε ₁ ² + ε ₂ ² + ε ₃ ^2. _{Here, | R-r k | (} k = 1,2,3) is the Euclidean distance between the position vector _r k of the position vector R of the receiving device 700B each speaker S501~S503 (k = 1,2,3) Is.

FIG. 19 is a block diagram showing a configuration example of the transmission device 600A of FIG. In FIG. 19, the visible light communication acoustic positioning controller 502A of the transmission device 600A replaces the light emission drive signal generation ROM 510 with the transmission signal processing circuit 110 of FIG. 4A as compared with the measurement controller 502 of FIG. 11 according to the second embodiment. It is characterized by that. Other configurations are the same as in FIG.

FIG. 20 is a block diagram showing a configuration example of the receiving device 700B of FIG. In FIG. 20, the receiving device 700B is
(1) The information receiving device 200 (including the moving image camera 21 and the received signal processing circuit 210) of FIG. 4C according to the first embodiment and
(2) The microphone 505 of FIG. 13 according to the second embodiment, the measurement processing unit 508, and the display 509 (the position information output device may be an output device such as a printer or a voice synthesizer). It is characterized by.

In FIG. 20, the sound signal measurement processing unit 508 sets the timing of the frame strobe signal from the moving image camera 201 to the camera (shutter) phase difference δ from the collation circuit 26 (phase difference between the frame strobe signal and the printable or information signal). The transmission time of the acoustic signal is known by correction with, the delay times d ₁ , d ₂ , and d ₃ are obtained from the timing of the received signal, and the current position of the receiving device 700B is calculated and displayed by the equation (15). Therefore, the measurement processing unit 508 generates the synchronization timing of the received acoustic signal based on the phase difference δ between the frame strobe signal from the moving image camera 21 and the frame strobe signal and the signal in the visible light signal. By calculating the propagation time of the acoustic signal from the time difference between the synchronization timing and the reception timing of the received acoustic signal and multiplying the propagation time by the propagation speed of the acoustic signal, the transmission device and the reception Calculate the distance to the device.

Here, the first embodiment has been cited and described as a visible light communication method that brings about a time synchronization effect for the third embodiment, but if it is a visible light communication method that generally employs synchronous communication, it is viewed from the internal clock. It can be seen that the third embodiment can be realized by obtaining time synchronization between the transmitters and receivers by setting the difference in reception timing as the phase difference δ.

Regarding flicker, by using the method described in the section "Reducing flicker" in the first embodiment, the modulation frequency of the modulation light source 501A is shifted to a higher frequency, and flicker during visible light communication is not perceived. It is possible to do so. By doing so, the main lighting fixture of the room can be used as a modulation light source.

FIG. 21A is a perspective view showing a configuration example of the direct lighting fixture 901 suspended from the ceiling 900, which is a configuration example of the transmission device 600A of FIG. As shown in FIG. 21A, for example, a plurality of speakers S501 to S503 can be incorporated into a lighting fixture 901 suspended from a ceiling 900 of a room to simultaneously execute visible light communication and acoustic positioning.

The lighting equipment of the building may be installed on the ceiling 900. At that time, if it is installed on the ceiling 900 as shown in FIG. 21A and acoustically positioned by the measured smart terminal device 503 of the receiving device 700B, the measured smart The camera mechanism of the terminal device 503 is directed upward, which may be difficult. In order to solve this, a configuration example of FIG. 21B is shown.

21B is a perspective view showing a configuration example of the indirect lighting system 902, which is a configuration example of the transmission device 600A of FIG. As shown in FIG. 21B, the reflection diffuser 501R is placed in the line-of-sight range from the modulation light source 501A suspended from the ceiling 900, and the modulated light from the modulation light source 501A, which is a ceiling lighting fixture, is roughly transmitted by the reflection diffuser 501R, for example. By reflecting the light in the horizontal direction and photographing the reflected light with the smart terminal device 503 to be measured, the service form in which the visible light communication and the acoustic positioning described here are simultaneously performed may be taken. In the case of this form, the reflection diffuser 501R may be replaced by a wall surface or the like in which the speakers S501 to S503 are embedded. The reflection diffuser 501R may have at least one of a reflection function and a diffusion function.

In the above-mentioned third embodiment, the acoustic signal is transmitted by using the frequency division multiplexing method of the second embodiment, but the present invention is not limited to this, and the acoustic signal is transmitted by using the time division multiplexing method of the second embodiment. You may send it.

Although the lighting fixture 901 and the lighting system 902 are described in FIGS. 21A and 21B, the present invention is not limited to this, and the present invention may be applied not only to the lighting device but also to a device for other purposes.

In the above-mentioned third embodiment, the modulation light source 501A of the point light source is used, but the present invention is not limited to this, and the modulation light source 501 of the surface light source of FIG. 10 may be used.

Embodiment 4.
FIG. 22 is a block diagram showing a configuration example of the information transmission system 800 according to the fourth embodiment of the present invention. In FIG. 22, the information transmission system 800 according to the fourth embodiment is characterized in that it differs from the information transmission system 300 of FIG. 4A in the following points.
(1) As the information transmission device 100, for example, a plurality of information transmission devices 100-1 to 100-4, which are four, are provided.
(2) A transmission information data generation circuit 801 that outputs transmission information data to a plurality of information transmission devices 100-1 to 100-4 is provided.
With the above configuration, the identification IDs are transmitted from the LED 14 which is the light source of the information transmitting devices 100-1 to 100-4 by the information transmission method according to the first embodiment, and the image is taken by the moving image camera 21 of the smartphone 400 which is the receiving device. It is characterized in that the three-dimensional position of the geometrical arrangement of each light source is positioned with a positioning accuracy of, for example, about several mm based on the image obtained, and the shooting coordinate position is determined based on the positioning accuracy. Hereinafter, the measurement principle for estimating and measuring the three-dimensional position and posture will be described below.

When four LED lights having known three-dimensional positions on the same plane are observed, the three-dimensional position and orientation of the camera can be determined (see, for example, Non-Patent Document 2). For example assume that four LEDi measurement points _p i ₌ is disposed _{(x i, y i, z} i) (i = 1,2,3,4). z i _{= 0} can be assumed without loss of generality. At this time, the following equation is obtained.

Here, s is a scaling parameter, a point corresponding to the (u _{i, v i)} is the measurement point on the image plane of the camera p _i. A is the eigenmatrix of the camera. R = (r ₁ , r ₂ , r ₃ ) is a rotation matrix, t is a translation vector, and H is a homography matrix. By using the OpenCV library, A is fixed in the camera calibration process. The 3D position and orientation of the camera are then acquired via R and t.

Next, the experiment of the fourth embodiment and the result thereof will be described below.

The following two proof-of-concept experiments were conducted.
[Experiment 1] Estimating the three-dimensional position and orientation.
[Experiment 2] Data transfer error rate.

The apparatus configuration used in the experiment is shown in FIG. As the LED of the LED14 array, OSB56A5111A type LED manufactured by OptoSupply was used. Further, the information transmission devices 100-1 to 100-4 and the transmission information data generation circuit 801 were composed of three NF 1948 type function generators manufactured by NF. For identification purposes, each LED includes its ID in communication information and transmits it. In the first embodiment, the signal to be transmitted is adjusted so that it becomes one symbol in five video frames by adding a guard frame to the signal taken at m = 4.

FIG. 23 is a front photographic image showing an arrangement example of the LED 14 array of FIG. 22. As shown in FIG. 23, each LED 14 was fixed to the center of a table tennis ball to form its illumination diffusion and placed at each corner of a 0.3 m square. The moving image camera 21 was composed of a point gray Flea3 type camera (USB3.0, 60 fps, resolution 1280 × 1024 pixels, global shutter). Further, by configuring the received signal processing circuit 210 with a personal computer and installing a program necessary for the experiment, the start frame from each LED 14 is specified, the delay time is estimated, and the data transferred from each LED 14 is decoded. It became.

FIG. 24 is a plan view showing a plurality of measurement points P1 to P9 and a camera posture in the experimental example of the information transmission system 800 of FIG. 22. Further, FIG. 25 is a table showing the average error value and standard deviation of the estimation of the three-dimensional position and posture at each measurement point P1 to P9, which is the experimental result of the fourth embodiment.

As shown in FIGS. 24 and 25, nine cameras were arranged at measurement points P1 to P9. The LED14 array was mounted on the wall surface at a height of 1.5 m from the floor surface, and its center was set as the coordinate origin. The cameras at the measurement points P1, P4, and P7 are provided so that their imaging surfaces face the wall surface. The postures of the cameras arranged at the remaining measurement points P2, P3, P5, P6, P8, and P9 were set so that all the LEDs 14 were within the angle of view as shown in FIG. 24.

The three-dimensional position and orientation of the camera were measured 100 times per measurement point in Experiment 1. In Experiment 2, data was transferred 5120 times at each measurement point for a randomly generated N-bit integer. Here, N was set to 8, 9, and 10. The data transfer performance was evaluated by counting the number of correct decodings from the delay time to the camera using equation (16).

Next, the experimental results will be described below with reference to FIGS. 25 to 27.

FIG. 25 shows the error mean value and the standard deviation of the three-dimensional position, and the posture was estimated by the proposed information transmission system of FIG. 22. Here, the absolute error average value of the position estimation on each axis was in the range of 0.165 to 42.7 mm, but the maximum standard deviation was 3.11 mm. Therefore, the positioning performance can be improved by performing calibration for eliminating the system error. The estimation result of the attitude that was less than 1.98 degrees means the absolute error value. Also, the standard deviation is 0.0889 degrees, which is considered acceptable for most indoor location recognition applications.

FIG. 26 is a graph showing the cumulative distribution function of the estimation error of the three-dimensional position, which is the experimental result of the fourth embodiment. As is clear from FIG. 26, it was confirmed that the error of 90% of the estimation of the three-dimensional position was in the range of 12.4 to 50.6 mm. From the experimental results, it is considered difficult to determine how the position and angle of the camera affect the estimation performance.

FIG. 27 is a graph showing the coding error rates at different measurement points P1 to P9, which are the experimental results of the fourth embodiment. FIG. 27 shows the coding error rate when the measurement point is changed. The proposed information transmission system of FIG. 22 has always succeeded in receiving an 8-bit integer except for the measurement point P9 ^{whose error rate was 1.95 × 10 -4.}

By increasing the number of bits, the error rate also increased. It is clear from FIG. 27 that the error rate is related to the position and orientation of the camera. If the distance between the LED14 array and the camera is long, or if they are not facing straight, the data transfer performance will be degraded. Since the proposed information transmission system of FIG. 22 can decode 8-bit integer data with almost no error, the data transfer speed when using 5 image frames of a 60 fps camera and one LED 14 is 60/5 ×. 8 = 96 bps. Utilizing the high speed as the visible light communication method of the first embodiment, it is possible to realize faster optical positioning than the existing system.

As described above, according to the fourth embodiment, a system for indoor positioning of a three-dimensional position and rapid data transfer using LED lighting has been proposed. Specifically, a plurality of information transmitting devices 100 are installed at known positions, known unique information is transmitted, the unique information is received by the smartphone 400 which is an information receiving device, and the information transmitting device 100 is based on the received contents. By optically knowing the apparent geometrical arrangement of each of the information transmitting devices 100 while identifying the information receiving device, the existing position of the information receiving device is determined. Here, using a camera having the same performance as a smartphone with a built-in camera, the proposed information transmission system performs 8-bit data transfer per LED using 5 image frames through a demonstration experiment of the concept, and the maximum is It was confirmed that a standard deviation of 3.11 mm achieved an estimation error of 3D positioning of 12.4 mm to 50.6 mm at 90%. If the number of LEDs that can be identified by one symbol is used, positioning can be performed in 5 video frames, that is, 83 milliseconds, which is much faster than the existing positioning system using a moving image camera. This method can be applied to indoor position recognition services for mobile devices using pure optical methods.

Summary of embodiments.
The gist of the embodiment according to the present invention is summarized as follows.
(1) According to the embodiment of the present invention, in an information transmission system including a moving image camera that shoots at a constant frame frequency and a modulated light emitting light source having a fundamental frequency of a waveform that is an integer m of the frame frequency. It is possible to provide an optical information transmission system that utilizes the fact that the signal detected by the camera has an information axis having m-dimensional orthogonal phase amplitudes.
(2) According to the embodiment of the present invention, reception is performed in an information transmission system including a moving image camera that shoots at a constant frame frequency and a modulated light emitting light source having a fundamental frequency of a waveform that is an integral fraction of the frame frequency. To provide an optical information transmission system that enables information transmission under a wide range of shooting conditions by predicting the modulation change of the transmission signal and correcting the decoding conditions according to the shutter opening of the camera used for be able to.
(3) According to the embodiment of the present invention, in an information transmission system including a moving image camera that shoots at a fixed frame period and a modulated light emitting light source having a basic period of a waveform at a time of 1 or more and an odd multiple of the frame period. It is possible to provide an optical information transmission system in which transmission information is added to a DC component in addition to the AC phase amplitude of the detection signal sampled in the moving image frame period.
(4) According to the embodiment of the present invention, in an information transmission system including a moving image camera that shoots at a fixed frame period and a modulated light emitting light source having a basic period of waveform at a time of 2 or more and even times the frame period. It is possible to provide an optical information transmission system that adds transmission information to the DC component and the Nyquist frequency component, which is 1/2 of the frame period, in addition to the AC phase amplitude of the detection signal sampled in the moving image frame period. it can.
(5) According to the embodiment of the present invention, light is emitted in an information transmission system including a moving image camera that shoots at a fixed frame cycle and a modulated light emitting light source having a basic period of waveform at a time of m times the frame cycle. When the operating phases of the camera and the camera do not match, the transmission waveform is transmitted in the information receiving device by adding another frame time to the basic cycle m frame and repeating the transmission information in the first frame time. It is possible to secure m effective reception frames while discarding frames including the symbol switching timing.
(6) According to the embodiment of the present invention, in an information transmission system including a moving image camera that shoots at a constant frame frequency and a modulated light emitting light source having a fundamental frequency that is an integral fraction of the frame frequency, it is generated from the light source. Utilizing the fact that the harmonic component that is an integral multiple of the frame frequency is detected as equivalent to the DC component, the physical DC component required for communication is kept constant, and information that should be transmitted to that harmonic component by DC. It is possible to provide an optical information transmission system that reduces visual flicker by sharing the above frequencies.
(7) According to the embodiment of the present invention, a system in which synchronous visible light communication and acoustic positioning are used in combination and executed at the same time becomes possible.
(8) According to the embodiment of the present invention, each light source transmits an identification ID from the light source by the information transmission method according to the first embodiment, and each light source is based on an image taken by a moving image camera of a smartphone as a receiving device. The three-dimensional position of the geometrical arrangement of the above can be positioned with a positioning accuracy of, for example, about several mm, and the shooting coordinate position can be determined based on the positioning accuracy.

As described in detail above, according to the present invention, it is possible to provide an information transmission system or the like capable of performing visible light communication at a higher speed than that of the prior art.

11 ... Bit division circuit,
12, 12A, 12B, 12C ... Modulation circuit,
13 ... Drive circuit,
14 ... LED,
21 ... Video camera,
22 ... DFT arithmetic circuit,
23 ... Signal separation circuit,
24 ... Codeword ROM,
25 ... Shutter opening correction circuit,
26 ... Matching circuit,
31 ... DC component bias modulator,
32 ... QAM modulator,
32-1-22-((m-1) / 2)) ... QAM modulator,
32- (m / 2) ... Amplitude modulator,
33 ... adder,
34 ... Inverse Fourier Transformer,
35 ... Low Pass Filter (LPF),
40 ... Clock generator,
41 ... Modulated waveform ROM,
42 ... DA converter,
51 ... Preamble,
52 ... Data,
53 ... Frame Check Sequence (FCS),
100, 100-1 to 100-4 ... Information transmitter,
110, 110A, 110B, 110C ... Transmission signal processing circuit,
200 ... Information receiving device,
210 ... Received signal processing circuit,
300 ... Information transmission system,
400 ... Smartphone,
410 ... Display,
501, 501A ... Modulation light source,
501R ... Reflective diffuser,
502 ... Measurement controller,
502A ... Visible light communication acoustic positioning controller,
503,503A ... Smart terminal device to be measured,
504 ... CMOS video camera,
505 ... Microphone,
506 ... AD converter,
507,507A ... Synchronous timing extraction processing unit,
508 ... Measurement processing unit,
509 ... Display,
510 ... Emission drive signal generation ROM,
511-513 ... Acoustic signal generation ROM,
515 ... Timing control circuit,
520-523 ... DA conversion / driver,
532 ... Correlation processing unit,
600, 600A ... Transmitter,
700, 700A, 700B ... Receiver,
800 ... Information transmission system,
801 ... Transmission information data generation circuit,
900 ... Ceiling,
901 ... Direct lighting equipment,
902 ... Indirect lighting system,
P1 to P9 ... Measurement points,
S501 to S503 ... Speaker.

Claims (20)

Information having an information receiving device having a moving image camera that receives a visible light signal and a light source that transmits a visible light signal modulated by a modulated signal having a basic frequency of a waveform that is an integral part of the frame frequency of the moving image camera. An information transmission device for an information transmission system including a transmission device.
The information transmitting device produces a virtual sine wave obtained by Fourier transforming m frame output signals output from the moving image camera when a visible light signal modulated by the modulation signal is input to the moving image camera. An information transmission device comprising a modulation means for generating the modulated signal by orthogonal amplitude modulation according to an input digital data signal. Further provided with a dividing means for dividing the input digital data signal into the first and second digital data signals,
The above modulation means
According (1) said first digital data signal, and quadrature amplitude modulation with reference to the virtual sine wave, and in accordance with (2) said second digital data signals, by a DC bias modulation,
The information transmission device according to claim 1, wherein the modulated signal is generated. The above integer m is an even number
The information transmitting device further includes a dividing means for dividing the input digital data signal into the first, second, and third digital data signals.
The above modulation means
(1) According to the first digital data signal, quadrature amplitude modulation is performed using the virtual sine wave having a fundamental frequency of a waveform that is an integer m of the frame frequency of the moving image camera.
(2) According to the second digital data signal, quadrature amplitude modulation is performed using the virtual sine wave having a basic frequency of a waveform (m / 2) equal to an integer m of the frame frequency of the moving image camera, and (3). ) By DC bias modulation according to the above third digital data signal
The information transmission device according to claim 1, wherein the modulated signal is generated. Said modulating means, harmonic components of integral multiples of a frame frequency output from the light source to be detected as a DC component equivalent, the information transmission according to claim 2 or 3, wherein the modulating apparatus. The digital data signal has a fundamental frequency of a waveform that is an integer m of the frame frequency of the moving image camera.
The digital data signal is configured by further adding one guard frame signal having a period of the fundamental frequency to m frame signals having a period of the fundamental frequency.
The guard frame signal has the same waveform as the first frame signal among the m frame signals.
The information transmitting device according to any one of claims 1 to 4. An information receiver with a video camera that receives visible light signals,
M frames output from the moving image camera when a visible light signal modulated by a modulated signal having a waveform having a waveform of 1 / m of the frame frequency of the moving image camera is input to the moving image camera. Using a virtual sine wave obtained by Fourier transforming the output signal, orthogonal amplitude modulation is performed according to the input digital data signal to generate the above-mentioned modulation signal, and the visible light signal modulated by the above-mentioned modulation signal is transmitted. An information receiving device for an information transmission system including an information transmitting device.
The visible light signal is captured by the moving image camera and received, and the m frame output signals output from the moving image camera are Fourier-transformed to demodulate the digital data signal. Information receiver. The information receiving device according to claim 6, wherein the demodulation means generates a synchronization timing signal based on the frame output signal, and demodulates the digital data signal based on the synchronization timing signal. The information receiving device according to claim 6 or 7, wherein the demodulation means uses a reference signal of the frame output signal corrected according to the shutter opening degree of the moving image camera when demodulating the frame output signal. The information receiving device according to any one of claims 6 to 8, which receives a visible light signal transmitted from the information transmitting device according to claim 4.
The demodulation means is an information receiving device characterized in that a harmonic component that is an integral multiple of the frame frequency output from the light source is detected as equivalent to a direct current component. The information receiving device according to any one of claims 6 to 8, which receives a visible light signal transmitted from the information transmitting device according to claim 4.
The demodulation means detects that the harmonic component that is an integral multiple of the frame frequency output from the light source is equivalent to the DC component, and that the harmonic component that has a remainder with respect to the frame frequency is equivalent to the surplus frequency. An information receiving device characterized by the fact that. The information receiving device according to any one of claims 6 to 8, which receives a visible light signal transmitted from the information transmitting device according to claim 5.
The demodulation means is an information receiving device characterized in that the guard frame is discarded and demodulated. The information transmitting device according to any one of claims 1 to 3 and
An information transmission system including the information receiving device according to any one of claims 6 to 8. The information transmitting device according to claim 4 and
An information transmission system including the information receiving device according to claim 9. The information transmitting device according to claim 5 and
An information transmission system including the information receiving device according to claim 11. An information receiver with a video camera that receives visible light signals,
When a visible light signal modulated by a modulation signal having a fundamental frequency of a waveform that is an integral fraction of the frame frequency of the moving image camera is input to the moving image camera, m frames output from the moving image camera. The above-mentioned modulated signal is generated by orthogonal amplitude modulation according to the input digital data signal using the virtual sine wave obtained by Fourier-transforming the output signal, and the visible light signal modulated by the above-mentioned modulated signal is transmitted. A program for an information receiving device for an information transmitting system including an information transmitting device.
A computer including a step of capturing and receiving the visible light signal by the moving image camera, Fourier transforming m frame output signals output from the moving image camera, and demodulating a digital data signal. Program executed by. A receiving device having a moving image camera for receiving a visible light signal and a receiving means for receiving an acoustic signal,
When a visible light signal modulated by a modulation signal having a fundamental frequency of a waveform that is an integral fraction of the frame frequency of the moving image camera is input to the moving image camera, m frames output from the moving image camera. Using a virtual sine wave obtained by Fourier transforming the output signal, the modulation signal is generated by orthogonal amplitude modulation according to the input digital data signal and a predetermined synchronization signal, and the visible signal modulated by the modulation signal is generated. A positioning system including a light source for transmitting an optical signal and a transmission device having a transmission means for transmitting an acoustic signal modulated according to the synchronization signal.
The receiving device receives the visible light signal and the acoustic signal, and receives the visible light signal and the acoustic signal.
The above receiving device
The synchronization timing of the received acoustic signal is generated based on the phase difference between the frame strobe signal from the moving image camera and the frame strobe signal and the signal in the visible light signal, and the synchronization timing and the reception are received. The propagation time of the acoustic signal is calculated from the time difference from the reception timing of the acoustic signal, and the distance between the transmitting device and the receiving device is calculated by multiplying the propagation time by the propagation speed of the acoustic signal. A positioning system characterized by having a measurement processing unit. The positioning system according to claim 16 is provided.
The transmission device is a plurality of speakers provided at different positions, and transmits a plurality of acoustic signals having different frequencies, or a plurality of acoustic signals having the same frequency and sequentially transmitted at a predetermined time difference. Equipped with multiple speakers
The measurement processing unit calculates each distance between the plurality of speakers and the receiving device based on the plurality of acoustic signals, and positions the receiving device based on the calculated distances. A featured positioning system. A luminaire comprising the transmitting device for the positioning system according to claim 16 or 17, and illuminating the luminaire. The transmitter for the positioning system according to claim 17 is provided.
With the above light source for lighting,
A lighting system having the plurality of speakers and provided with means for reflecting or diffusing visible light from the light source to the receiving device. A plurality of information transmitting devices according to any one of claims 1 to 5 are installed at known positions, known unique information is transmitted, and the unique information is transmitted to any one of claims 6 to 10. By receiving with the information receiving device described in the above, identifying the information transmitting device from the received contents, and optically knowing the apparent geometrical arrangement of each of the information transmitting devices, the existing position of the information receiving device is obtained. A positioning system characterized by positioning.

Images