JP4992129B2 - Optical communication device - Google Patents

Description

  The present invention relates to an optical communication apparatus, and more particularly to an optical communication apparatus that receives signal light and performs optical communication.

Conventionally, an apparatus for performing optical communication is known (for example, Non-Patent Document 1). In the apparatus described in Non-Patent Document 1, an image sensor readout line and a communication high-speed readout line are provided for the image sensor pixel, and either a communication signal or an image signal is switched from the image sensor pixel. Output.
Keiichiro Kagawa, Tomohiro Nishimura, Takao Hirai, Atsushi Ota, Masahiro Nushita, Koji Yamazaki, Masaji Yamada, Shozo Sugishita, Kunihiro Watanabe, “Development of Vision Chip for Optical Wireless LAN Using BiCMOS Process”, Institute of Image Information and Television Engineers Information Sensing Consumer Electronics Study Group, ITE Technical Report 26 (26), pp. 35-40 (2002)

  However, in the description of Non-Patent Document 1 described above, since a communication signal and an image signal cannot be output simultaneously from the same pixel, when a communication signal is output, an image signal of coordinates at which communication is performed is output. However, there is a problem in that the image is missing and the image quality obtained from the image signal is degraded.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an optical communication apparatus capable of performing optical communication without degrading the image quality of a captured image obtained.

In order to achieve the above object, an optical communication apparatus according to the present invention outputs a plurality of first photoelectric conversion elements that output a signal corresponding to the amount of received light, a signal corresponding to the amount of received light, and the amount of received light and a second photoelectric conversion element is the response speed of the plurality faster than the first photoelectric conversion element with respect to a change in, the first the first photoelectric conversion element and the second photoelectric conversion element of the plurality such that the pair Obtained based on an image sensor in which a photoelectric conversion element and the plurality of second photoelectric conversion elements are two-dimensionally arranged on one substrate, and a signal output from the plurality of first photoelectric conversion elements of the image sensor Based on a signal output from the second photoelectric conversion element located in the region extracted by the extraction unit and the extraction unit that extracts the region on the substrate receiving the signal light from the captured image And set a reference value for the intensity of the signal light. A reference value setting means for, based on the set the reference value by the reference value setting means, the digital converting a signal output from the second photoelectric conversion elements located has been within the region extracted by said extracting means Then , an optical communication output means for outputting the digitally converted signal as an optical communication signal represented by the signal light, and an imaging obtained based on signals output from the plurality of first photoelectric conversion elements of the imaging element Image output means for outputting an image.

  According to the optical communication device of the present invention, on the substrate that receives the signal light from the captured image obtained based on the signals output from the plurality of first photoelectric conversion elements of the imaging element by the extraction unit. Extract regions.

  Then, the optical communication output means outputs the signal output from the second photoelectric conversion element located in the region extracted by the extraction means as an optical communication signal represented by the signal light, and the image output means captures the image. A captured image obtained based on signals output from the plurality of first photoelectric conversion elements of the element is output.

  In this manner, the captured image obtained based on the signals from the plurality of first photoelectric conversion elements is output, and the signal output from the second photoelectric conversion element is output as an optical communication signal represented by the signal light. Thus, optical communication can be performed without degrading the image quality of the obtained captured image.

  Here, the second photoelectric conversion element has a faster response speed to the change in the amount of received light than the first photoelectric conversion element, and can capture a high-speed change in signal light. can do.

  The optical communication apparatus according to the present invention further includes a calculation unit that calculates a disturbance light component based on a signal output from the first photoelectric conversion element of the imaging element, and the optical communication output unit is extracted by the extraction unit. The disturbance light component calculated by the calculation means is removed from the signal output from the second photoelectric conversion element located in the region thus generated to generate an optical communication signal represented by the signal light, and the generated optical communication signal is Can be output. Thereby, the disturbance light component can be removed and an optical communication signal can be obtained, so that highly accurate optical communication can be realized.

  The optical communication apparatus according to the present invention further includes setting means for setting a disturbance light component based on a signal output from at least one of the first photoelectric conversion element and the second photoelectric conversion element of the imaging element, and outputs the optical communication. The means removes the disturbance light component set by the setting means from the signal output from the second photoelectric conversion element located in the region extracted by the extraction means, and generates an optical communication signal represented by the signal light. The generated optical communication signal can be output. Thereby, the disturbance light component can be removed and an optical communication signal can be obtained, so that highly accurate optical communication can be realized.

  The optical communication output unit according to the present invention removes the disturbance light component by passing only a predetermined frequency band component from the signal output from the second photoelectric conversion element located in the region extracted by the extraction unit. An optical communication signal represented by the signal light can be generated based on the signal output from the filter, and the generated optical communication signal can be output. Thus, components other than the signal light can be removed to obtain an optical communication signal, so that highly accurate optical communication can be realized.

Optical communication apparatus according to the present invention, based on the signal output from the second photoelectric conversion elements located in the region extracted by the Extraction means, reference value setting means for setting a reference value of the intensity of the signal light The optical communication output means digitally converts the signal output from the second photoelectric conversion element located in the region extracted by the extraction means based on the reference value set by the reference value setting means , you output digital converted signal as an optical communication signal. As a result, an optical communication signal digitally converted based on a reference value considering the intensity of the signal light can be obtained, so that highly accurate optical communication can be realized.

  The extraction unit can extract a region on the substrate receiving signal light from the captured image at a predetermined period. As a result, it is possible to track the area receiving the signal light.

  The optical communication apparatus further includes an estimation unit that estimates at least one of a movement amount and a movement direction of the region based on the region extracted in the past by the extraction unit, and the extraction unit includes the estimation unit. Based on at least one of the movement amount and the movement direction estimated by the above, an area on the substrate that receives the signal light can be extracted from the captured image. As a result, it is possible to easily extract the region receiving the signal light.

  About said 2nd photoelectric conversion element, the charge storage time in photoelectric conversion can be made shorter than a 1st photoelectric conversion element. Thereby, the response speed with respect to the change in the amount of received light can be increased.

  As described above, according to the optical communication device of the present invention, the captured image obtained based on the signals from the plurality of first photoelectric conversion elements is output, and the signal output from the second photoelectric conversion element is output. By outputting as an optical communication signal represented by the signal light, there is an effect that optical communication can be performed without degrading the image quality of the obtained captured image.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a case where the present invention is applied to an optical communication device of a driving support system that is mounted on a vehicle and performs driving support based on received data received by optical communication will be described as an example.

  As shown in FIG. 1, the driving support system 10 according to the first embodiment captures an image of the front of the vehicle, performs optical communication, and receives and receives data received by the optical communication device 12. And a driving support device 14 that performs driving support based on the captured image.

  The optical communication device 12 receives the signal light emitted from the optical transmission device 16 and receives the optical communication signal. The optical transmission device 16 includes, for example, a light emitting element 18 composed of LEDs, and a communication control unit 20 that turns on and off the light emitting element 18 in accordance with a data string of transmission data.

  The communication control unit 20 of the optical transmission device 16 turns on and off the light emitting element 18 so that information indicating the vehicle speed or acceleration / deceleration of the mounted vehicle is transmitted by signal light. The light emitting element 18 is switched (on / off) at a high speed in order to transmit a large amount of data.

  The optical communication device 12 outputs the reception data represented by the optical communication signal received by optical communication from the optical transmission device 16 mounted on another vehicle to the driving support device 14 as a reception data string. In this way, the optical communication device 12 realizes wireless communication using light instead of wired communication or wireless communication using radio waves. Since wired communication requires wiring for signal transmission, convenience decreases as the communication distance increases. In addition, wireless communication using radio waves is a barrier to the allocation of bands according to the Radio Law, and the use of radio waves increases the cost of radio transceivers. On the other hand, the optical communication used in this embodiment is realized by an inexpensive system composed of a light emitting element and a camera, and has no legal barrier, so it can be used for wired communication or wireless communication using radio waves. Has superiority.

  The driving support device 14 of the driving support system 10 performs image processing such as pattern matching on the captured image captured by the optical communication device 12 to detect the position of the vehicle ahead, and from the optical communication device 12. The vehicle speed or acceleration of the preceding vehicle is acquired from the received data. Then, the driving support device 14 determines the possibility of collision with the preceding vehicle based on the acquired position of the preceding vehicle and the vehicle speed or acceleration, and based on the determined possibility of collision, Present and control driving.

  As illustrated in FIG. 2, the optical communication device 12 includes an image sensor 24 and a control device 26 that performs drive control of the image sensor 24 and performs predetermined image processing.

  As shown in FIG. 3, the image sensor 24 has a plurality of photoelectric conversion elements 28 </ b> A for image pixels and a plurality of photoelectric conversion elements 28 </ b> B for optical communication pixels that output signals according to the amount of received light on a single substrate. A two-dimensionally arranged pixel array 28 and a digital conversion unit 30 that converts an analog signal output from the photoelectric conversion element 28B for the optical communication pixel into a digital signal are provided.

  In the pixel array 28, for example, photoelectric conversion elements 28A for image pixels and photoelectric conversion elements 28B for optical communication pixels are alternately arranged for each line. As shown in FIG. 4, the photoelectric conversion elements 28 </ b> A for image pixels and the photoelectric conversion elements 28 </ b> B for optical communication pixels may be arranged in a lattice pattern, or may be arranged in other arrangement patterns. . However, as in the two specific examples shown in FIGS. 3 and 4, the photoelectric conversion element 28 </ b> A for the image pixel and the photoelectric conversion element 28 </ b> B for the optical communication pixel are arranged in a pair (proximity structure), It is preferable that the optical axis for the photoelectric conversion elements 28A for the plurality of image pixels and the optical axis for the photoelectric conversion elements 28B for the plurality of optical communication pixels substantially coincide.

  In addition, the imaging device 24 includes only one lens (not shown) as a common lens for the photoelectric conversion elements 28A for a plurality of image pixels and the photoelectric conversion elements 28B for a plurality of optical communication pixels. It is used.

  Since the light emitting element 18 of the optical transmission device 16 is turned on and off at high speed and a general photoelectric conversion element for capturing an image cannot respond to the on / off speed of the light emitting element 18, as the photoelectric conversion element 28 </ b> B for optical communication, A photoelectric conversion element that has a short charge accumulation time in photoelectric conversion and can respond to high-speed on / off is used.

  Further, a signal output from the photoelectric conversion element 28 </ b> A for image pixels is output as an image signal to the control device 26, and a coordinate value is input from the control device 26 to the pixel array 28 of the image sensor 24. The pixel array 28 turns on a switch (not shown) corresponding to the photoelectric conversion element 28B for the optical communication pixel located at the position represented by the coordinate value input from the control device 26, and the photoelectric conversion element 28B with the switch turned on. The analog signal output from is output to the digital conversion unit 30.

  The digital conversion unit 30 converts the analog signal input from the photoelectric conversion element 28B for the optical communication pixel into a digital signal, and outputs the digital signal to the control device 26 as an optical communication signal.

  As shown in FIG. 5, the digital conversion unit 30 includes a DC component removing device 34 for deleting or reducing disturbance light components, an amplifier 36 for amplifying a weak signal, and a digital signal for converting an analog signal into a digital signal. And a converter 38.

  The DC component removing device 34 is a device that removes disturbance light (sunlight, streetlight, etc.) components that are very low frequency (substantially DC) compared to the optical communication signal, and extracts only the optical communication signal modulated to a high frequency. is there. The digital converter 38 is configured using, for example, a comparator, converts the analog signal up to the previous stage into a 1-bit digital signal, and outputs the digital signal.

  The amplifier 36 may be provided when the output of the DC component removing device 34 is weak, and may not be provided when the signal intensity is sufficient. Further, an amplifier 36 may be provided in the previous stage of the DC component removing device 34.

  As shown in FIG. 6, the direct current component removing device 34 outputs a difference between the analog signal input from the photoelectric conversion element 28 </ b> B for the optical communication pixel and the intensity level of the disturbance light input from the control device 26. A circuit 40 is provided. The signal output from the photoelectric conversion element 28B for the optical communication pixel includes an intensity signal due to disturbance light such as sunlight other than the signal light. Therefore, the difference circuit 40 obtains the intensity level of the disturbance light calculated based on the image signal output from the photoelectric conversion element 28A for the image pixel, and as shown in FIG. The difference between the input signal from the photoelectric conversion element 28B and the intensity level of the disturbance light (input signal from the photoelectric conversion element 28B-intensity level of the disturbance light) is taken, as shown in FIG. A signal from which the light component has been removed is output.

  In addition, as shown in FIG. 8B, the digital converter 38 configured by a comparator as shown in FIG. 8A has a reference value (threshold value) in which the level of the input analog signal is set as input. 1 is output when the level is exceeded, and 0 is output when the level is below the reference value level. As described above, the digital converter 38 converts the analog signal output via the DC component removing device 34 and the amplifier 36 into a 1-bit digital signal and outputs it as an optical communication signal.

  The control device 26 of the optical communication device 12 is configured by a computer and includes a CPU, a ROM that stores a program of an optical communication processing routine described later, a RAM that stores data, and a bus that connects these. ing. The control device 26 will be described in terms of functional blocks divided for each function realization means determined based on hardware and software. As shown in FIG. 9, the image signal is output from each of the photoelectric conversion elements 28A for a plurality of image pixels. The region on the substrate that receives the signal light from the optical transmitter 16 from the image signal input unit 44 that inputs the signal light and the captured image obtained based on the plurality of image signals input by the image signal input unit 44. A light extraction unit 46 that extracts and outputs coordinate values in the extracted region to the pixel array 28, and a disturbance level calculation that calculates the intensity level of disturbance light based on the image signal input by the image signal input unit 44 Unit 48, an image output unit 50 that outputs a captured image obtained based on the plurality of image signals input by the image signal input unit 44, and the digital conversion unit 30. The optical communication signal input / output unit 52 that inputs the input optical communication signal and outputs it to the outside, and the reference used when digitally converting the optical communication signal into an optical communication signal based on the image signal input by the image signal input unit 44 And a reference value determination unit 54 for determining a value level.

  When the signal light from the light emitting element 18 of the optical transmission device 16 is directly incident on the image pickup device 24 of the optical communication device 12 from the front, the intensity of the received light is large, so In the photoelectric conversion element 28A, overflow (saturation) due to excessive light incidence occurs, and only the region where the signal light of the captured image is received appears in white. Therefore, the light extraction unit 46 extracts a region on the substrate that receives the signal light from the captured image obtained based on the image signal, using a saturated region as a clue. In addition, the light extraction unit 46 outputs the coordinate value in the extracted area to the pixel array 28, and the light extraction unit 46 receives the light in the photoelectric conversion element 28B for the optical communication pixel located at the position represented by the coordinate value in the extracted area. A signal corresponding to the light is output.

  In addition, the light extraction unit 46 tracks the region receiving the signal light by repeatedly extracting the region on the substrate receiving the signal light at a predetermined period. In addition, in the captured image, by utilizing the fact that the region receiving the signal light does not move greatly in a short time, the light extraction unit 46 determines the region receiving the signal light as the region extracted last time. Search preferentially from the vicinity. As a result, the processing efficiency can be improved and the calculation cost can be reduced as compared with the case where a region receiving the signal light is searched and extracted from the entire captured image obtained from the image signal each time.

  Further, when tracing the region receiving the signal light, the light extraction unit 46 observes the movement amount and the moving direction of the region receiving the signal light by image processing as shown in FIG. Based on the observed movement amount and movement direction, the movement destination at the next timing is estimated, and the area receiving the signal light is preferentially searched from the estimated movement destination area. This further improves the processing efficiency. Note that only one of the movement amount and the movement direction of the light receiving area is observed, and the movement destination at the next timing is estimated based on either the observed movement amount or the movement direction. Also good.

  The disturbance level calculation unit 48 uses the image signal input by the image signal input unit 44 and, as will be described below, the disturbance light component included in the signal output from the photoelectric conversion element 28B for the optical communication pixel. Calculate the intensity level.

The image signal output from the photoelectric conversion element 28A for the image pixel represents the intensity level of disturbance light components such as sunlight, but the photoelectric conversion element 28A for the image pixel and the photoelectric conversion element 28B for the optical communication pixel. Since the structure is different from, the output signal intensity differs even when the same amount of light is applied. Therefore, the intensity of the signal output from each of the two types of photoelectric conversion elements 28A and 28B when the same light is incident is measured in advance, and the intensity of the output signal of each of the two types of photoelectric conversion elements 28A and 28B is the same. A conversion coefficient α for obtaining the above is obtained. And the intensity level of the disturbance light component contained in the signal output from the photoelectric conversion element 28B for optical communication pixels is calculated by the following equation.
The intensity level of the disturbance light component = α × the intensity level of the image signal. The reference value determination unit 54 receives the signal light of the picked-up image obtained based on the image signal input by the image signal input unit 44. Based on this, the intensity level of the signal light is calculated, and the level of the reference value input to the digital converter 38 is determined according to the intensity level of the signal light.

  When the disturbance light component based on the image signal is removed, it is necessary to arbitrarily set the reference value as shown in FIG. For example, when the reference value when the incident signal light is sufficiently strong (saturated) is set as shown in FIG. 11A, if the intensity of the signal light becomes weak as shown in FIG. It becomes impossible to convert. Therefore, the reference value determination unit 54 monitors the intensity (brightness) of the received signal light from the captured image obtained based on the image signal, and according to the intensity of the received signal light as shown in FIG. Even if the intensity of the signal light is weakened by dynamically changing the level of the reference value, the analog signal from the photoelectric conversion element 28B for the optical communication pixel is accurately converted into a digital signal. Control.

  Next, the operation of the driving support system 10 according to the first embodiment will be described. Hereinafter, a case where signal light is emitted from the light transmission device 16 mounted on the preceding vehicle when the host vehicle on which the driving support system 10 is mounted is traveling will be described as an example.

  In the optical communication device 12 of the driving support system 10, the optical communication processing routine shown in FIG. 13 is executed. First, in step 100, 0 is set as an initial value to a variable t indicating an elapsed time after outputting the captured image, and in step 102, the image output from each of the photoelectric conversion elements 28A for the plurality of image pixels. Get the signal.

  In step 104, the captured image representing the preceding vehicle obtained based on the image signal acquired in step 102 is output to the driving support device 14, and in the next step 106, the image signal acquired in step 102 is output. The disturbance light intensity level included in the signal output from the photoelectric conversion element 28B for the optical communication pixel is calculated, and the calculated disturbance light intensity level is input to the digital conversion unit 30.

  In step 108, an area on the substrate receiving the signal light from the optical transmitter 16 is extracted from the captured image obtained based on the image signal acquired in step 102. In step 109, the image is captured. The intensity of the received signal light is calculated from the region extracted in step 108 of the image, the reference value level for digital conversion is determined according to the calculated intensity of the signal light, and the digital converter 38 The determined reference value level is input and set as the reference value level.

  In step 110, the coordinate value in the region on the substrate extracted in step 108 is output to the pixel array 28, and the photoelectric signal for the optical communication pixel located at the position represented by the coordinate value in the extracted region is output. The conversion element 28B is instructed to output a signal. As a result, the signal output from the photoelectric conversion element 28B located at the location represented by the designated coordinate value is input to the digital conversion unit 30 as an analog signal, and the disturbance light component of the disturbance light level input in step 106 is input. Are removed, converted into a digital signal, and output from the image sensor 24 as an optical communication signal.

  In the next step 112, an optical communication signal is acquired from the digital conversion unit 30, and in step 114, the acquired optical communication signal is output to the driving support device 14 as reception data. In step 116, the variable t representing the elapsed time is incremented. In step 118, it is determined whether the elapsed time t is less than the time F corresponding to a predetermined captured image output rate. However, if the time corresponding to the captured image output rate has not been reached, the process returns to step 112 to acquire the optical communication signal at the next timing, while the elapsed time is the time corresponding to the captured image output rate. If it reaches, step 120 is entered. For example, 33 ms (30 fps) or 16 ms (60 fps) may be set as the time corresponding to the captured image output rate.

  In step 120, based on the movement amount and movement direction of the region receiving the signal light extracted in the past, the movement destination of the region receiving the signal light is estimated, and the process returns to step 100, and the image is again displayed. Get the signal. Further, in step 108, the search is preferentially performed from the movement destination estimated in step 120, and the region receiving the signal light is extracted.

  By executing the optical communication processing routine described above, the region receiving the signal light is tracked at the same cycle as the cycle of the captured image output rate, and the photoelectric conversion element 28B positioned at the coordinates in the tracked region. The optical communication signal is acquired, and a received data string represented by the optical communication signal is output to the driving support device 14. In addition, each of the captured images obtained based on the image signal acquired at the cycle of the captured image output rate is output to the driving support device 14.

  Then, the driving support device 14 determines the possibility of collision with the preceding vehicle based on the vehicle speed or acceleration of the preceding vehicle represented by the received data string and the position of the preceding vehicle detected from the captured image. Based on the possibility of collision, an alarm is presented and driving control is performed as driving assistance.

  As described above, according to the optical communication device of the driving support system according to the first embodiment, the captured image obtained based on the signals from the photoelectric conversion elements for a plurality of image pixels is output, By outputting the signal output from the photoelectric conversion element for the optical communication pixel as an optical communication signal represented by the signal light, optical communication can be performed without degrading the image quality of the obtained captured image.

  In addition, the photoelectric conversion element for the image pixel configured to aim at the image quality of the captured image and the photoelectric conversion element for the optical communication pixel configured to achieve a high-speed response improve the image quality of the captured image and achieve high accuracy. Optical communication can be performed.

  In addition, by extracting a region receiving signal light from the captured image and acquiring an optical communication signal based on a signal output from the photoelectric conversion element for the optical communication pixel located in the region, optical communication is performed. The signal can be received with high accuracy.

  In addition, since the disturbance light component calculated based on the image signal can be removed from the signal from the photoelectric conversion element for the optical communication pixel to obtain an optical communication signal, high-precision light can be obtained using the image signal. Communication can be realized. In addition, since an optical communication signal digitally converted in consideration of the intensity of signal light obtained from the captured image can be obtained, high-accuracy optical communication can be realized using the captured image.

  Further, by estimating the movement destination of the region receiving the signal light based on the light receiving region extracted in the past, it is possible to easily extract the region receiving the signal light at the next timing. .

  In the above embodiment, the case where warning presentation or driving control according to collision risk is performed as driving assistance has been described as an example. However, the present invention is not limited to this, and is based on captured images and received data. Thus, lane maintenance assistance (lane keeping assistance) or pedestrian detection by white line detection may be performed.

  In the optical communication device, the case where the image sensor and the control device are separately provided has been described as an example. However, the control device may be provided in the image sensor.

  Moreover, although the case where the signal output from the photoelectric conversion element for optical communication pixels was digitally converted and output as an optical communication signal was described as an example, the signal output from the photoelectric conversion element for optical communication pixels was You may make it output outside as an optical communication signal with an analog signal.

  Next, a second embodiment will be described. In addition, about the part which becomes the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In the second embodiment, the disturbance light component is mainly removed by removing the low frequency component of the analog signal output from the photoelectric conversion element for the optical communication pixel. Is different.

  As illustrated in FIG. 14, the DC removal device 234 of the digital conversion unit of the image sensor according to the second embodiment includes a high-pass filter 240.

  The signal output from the photoelectric conversion element 28B for the optical communication pixel includes an intensity signal due to disturbance light such as sunlight other than the signal light, and the intensity signal due to disturbance light is compared to the optical communication signal. The frequency is low. Therefore, as shown in FIGS. 15A and 15B, the disturbance light component is removed by removing the low-frequency component from the signal output from the photoelectric conversion element 28B for the optical communication pixel by the high-pass filter 240. To do.

  The high-pass filter 240 passes only components having a frequency higher than the frequency represented by the input passband setting parameter in the input signal. For example, when it is known that the modulation frequency of incident signal light is 1 MHz, by setting only the frequency component of 500 kHz or higher by setting the passband setting parameter to 500 kHz, optical communication such as disturbance light components Remove unrelated components.

  In this way, the reference value level for digital conversion is set to 0 by removing low-frequency components such as disturbance light components from the signal output from the photoelectric conversion element for optical communication pixels using a high-pass filter. You can do it.

  In the above embodiment, the case where a high-pass filter is used as the direct current removing device has been described as an example, but a band-pass filter may be used. In this case, a pass band setting parameter representing a frequency band of an arbitrary width is input and set to the band pass filter, and only the component of the frequency band of an arbitrary width is passed by the band pass filter. For example, when the modulation frequency of the incident signal light is known as 1 MHz, only a component having a frequency in the range of 500 kHz to 1.5 MHz is set by inputting a passband setting parameter representing a frequency band of 500 kHz to 1.5 MHz. To remove signals that are not necessary for communication, such as disturbance light components.

  Next, a third embodiment will be described. In addition, about the part which becomes the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The third embodiment is mainly different from the first embodiment in that an analog signal output from a photoelectric conversion element for an optical communication pixel is digitally converted into a multi-bit digital signal.

  In the optical communication device 12 according to the third embodiment, the digital converter 38 is configured by an A / D converter. As shown in FIG. 16, in the case of transmitting an optical communication signal indicating information (for example, five levels of information from 0 to 4) according to the brightness at the time of on-off as well as on-off by the optical transmission device. Since the received signal is multi-valued, the analog signal is digitally converted into a multi-bit digital signal using an A / D converter.

  A plurality of types of reference value levels are calculated based on a captured image obtained based on a signal from a photoelectric conversion element for an image pixel or a signal from a photoelectric conversion element for an optical communication pixel. The reference value level is set in the A / D converter. The A / D converter digitally converts an analog signal into a multi-bit digital signal based on a plurality of types of set reference value levels.

  As a result, it is possible to receive a large amount of data corresponding to the number of steps of brightness compared to 1 bit.

  Next, a fourth embodiment will be described. In addition, since the structure of the driving assistance system which concerns on 4th Embodiment is the structure similar to 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

  The fourth embodiment is mainly different from the first embodiment in that pattern matching is performed on a captured image and a region receiving signal light is extracted.

  In the control device 26 of the optical communication device 12 according to the fourth embodiment, a circular pattern is formed in a captured image obtained by the light extraction unit 46 based on a signal from the photoelectric conversion element 28A for image pixels. Searching is performed, and a region representing the searched pattern is extracted as a region receiving signal light. Since the light emitting element 18 of the optical transmission device 16 is imaged as a circle having a certain brightness on the captured image, the region representing the circular pattern searched for in the captured image receives the signal light. It becomes an area.

  Thereby, even if the light emitting element 18 exists at a long distance, the region receiving the signal light can be extracted by extracting the region by pattern matching.

  In the above embodiment, the case where a region representing a circular pattern is extracted from a captured image has been described as an example. However, the present invention is not limited to this, and a vehicle or a traffic light is detected from the captured image and detected. An area representing a circular pattern may be extracted from the areas representing the vehicles and traffic lights. Accordingly, it is possible to prevent the extraction of all circular patterns having a certain level of brightness representing street lamps or the like from the captured image, and it is possible to accurately extract the region receiving the signal light.

  In the case where a plurality of light emitting elements on the transmission side are arranged and the arrangement is known on the optical communication apparatus side, the signal light from the light emitting elements is extracted from the captured image by circular pattern matching. Light receiving region candidates may be extracted, and pattern matching of the arrangement may be further performed from the extracted light receiving region candidates to extract regions that receive signal light. For example, as shown in FIG. 17, when a plurality of communication light emitting elements are arranged in a certain arrangement in the tail lamp of a vehicle, the optical communication apparatus performs pattern matching of this arrangement to perform imaging. A region receiving signal light is extracted from the image. As a result, the region receiving the signal light can be extracted with higher accuracy.

  Next, a fifth embodiment will be described. In addition, since the structure of the driving assistance system which concerns on 5th Embodiment is the structure similar to 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

  The fifth embodiment is different from the first embodiment mainly in that the photoelectric conversion elements for image pixels are arranged on the substrate so as to enclose the photoelectric conversion elements for optical communication pixels. Yes.

  As shown in FIG. 18A, in the pixel array 28, photoelectric conversion elements 28A for image pixels are arranged so as to enclose photoelectric conversion elements 28B for optical communication pixels, and a plurality of image pixel photoelectric conversion elements 28A are arranged. The photoelectric conversion element 28A and the photoelectric conversion elements 28B for a plurality of optical communication pixels are arranged on one substrate.

  Thereby, the deviation between the optical axis of the photoelectric conversion elements for the plurality of image pixels and the optical axis of the photoelectric conversion elements 28 for the plurality of optical communication pixels can be substantially matched.

  Although the case where the photoelectric conversion elements for image pixels are arranged on the substrate so as to enclose the photoelectric conversion elements for optical communication pixels has been described as an example, the photoelectric conversion elements for optical communication pixels are You may arrange | position on a board | substrate so that the photoelectric conversion element for pixels may be included.

  In the above embodiment, the case where the size of the photoelectric conversion element for the image pixel is the same as the size of the photoelectric conversion element for the optical communication pixel is described as an example. However, as shown in FIG. The size of the photoelectric conversion element for the pixel and the size of the photoelectric conversion element for the optical communication pixel may be different. For example, the size of the photoelectric conversion element for the optical communication pixel is the size of the photoelectric conversion element for the image pixel. It may be smaller.

  Next, a sixth embodiment will be described. In addition, about the part which becomes the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In the sixth embodiment, the intensity level of disturbance light is set based on the signal from the photoelectric conversion element for the optical communication pixel, and the signal from the photoelectric conversion element for the optical communication pixel. The difference from the first embodiment is that a reference value level for digital conversion is set.

  In the sixth embodiment, a signal output from the photoelectric conversion element 28B for the optical communication pixel of the pixel array 28 is input to the control device 626 shown in FIG.

  The control device 626 is on the substrate that receives the signal light from the optical transmission device 16 from the image signal input unit 44 and a captured image obtained based on the plurality of image signals input by the image signal input unit 44. A light extraction unit 646 that extracts a region, outputs coordinate values in the extracted region to the pixel array 28, and outputs coordinate values outside the extracted region to the pixel array 28, and photoelectric conversion for the optical communication pixel A disturbance level setting unit 648 for setting the intensity level of disturbance light based on the signal output from the element 28B, an image output unit 50, an optical communication signal input / output unit 52, and a photoelectric conversion element 28B for an optical communication pixel. And a reference value setting unit 654 for setting a reference value level used when digitally converting to an optical communication signal based on a signal output from.

  The disturbance level setting unit 648 uses the signal output from the photoelectric conversion element 28B for the optical communication pixel corresponding to the coordinate value outside the region extracted by the light extraction unit 646, and the photoelectric conversion element 28B for the optical communication pixel. The intensity level of the disturbance light component included in the signal output from is calculated, and the calculated intensity level of the disturbance light is set as an initial setting for the difference circuit 40.

  The reference value determination unit 654 calculates the intensity level of the signal light using the signal output from the photoelectric conversion element 28B for the optical communication pixel corresponding to the coordinate value in the region extracted by the light extraction unit 646, A reference value level for digital conversion is determined according to the intensity level of the signal light, and the determined reference value level is set as an initial setting for the digital converter 38.

  Next, the operation of the driving support system according to the sixth embodiment will be described. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted. First, in the optical communication device of the driving support system, an initial setting process routine shown in FIG. 20 is executed. In step 102, an image signal output from each of the photoelectric conversion elements 28A for the plurality of image pixels is acquired, and in step 108, optical transmission is performed from the captured image obtained based on the image signal acquired in step 102. A region on the substrate receiving the signal light from the device 16 is extracted, and in step 110, the coordinate value in the region on the substrate extracted in step 108 is output to the pixel array 28 for extraction. An instruction is given to output a signal from the photoelectric conversion element 28B for the optical communication pixel located at the position represented by the coordinate value in the area.

  In the next step 670, the signal output from the photoelectric conversion element 28B for the optical communication pixel located at the position represented by the coordinate value in the extracted region is acquired. In step 672, the signal acquired in the step 670 is acquired. The reference value level of the signal light is determined based on the above, and the reference value level is set for the digital converter 38.

  In step 674, the coordinate value outside the region on the substrate extracted in step 108 is output to the pixel array 28, and the photoelectric signal for the optical communication pixel located at the location represented by the coordinate value outside the extracted region is output. The conversion element 28B is instructed to output a signal. In the next step 676, the signal output from the photoelectric conversion element 28B for the optical communication pixel located at the location represented by the coordinate value outside the extracted area is acquired, and in step 678, the signal is acquired in step 676. Based on the received signal, the intensity level of disturbance light is calculated, the calculated intensity level of disturbance light is set in the difference circuit 40, and the initial setting processing routine is terminated.

  Then, the optical communication processing routine shown in FIG. 21 is executed in the optical communication apparatus. First, in step 100, 0 is set as an initial value for a variable t indicating elapsed time. In step 102, an image signal output from each of the plurality of image pixel photoelectric conversion elements 28A is acquired. In step 104, the captured image obtained based on the image signal acquired in step 102 is output to the driving support device 14.

  In step 108, a region on the substrate receiving the signal light is extracted from the captured image obtained based on the image signal acquired in step 102. In step 110, the region extracted in step 108 is extracted. The coordinate value in the area on the substrate is output to the pixel array 28. In the next step 112, an optical communication signal is acquired from the digital conversion unit 30, and in step 114, the acquired optical communication signal is output to the driving support device 14 as reception data.

  In step 116, the variable t representing the elapsed time is incremented. In step 118, it is determined whether the elapsed time t is less than the time F corresponding to a predetermined captured image output rate. However, when the time corresponding to the captured image output rate has not been reached, the process returns to step 112. On the other hand, when the elapsed time reaches the time corresponding to the captured image output rate, the process proceeds to step 120. To do.

  In step 120, based on the movement amount and movement direction of the region receiving the signal light extracted in the past, the movement destination of the region receiving the signal light is estimated, and the process returns to step 100, and the image is again displayed. Get the signal.

  As described above, according to the optical communication device of the driving support system according to the sixth embodiment, the disturbance light component set based on the signal from the photoelectric conversion element for the optical communication pixel is converted into the optical communication pixel. Since the optical communication signal can be obtained by removing the signal from the photoelectric conversion element for use, high-accuracy optical communication can be realized. In addition, a reference value level is set in consideration of the intensity of signal light obtained from a signal from a photoelectric conversion element for an optical communication pixel, and an optical communication signal that is digitally converted based on the set reference value level is obtained. Therefore, highly accurate optical communication can be realized.

  In the above embodiment, the case where the disturbance light component is set based on the signal from the photoelectric conversion element for the optical communication pixel is described as an example, but the signal from the photoelectric conversion element for the optical communication pixel is The disturbance light component may be set based on the captured image obtained based on the image signal. In this case, for example, the disturbance light component is calculated based on the signal from the photoelectric conversion element for the optical communication pixel, and the disturbance light component is calculated using the captured image in the same manner as in the first embodiment. The calculated disturbance light components may be averaged, and the average value of the disturbance light components may be set for the difference circuit.

  In addition, the case where the reference value level for digital conversion is set based on the signal from the photoelectric conversion element for the optical communication pixel has been described as an example, but the signal from the photoelectric conversion element for the optical communication pixel and the image signal are The reference value level may be set based on the captured image obtained based on the image. In this case, for example, the reference value level is determined based on the signal from the photoelectric conversion element for the optical communication pixel, and the reference value level is set using the captured image as in the first embodiment. It is also possible to determine the average value of both of the determined reference value levels and set the average value of the reference value levels in the digital converter.

  Further, the case where the optical communication device is mounted on the vehicle has been described as an example. However, the present invention is not limited to this. For example, the optical communication device may be mounted on a traffic light or the like.

  In addition, the case where a photoelectric conversion element with a short charge accumulation time in photoelectric conversion is used as the photoelectric conversion element for the optical communication pixel is described as an example, but the present invention is not limited to this. However, a photoelectric conversion element that is faster than the photoelectric conversion element for image pixels may be used. For example, a photoelectric conversion element that outputs a signal according to the amount of received light as it is without accumulating a signal according to the amount of received light (charge accumulation time is zero) may be used as a photoelectric conversion element for an optical communication pixel. Good.

It is the schematic which shows the structure of the driving assistance system which concerns on the 1st Embodiment of this invention, and an optical transmitter. It is a block diagram which shows the structure of the optical communication apparatus which concerns on the 1st Embodiment of this invention. It is an image figure which shows arrangement | positioning of the photoelectric conversion element of the pixel array of the optical communication apparatus which concerns on the 1st Embodiment of this invention. It is an image figure which shows the other example of arrangement | positioning of the photoelectric conversion element of a pixel array. It is a block diagram which shows the structure of the digital conversion part of the optical communication apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the direct current | flow removal apparatus of the optical communication apparatus which concerns on the 1st Embodiment of this invention. (A) The graph which shows the change of the input signal, (B) The graph which shows the change of the signal which removed the disturbance light component. (A) The figure which shows the comparator of a digital converter, (B) It is a graph which shows a mode that an analog signal is digitally converted. It is a block diagram which shows the structure of the control apparatus of the optical communication apparatus which concerns on the 1st Embodiment of this invention. It is an image figure which shows a mode that the movement destination of a light reception area | region is estimated based on the movement amount and movement direction of a light reception area | region. (A) A graph showing a state of digital conversion based on the reference value when the incident signal light is sufficiently strong, and (B) a state of digital conversion based on the reference value when the intensity of the signal light becomes weak. It is a graph which shows. It is a graph which shows a mode that the level of a reference value is changed dynamically according to the intensity | strength of the signal light to receive. It is a flowchart which shows the content of the optical communication processing routine in the control apparatus of the optical communication apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the direct current | flow removal apparatus of the optical communication apparatus which concerns on the 2nd Embodiment of this invention. (A) The graph which shows the change of the input signal, (B) The graph which shows the change of the signal which removed the disturbance light component. It is an image figure which shows a mode that it converts into a multibit optical communication signal digitally. It is an image figure which shows the tail lamp by which the several light emitting element for communication is arrange | positioned. (A) The image figure which shows the other arrangement pattern of the photoelectric conversion element of a pixel array, (B) The image figure which shows the other arrangement pattern of the photoelectric conversion element of a pixel array. It is a block diagram which shows the structure of the control apparatus of the optical communication apparatus which concerns on the 6th Embodiment of this invention. It is a flowchart which shows the content of the initialization process routine in the control apparatus of the optical communication apparatus which concerns on the 6th Embodiment of this invention. It is a flowchart which shows the content of the optical communication processing routine in the control apparatus of the optical communication apparatus which concerns on the 6th Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Driving support system 12 Optical communication apparatus 14 Driving support apparatus 16 Optical transmission apparatus 18 Light emitting element 24 Imaging element 26,626 Control apparatus 28 Pixel array 28A Photoelectric conversion element 28B for image pixels Photoelectric conversion element 30 for optical communication pixels Digital conversion Units 34, 234 DC component removal device 38 digital converter 40 difference circuit 46, 646 light extraction unit 48, 648 disturbance level calculation unit 50 image output unit 52 optical communication signal input / output units 54, 654 reference value determination unit 240 high pass filter

Claims (5)

A plurality of first photoelectric conversion elements that output a signal according to the amount of received light, and a plurality of first photoelectric conversion elements that output a signal according to the amount of received light and have a response speed with respect to a change in the amount of received light that is faster than that of the first photoelectric conversion element. and a second photoelectric conversion element, the first photoelectric conversion element and the second photoelectric conversion element and the plurality of first photoelectric conversion elements and the plurality of second photoelectric conversion element so that the pairs, one substrate An image sensor two-dimensionally arranged on top;
Extraction means for extracting a region on the substrate receiving signal light from a captured image obtained based on signals output from the plurality of first photoelectric conversion elements of the imaging element;
Reference value setting means for setting a reference value of the intensity of the signal light based on the signal output from the second photoelectric conversion element located in the region extracted by the extraction means;
Based on the reference value set by the reference value setting means, wherein a signal output from the second photoelectric conversion elements located on the extracted within the region by the extracting means to digital conversion, said digital conversion optical communication output means for outputting a signal as an optical communication signal represented by the signal light,
Image output means for outputting a captured image obtained based on signals output from the plurality of first photoelectric conversion elements of the image sensor;
An optical communication device including: A calculation means for calculating a disturbance light component based on a signal output from the first photoelectric conversion element of the imaging element;
The optical communication output means removes the disturbance light component calculated by the calculation means from the signal output from the second photoelectric conversion element located in the region extracted by the extraction means, and the signal The optical communication apparatus according to claim 1, wherein an optical communication signal represented by light is generated, and the generated optical communication signal is output. Further comprising setting means for setting a disturbance light component based on a signal output from at least one of the first photoelectric conversion element and the second photoelectric conversion element of the imaging element;
The optical communication output means removes the disturbance light component set by the setting means from the signal output from the second photoelectric conversion element located in the region extracted by the extraction means, and the signal The optical communication apparatus according to claim 1, wherein an optical communication signal represented by light is generated, and the generated optical communication signal is output.   The optical communication output means passes only a predetermined frequency band component from the signal output from the second photoelectric conversion element located in the region extracted by the extraction means, and removes the disturbance light component. The optical communication apparatus according to claim 1, further comprising: a filter, generating an optical communication signal represented by the signal light based on a signal output from the filter, and outputting the generated optical communication signal. The second photoelectric conversion element, the optical communication apparatus according to any one of the photoelectric claim charge accumulation time in the conversion shorter than the first photoelectric conversion element 1 to claim 4.

Images