JP2013081145A - Optical communication device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain highly accurate information indicating the relative relation between transmission means and imaging means by a device alone.SOLUTION: A reception control unit 14 obtains a prescribed number of frame images having imaged a region including a light-emitting part 52 of an optical signal transmission device 50 for transmitting communication information C indicating the height H of the light-emitting part 52 by an optical signal, detects a light-emitting part region from each frame image, observes change in a brightness value of the light-emitting part region between frames, and obtains the height H of the light-emitting part 52 indicated by the communication information C. An image information extraction unit 16 detects the light-emitting part region from a frame image I, extracts the number of longitudinal pixels n of the light-emitting part region as image information i. A relative information calculation unit 18 calculates a distance L between the optical signal transmission device 50 and an optical communication device 10 by use of the height H of the light-emitting part 52, the number of longitudinal pixels n of the light-emitting part region, a lens focal distance f which is a known value, and the size "a" of one pixel of an imaging element which is a known value.

Description

  The present invention relates to an optical communication device and a program, and more particularly to an optical communication device and a program that simultaneously acquire communication information indicated by an image and an optical signal.

  2. Description of the Related Art Conventionally, there has been proposed an optical communication apparatus capable of simultaneously acquiring communication information indicated by an image and an optical signal by imaging a transmission unit that transmits an optical signal with an imaging unit (see, for example, Patent Documents 1 and 2). ). In the optical communication device described in Patent Document 1, the moving speed or acceleration of the transmission unit is acquired as communication information.

  In addition, as a system for measuring the distance to the detected object from the image captured by the camera, based on the image captured by using two or more cameras, it is possible to reach the object by a technique such as triangulation. A system for measuring a distance has been proposed (for example, Patent Documents 3 and 4). In order to perform distance measurement with higher accuracy, an object detection apparatus that measures a distance to an object using a distance measuring device such as a laser radar or a millimeter wave radar in addition to a camera has been proposed (for example, Patent Documents 5 and 6).

JP 2009-27480 A International Publication No. 2010/032842 Pamphlet Japanese Patent No. 4341564 Japanese Patent No. 4586571 JP 2006-234513 A JP 2007-240208 A

  However, in the optical communication devices described in Patent Documents 1 and 2, the acquired image and the communication information are individually used without being related, and it is considered that the image and the communication information are integrated and used. It has not been.

  Further, in the techniques described in Patent Documents 3 to 6, in order to measure information indicating the relative relationship between the camera and the object, such as the distance to the object, two or more cameras or measurements other than the camera are measured. There is a problem that it is necessary to use a distance device, which leads to an increase in size and cost of the system.

  The present invention has been made to solve the above-described problems, and provides an optical communication device and a program that can acquire information indicating the relative relationship between a transmission unit and an imaging unit with high accuracy by a single unit. The purpose is to do.

  To achieve the above object, the optical communication device of the first invention controls the light emitting unit to transmit communication information indicating at least one of the position, size, and shape of the light emitting unit using an optical signal. And communication information indicated by the optical signal transmitted by the transmission unit based on a change in pixel values between frames of a plurality of frame images captured by the imaging unit. Image information for extracting image information indicating at least one of a position, a size, and a shape of a light emitting part region indicating the light emitting part from at least one frame image of the plurality of frame images, and extracting communication information Based on the extraction means, the communication information extracted by the communication information extraction means, and the image information extracted by the image information extraction means, the transmission means and the imaging means It is configured to include a calculating means for calculating the relative information that indicates the relationships between the.

  According to the optical communication apparatus of the first aspect of the invention, the imaging unit controls the light emitting unit, thereby transmitting communication information indicating at least one of the position, size, and shape of the light emitting unit using an optical signal. An area including the image is captured. Then, the communication information extraction unit extracts the communication information indicated by the optical signal transmitted by the transmission unit based on the change in the pixel value between the frames of the plurality of frame images captured by the imaging unit. Thereby, communication information and an image can be acquired simultaneously.

  The image information extraction means extracts image information indicating at least one of the position, size, and shape of the light emitting area indicating the light emitting area from at least one frame image of the plurality of frame images, and the calculating means Then, based on the communication information extracted by the communication information extraction unit and the image information extracted by the image information extraction unit, relative information indicating a relative relationship between the transmission unit and the imaging unit is calculated.

  As described above, based on communication information that is accurate information about the light emitting unit transmitted from the transmission unit, and image information extracted from the light emitting unit region on the image obtained by imaging the region including the light emitting unit. Since the relative information indicating the relative relationship between the transmission means and the imaging means is calculated, the relative information can be obtained with high accuracy by the apparatus alone.

  The optical communication device according to a second aspect of the invention outputs a plurality of first pixels that output a signal corresponding to the amount of received light, a signal corresponding to the amount of received light, and a response speed with respect to a change in the amount of received light. An image pickup means including an image pickup device in which a plurality of second pixels faster than one pixel are two-dimensionally arranged on one substrate, and the position, size, and shape of the light-emitting portion are controlled by controlling the light-emitting portion. An imaging unit that captures an area including a transmission unit that transmits at least one of communication information using an optical signal, and a light emission that indicates the light emitting unit on a frame image captured using the first pixel of the imaging unit Communication information extraction means for extracting communication information indicated by the optical signal transmitted by the transmission means based on a signal output from the second pixel at a position corresponding to a partial area; and the light emitting section from the frame image Light emitting area showing Image information extracting means for extracting image information indicating at least one of position, size, and shape, communication information extracted by the communication information extracting means, and image information extracted by the image information extracting means Based on this, it is configured to include calculation means for calculating relative information indicating the relative relationship between the transmission means and the imaging means.

  According to the optical communication device of the second invention, the imaging means outputs a plurality of first pixels that output a signal corresponding to the amount of received light, a signal corresponding to the amount of received light, and a response speed with respect to a change in the amount of received light Is configured to include an image sensor in which a plurality of second pixels faster than the first pixel are two-dimensionally arranged on one substrate. The imaging unit controls the light emitting unit, thereby imaging an area including a transmission unit that transmits communication information indicating at least one of the position, size, and shape of the light emitting unit by an optical signal. Then, the communication information extraction means is based on the signal output from the second pixel at the position corresponding to the light emitting area indicating the light emitting area on the frame image captured using the first pixel of the imaging means. The communication information indicated by the optical signal transmitted by is extracted. Thereby, the communication information obtained using the second pixel and the image captured using the first pixel can be acquired simultaneously.

  Further, the image information extracting means extracts image information indicating at least one of the position, size, and shape of the light emitting area indicating the light emitting section from the frame image, and the calculating means is extracted by the communication information extracting means. Based on the communication information obtained and the image information extracted by the image information extraction means, relative information indicating the relative relationship between the transmission means and the imaging means is calculated.

  As described above, based on communication information that is accurate information about the light emitting unit transmitted from the transmission unit, and image information extracted from the light emitting unit region on the image obtained by imaging the region including the light emitting unit. Since the relative information indicating the relative relationship between the transmission means and the imaging means is calculated, the relative information can be obtained with high accuracy by the apparatus alone. In addition, since multiple frames of images are not required to acquire communication information, high-speed processing is possible.

  In the second invention, the imaging element includes a first pixel having a first photoelectric conversion element and a first circuit for processing an electric charge generated by the first photoelectric conversion element as an image signal, and a second photoelectric conversion element. Or a second pixel having a second circuit that processes a charge generated by the second photoelectric conversion element as a communication signal, or a common photoelectric conversion element and the common photoelectric conversion element A first pixel having a third circuit for processing the generated charge as an image signal, and a fourth circuit for processing the common photoelectric conversion element and the charge generated by the common photoelectric conversion element as a communication signal. 1 pixel can be included.

  Further, the calculating means in the first and second inventions is based on the size of the light emitting part extracted by the communication information extracting means and the size of the light emitting part region extracted by the image information extracting means, or Calculating a distance between the transmission unit and the imaging unit based on the position of the light emitting unit extracted by the communication information extracting unit and the position of the light emitting unit region extracted by the image information extracting unit; or Based on the shape of the light emitting unit extracted by the communication information extracting unit and the shape of the light emitting unit region extracted by the image information extracting unit, the inclination of the transmitting unit with respect to the imaging unit can be calculated.

  In the first and second aspects of the invention, when the transmission unit includes a plurality of light emitting units, the communication information is transmitted as an interval between the plurality of light emitting units, and the image information extracting unit is configured to transmit the frame image. A plurality of light emitting portion areas can be detected from above, and intervals between the plurality of light emitting portion areas can be extracted. If a plurality of light emitting units are regarded as one light emitting unit, the interval between the light emitting units can be referred to as the size of the light emitting unit. If attention is paid to each light emitting unit, the interval between the light emitting units is relative to one light emitting unit. It can be said that it is the position of another light emission part.

  Further, the communication information in the first and second inventions further includes information indicating the moving speed of the transmitting means, and the moving speed of the imaging means detected by the detecting means for detecting the moving speed of the imaging means. Based on the moving speed of the transmitting means extracted by the communication information extracting means, a relative speed between the transmitting means and the imaging means is calculated, and based on the calculated relative speed and the relative information. In addition, a prediction unit that predicts a collision between the transmission unit and the imaging unit may be included.

  Further, the prediction means calculates a predicted collision time between the transmission means and the imaging means based on the relative speed and a distance between the transmission means and the imaging means calculated as the relative information. Can do.

  Further, the optical communication device according to the first and second aspects of the invention is a setting unit that sets a parameter for controlling at least one of imaging by the imaging unit and image processing on the captured image based on the relative information. Can be configured.

  The communication information in the first and second inventions further includes information indicating the size of the object on which the transmission unit is mounted, with the light emitting unit as a reference, and is extracted by the communication information extraction unit. The object on the frame image is indicated based on the information indicating the size of the object, the position of the light emitting area extracted by the image information extracting unit, and the relative information calculated by the calculating unit. A detection means for detecting the object region can be included. The size of the object relative to the light emitting unit can be, for example, the height from the light emitting unit to the upper side or the lower side of the object.

  The communication information in the first and second inventions includes information indicating the color of the object, information indicating the color of the object extracted by the communication information extracting means, and detection by the detecting means. An adjustment unit that adjusts at least one of the white balance and the exposure control value of the imaging unit based on the color information of the object area that has been performed can be configured.

  Further, the communication information in the first and second inventions further includes information indicating a width of the moving body on which the transmission unit is mounted, and information indicating the width of the moving body extracted by the communication information extracting unit. Based on the width of the moving object region indicating the moving object on the frame image and the relative information calculated by the calculating means, the moving object on which the optical communication device is mounted is mounted on the transmitting means. It can comprise including the determination means which determines the angle of the moving direction for moving avoiding a mobile body.

  Further, a control means for controlling the moving direction of the moving body on which the optical communication device is mounted so that the inclination of the transmitting means with respect to the imaging means calculated by the calculating means in the first and second inventions is eliminated. Can be configured.

  According to a third aspect of the present invention, there is provided an optical communication program for transmitting, by an optical signal, communication information indicating at least one of a position, a size, and a shape of a light emitting unit by controlling a light emitting unit with a computer. Communication information extraction means for extracting communication information indicated by an optical signal transmitted by the transmission means based on a change in pixel values between frames of a plurality of frame images captured by an imaging means for imaging an area including Image information extraction means for extracting image information indicating at least one of a position, a size, and a shape of a light emitting portion area indicating the light emitting portion from at least one frame image of a plurality of frame images, and the communication information extracting means Based on the communication information extracted by the image information and the image information extracted by the image information extraction means, the relative relationship between the transmission means and the imaging means is determined. Is a program for functioning as a calculating means for calculating to relative information.

  According to a fourth aspect of the invention, there is provided an optical communication program for causing a computer to output a plurality of first pixels that output a signal corresponding to the amount of received light, a signal corresponding to the amount of received light, and a response speed for a change in the amount of received light Includes an imaging device in which a plurality of second pixels faster than the first pixel are two-dimensionally arranged on a single substrate, and by controlling the light emitting unit, at least the position, size, and shape of the light emitting unit A position corresponding to the light emitting unit region indicating the light emitting unit on the frame image captured using the first pixel of the imaging unit that captures the region including the transmission unit that transmits the communication information indicating one by an optical signal. Communication information extracting means for extracting communication information indicated by the optical signal transmitted by the transmitting means based on a signal output from the second pixel; a position of a light emitting portion area indicating the light emitting portion from the frame image; big And image information extracting means for extracting image information indicating at least one of the shapes, communication information extracted by the communication information extracting means, and image information extracted by the image information extracting means, A program for functioning as a calculation unit that calculates relative information indicating a relative relationship between a transmission unit and the imaging unit.

  As described above, according to the optical communication device and the program of the present invention, the communication information that is accurate information about the light emitting unit transmitted from the transmission unit and the image obtained by imaging the region including the light emitting unit. Since the relative information indicating the relative relationship between the transmission unit and the imaging unit is calculated based on the image information extracted from the light emitting unit area, the relative information can be obtained with high accuracy by the apparatus alone. Is obtained.

It is the schematic which shows the structure of the optical communication system and optical communication apparatus which concern on 1st Embodiment. (A) Schematic which shows the structure of the image pick-up element of the optical communication apparatus which concerns on 1st Embodiment, (b) It is a figure for demonstrating the light emission part area | region on the imaged frame image. It is a figure for demonstrating the detection of the light emission part area | region from a frame image, and extraction of communication information. It is a flowchart which shows the content of the optical communication processing routine in the optical communication apparatus which concerns on 1st Embodiment. It is the schematic which shows the structure of the optical signal transmitter of the optical communication system which concerns on 2nd Embodiment. It is a figure for demonstrating extraction of the image information in 2nd Embodiment. It is the schematic which shows the structure of the optical communication system and optical communication apparatus which concern on 3rd Embodiment. It is a figure for demonstrating extraction of the image information in 3rd Embodiment. It is the schematic which shows the structure of the optical communication system and optical communication apparatus which concern on 4th Embodiment. It is a flowchart which shows the content of the optical communication processing routine in the optical communication apparatus which concerns on 4th Embodiment. It is the schematic which shows the structure of the optical communication system and optical communication apparatus which concern on 5th Embodiment. It is a flowchart which shows the content of the optical communication processing routine in the optical communication apparatus which concerns on 5th Embodiment. It is the schematic which shows the structure of the optical communication system and optical communication apparatus which concern on 6th Embodiment. It is a figure for demonstrating the detection of the vehicle area | region in 6th Embodiment. It is a flowchart which shows the content of the optical communication processing routine in the optical communication apparatus which concerns on 6th Embodiment. It is the schematic which shows the structure of the optical communication system and optical communication apparatus which concern on 7th Embodiment. It is a figure for demonstrating arrangement | positioning of the light emission part of the optical signal transmitter in 7th Embodiment. It is a figure for demonstrating calculation of the avoidance steering angle in 7th Embodiment. It is a flowchart which shows the content of the optical communication processing routine in the optical communication apparatus which concerns on 7th Embodiment. It is a figure for demonstrating the other example of calculation of the avoidance steering angle in 7th Embodiment. It is a flowchart which shows the content of the optical communication processing routine in the optical communication apparatus which concerns on 8th Embodiment. It is a figure for demonstrating the determination of the presence or absence of the inclination of the optical signal transmitter with respect to the optical communication apparatus in 8th Embodiment. It is the schematic which shows the structure of the optical communication system and optical communication apparatus which concern on 9th Embodiment. It is a flowchart which shows the content of the optical communication processing routine in the optical communication apparatus which concerns on 9th Embodiment. It is a figure for demonstrating the other example of the steering control in 9th Embodiment. It is the schematic which shows the structure of the image pick-up element of the optical communication apparatus comprised only with the pixel for an image. It is the schematic which shows the structure of the image pick-up element of the optical communication apparatus which concerns on 10th Embodiment. It is a flowchart which shows the content of the optical communication processing routine in the optical communication apparatus which concerns on 10th Embodiment. It is the schematic which shows the structure of the image pick-up element of the optical communication apparatus which concerns on 11th Embodiment. It is a flowchart which shows the content of the optical communication processing routine in the optical communication apparatus which concerns on 11th Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, the optical signal transmitted from the optical signal transmission device mounted on the preceding vehicle is imaged by the optical communication device mounted on the own vehicle, and the optical signal transmission device and the optical communication device An example in which the present invention is applied to an optical communication device used in an optical communication system that calculates relative information indicating a relative relationship will be described.

  As shown in FIG. 1, the optical communication system 1 according to the first embodiment includes an optical signal transmission device 50 and an optical communication device 10.

  The optical signal transmission device 50 includes, for example, a light emitting unit 52 composed of LEDs, and a transmission control unit 54 that turns the light emitting unit 52 on and off according to a data string indicating the communication information C. Note that the transmission control unit 54 may change the light intensity of the light emitting unit 52 according to the data string. The optical signal to be transmitted may be a digital waveform or an analog waveform as long as it is a modulation method capable of transmitting information.

  The communication information C transmitted as an optical signal by the optical signal transmission device 50 in the first embodiment can be information indicating the size of the light emitting unit 52. The information indicating the size of the light emitting unit 52 can be, for example, the height or width of the light emitting unit 52.

  The optical communication device 10 includes an image sensor 12, a CPU, a ROM that stores a program of an optical communication processing routine described later, a RAM that stores data, and a computer that includes a bus that connects these. When this computer is described by function blocks divided for each function realizing means determined based on hardware and software, the image sensor 12 is driven and a frame image is based on the pixel data received by the image sensor 12. A reception control unit 14 that performs processing for obtaining I and communication information C, an image information extraction unit 16 that extracts image information i of a light emitting unit region indicating the light emitting unit 52 on the frame image I, and image information i. Based on the communication information C, it can be expressed by a configuration including a relative information calculation unit 18 that calculates relative information indicating a relative relationship between the optical signal transmission device 50 and the optical communication device 10. The reception control unit 14 is an example of the communication information extraction unit of the present invention.

  As shown in FIG. 2A, the image sensor 12 has a plurality of pixels that output a signal corresponding to the amount of light received through a lens (not shown) arranged two-dimensionally on a substrate. It is configured.

  As shown in FIG. 3A, the reception control unit 14 acquires a predetermined number of frame images, and as shown in FIG. A light emitting area indicating the light emitting section 52 on the image is detected. For the detection of the high luminance area, for example, the brightest area in each frame image may be detected, an area having a luminance value equal to or higher than a predetermined value may be detected, and a change in luminance value between frames is large. An area may be detected. Then, the detected light emitting area is traced between frames, and a change in luminance value of the light emitting area between frames is observed and acquired as communication information C as shown in FIG.

  Further, the reception control unit 14 acquires at least one frame image of the predetermined number of frame images as the frame image I. The predetermined number of frame images acquired by the reception control unit 14 can be appropriately determined according to the length of the optical signal. In addition, as the frame image acquired as the frame image I, it is desirable to select a frame image captured last among a predetermined number of frame images.

  The image information extraction unit 16 detects a light emitting unit region from the frame image I acquired by the reception control unit 14 and extracts image information i indicating the size of the light emitting unit region. For example, as the size of the light emitting portion area, the vertical or horizontal size of the light emitting portion area can be extracted in units of pixels.

  The relative information calculation unit 18 compares the optical signal transmission device 50 with the optical communication device 10 based on the image information i extracted by the image information extraction unit 16 and the communication information C extracted by the reception control unit 14. Relative information indicating the relationship is calculated. The relative information in the first embodiment can be the distance between the optical signal transmission device 50 and the optical communication device 10.

  Next, an optical communication processing routine executed by the optical communication device 10 according to the first embodiment will be described with reference to FIG. Here, the height H of the light emitting unit 52 is transmitted as the communication information C, and the distance between the optical signal transmission device 50 and the optical communication device 10, that is, the distance between the preceding vehicle and the host vehicle is calculated as the relative information. Will be described.

  In step 100, a predetermined number of frame images are acquired, and then in step 102, a light emitting unit 52 on the frame image is detected by detecting a high luminance area from each frame image acquired in step 100. Detect subregions. Then, the detected light emitting area is traced between frames, the change in the luminance value of the light emitting area between frames is observed, and the height H of the light emitting section 52 is acquired as communication information C.

  Next, in step 104, the light emitting area is detected from the frame image I which is the final frame of the predetermined number of frame images obtained in step 100, and the number n of vertical pixels in the light emitting area (FIG. 2) is obtained as image information i. (B)) is extracted.

  Next, in step 106, the height H of the light emitting unit 52 acquired in step 102, the number of vertical pixels n of the light emitting unit region on the frame image I extracted in step 104, and the focal length of the lens that is a known value. The distance L between the optical signal transmission device 50 and the optical communication device 10 is expressed by the following equation (1) using f and a size a (FIG. 2A) of one pixel of the imaging element that is a known value. Calculate and output, and end the process.

  As described above, according to the optical communication device of the first embodiment, communication information that is accurate information indicating the size of the light emitting unit transmitted from the optical communication device and an area including the light emitting unit are imaged. In order to calculate the distance between the optical signal transmission device and the optical communication device based on the image information extracted from the image obtained in this way, the distance between the optical signal transmission device and the optical communication device with high accuracy by a single device Can be obtained.

  Next, a second embodiment will be described. The optical communication system according to the second embodiment includes an optical signal transmission device and an optical communication device as in the optical communication system 1 according to the first embodiment. Since it is the same as the optical communication apparatus 10 of the first embodiment, an optical signal transmission apparatus different from the first embodiment will be described. In addition, about the structure similar to the optical signal transmission apparatus 50 of 2nd Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  As illustrated in FIG. 5, the optical signal transmission device 250 according to the second embodiment includes two light emitting units 252 a and 252 b and a transmission control unit 54.

  The communication information C transmitted as an optical signal by the optical signal transmission device 250 in the second embodiment can be the interval H between the light emitting unit 252a and the light emitting unit 252b. In this case, together with the communication information C, the light emitting units 252a and 252b may be identified, and identification information indicating the correspondence between the light emitting units 252a and 252b may be transmitted. The communication information C may be transmitted from only one of the light emitting unit 252a and the light emitting unit 252b, or may be transmitted from both.

  Next, an optical communication processing routine in the second embodiment will be described. The optical communication processing routine of the second embodiment is different from the optical communication processing routine (FIG. 4) of the first embodiment only in the processing of step 104, and this point will be described.

  In step 104, a plurality of light emitting portion areas are detected from the frame image I, and based on the identification information acquired from each light emitting portion area, the light emitting portion area indicating the light emitting portion 252a and the light emitting portion area indicating the light emitting portion 252b. Take correspondence with. Then, as shown in FIG. 6, the number of pixels n between the light emitting region is extracted as image information i.

  As described above, according to the optical communication device of the second embodiment, communication information that is accurate information indicating the interval between two light emitting units transmitted from the optical communication device, and a region including the light emitting unit In order to calculate the distance between the optical signal transmission device and the optical communication device based on the image information extracted from the image obtained by imaging the optical signal transmission device and the optical communication device Can get the distance.

  In the first embodiment, since the size of one light emitting unit is used as communication information, the light emitting unit itself needs to have a certain size, for example, one light emitting unit is constituted by a plurality of LEDs. On the other hand, in 2nd Embodiment, since the space | interval between light emission parts is used as communication information, it can also be set as the point light source which comprised each of several light emission parts by one LED.

  Next, a third embodiment will be described. As shown in FIG. 7, the optical communication system 3 according to the third embodiment includes an optical signal transmission device 350 and an optical communication device 10 as in the optical communication system 1 according to the first embodiment. However, since the optical communication device is the same as that of the optical communication device 10 of the first embodiment, the optical signal transmission device 350 different from the first embodiment will be described. In addition, about the structure similar to the optical signal transmission apparatus 50 of 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  As shown in FIG. 7, the optical signal transmission device 350 according to the third embodiment includes a light emitting unit 352 and a transmission control unit 54.

  The communication information C transmitted as an optical signal by the optical signal transmission device 350 in the third embodiment can be set as an interval H between the light emitting unit 352 and a predetermined reference position. The predetermined reference position can be, for example, the lens center (optical axis) of the optical communication device 10. Alternatively, the upper side or the lower side of the vehicle on which the optical signal transmission device 350 is mounted may be used as the reference position.

  Next, an optical communication processing routine in the third embodiment will be described. The optical communication processing routine of the third embodiment is different from the optical communication processing routine (FIG. 4) of the first embodiment only in the processing of step 104, and this point will be described.

  In step 104, the light emitting area is detected from the frame image I, and as shown in FIG. 8, the number n of pixels between the light emitting area and the reference position (image center) is extracted as image information i. In this case, it is preferable to correct the reference position on the frame image according to the attitude angle of the camera. When the upper side or the lower side of the vehicle is used as the reference position, the frame image I is subjected to image processing to detect the reference position, and then the number of pixels between the light emitting area and the reference position is extracted. It is good to.

  As described above, according to the optical communication device of the third embodiment, communication information that is accurate information indicating the interval between the light emitting unit and the reference position transmitted from the optical communication device, and the light emitting unit are included. Since the distance between the optical signal transmission device and the optical communication device is calculated based on the image information extracted from the image obtained by imaging the region, the optical signal transmission device and the optical communication device can be accurately used as a single device. And get the distance. Further, similarly to the second embodiment, a point light source in which the light emitting unit is configured by one LED can be used.

  Next, a fourth embodiment will be described. In addition, about the structure same as the optical communication system 1 of 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  As shown in FIG. 9, the optical communication system 4 according to the fourth embodiment includes an optical signal transmission device 450 and an optical communication device 410.

  The optical signal transmission device 450 transmits, as the communication signal C, the speed Va of the vehicle (front vehicle) on which the optical signal transmission device 450 is mounted in addition to information indicating the size of the light emitting unit 52.

  The optical communication device 410 includes an image sensor 12 and a computer. The computer includes a reception control unit 14, an image information extraction unit 16, a relative information calculation unit 18, and a speed included in the communication information C. Based on Va and the speed Vb of the vehicle (own vehicle) on which the optical communication device 410 is mounted, the relative speed calculation unit 20 that calculates the relative speed ΔV between the preceding vehicle and the own vehicle, and the relative information and the relative speed ΔV Based on this, it can be expressed by a configuration including a collision prediction calculation unit 22 that calculates collision prediction information related to the collision prediction between the host vehicle and the preceding vehicle. The relative speed calculation unit 20 and the collision prediction calculation unit 22 are examples of the prediction unit of the present invention.

  The relative speed calculation unit 20 includes the speed Va of the preceding vehicle included in the communication information C extracted by the reception control unit 14 and the speed Vb of the host vehicle detected by a speed sensor (not shown) that detects the speed of the host vehicle. (Vb−Va) is calculated as a relative speed ΔV between the host vehicle and the preceding vehicle. When the relative speed ΔV is a negative value, it indicates that the vehicle ahead is moving away from the host vehicle.

  The collision prediction calculation unit 22 is based on the distance L between the preceding vehicle and the vehicle amount, which is the relative information calculated by the relative information calculation unit 18, and the relative speed ΔV calculated by the relative speed calculation unit 20. Collision prediction information relating to collision prediction between the vehicle and the preceding vehicle is calculated. The collision prediction information can be, for example, information indicating the possibility of collision between the host vehicle and the preceding vehicle, or a collision time.

  Next, an optical communication processing routine executed by the optical communication device 410 according to the fourth embodiment will be described with reference to FIG. Here, the case where the height H of the light emitting unit 52 and the speed Va of the preceding vehicle are transmitted as the communication information C, and the collision time between the preceding vehicle and the host vehicle is calculated as the collision prediction information will be described. In addition, about the process same as the optical communication processing routine of 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  In step 100, a predetermined number of frame images are acquired. Next, in step 400, a high-intensity region is detected from each frame image acquired in step 100, thereby indicating the light emission unit 52 on the frame image. Detect subregions. Then, the detected light emitting area is tracked between frames, the change in the luminance value of the light emitting area between frames is observed, and the height H of the light emitting section 52 and the speed Va of the vehicle ahead are acquired as communication information C. To do.

  Next, in step 104, the number n of vertical pixels in the light emitting area is extracted as the image information i. Next, in step 106, the distance L between the optical signal transmission device 50 and the optical communication device 10 is calculated.

  Next, in step 402, the speed Vb of the host vehicle is taken from the speed sensor, and the difference (Vb−Va) between the speed Va of the front vehicle acquired in step 400 and the speed Vb of the host vehicle is It is calculated as a relative speed ΔV with the vehicle.

  Next, in step 404, based on the distance L calculated in step 106 and the relative speed ΔV calculated in step 402, the collision time t (L / ΔV) is calculated and output, and the process ends. .

  In the optical communication processing routine described above, the case where the collision time is calculated as the collision prediction information has been described. However, the present invention is not limited to this. For example, when the inclination of the optical signal transmission device 50 with respect to the optical communication device 10 is calculated as relative information, whether or not there is a possibility that the host vehicle and the preceding vehicle collide with the calculated relative speed information. It is also possible to calculate information indicating.

  As described above, according to the optical communication device of the fourth embodiment, as in the first embodiment, the own vehicle and the preceding vehicle are used using the relative information acquired with high accuracy by the device alone. Therefore, the collision prediction information can also be calculated with high accuracy by the apparatus alone.

  In the fourth embodiment, the case where the collision prediction information is calculated using the relative information calculated in the optical communication system of the first embodiment has been described. However, the second or third embodiment is described. You may apply to the optical communication system of this.

  Next, a fifth embodiment will be described. In addition, about the structure same as the optical communication system 1 of 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  As illustrated in FIG. 11, the optical communication system 5 according to the fifth embodiment includes an optical signal transmission device 50 and an optical communication device 510.

  The optical communication device 510 includes an image sensor 12 and a computer. The computer receives the reception control unit 14, the image information extraction unit 16, the relative information calculation unit 18, and the reception based on the relative information. It can be expressed by a configuration including a parameter setting unit 24 for setting parameters of the control unit 14.

  The parameter setting unit 24 sets parameters necessary for image sensor drive control and image processing by the reception control unit 14 based on the relative information calculated by the relative information calculation unit 18. The parameter may be, for example, an amplification degree with respect to a signal received by the image sensor 12, an imaging cycle, a field angle of the lens, and the like.

  Next, an optical communication processing routine executed by the optical communication apparatus 510 according to the fifth embodiment will be described with reference to FIG. Here, a case will be described in which the amplification degree of the received signal is determined as a parameter based on the distance L between the optical signal transmission device 50 and the optical communication device 510 calculated as relative information. In addition, about the process same as the optical communication processing routine of 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  Steps 100 to 106 are executed to calculate the distance L between the optical signal transmission device 50 and the optical communication device 510.

  Next, in step 500, the amplification level of the received signal is set according to the distance L calculated in step 106. The range of the signal amplitude that can be input to the reception control unit 14 is finite, but the amplitude of the reception signal varies depending on the distance L, and thus may deviate from the range. Therefore, by determining the amplification degree of the reception signal according to the distance L, the reception control unit 14 is adjusted so that the reception signal in the optimum range that can exhibit the best performance is input. Specifically, the greater the distance L, the larger the amplification degree, and the smaller the distance L, the smaller the amplification degree.

  In the optical communication processing routine described above, the case where the amplification degree of the reception signal is set as a parameter has been described. However, the present invention is not limited to this. For example, when the distance L is small, it can be said that the preceding vehicle and the host vehicle are close to each other and the degree of danger is high. In such a situation, more information is often needed so that it can respond quickly to changes in the situation. Therefore, when the distance L is small, the parameter can be set to increase the imaging cycle. On the other hand, when the distance L is large, since the responsiveness to the change in the situation is low, the parameter can be set so as to reduce the imaging cycle. The imaging cycle in the case of lowering can be set to a minimum cycle necessary for recognition of an optical signal, for example, thereby reducing the calculation load.

  In addition, when the distance L is small and the angle of view of the lens is telephoto, there is a possibility that the light emitting part region occupies almost the frame image I. On the other hand, when the distance L is large and the angle of view is wide, the light emitting area on the frame image I becomes too small, and the communication information C and image information i may not be extracted properly. . Therefore, when the distance L is small, the angle of view is adjusted to the wide angle side, and when the distance L is large, the parameter is set so that the angle of view is adjusted to the telephoto side. It can adjust so that a light emission part area | region may become an appropriate magnitude | size.

  As described above, according to the optical communication apparatus of the fifth embodiment, as in the first embodiment, it is possible to acquire relative information with high accuracy by the apparatus alone, and furthermore, this relative information. Is used to set parameters necessary for image sensor drive control and image processing, so that the accuracy of an image for calculating relative information is improved, and relative information can be calculated with higher accuracy.

  In the fifth embodiment, the case where the parameter is determined using the relative information calculated in the optical communication system of the first embodiment has been described. However, the collision prediction calculated in the fourth embodiment is described. You may make it determine a parameter using time.

  Next, a sixth embodiment will be described. In addition, about the structure same as the optical communication system 1 of 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  As illustrated in FIG. 13, the optical communication system 6 according to the sixth embodiment includes an optical signal transmission device 650 and an optical communication device 610.

The optical signal transmission device 650 includes two light emitting units 652a and 652b. As communication information C, as shown in FIG. 14A, an interval H between the light emitting unit 652a and the light emitting unit 652b, a light emitting unit 652a, A height D ₁ from 652b to the upper side of the vehicle and a height D ₂ from the light emitting units 652a and 652b to the lower side of the vehicle are transmitted.

  The optical communication device 610 includes an image sensor 12 and a computer. The computer includes a reception control unit 14, an image information extraction unit 616, a relative information calculation unit 618, and a region detection unit 26. Can be represented by a configuration.

As illustrated in FIG. 14B, the image information extraction unit 616 detects two light emitting unit regions from the frame image I acquired by the reception control unit 14, the number of pixels n between the light emitting unit regions, and each light emission Pixel coordinates L (x _L , y _L ) and R (x _R , y _R ) of the center pixel of the partial area are extracted as image information i.

  The relative information calculation unit 618 can be represented by a distance calculation unit 618a and a height calculation unit 618b. The distance calculation unit 618a calculates the distance L between the optical signal transmission device 650 and the optical communication device 610 by the same processing as the relative information calculation unit 18 of the second embodiment.

In addition, the height calculation unit 618b uses D ₁ and D ₂ acquired by the reception control unit 14, the distance L calculated by the distance calculation unit 618a, the focal length f of the lens that is a known value, and the known value. Using the size a for one pixel of an image sensor, the height d ₁ from the light emitting area on the frame image I to the upper side of the vehicle, and the light emitting area from the light emitting area on the frame image I by the following equations (2) and (3) calculating the height d ₂ until the vehicle lower side.

The area detection unit 26 includes pixel coordinates L (x _L , y _L ) and R (x _R , y _R ) of the light emitting unit area extracted by the image information extraction unit 616, and the height calculated by the height calculation unit 618 b. using the _d 1, _{d 2,} as shown in FIG. 14 (c), pixel coordinates _{a (x L, y L -d} 1), B (x R, y R -d 1), C (x L , Y _L + d ₂ ) and D (x _R , y _R + d ₂ ). Here, the origin of coordinates (0, 0) is set at the upper left of the frame image I. As shown in FIG. 4D, the area surrounded by the calculated pixel coordinates A, B, C, D is output as the final vehicle area. The final output may be each coordinate value of ABCD, or may be an image in which a rectangular frame indicated by ABCD is drawn on the frame image I.

  Next, an optical communication processing routine executed by the optical communication device 610 according to the sixth embodiment will be described with reference to FIG. In addition, about the process same as the optical communication processing routine of 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

In step 100, a predetermined number of frame images are acquired. Next, in step 600, a light emitting unit area is detected from each frame image acquired in step 100, and the light emitting unit 652a and the light emitting unit are used as communication information C. the distance H between 652b, the light emitting portion 652a, the height _{D 1} of the from 652b to the vehicle upper side, and the light emitting unit 652a, obtains the height _{D 2} to the vehicle lower side from 652b.

Next, in step 602, the number n of pixels between the light emitting area and the pixel coordinates L (x _L , y _L ) and R (x _R , y _R ) of the center pixel of each light emitting area are used as image information i. Next, in step 106, the distance L between the optical signal transmission device 650 and the optical communication device 610 is calculated.

Next, in step 604, D ₁ and D ₂ acquired in step 600, the distance L calculated in step 106, the focal length f of the lens that is a known value, and the imaging element that is a known value. Using the size a for one pixel, the height d ₁ from the light emitting area to the upper side of the vehicle on the frame image I and the height from the light emitting area to the lower side of the vehicle according to the expressions (2) and (3) It is to calculate the _{d 2.}

Next, in step 606, the pixel coordinates L (x _L , y _L ) and R (x _R , y _R ) of the light emitting area extracted in step 602 and the height d ₁ calculated in step 604 are displayed. , D ₂ , pixel coordinates A (x _L , y _L −d ₁ ), B (x _R , y _R −d ₁ ), C (x _L , y _L + d ₂ ), D (x _R , y _R + d ₂ ) is calculated, the area surrounded by the calculated pixel coordinates A, B, C, and D is output as a vehicle area, and the process ends.

  As described above, according to the optical communication device of the sixth embodiment, as in the first embodiment, it is possible to acquire relative information with high accuracy by a single device, and furthermore, this relative information. The object area on the image can be accurately detected using the information on the light emitting part area on the frame image.

  In the sixth embodiment, the case where an optical signal transmission device including two light emitting unit regions is used has been described. However, the present invention can be applied even when there is one light emitting unit.

  In the sixth embodiment, the case where the object region detected from the image is output based on the calculated relative information has been described. However, instead of outputting the detection result, or outputting the detection result. At the same time, the white balance of the image captured by the optical communication device may be adjusted using the color information of the detected object region. For example, when a sunset hits a vehicle in front of a white vehicle body color, the vehicle body of the vehicle in front is imaged in red when captured by a camera. That is, the color of the original object cannot be properly captured by the camera due to the hue of the illumination (here, the sun). In order to eliminate such a phenomenon, white balance is adjusted.

  In this case, vehicle color information (white, red, blue, etc.) is added as communication information C transmitted from the optical signal transmitter. Then, the color information of the object area detected from the image in the processing in the above embodiment is compared with the vehicle body color included in the communication information C. If the colors are different as a result of the comparison, the white balance is adjusted. For example, when the vehicle body color included in the communication information C is “white”, but the color of the object area on the image is “red”, the color information in the object area is “white”. Adjust the white balance so that “

  In addition to adjusting the white balance, exposure control or the like may be performed by comparing the vehicle body color transmitted as the communication information C with the color of the object area.

  Next, a seventh embodiment will be described. In addition, about the structure same as the optical communication system 1 of 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  As illustrated in FIG. 16, the optical communication system 7 according to the seventh embodiment includes an optical signal transmission device 750 and an optical communication device 710.

  The optical signal transmission device 750 includes two light emitting units 752a and 752b, and transmits the interval H between the light emitting unit 752a and the light emitting unit 752b as communication information C. In addition, as shown in FIG. 17, the light emitting units 752a and 752b are disposed at both end positions in the vehicle width direction of the preceding vehicle as viewed from the optical communication device 710 side. That is, the light emitting units 752a and 752b are arranged so that the interval H transmitted as the communication information C becomes information indicating the vehicle width of the preceding vehicle.

  The optical communication device 710 includes an image sensor 12 and a computer. The computer includes a reception control unit 14, an image information extraction unit 16, a relative information calculation unit 18, and an avoidance steering angle determination unit 28. Can be expressed in a configuration including

  The avoidance steering angle determination unit 28 uses the distance L between the optical signal transmission device 750 and the optical communication device 710 calculated by the relative information calculation unit 18 and the vehicle width H of the preceding vehicle acquired by the reception control unit 14. An avoidance steering angle θ that is a reference for preventing collision with the preceding vehicle in an emergency is calculated. For example, when the central axes of the optical communication device 710 and the preceding vehicle are aligned, the relationship among the distance L, the vehicle width H, and the avoidance steering angle θ is as shown in FIG. The avoidance steering angle θ is calculated using the distance L and the vehicle width H according to the following equation (4). The calculated avoidance steering angle θ is output to a steering control device that controls the steering angle of the host vehicle.

  Next, an optical communication processing routine executed by the optical communication device 710 according to the seventh embodiment will be described with reference to FIG. In addition, about the process same as the optical communication processing routine of 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  In step 100, a predetermined number of frame images are acquired. Next, in step 700, the light emitting area is detected from each frame image acquired in step 100, and the vehicle width ( The distance H between the light emitting unit 652a and the light emitting unit 652b) is acquired.

  Next, in step 702, the number n of pixels between the light emitting region is extracted as the image information i, and then in step 106, the distance L between the optical signal transmission device 750 and the optical communication device 710 is calculated.

  Next, in step 704, using the vehicle width H acquired in step 700 and the distance L calculated in step 106, the avoidance steering angle θ is calculated and output according to the equation (4). The process ends.

  Note that, as shown in FIG. 20A, when the center axis between the optical communication device 710 and the preceding vehicle is displaced, as shown in FIG. 20B, from the image center of the frame image I. The avoidance steering angle θ can be calculated from the following equation (5) by extracting the number of pixels m up to the light emitting area.

Further, ± α may be adjusted to the calculated avoidance steering angle θ. It is also possible to calculate both the avoidance steering angle θ _L when avoiding from the left side of the preceding vehicle and the avoidance steering angle θ _R when avoiding from the right side of the preceding vehicle. In this case, when the avoidance steering angle becomes large, the steering wheel becomes a sharp handle. Therefore, it is preferable to select and output the smaller avoidance steering angle.

  As described above, according to the optical communication apparatus of the seventh embodiment, as in the first embodiment, it is possible to acquire relative information with high accuracy by the apparatus alone, and furthermore, this relative information. Can be used to accurately calculate the avoidance steering angle.

  In the seventh embodiment, two light emitting units are arranged at both end positions in the vehicle width direction of the preceding vehicle, and the interval between the light emitting unit regions on the frame image is extracted as an interval corresponding to the vehicle width. Although the case has been described, the vehicle width may be extracted by detecting the vehicle ahead by image processing from the frame image. In this case, it is not necessary to consider the arrangement of the light emitting units.

  Next, an eighth embodiment will be described. Note that the optical communication system according to the eighth embodiment includes an optical signal transmission device and an optical communication device, similarly to the optical communication system 1 according to the first embodiment. Since the configuration of each device is the same as that of the first embodiment, description thereof is omitted. In the eighth embodiment, the communication information C transmitted from the optical signal transmission device 50 is information indicating the shape of the light emitting unit 52 (square, rectangle, circle, ellipse, equilateral triangle, right triangle, etc.). Send. In addition, the relative information calculation unit 18 of the optical communication device 10 calculates the presence / absence of the inclination of the light emitting unit 52 with respect to the optical communication device 10, that is, the presence / absence of the inclination of the preceding vehicle with respect to the host vehicle, as relative information.

  Next, an optical communication processing routine executed by the optical communication device 10 according to the eighth embodiment will be described with reference to FIG. Here, a case where the shape of the light emitting unit 52 transmitted by the communication information C is “square” will be described.

  In step 100, a predetermined number of frame images are acquired. Next, in step 800, a light emitting unit area is detected from each frame image acquired in step 100, and the shape of the light emitting unit 52 is acquired as communication information C. .

  Next, in step 802, the light emitting area is detected from the frame image I which is the final frame of the predetermined number of frame images acquired in step 100, and the image information i is, for example, shown in FIGS. As shown, the shape of the light emitting part region is extracted. More specifically, the length (number of pixels) of each side of the light emitting area is extracted as the shape of the light emitting area.

  Next, in step 804, based on the shape of the light emitting unit region extracted in step 802, the presence / absence of the inclination of the light emitting unit 52 with respect to the optical communication device 10 is calculated and output. For example, as shown in FIG. 22D, the number of pixels on the A side (left side) and the number of pixels on the B side (right side) of the light emitting section region are compared. When the number of pixels on the A side = the number of pixels on the B side, or when the difference between the number of pixels on the A side and the number of pixels on the B side is within a predetermined range that can be made almost zero, as shown in FIG. In addition, the light emitting area on the frame image looks square, and it is determined that there is no inclination. When the difference between the number of pixels on the A side and the number of pixels on the B side is equal to or greater than a predetermined value, the light emitting area on the frame image looks trapezoidal as shown in FIG. It is determined that there is an inclination.

  As described above, according to the optical communication device of the eighth embodiment, the communication information that is accurate information indicating the shape of the light emitting unit transmitted from the optical communication device and the region including the light emitting unit are imaged. In order to calculate the presence or absence of the tilt of the optical signal transmission device with respect to the optical communication device based on the image information extracted from the image obtained in this way, the tilt of the optical signal transmission device with respect to the optical communication device can be calculated with high accuracy by a single device. Can be acquired.

  In the eighth embodiment, the case where the presence / absence of the right / left inclination of the light emitting portion region is determined has been described, but the presence / absence of the vertical inclination can also be determined by a similar method.

  Next, a ninth embodiment will be described. In addition, about the structure same as the optical communication system 1 of 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  As shown in FIG. 23, the optical communication system 9 according to the ninth embodiment includes an optical signal transmission device 950 and an optical communication device 910.

  The optical signal transmission device 950 transmits the shape of the light emitting unit 52 as the communication information C.

  The optical communication device 910 includes an image pickup device 12 and a computer. The computer includes a reception control unit 14, an image information extraction unit 16, a relative information calculation unit 18, and a follow-up control unit 30. Can be represented by a configuration.

  The relative information calculation unit 18 calculates the presence or absence of the inclination of the optical signal transmission device 950 with respect to the optical communication device 910, as in the eighth embodiment.

  The follow-up control unit 30 generates a control signal for following the vehicle ahead based on the presence or absence of the inclination calculated by the relative information calculation unit 18, and outputs the control signal to the follow-up control device that drives the steering device or the like.

  Next, an optical communication processing routine executed by the optical communication apparatus 910 according to the ninth embodiment will be described with reference to FIG. Note that the same processes as those in the optical communication process routines of the first and eighth embodiments are denoted by the same reference numerals, and detailed description thereof is omitted.

  Through steps 100 and 800 to 804, the presence / absence of the inclination of the optical signal transmission device 950 relative to the optical communication device 910 is calculated.

  Next, in step 900, it is determined whether or not the calculation result in step 804 has an inclination. If there is an inclination, the process proceeds to step 902 where a control signal for steering leftward or rightward is generated according to whether the inclination is leftward or rightward, and is output to the tracking control device. On the other hand, if there is no inclination, the process proceeds to step 904 to stop generating a control signal for controlling the steering.

  Next, in step 906, it is determined whether or not the next frame image has been acquired by the optical communication device 910. If acquired, the process returns to step 802, the shape of the light emitting area is extracted from the newly acquired frame image, and the subsequent processing is repeated. If the next frame image is not acquired, the process is terminated.

  As described above, according to the optical communication apparatus of the ninth embodiment, as in the first embodiment, it is possible to acquire relative information with high accuracy by the apparatus alone, and furthermore, this relative information. Can be used to perform follow-up control to follow the vehicle ahead.

  In the ninth embodiment, the case where there is one light emitting unit of the optical signal transmission device has been described. However, a plurality of light emitting units may be provided. In this case, as shown in FIG. 25, it is possible to calculate the presence / absence of an inclination by comparing the sizes (total number of pixels) of the light emitting area extracted from the frame image. For example, as shown in FIG. 5A, if the left light emitting area is larger than the right light emitting area, a control signal for steering leftward is generated and output to the tracking control device. To do. In the next frame image, as shown in FIG. 4B, if the left light emitting area is still larger than the right light emitting area, a control signal for continuing steering to the left is generated. Output to the tracking control device. In the next frame image, as shown in FIG. 5C, when the left and right light emitting unit areas have the same size, generation of a control signal for controlling steering is stopped.

  Next, a tenth embodiment will be described. Note that the optical communication system according to the tenth embodiment includes an optical signal transmission device and an optical communication device, similarly to the optical communication system 1 according to the first embodiment. Since the configuration of each device is different only in the configuration of the image sensor of the optical communication device, this point will be described.

  As shown in FIG. 26A, in a normal imaging device, a plurality of pixels that output signals according to the amount of light received through a lens (not shown) are two-dimensionally formed on one substrate. It is arranged and arranged. In this pixel, as shown in FIG. 4B, the photoelectric conversion unit that converts received light into electric charges, and the electric charges generated by the photoelectric conversion units are converted into voltage or current. And an in-pixel circuit for performing various processing such as amplification and sending to a readout circuit or the like.

  As shown in FIG. 27A, the imaging device of the optical communication apparatus according to the tenth embodiment includes a plurality of first pixels that output a signal corresponding to the amount of light received through the lens, A plurality of second pixels that output a signal according to the amount and have a response speed with respect to a change in the amount of received light that is faster than the first pixel are alternately arranged two-dimensionally on a single substrate. The first pixel is a normal image pixel that captures a change in light in a frame, and the second pixel is a communication pixel that captures a change in light continuously.

  As shown in FIG. 2B, the first pixel has an image photoelectric conversion element and an image pixel internal circuit that processes charges generated by the image photoelectric conversion element. The charge generated by the image photoelectric conversion element is processed by the image pixel circuit, and is sent to the readout line as an image signal. The second pixel has a communication photoelectric conversion element and a communication pixel circuit that processes charges generated by the communication photoelectric conversion element. The charge generated by the communication photoelectric conversion element is processed by the communication pixel circuit and sent to the readout line as a communication signal.

  In the first to ninth embodiments, the communication information C is acquired by detecting a change in luminance value from a plurality of frame images. On the other hand, in the optical communication device 10 according to the tenth embodiment, the light emitting area is extracted from one frame image captured using the image pixels, and the communication pixels corresponding to the coordinates of the light emitting area are used. Communication information C is acquired based on the acquired optical signal.

  Next, an optical communication processing routine executed by the optical communication apparatus 10 according to the tenth embodiment will be described with reference to FIG.

  In step 1000, one frame image I is acquired. Next, in step 1002, a light emitting portion area is detected from the frame image I acquired in step 1000. Next, in step 1004, a communication pixel corresponding to the coordinates of the light emitting unit area detected in step 1002 is selected. Next, in step 1006, communication information C is acquired based on the optical signal acquired by the communication pixel selected in step 1004.

  Next, in step 1008, image information i is extracted based on the light emitting area of the frame image I detected in step 1002. Next, in step 1010, based on the communication information C acquired in step 1006 and the image information i extracted in step 1008, relative information between the optical communication device and the optical signal transmission device is calculated and output. The process ends.

  As described above, according to the optical communication apparatus of the tenth embodiment, in addition to the effects of the first to ninth embodiments, it is possible to calculate relative information at a higher speed.

  Next, an eleventh embodiment will be described. The optical communication system according to the eleventh embodiment includes an optical signal transmission device and an optical communication device, similarly to the optical communication system 1 according to the first embodiment. Since the configuration of each device is different only in the configuration of the image sensor of the optical communication device, this point will be described. Note that detailed description of portions common to the configuration of the image sensor in the tenth embodiment is omitted.

  As shown in FIG. 29A, the image sensor of the optical communication apparatus according to the eleventh embodiment is an image / communication in which the image pixel and the communication pixel described in the tenth embodiment are integrated. The integrated pixels are two-dimensionally arranged on one substrate.

  As shown in FIG. 5B, the image / communication integrated pixel has a common photoelectric conversion element, an image pixel circuit, and a communication pixel circuit that are used in common by the image pixel and the communication pixel. doing. That is, an image pixel is constituted by the common photoelectric conversion element and the image pixel circuit, and a communication pixel is constituted by the common photoelectric conversion element and the communication pixel circuit. In the image / communication integrated pixel, the imaging mode and the communication mode are switched by a control signal from the reception control unit 14. When an operation in the imaging mode is instructed by the control signal, the electric charge generated by the common photoelectric conversion element is processed by the image pixel circuit and sent as an image signal to the readout line. When the operation in the communication mode is instructed by the control signal, the charge generated by the common photoelectric conversion element is processed by the communication pixel circuit and sent as a communication signal to the readout line. .

  Next, an optical communication processing routine executed by the optical communication device 10 according to the eleventh embodiment will be described with reference to FIG. Note that the same processes as those in the optical communication process routine of the tenth embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  In step 1100, a control signal is transmitted so that all image / communication integrated pixels operate in the imaging mode, and one frame image I is acquired in the imaging mode. Next, in step 1002, the light emitting area is detected from the frame image I acquired in step 1000. Next, at step 1102, the control signal is switched so that the image / communication integrated pixel corresponding to the coordinates of the light emitting section area detected at step 1002 operates in the communication mode. Next, in step 1104, communication information C is acquired based on the optical signal acquired by the image / communication integrated pixel switched to the communication mode in step 1102. Next, steps 1008 and 1010 are executed to calculate relative information.

  As described above, according to the optical communication apparatus of the eleventh embodiment, relative information can be calculated at a higher speed as in the tenth embodiment.

  In the above embodiment, the case where the optical communication device is mounted on a vehicle has been described as an example. However, the present invention is not limited to this, and may be applied to, for example, a factory line.

  Moreover, it is good also as a structure which combined each said embodiment suitably.

  In the above embodiment, the program has been described as an embodiment in which the program is installed in advance. However, the program can be provided by being stored in a storage medium such as a CDROM. Moreover, you may comprise with the hardware which implement | achieves each function.

1, 3, 4, 5, 6, 7, 9 Optical communication system 10, 410, 510, 610, 710, 910 Optical communication device 12 Image sensor 14 Reception control unit 16, 616 Image information extraction unit 18, 618 Relative information calculation Unit 20 relative speed calculation unit 22 collision prediction calculation unit 24 parameter setting unit 26 region detection unit 28 avoidance steering angle determination unit 30 follow-up control units 50, 250, 350, 450, 650, 750, 950 optical signal transmission devices 52, 252, 352, 652, 752 Light emitting unit 54 Transmission control unit

Claims (15)

An imaging unit that images a region including a transmission unit that transmits communication information indicating at least one of a position, a size, and a shape of the light emitting unit by an optical signal by controlling the light emitting unit;
Communication information extraction means for extracting communication information indicated by the optical signal transmitted by the transmission means based on a change in pixel values between frames of a plurality of frame images captured by the imaging means;
Image information extracting means for extracting, from at least one frame image of the plurality of frame images, image information indicating at least one of a position, a size, and a shape of a light emitting portion region indicating the light emitting portion;
Calculation means for calculating relative information indicating a relative relationship between the transmission means and the imaging means based on the communication information extracted by the communication information extraction means and the image information extracted by the image information extraction means; ,
An optical communication device including: A plurality of first pixels that output a signal according to the amount of received light, and a plurality of second pixels that output a signal according to the amount of received light and have a response speed to the change in the amount of received light that is faster than the first pixel, An imaging unit including an imaging device arranged two-dimensionally on a single substrate, and by controlling the light emitting unit, communication information indicating at least one of the position, size, and shape of the light emitting unit is obtained by an optical signal. Imaging means for imaging an area including transmission means for transmitting;
Based on a signal output from the second pixel at a position corresponding to a light emitting unit area indicating the light emitting unit on a frame image captured using the first pixel of the imaging unit, the transmission unit transmits the signal. Communication information extracting means for extracting communication information indicated by the optical signal;
Image information extracting means for extracting, from the frame image, image information indicating at least one of a position, a size, and a shape of a light emitting portion region indicating the light emitting portion;
Calculation means for calculating relative information indicating a relative relationship between the transmission means and the imaging means based on the communication information extracted by the communication information extraction means and the image information extracted by the image information extraction means; ,
An optical communication device including:   A first pixel having a first circuit that processes the image sensor as a signal for an image using a first photoelectric conversion element and a charge generated by the first photoelectric conversion element; a second photoelectric conversion element; and the second photoelectric conversion Or a second pixel having a second circuit that processes the charge generated by the element as a communication signal, or a common photoelectric conversion element and the charge generated by the common photoelectric conversion element as an image signal And a first pixel having a fourth circuit for processing the common photoelectric conversion element and a charge generated by the common photoelectric conversion element as a communication signal. The optical communication apparatus according to claim 2.   The calculating means is based on the size of the light emitting part extracted by the communication information extracting means and the size of the light emitting area extracted by the image information extracting means, or the light emission extracted by the communication information extracting means. A distance between the transmission unit and the imaging unit is calculated based on the position of the unit and the position of the light emitting unit region extracted by the image information extraction unit, or the light emission extracted by the communication information extraction unit The inclination of the said transmission means with respect to the said imaging means is calculated based on the shape of a part and the shape of the said light emission part area | region extracted by the said image information extraction means. Optical communication device. In the case where the transmission unit includes a plurality of light emitting units, the transmission unit transmits the interval between the plurality of light emitting units as the communication information,
5. The optical communication device according to claim 1, wherein the image information extraction unit detects a plurality of light emitting unit regions from the frame image and extracts an interval between the plurality of light emitting unit regions. . The communication information further includes information indicating a moving speed of the transmission unit,
Based on the moving speed of the imaging means detected by the detecting means for detecting the moving speed of the imaging means and the moving speed of the transmitting means extracted by the communication information extracting means, the transmitting means and the imaging means 6. A prediction means for predicting a collision between the transmission means and the imaging means based on the calculated relative speed and the relative information is included. 2. An optical communication device according to item 1.   7. The prediction unit calculates a predicted collision time between the transmission unit and the imaging unit based on the relative speed and a distance between the transmission unit and the imaging unit calculated as the relative information. The optical communication device described.   8. The apparatus according to claim 1, further comprising a setting unit configured to set a parameter for controlling at least one of imaging by the imaging unit and image processing on the captured image based on the relative information. Optical communication equipment. The communication information further includes information indicating a size of an object on which the transmission unit is mounted, with the light emitting unit as a reference,
Based on the information indicating the size of the object extracted by the communication information extraction unit, the position of the light emitting area extracted by the image information extraction unit, and the relative information calculated by the calculation unit, the frame The optical communication apparatus according to claim 1, further comprising a detection unit that detects an object region indicating the object on the image. The communication information further includes information indicating the color of the object,
Based on the information indicating the color of the object extracted by the communication information extracting means and the color information of the object area detected by the detecting means, at least one of the white balance and the exposure control value of the imaging means is obtained. The optical communication apparatus according to claim 9, further comprising adjustment means for adjusting. The communication information further includes information indicating a width of a mobile body on which the transmission unit is mounted,
Based on the information indicating the width of the moving object extracted by the communication information extracting means, the width of the moving object region indicating the moving object on the frame image, and the relative information calculated by the calculating means. The moving body on which the communication device is mounted includes a determining unit that determines an angle in a moving direction for moving while avoiding the moving body on which the transmitting unit is mounted. The optical communication device described.   12. The control unit according to claim 1, further comprising a control unit that controls a moving direction of the moving body on which the optical communication device is mounted so that the inclination of the transmission unit with respect to the imaging unit calculated by the calculation unit is eliminated. An optical communication device according to claim 1. Computer
By controlling the light emitting unit, a plurality of frames captured by an imaging unit that captures an area including a transmitting unit that transmits communication information indicating at least one of the position, size, and shape of the light emitting unit by an optical signal Communication information extraction means for extracting communication information indicated by the optical signal transmitted by the transmission means based on a change in pixel value between frames of an image;
Image information extracting means for extracting, from at least one frame image of the plurality of frame images, image information indicating at least one of a position, a size, and a shape of a light emitting portion area indicating the light emitting portion; and the communication information extraction Based on the communication information extracted by the means and the image information extracted by the image information extraction means, it functions as a calculation means for calculating relative information indicating a relative relationship between the transmission means and the imaging means. Optical communication program. Computer
A plurality of first pixels that output a signal according to the amount of received light, and a plurality of second pixels that output a signal according to the amount of received light and have a response speed to the change in the amount of received light that is faster than the first pixel, Transmitting means for transmitting communication information indicating at least one of the position, size, and shape of the light emitting part by an optical signal by controlling the light emitting part by including an image pickup device arranged two-dimensionally on one substrate. The transmission based on a signal output from the second pixel at a position corresponding to the light emitting part region indicating the light emitting part on the frame image picked up using the first pixel of the image pickup means for picking up an image including the region Communication information extracting means for extracting communication information indicated by the optical signal transmitted by the means;
Image information extracting means for extracting image information indicating at least one of a position, a size, and a shape of a light emitting portion area indicating the light emitting portion from the frame image; and communication information extracted by the communication information extracting means; An optical communication program for functioning as calculation means for calculating relative information indicating a relative relationship between the transmission means and the imaging means based on the image information extracted by the image information extraction means.   The optical communication program for functioning a computer as each means which comprises the optical communication apparatus of any one of Claims 1-12.

Images