WO2016104235A1

WO2016104235A1 - Stereo imaging device and moving body

Info

Publication number: WO2016104235A1
Application number: PCT/JP2015/085013
Authority: WO
Inventors: 敦司山下; 山本　信一; 野崎　昭俊; 修丹内
Original assignee: コニカミノルタ株式会社
Priority date: 2014-12-26
Filing date: 2015-12-15
Publication date: 2016-06-30
Also published as: JPWO2016104235A1

Abstract

The present invention provides a stereo imaging device that is compact, light, and capable of recognizing a variety of obstacles, and a moving body having the same. Configuring a stereo imaging device from a first imaging device having a first array optical system provided with various optical filters and a second imaging device having a second array optical system results in a longer baseline than that of an imaging device having a single array optical system and thus makes it possible to detect a variety of obstacles from images having passed through the various optical filters while maintaining distance measurement accuracy at a long distance.

Description

Stereo imaging apparatus and moving body

The present invention relates to a stereo imaging device including a plurality of compound-eye imaging devices and a moving body equipped with such stereo imaging devices.

In recent years, an obstacle detection device for detecting an obstacle ahead of a traveling direction has been developed in an unmanned moving body or a vehicle. For example, the obstacle detection apparatus disclosed in Patent Document 1 detects an obstacle by scanning infrared rays projected in the traveling direction, but the detection range remains within the infrared scan range and is relatively narrow. There is.

On the other hand, the obstacle detection device disclosed in Patent Document 2 detects an obstacle using a millimeter wave radar, but has a problem that it is difficult to identify the shape and size of the obstacle. On the other hand, the obstacle detection device disclosed in Patent Document 3 calculates, by template matching, positional deviation (parallax) of the same object on a plurality of images captured at the same time by using a stereo camera. The position of the object in the real space can be calculated based on the parallax that has been obtained, using a known conversion formula, and the above-described problems can be solved.

JP 2014-24431 A JP 2013-86560 A JP 2009-146217 A

However, the technique disclosed in Patent Document 3 has a problem that it is difficult to identify a subject under a reflecting object or backlight condition. Although it is conceivable to mount the above-described three types of obstacle detection devices and use them according to the conditions, the cost increases and the weight increases. In particular, when an obstacle detection device is mounted on an unmanned air vehicle or the like, there is an actual situation that an obstacle detection device that is as small and light as possible is desired in order to secure one cruising distance.

The present invention has been made in view of such problems, and an object of the present invention is to provide a stereo imaging device capable of recognizing various obstacles while being small and light and a moving body having the same.

In order to realize at least one of the above-described objects, a stereo imaging device reflecting one aspect of the present invention is provided.
A first array optical system having a plurality of first imaging optical systems arranged with different optical axes and an image formed by each of the plurality of first imaging optical systems are photoelectrically converted and an image signal is output. A first imaging device having a first solid-state imaging device;
A second array optical system having a plurality of second imaging optical systems arranged with different optical axes and an image formed by each of the plurality of second imaging optical systems are photoelectrically converted and an image signal is output. A second imaging device having a second solid-state imaging device, and a stereo imaging device comprising:
The optical axes of the first imaging optical system and the second imaging optical system are parallel, and the first imaging device and the second imaging device are arranged apart from each other in a direction orthogonal to the optical axis,
Each of the first imaging optical systems includes any one of a filter without optical correction, a red filter, a green filter, a blue filter, an ND filter, a near-infrared filter, and a polarizing filter, and the first There are at least two types of filters in one array optical system,
Each filter of the second imaging optical system is the same type as the filter of the first array optical system,
Based on the image signal of the image formed by the first imaging optical system and the image signal of the image formed by the second imaging optical system having the same type of filter, three-dimensional information of the subject is obtained.

According to the present invention, it is possible to provide a stereo imaging device capable of recognizing various obstacles while being small and light, and a movable body having the same.

It is a perspective view of the unmanned air vehicle as a moving body concerning this embodiment. It is a figure which shows schematic structure of 1st imaging device CA1. It is a figure which shows an example of the arrangement | sequence of the filter element in the color filter used for 1st imaging device CA1. 1 is a block diagram showing a configuration of an unmanned air vehicle 100. FIG. It is a block diagram which shows the processing flow in the image process part PR. It is a block diagram which shows another processing flow in the image process part PR. It is a block diagram which shows another processing flow in the image process part PR. It is a block diagram which shows another processing flow in the image process part PR. It is a block diagram which shows another processing flow in the image process part PR. It is a block diagram which shows the processing flow in the image process part PR concerning a 2nd process aspect. It is a block diagram which shows another processing flow in the image process part PR. It is a block diagram which shows another processing flow in the image process part PR. It is a block diagram which shows another processing flow in the image process part PR. It is a block diagram which shows another processing flow in the image process part PR. It is a perspective view of the unmanned aerial vehicle as a moving body concerning another embodiment. It is a block diagram which shows the structure of unmanned air vehicle 100 '. It is a figure which shows the example of the vehicle carrying a stereo imaging device.

Hereinafter, embodiments according to the present invention will be described. FIG. 1 is a perspective view of an unmanned air vehicle as a moving body according to the present embodiment. In the unmanned air vehicle 100 shown in FIG. 1, the main body 101 supported by the legs 102 has four arms 103A to 103D horizontally implanted at equal intervals in the circumferential direction. Motors 104A to 104D, which are propulsive force generators, are attached to the tips of the arms 103A to 103D, and propellers 105A to 105D are rotatably attached to the rotation shafts of the motors 104A to 104D that face in the vertical direction. It has been.

On the upper surface of the main body 101, the first image pickup device CA1 and the second image pickup device CA2 are installed with their respective optical axes parallel to each other and spaced apart in the direction perpendicular to the optical axis and facing the traveling direction (arrow direction). Yes.

FIG. 2 is a diagram illustrating a schematic configuration of the first imaging device CA1. In FIG. 2, the first imaging device CA1 photoelectrically converts a subject image formed by each of the single-eye optical system IL and the single-eye optical system IL arranged in three rows and three columns with the optical axes parallel to each other. A solid-state imaging device (first solid-state imaging device) SR having nine conversion regions (which may be integrated) Ia, and an optical filter CF disposed between the single-eye optical system IL and the photoelectric conversion region Ia; Have. The optical filter CF is divided into nine filter elements CFa according to the individual eye optical system IL. The optical filter CF may be disposed on the subject side with respect to the single-eye optical system IL. A plurality of types of optical filters CF and a plurality of single-eye optical systems IL constitute a first array optical system. Although not shown, in the second imaging device CA2 having the same configuration, the same type of optical filter CF and a plurality of single-eye optical systems IL constitute a second array optical system, and a subject image formed thereby. The solid-state imaging element SR that photoelectrically converts the second to the second solid-state imaging element.

FIG. 3 is a diagram illustrating an example of the arrangement of filter elements in the optical filter used in the first imaging device CA1. The optical filter CF includes a filter element CFa (hereinafter referred to as NO) that is a transparent glass, a filter element CFa that transmits red light (hereinafter referred to as R), and a filter element CFa (hereinafter referred to as R) that transmits green light. G), a filter element CFa that transmits blue light (hereinafter referred to as B), a filter element CFa that transmits near infrared light (hereinafter referred to as IR), and a first attenuation A filter element (ND filter) CFa having a rate (hereinafter referred to as ND1), a filter element (ND filter) CFa (hereinafter referred to as ND2) having a second attenuation rate higher than the first attenuation rate, and a transmission It has a filter element (polarization filter) CFa (hereinafter referred to as PF) whose axis is 0 degrees and a filter element (polarization filter) CFa (hereinafter referred to as SF) whose transmission axis is 90 degrees. An image forming optical system (first image forming optical system or second image forming optical system) is configured by one type of filter element CFa and the corresponding single-eye optical system IL. It is sufficient that at least two types of filter elements are provided depending on the application.

The second imaging device CA2 has the same configuration as the first imaging device CA1 shown in FIGS. However, the arrangement of the filter elements CFa may be the same as that of the first imaging device, an arrangement having a mirror image relationship, or a random arrangement. The “filter without optical correction” refers to a filter that is not substantially optically corrected for transmitted light. In addition to the transparent glass shown in FIG. 3, a transparent resin material may be used, and no filter element is provided. In some cases, it is possible to capture wavelength information in a wider band than in each case of RGB.

FIG. 4 is a block diagram showing the configuration of the unmanned air vehicle 100. As shown in FIG. Image signals output from the first imaging device CA1 and the second imaging device CA2 are image processing units PR that are mounted on the main body 101 of the unmanned air vehicle 100 and connected to the first imaging device CA1 and the second imaging device CA2. The subject distance information acquisition unit PR1 in the image processing unit PR obtains three-dimensional information of the imaging region, thereby obtaining subject information related to the obstacle. Further, based on the input image signal, a visible image is formed by the monitoring visible image information acquisition unit PR2 in the image processing unit PR, and the data is transmitted to an external monitor (display device) MT. The visible image captured from the unmanned air vehicle 100 can be observed by the operator through the monitor MT. In particular, a color visible light image can be formed by using a light beam transmitted through the filter elements (R, G, B) of the first imaging device CA1 and / or the second imaging device CA2, and this is displayed on the monitor MT. By doing so, the operation of the operator can be supported. A monitor image may be displayed using a light beam that has passed through another filter element.

Further, the subject information output from the image processing unit PR is output to a drive control unit (movement control unit) DR disposed in the main body 101, and based on this, motors 104A to 104D are used so as not to collide with obstacles. It is possible to realize autonomous flight by controlling.

Next, processing including autonomous flight control in the image processing unit PR will be described with reference to the drawings. FIG. 5 is a block diagram illustrating a processing flow in the image processing unit PR. Here, it is assumed that the unmanned air vehicle 100 detects an object ahead in the moving direction as an obstacle, and performs a flight so as to avoid the obstacle. In the flight state, the first imaging device CA1 and the second imaging device CA2 respectively receive the light beams transmitted through the filter elements CFa and the individual eye optical systems IL by the photoelectric conversion region Ia in the solid-state imaging device SR. Perform photoelectric conversion. After photoelectric conversion, each signal is output from the solid-state imaging device SR, stored in a memory MR (not shown) provided in the image processing unit PR, and called up as necessary.

(First processing mode)
In the first processing mode, a target for distance measurement is extracted in advance, and distance measurement is performed on the target to create a distance map. In addition, when a reflector is detected, distance information of the reflector itself is not detected. First, when the image processing unit PR is to determine the exposure time (exposure amount) by the light flux that has passed through the filter element (NO) of the transparent glass by a predetermined program or the operation of the operator, the process P101 , P102, a signal corresponding to the light beam that has passed through the filter element (NO) of the transparent glass is input independently from each of the first imaging device CA1 and the second imaging device CA2, and the exposure time is calculated from the value. This is determined and fed back to control the exposure time of the solid-state imaging device SR individually. After imaging with the determined exposure time, signals output from the photoelectric conversion regions Ia of the first imaging device CA1 and the second imaging device CA2 are stored in the memory MR.

Next, in the process P103, the image processing unit PR receives from the memory MR a signal corresponding to the light beam that has passed through the filter element (NO) of the transparent glass according to each of the first imaging device CA1 and the second imaging device CA2, and the first The signal corresponding to the light beam that has passed through the filter element (ND1) having the attenuation factor of 1 and the signal corresponding to the light beam that has passed through the filter element (ND2) having the second attenuation factor are input, and the contrast is the highest. An image is selected, and a subject is discriminated from a signal value corresponding to the image. For example, a subject with high brightness such as the sun may appear under imaging conditions such as under backlighting, but the pixel value of the photoelectric conversion region Ia to which the light beam that has passed through the filter element (NO) of the transparent glass is incident is as described above. Even in an imaging state that saturates, a subject image having a contrast that does not cause whiteout is obtained based on a signal corresponding to the light beam that has passed through the filter element (ND1 or ND2) having the first attenuation factor or the second attenuation factor. Therefore, it is possible to obtain an image with the optimum brightness for obstacle detection. For this reason, it is possible to prevent erroneous detection. Note that any two of the filter elements (NO, ND1, ND2) may be selected, and the same operation as described above may be performed from the obtained signal. Alternatively, any two or three of the filter elements (NO, ND1, ND2) may be selected to create an image with a wide dynamic range.

In parallel with the above, in the process P104, the image processing unit PR outputs a signal corresponding to the light beam that has passed through the filter element (PF) and a signal corresponding to the light beam that has passed through the filter element (SF) from the memory MR. Then, the difference between the two signal values is obtained, and it is determined from these values whether the subject contains a reflector. That is, if the difference between the two signal values exceeds the threshold value, it can be seen that the reflected light has entered, so the image processing unit PR recognizes the region range exceeding the threshold value as a reflector. Note that the signal used for the determination of the reflector may be only the signal in the first imaging device CA1, only the signal in the second imaging device CA2, or from both the first imaging device CA1 and the second imaging device CA2. This signal may be used.

In parallel with the above, the image processing unit PR inputs a signal corresponding to the light beam that has passed through the filter element (IR) that transmits near-infrared light from the memory MR in the process P105, and the subject is lower than the threshold value. Identify and extract. For example, overhead lines installed on the forest may be difficult to discern with visible light because the color tone is close to the background, but plants generally reflect near-infrared light, so only the signal of the overhead line part Becomes lower. Therefore, an overhead line can be identified from the forest by grasping near-infrared light.

In parallel with the above, in the process P106, the image processing unit PR outputs from the memory MR a signal corresponding to the red light beam that has passed through the filter element (R) and a signal corresponding to the green light beam that has passed through the filter element (G). And at least one of the signals corresponding to the blue luminous flux that has passed through the filter element (B) and the signal corresponding to the luminous flux that has passed through the filter element (IR) that transmits near-infrared light, Calculate to identify the subject.

For example, an NDVI (normalized difference vegetation index) formula can be used to recognize an object. When the luminance value of the signal corresponding to the red light beam passing through the filter element (R) is Rs and the luminance value of the signal corresponding to the near-infrared light beam passing through the filter element (IR) is IRs, (1) If the NDVI value represented by the formula is within a predetermined range, it can be determined that the imaged subject is a plant. Although only a signal corresponding to the near-infrared light beam that has passed through the filter element (IR) may be identified as a plant, using the equation (1) makes it possible to identify an object with high accuracy. A signal corresponding to the green light beam that has passed through the filter element (G) or a signal corresponding to the blue light beam that has passed through the filter element (B) may be used as appropriate.
(NDVI) = (IRs−Rs) / (IRs + Rs) (1)

Further, the image processing unit PR applies the signal corresponding to the red light beam that has passed through the filter element (R) from the memory MR and the filter element (G) to the region other than the reflector determined in process P104. ) And a signal corresponding to the blue luminous flux that has passed through the filter element (B) are respectively input to generate two sets of visible light image data, and based on the parallax information A stereo image of each color is formed and distance measurement is performed. The distance measuring technique using a stereo image is disclosed in Japanese Patent Application Laid-Open No. 2009-186228, and thus details are not described. Thereafter, the image processing unit PR creates a visible light distance map for each pixel in process P108, and obtains three-dimensional information of the subject. The “distance map” refers to an object image having distance information.

In parallel with this, the image processing unit PR outputs a signal corresponding to the light beam that has passed through the filter element (NO) of the transparent glass from the memory MR in the process P109 for the region other than the reflector determined in the process P104. And a signal corresponding to the light beam that has passed through the filter element (ND1) having the first attenuation factor and a signal corresponding to the light beam that has passed through the filter element (ND2) having the second attenuation factor are input respectively. Then, a stereo image is formed for the image having the highest contrast, and the distance is measured. Thereafter, the image processing unit PR creates a backlight distance map for each pixel in process P110, and obtains three-dimensional information of the subject. In addition, you may perform the same thing as the above from the signal obtained by selecting any two of filter elements (NO, ND1, ND2). Alternatively, any two or three of the filter elements (NO, ND1, ND2) may be used to create an image with a wide dynamic range, form a stereo image, and perform distance measurement.

In parallel with this, in the process P111, the image processing unit PR obtains the boundary portion of the reflector determined in the process P104, and the light flux that has passed through the filter element (PF) from the memory MR with respect to this boundary portion. Or a signal corresponding to the light beam that has passed through the filter element (SF) is input to form a stereo image and perform distance measurement. Thereafter, in process P112, the image processing unit PR creates a distance map including the reflector using the obtained distance measurement value of the boundary as a distance measurement value of the reflector, and obtains three-dimensional information of the subject.

In parallel with this, the image processing unit PR outputs a signal corresponding to the near-infrared light beam that has passed through the filter element (IR) from the memory MR in the process P113 for the region other than the reflector determined in the process P104. Is input, a near-infrared stereo image is formed, and distance measurement is performed. Thereafter, the image processing unit PR creates a near-infrared distance map for each pixel in process P114, and obtains three-dimensional information of the subject.

In parallel with this, the image processing unit PR outputs a signal corresponding to the red light beam that has passed through the filter element (R) from the memory MR in the process P115 for the region other than the reflector determined in the process P104. Inputs a signal corresponding to the green light beam that has passed through the filter element (G), a signal that corresponds to the blue light beam that has passed through the filter element (B), and a signal that corresponds to the near-infrared light beam that has passed through the filter element (IR). Then, visible light and near-infrared stereo images are formed, and distance measurement is performed. Thereafter, the image processing unit PR creates a visible light and near-infrared distance map for each pixel in process P116, and obtains three-dimensional information of the subject.

Thereafter, in process P117, the image processing unit PR superimposes the distance maps obtained in processes P108, P110, P112, P114, and P116, and the distance measurement information of the reflector itself that cannot guarantee the accuracy (the distance measurement information of the reflector). To avoid this when there is an object recognized by two or more distance maps, for example. Subject information is output to the drive controller DR. In addition to this, the image processing unit PR avoids the case where there is an object closest to the unmanned air vehicle 100 based on the distance map obtained in the processes P108, P110, P112, P114, and P116. As described above, the subject information may be output to the drive control unit DR.

FIG. 6 is a block diagram showing another example of the processing flow in the image processing unit PR. The image processing unit PR receives a red light beam that has passed through the filter element (R), a green light beam that has passed through the filter element (G), and a blue light that has passed through the filter element (B) by a predetermined program or an operator's operation. When the exposure time is to be determined by the luminous flux, the red luminous flux that has passed through the filter element (R) independently from each of the first imaging device CA1 and the second imaging device CA2 in the processes P101A and P102A. , A signal corresponding to the green light beam that has passed through the filter element (G), and a signal corresponding to the blue light beam that has passed through the filter element (B), and a luminance value is calculated from these values. The exposure time is determined and fed back to control the exposure time of the solid-state imaging element SR individually. Since the processes P103 to P117 are the same as the processes in FIG.

FIG. 7 is a block diagram showing another example of the processing flow in the image processing unit PR. When the exposure time is determined by the near-infrared light beam that has passed through the filter element (IR) by a predetermined program or an operator's operation, the image processing unit PR performs the first in the processing P101B and P102B. A signal corresponding to the near-infrared light beam that has passed through the filter element (IR) is independently input from the imaging device CA1 and the second imaging device CA2, and the exposure time is determined from this value and fed back. Thus, the exposure time of the solid-state image sensor SR is individually controlled. Since the processes P103 to P117 are the same as the processes in FIG.

FIG. 8 is a block diagram showing another example of the processing flow in the image processing unit PR. The image processing unit PR has passed through the filter element (ND1) having the first attenuation factor and the filter element (ND2) having the second attenuation factor by a predetermined program or an operator's operation. When the exposure time is to be determined by at least one of the luminous fluxes, the processing P101C and P102C have the first attenuation rate independently from the first imaging device CA1 and the second imaging device CA2. A signal corresponding to the light beam passing through the filter element (ND1) and / or a signal corresponding to the light beam passing through the filter element (ND2) having the second attenuation factor is input, and the exposure time is determined based on this value. This is determined and fed back to control the exposure time of the solid-state imaging device SR individually. Since the processes P103 to P117 are the same as the processes in FIG.

FIG. 9 is a block diagram showing another example of the processing flow in the image processing unit PR. The image processing unit PR determines the exposure time based on a signal corresponding to the light beam that has passed through the filter element (PF) or a signal corresponding to the light beam that has passed through the filter element (SF) by a predetermined program or an operator's operation. In this case, in the processes P101D and P102D, a signal corresponding to the light beam that has passed through the filter element (PF) or the filter element (independently from the first image pickup apparatus CA1 and the second image pickup apparatus CA2). The signal corresponding to the light flux that has passed SF) is input, the exposure time is determined based on this value, and this is fed back to individually control the exposure time of the solid-state imaging device SR. Since the processes P103 to P117 are the same as the processes in FIG.

For example, when the unmanned aerial vehicle 100 moves toward the water surface, there is a possibility that most of the imaging region is occupied by the reflector (water surface). In such a case, since accurate ranging becomes extremely difficult and the unmanned flying object may land on the water surface, when the reflector occupies most of the subject (for example, 80% or more), It is preferable that the drive control unit DR temporarily stops the unmanned air vehicle 100.

(Second processing mode)
In the second processing mode, an image signal obtained by the same type of filter is used, and distance measurement is performed on all captured subjects to form a distance map, and necessary ones are extracted. Further, when a reflector is detected, it is not used even if distance information of the reflector itself is detected. FIG. 10 is a block diagram illustrating a processing flow in the image processing unit PR according to the second processing mode. First, when the exposure time is to be determined by the light beam that has passed through the filter element (NO) of the transparent glass by a predetermined program or an operator's operation, the image processing unit PR, in processing P201, P202, A signal corresponding to the light beam that has passed through the filter element (NO) of the transparent glass is input independently from each of the first imaging device CA1 and the second imaging device CA2, and the exposure time is determined from the value. And the exposure time of the solid-state image sensor SR is individually controlled. After imaging with the determined exposure time, signals output from the photoelectric conversion regions Ia of the first imaging device CA1 and the second imaging device CA2 are stored in the memory MR.

Next, in the process P203, the image processing unit PR, for each of the first imaging device CA1 and the second imaging device CA2, outputs a signal corresponding to the red light beam that has passed through the filter element (R) and the filter element (G ) And a signal corresponding to the blue light beam that has passed through the filter element (B) are respectively input, a stereo image of each color is formed, a distance measurement is performed, and an RGB distance measurement is performed. Get the data. If the subject includes a reflector, the RGB distance measurement data also includes the actual distance measurement data.

In parallel with this, in the process P204, the image processing unit PR outputs a signal corresponding to the light beam that has passed through the filter element (NO) of the transparent glass for each of the first imaging device CA1 and the second imaging device CA2 from the memory MR. And a signal corresponding to the light beam that has passed through the filter element (ND1) having the first attenuation factor and a signal corresponding to the light beam that has passed through the filter element (ND2) having the second attenuation factor are input respectively. From the values corresponding to the same kind of filter elements, a stereo image is formed, and distance measurement is performed to obtain ND distance measurement data. The ND distance measurement data also includes the actual distance measurement data when the subject includes a reflector. Note that any two of the filter elements (NO, ND1, ND2) may be selected, and the same operation as described above may be performed from the obtained signal. Alternatively, any two or three of the filter elements (NO, ND1, ND2) may be used to create an image with a wide dynamic range, form a stereo image, and perform distance measurement.

In parallel with the above, in the process P205, the image processing unit PR receives, from the memory MR, a signal corresponding to the light beam that has passed through the filter element (PF) and the filter for each of the first imaging device CA1 and the second imaging device CA2. A signal corresponding to the light beam that has passed through the element (SF) is input, a stereo image is formed from values corresponding to the same type of filter element, and distance measurement is performed to obtain polarization distance measurement data. When the subject includes a reflector, the polarization distance measurement data also includes the actual distance measurement data.

In parallel with the above, in the process P206, the image processing unit PR receives a signal corresponding to the near-infrared light beam that has passed through the filter element (IR) from the memory MR for each of the first imaging device CA1 and the second imaging device CA2. Are input, a stereo image is formed, and distance measurement is performed to obtain near-infrared distance measurement data. Near-infrared distance measurement data also includes actual distance measurement data when the subject includes a reflector.

Next, in process P207, the image processing unit PR receives a signal corresponding to the light beam that has passed through the filter element (PF) and a signal that corresponds to the light beam that has passed through the filter element (SF) from the memory MR, and The difference between the two signal values is obtained, and it is determined from these values whether the subject contains a reflector. That is, if the difference between the two signal values exceeds the threshold value, it can be seen that the reflected light has entered, so the image processing unit PR recognizes the region range that exceeds the threshold value as a reflector. Note that the signal used for the determination of the reflector may be only the signal in the first image pickup device CA1, only the signal in the second image pickup device CA2, or both the first image pickup device CA1 and the second image pickup device CA2. The signal from

Further, the image processing unit PR obtains a boundary portion of the reflector in the subject in the process P208, and the boundary value of the distance measurement value of the reflector is obtained from the polarization distance measurement data obtained in the process P205. In step P209, a distance map of the reflector is created and the three-dimensional information of the subject is obtained.

Further, the image processing unit PR removes the distance measurement data corresponding to the reflector determined in the process P207 from the visible light distance measurement data obtained in the process P203, and in the process P210, the distance map for visible light. And 3D information of the subject is obtained.

Further, in process P211, the image processing unit PR receives from the memory MR a signal corresponding to the light beam that has passed through the filter element (NO) of the transparent glass in accordance with each of the first imaging device CA1 and the second imaging device CA2. The signal corresponding to the light beam that has passed through the filter element (ND1) having the attenuation factor of 1 and the signal corresponding to the light beam that has passed through the filter element (ND2) having the second attenuation factor are input, and the contrast is the highest. An image is selected, and ND ranging data corresponding to the image is selected from the ND ranging data obtained in process P204. In process P212, the image processing unit PR removes distance measurement data corresponding to the reflector determined in process P207 from the selected ND distance measurement data, creates a distance map for backlighting, and creates a three-dimensional image of the subject. Ask for information. Note that any two of the filter elements (NO, ND1, ND2) may be selected, and the same operation as described above may be performed from the obtained signal. Alternatively, any two or three of the filter elements (NO, ND1, ND2) may be used to create an image with a wide dynamic range, form a stereo image, and perform distance measurement.

Further, the image processing unit PR removes the distance measurement data corresponding to the reflector determined in the process P207 from the near infrared distance measurement data obtained in the process P206, and in the process P213, the near infrared distance. A map is created and 3D information of the subject is obtained.

Further, in process P214, the image processing unit PR receives from the memory MR a signal corresponding to the red light beam that has passed through the filter element (R), a signal corresponding to the green light beam that has passed through the filter element (G), and the filter element. Input at least one of the signals corresponding to the blue luminous flux that has passed through (B) and the signal corresponding to the luminous flux that has passed through the filter element (IR) that transmits near-infrared light, and calculate these to calculate the subject. Is identified.

Further, in process P215, the image processing unit PR receives from the memory MR a signal corresponding to the red light beam that has passed through the filter element (R), a signal corresponding to the green light beam that has passed through the filter element (G), and a filter element. A signal corresponding to the blue light beam that has passed through (B) and a signal corresponding to the near-infrared light beam that has passed through the filter element (IR) are input to form visible light and a near-infrared stereo image, and distance measurement is performed. Do. Thereafter, the image processing unit PR removes the distance measurement data corresponding to the reflector determined in process P207 from the obtained distance measurement data, creates a vegetation distance map, and obtains the three-dimensional information of the subject. Ask.

Thereafter, in process P216, the image processing unit PR superimposes the distance maps obtained in processes P210, P212, P209, P213, and P215. At this time, the distance measurement information of the reflector that is missing in each distance measurement map is replaced with the distance measurement information of the boundary portion of the reflector, and the distance map is complemented. For example, when there is an object recognized by two or more distance maps, the subject information is output to the drive control unit DR so as to avoid this. In addition to this, the image processing unit PR avoids the case where there is an object closest to the unmanned air vehicle 100 based on the distance map obtained in the processes P210, P212, P209, P213, and P215. As described above, the subject information may be output to the drive control unit DR.

FIG. 11 is a block diagram showing another example of the processing flow in the image processing unit PR. The image processing unit PR receives a signal corresponding to the red light beam that has passed through the filter element (R), a signal corresponding to the green light beam that has passed through the filter element (G), and a filter by a predetermined program or operator operation. When the exposure time is to be determined based on the signal corresponding to the blue light beam that has passed through the element (B), the first imaging device CA1 and the second imaging device CA2 are independent from each other in the processes P201A and P202A. The signal corresponding to the red light beam that has passed through the filter element (R), the signal that corresponds to the green light beam that has passed through the filter element (G), and the signal that corresponds to the blue light beam that has passed through the filter element (B) are input. Then, the brightness value is calculated from these values to determine the exposure time, and this is fed back to individually control the exposure time of the solid-state image sensor SR. Going on. Note that processing P203 to P216 is the same as the processing in FIG.

FIG. 12 is a block diagram showing another example of the processing flow in the image processing unit PR. When the image processing unit PR is to determine the exposure time based on a signal corresponding to the near-infrared light beam that has passed through the filter element (IR) by a predetermined program or an operator's operation, the processing P201B, P202B , The signal corresponding to the near-infrared luminous flux that has passed through the filter element (IR) is input independently from each of the first imaging device CA1 and the second imaging device CA2, and the exposure time is determined from this value. This is fed back to individually control the exposure time of the solid-state image sensor SR. Note that processing P203 to P216 is the same as the processing in FIG.

FIG. 13 is a block diagram illustrating another example of the processing flow in the image processing unit PR. The image processing unit PR receives a signal corresponding to the light beam that has passed through the filter element (ND1) having the first attenuation factor and a filter element (ND2 having the second attenuation factor) by a predetermined program or an operator's operation. ), The exposure time is determined by at least one of the signals corresponding to the light beam that has passed through the first imaging device CA1 and the second imaging device CA2 in the processes P201C and P202C. The signal corresponding to the light beam that has passed through the filter element (ND1) having the first attenuation factor and the signal corresponding to the light beam that has passed through the filter element (ND2) having the second attenuation factor are input, and this The exposure time is determined based on the value, and this is fed back to individually control the exposure time of the solid-state image sensor SR. Note that processing P203 to P216 is the same as the processing in FIG.

FIG. 14 is a block diagram showing another example of the processing flow in the image processing unit PR. The image processing unit PR determines the exposure time based on a signal corresponding to the light beam that has passed through the filter element (PF) or a signal corresponding to the light beam that has passed through the filter element (SF) by a predetermined program or an operator's operation. If this is the case, in the processes P201D and P202D, a signal corresponding to the light beam that has passed through the filter element (PF) or the filter element (independently from the first image pickup apparatus CA1 and the second image pickup apparatus CA2). The signal corresponding to the light flux that has passed SF) is input, the exposure time is determined based on this value, and this is fed back to individually control the exposure time of the solid-state imaging device SR. Note that processing P203 to P216 is the same as the processing in FIG.

According to the present embodiment described above, by providing the first array optical system and the second optical system, it is possible to reduce the size and weight as compared with a combination of these units in separate units, and to increase the functionality. By arranging the first image pickup device having the first array optical system and the second image pickup device having the second array optical system in a stereo-capable arrangement, the base line length is longer than that of the image pickup device having a single array optical system. Can be ensured, and distance measurement accuracy at a long distance can be ensured. In the single-eye optical system IL having the filter element (NO) of the transparent glass, the light of all wavelength regions sensitive to the solid-state imaging element SR can be condensed, and the filter elements (ND1, ND2 having different attenuation factors). ), It is possible to obtain a subject image with a wide dynamic range from low luminance to high luminance by one imaging. In addition, the red, green, and blue filter elements (R, G, and B) can measure an object in the visible light region, and a visible image to be displayed to the operator of the unmanned air vehicle 100 can be obtained. The subject can be specified from the intensity difference or ratio. The near-infrared filter element (IR) can extract vegetation such as forests and discriminate overhead lines and the like. A reflector can be detected by detecting a light amount difference by using two types of polarizing filter elements (SF, PF) having different transmission axes. By combining a plurality of functions of each individual eye, it is possible to obtain a desired image for obstacle detection while being small and light. In the above embodiment, the visible light stereo image is formed using the signal obtained by transmitting the red, green, and blue filter elements (R, G, B). However, at least one color filter is used. A single-color or two-color stereo image may be formed using the element. Further, the image processing unit PR may first detect whether or not a reflector exists in the subject, and when the reflector is detected, distance measurement of the corresponding area may not be performed.

FIG. 15 is a perspective view of an unmanned air vehicle as a moving body according to another embodiment. The unmanned aerial vehicle 100 ′ of the present embodiment has a function capable of capturing an image of a structure such as a bridge while autonomously flying. In the unmanned air vehicle 100 ′ shown in FIG. 15, a shaft 106 that can rotate with respect to the main body 101 is provided with respect to the embodiment shown in FIG. 1, and a frame 107 is attached to the upper end of the shaft 106. The frame 107 mounts the high-pixel camera HCA and the third imaging device CA3 and the fourth imaging device CA4 arranged on both sides thereof so that the optical axes are parallel. The third image pickup device CA3 and the fourth image pickup device CA4 have the same configuration as the first image pickup device CA1 and the second image pickup device CA2, but are preferably directed in different directions (for example, 90 degrees).

FIG. 16 is a block diagram showing the configuration of the unmanned air vehicle 100 '. The image signals output from the third imaging device CA3 and the fourth imaging device CA4 are input to the sub-image processing unit PR ′ arranged in the main body 101 of the unmanned air vehicle 100 ′, and are stored in the sub-image processing unit PR ′. In the inspection object distance information acquisition unit PR3, the distance to the structure as the inspection object can be acquired. In addition, a self-location information acquisition unit that can acquire self-location information (GPS signals may be used) of the unmanned air vehicle 100 ′ by pattern-matching images captured by the third imaging device CA3 and the fourth imaging device CA4. PR4 is provided in the sub-image processing unit PR ′. These outputs are output to the drive control unit DR.

The unmanned air vehicle 100 ′ flies so as to avoid an obstacle ahead in the moving direction based on signals from the first imaging device CA1 and the second imaging device CA2, as in the above-described embodiment. The object distance information acquisition unit PR3 can acquire the distance to the structure as the inspection object, and the drive control unit DR can perform feedback control so that the distance is constant, and the self-position information acquisition unit PR4 It is possible to fly along a predetermined route with the signal from. Further, during the flight, the structure is photographed by the high pixel camera HCA, and the image is transmitted to the monitor MT so that the operator can observe and record it at the transmission destination. Since the other configuration is the same as that of the above-described embodiment, the description thereof is omitted.

For example, when an unmanned air vehicle 100 'is approached for inspecting an infrastructure structure such as a bridge, and the inspection image is obtained by imaging the bridge with a high-pixel camera HCA for inspection, it collides with a bridge that is an inspection object. It is necessary to keep a certain distance so as not to. Further, there are cases where the moving direction of the unmanned air vehicle 100 ′ and the direction of the inspection object are different from each other, such as being perpendicular to each other. According to the present embodiment, the first image pickup device CA1 and the second image pickup device CA2 are directed forward in the traveling direction of the unmanned air vehicle 100 ′, and another set of the third image pickup device CA3 and the fourth image pickup device CA4 are inspected. The unmanned air vehicle 100 ′ capable of autonomous control can be realized by appropriately maintaining the distance from the inspection object while detecting the obstacle in the traveling direction.

According to the present embodiment, if the exposure time (exposure amount) is determined by the light flux that has passed through the filter element of the transparent glass, it is possible to capture a relatively low-brightness subject at a high shutter speed, and to detect blurring of moving objects and subjects. Thus, an image formed with a light beam that has passed through the filter element of the transparent glass can be an image without black crushing. In addition, an image formed with a light beam transmitted through another optical filter element is slightly underexposed, and an image without whiteout can be obtained. As a result, ranging accuracy is improved for both low-luminance subjects and high-luminance subjects.

If the exposure time (exposure amount) is determined by at least one of the light beams transmitted through the red filter element, the green filter element, and the blue filter element, the ranging performance in the visible light region is improved and the unmanned flight is performed. Visible visible image information can be transmitted to the body operator.

If the exposure time (exposure amount) is determined by the light beam transmitted through the near-infrared filter element, for example, the light reflected by vegetation or the like can be accurately imaged, and the extraction ability of vegetation or the like is improved.

If the exposure time (exposure amount) is determined by the light beam that has passed through the polarizing filter element, it is possible to accurately capture the object light from the reflector and the like, and the extraction ability of the reflector and the like is improved.

If the exposure time (exposure amount) is determined by the light beam that has passed through the filter element that can attenuate the incident light, the amount of light from the backlight or high-intensity subject can be suppressed, and as a result, whiteout of the image can be avoided. Is possible.

In addition, if an image reflected on a reflector such as the water surface, transparent acrylic, or glass is used as a distance measurement target for a stereo image, the distance of a virtual object that exists farther than that will be detected. The position cannot be detected. In order to prevent such erroneous detection, it is important to recognize the reflector. As a means for recognizing the reflector, if the reflector is photographed through two or more types of polarizing filter elements having different transmission axes, the reflected light is polarized and thus obtained through the polarizing filter elements having different transmission axes. The brightness of the image is different from each other at the part of the reflector. If a difference in image luminance between polarization filter elements is detected in this way, it can be identified as a reflector. The combination of polarizing filter elements having different transmission axes may be in the same array optical system or in different array optical systems.

If the reflector can be detected in this way, it is possible to obtain accurate three-dimensional information of the subject by not adopting distance information based on the light flux from the reflector or by not performing distance measurement processing on the light flux from the reflector. it can.

However, even if the reflector can be specified, the distance to the reflector remains unclear if it remains as it is, so the non-reflector existing at the boundary of the reflector is replaced with a filter element made of plain glass, red light, green light. , A filter element that transmits blue light, a filter element that transmits near-infrared light, a filter element having a different attenuation factor, an imaging optical system having any one of polarizing filter elements, or an image having different types of filter elements A distance measurement process is performed based on a combination of signals from the optical system, and the distance information obtained here is used as the distance of the reflector, thereby avoiding a situation in which no distance information exists.

In addition, if there is a forest in the background of an overhead line such as an electric wire, it is difficult to detect the overhead line from a normal visible image, but if there is an imaging optical system provided with a filter element that transmits near-infrared light, Corresponding vegetation images are obtained, and in this case, it can be estimated that a subject with a low luminance value (pixel value) of a pixel of the solid-state image sensor is an electric wire. It becomes easy and accurate obstacle detection becomes possible.

If the area that can be measured is specified before the distance measurement process, the calculation load becomes smaller than the distance measurement for all the pixel areas, and the process can be performed in a short time.

By using a filter element that can attenuate incident light even under imaging conditions where the pixel value is saturated due to backlight or the like, it is possible to obtain an image with contrast that does not cause whiteout, and an image with optimal brightness for obstacle detection Therefore, it is possible to prevent false detection.

Before calculating the distance from the parallax information of the stereo image, if the subject is specified or the subject information is obtained from the characteristics of the light flux that has passed through each filter element, an optimum distance image can be obtained for each subject accordingly. It is possible to reduce the calculation load and time compared to the case where the distance measurement processing is performed collectively.

However, even if there are reflectors and the like that cannot be measured, the processing configuration in the image processing algorithm may be simplified if the distance is collectively measured. In such a case, an optimum distance image can be obtained for each subject by specifying the subject or obtaining subject information based on the characteristics of the light flux that has passed through each filter element after measuring all the subjects at once.

By displaying a color visible image obtained through a filter element that transmits red light, green light, and blue light, it is possible to transmit effective information necessary for the operation of the unmanned air vehicle operator. Furthermore, it is possible to superimpose and display images obtained through a filter element made of plain glass or a filter element that attenuates incident light, so that an image with a wide dynamic range without whiteout or blackout can be obtained even in backlight. Can be provided. In addition, if the image obtained through the polarizing filter element is displayed, it is possible to indicate the position of the reflector to be careful in obstacle detection. These images may be displayed in combination as appropriate.

By installing such a stereo imaging device with various functions in an unmanned air vehicle, obstacle detection under a wide range of conditions becomes possible.

For example, when an unmanned aerial vehicle moves toward the water surface, there is a possibility that most of the imaging area is occupied by a reflector (water surface). In such a case, accurate ranging becomes extremely difficult, and the unmanned flying object may land on the water surface. In such a case, it is preferable to temporarily stop the moving object.

The present invention is not limited to the embodiments described in the present specification, and includes other embodiments and modifications based on the embodiments and technical ideas described in the present specification. It is obvious to For example, the filters included in each of the second imaging optical systems of the second imaging device are the same type as the filters included in the first array optical system of the first imaging device, but the filters included in each of the second imaging optical systems. Is not limited to the case where the filters of each of the first imaging optical systems are completely the same, and the filters of each of the second imaging optical systems of the second imaging device have each of the first imaging optical systems. In addition to having the same filter, an extra filter (a filter that is not in the first imaging optical system but only in the second imaging optical system) may be provided.

For example, the stereo imaging device of the present invention can be mounted not only on an unmanned air vehicle but also on a vehicle. For example, in FIG. 17, the first imaging device CA1 and the second imaging device CA2 are arranged on both sides of the room mirror RM of the vehicle VH, and the imaging is performed with the optical axis directed toward the front of the vehicle. Obstacles can be detected and driving assistance can be realized. Note that the first imaging device CA1 and the second imaging device CA2 may be provided in a headlamp or a bumper.

100, 100 'Unmanned air vehicle 101 Main body 102 Legs 103A-103D Arm 104A-104D Motor 105A-105D Propeller 102 Main body 106 Axis 107 Frame CA1 First imaging device CA2 Second imaging device CA3 Third imaging device CA4 Fourth imaging device CF optical filter CFa filter element DR drive control unit HCA high pixel camera Ia photoelectric conversion region IL single-eye optical system MR memory MT monitor PR image processing unit PR ′ sub-image processing unit PR1 subject distance information acquisition unit PR2 acquisition of visible image information for monitoring Part PR3 inspection object distance information acquisition part PR4 self-position information acquisition part RM room mirror SR solid-state image sensor VH Vehicle

Claims

A first array optical system having a plurality of first imaging optical systems arranged with different optical axes and an image formed by each of the plurality of first imaging optical systems are photoelectrically converted and an image signal is output. A first imaging device having a first solid-state imaging device;
A second array optical system having a plurality of second imaging optical systems arranged with different optical axes and an image formed by each of the plurality of second imaging optical systems are photoelectrically converted and an image signal is output. A second imaging device having a second solid-state imaging device, and a stereo imaging device comprising:
The optical axes of the first imaging optical system and the second imaging optical system are parallel, and the first imaging device and the second imaging device are arranged apart from each other in a direction orthogonal to the optical axis,
Each of the first imaging optical systems includes any one of a filter without optical correction, a red filter, a green filter, a blue filter, an ND filter, a near-infrared filter, and a polarizing filter, and the first There are at least two types of filters in one array optical system,
Each filter of the second imaging optical system is the same type as the filter of the first array optical system,
A stereo having the same type of filter and obtaining three-dimensional information of a subject based on the image signal of the image formed by the first imaging optical system and the image signal of the image formed by the second imaging optical system Imaging device.
There are at least two types of filters of the first array optical system, and at least one of the filter without optical correction, the red filter, the green filter, and the blue filter, the filter without optical correction, The stereo imaging device according to claim 1, comprising at least one of an ND filter, the near-infrared filter, and the polarizing filter.
The first array optical system and the second array optical system include the first imaging optical system and the second imaging optical system including the optical correction-free filter,
The exposure amount at the time of imaging is determined by the image signal of the image formed by the first imaging optical system including the optical correction-free filter and / or the second imaging optical system including the optical correction-free filter. The stereo imaging device according to claim 1, wherein the stereo imaging device is determined based on the image signal of the formed image.
The first array optical system and the second array optical system include the first imaging optical system and the second imaging optical system including the red filter, the green filter, and the blue filter,
The exposure amount at the time of imaging is the image first signal of the image formed by the first imaging optical system including the red filter, the green filter, and the blue filter, and / or the red filter, and the green color. The stereo imaging device according to claim 1, wherein the stereo imaging device is determined by the image signal of an image formed by each of the second imaging optical systems including a filter and the blue filter.
The first array optical system and the second array optical system include the first imaging optical system and the second imaging optical system including the ND filter,
The exposure amount at the time of imaging is the image signal of the image formed by the first imaging optical system provided with the ND filter and / or the image formed by the second imaging optical system provided with the ND filter. The stereo imaging device according to claim 1, wherein the stereo imaging device is determined by the image signal.
The first array optical system and the second array optical system include the first imaging optical system and the second imaging optical system including the near-infrared filter,
The exposure amount at the time of imaging is determined by the image signal of the image formed by the first imaging optical system including the near-infrared filter and / or the second imaging optical system including the near-infrared filter. The stereo imaging device according to claim 1, wherein the stereo imaging device is determined based on the image signal of the formed image.
The first array optical system and the second array optical system include the first imaging optical system and the second imaging optical system including the polarizing filter,
The exposure amount at the time of imaging is the image signal of the image formed by the first imaging optical system provided with the polarizing filter and / or the image formed by the second imaging optical system provided with the polarizing filter. The stereo imaging device according to claim 1, wherein the stereo imaging device is determined by the image signal.
The said 1st image formation optical system and the said 2nd image formation optical system have at least any one among the said ND filter, the said near-infrared filter, and the said polarization filter, respectively. Stereo imaging device.
The first array optical system and the second array optical system include the first imaging optical system and the second imaging optical system including the polarizing filter having transmission axes different from each other,
The image signals of the two images formed by the first imaging optical system having the polarizing filters having different transmission axes, or the second imaging optical system having the polarizing filters having different transmission axes. The image signal of two formed images is compared, and when both signals have a luminance difference equal to or greater than a threshold value, the region having the luminance difference is determined as a reflector. Stereo imaging device.
The stereo imaging according to claim 9, wherein when the processing unit detects the reflector, distance measurement is not performed on the subject, or even if distance measurement is performed on the subject, stereo imaging is not used. apparatus.
When it is determined as the reflector, the boundary portion of the reflector is extracted, and the image signal of the image formed by the first imaging optical system having the same type of filter and the second imaging optical system are used. The stereo imaging device according to claim 9 or 10, wherein three-dimensional information of the boundary portion is obtained based on the image signal of the formed image, and the three-dimensional information of the boundary portion is used as the three-dimensional information of the reflector.
The first array optical system and the second array optical system include the first imaging optical system and the second imaging optical system including the near-infrared filter,
The image signal of the image formed by the first imaging optical system including the near infrared filter and / or the image of the image formed by the second imaging optical system including the near infrared filter. The stereo imaging device according to claim 1, wherein the subject is determined by extracting a region having a luminance value lower than a threshold value from the signal.
The stereo imaging device according to claim 12, wherein the subject is an overhead line.
The first array optical system and the second array optical system include the first imaging optical system including at least one of the red filter, the green filter, and the blue filter and the near infrared filter, and the first array optical system. Having two imaging optics,
The image signal of the image formed by the first imaging optical system and / or the red filter, comprising at least one of the red filter, the green filter, and the blue filter and the near-infrared filter, 14. The subject is discriminated by comparing the image signals of the image formed by the second imaging optical system including at least one of a green filter and the blue filter and the near-infrared filter. The stereo imaging device according to any one of the above.
The stereo imaging device according to claim 14, wherein the subject is a plant.
The first array optical system and the second array optical system include the first imaging optical system and the first array optical system each including at least two of the non-optical correction filter and the plurality of ND filters having different attenuation factors. Having two imaging optics,
The image signal of the image formed by the first imaging optical system including at least two of the ND filters having different optical correction and the plurality of ND filters having different attenuation factors, the filter without optical correction, and the attenuation rate An image having the highest contrast is obtained by individually acquiring the image contrast based on the image signal of the image formed by the second imaging optical system including at least two of the plurality of ND filters having different values. The stereo imaging device according to claim 1, wherein three-dimensional information of a subject is obtained using the image signal.
The first array optical system and the second array optical system include the first imaging optical system and the first array optical system each including at least two of the non-optical correction filter and the plurality of ND filters having different attenuation factors. Having two imaging optics,
The image signal of the image formed by the first imaging optical system including at least two of the ND filters having different optical correction and the plurality of ND filters having different attenuation factors, the filter without optical correction, and the attenuation rate Based on the image signal of the image formed by the second imaging optical system having at least two of the plurality of ND filters with different values, images having a wide dynamic range were individually created. The stereo imaging device according to any one of claims 1 to 16, wherein three-dimensional information of a subject is obtained using an image signal of an image having a wide dynamic range.
The stereo imaging device has a processing unit,
The processing unit is formed by the image signal of the image formed by the first imaging optical system and the second imaging optical system formed by the first imaging optical system including the same type of filter in the first imaging device and the second imaging device. After calculating the three-dimensional information of the subject from the parallax information of the image signal of the captured image, the first imaging optical system provided with a predetermined type of filter in the first imaging device and the second imaging device. The usable subject region is selected from the calculated three-dimensional information of the subject based on the image signal of the formed image and / or the image signal of the image formed by the second imaging optical system. The stereo imaging device according to any one of 1 to 17.
The stereo imaging device has a processing unit,
The processing unit includes the image signal and / or the second imaging optics of an image formed by the first imaging optical system including a predetermined type of filter in the first imaging device and the second imaging device. Based on the image signal of the image formed by the system, select a usable subject area,
The image signal of the image formed by the first imaging optical system including the same type of filter in the first imaging device and the second imaging device, and the image of the image formed by the second imaging optical system The stereo imaging device according to claim 1, wherein three-dimensional information of a usable subject area is calculated from parallax information of an image signal.
A stereo imaging device that transmits an image obtained by imaging by the stereo imaging device according to any one of claims 1 to 19 to a device other than the stereo device.
A moving body comprising the stereo imaging device according to any one of claims 1 to 19.
The moving body has a movement control unit that controls movement of the moving body,
The image signals of the two images formed by the first imaging optical system having the polarizing filters having different transmission axes, or the second imaging optical system having the polarizing filters having different transmission axes. The moving body according to claim 21, wherein the image signals of the two formed images are compared, and when there is a luminance difference equal to or greater than a threshold value, the moving body is temporarily stopped.
A third array optical system having a plurality of third imaging optical systems arranged with different optical axes and an image formed by each of the plurality of third imaging optical systems are photoelectrically converted and an image signal is output. A third imaging device having a third solid-state imaging device;
A fourth array optical system having a plurality of fourth imaging optical systems arranged with different optical axes and an image formed by each of the plurality of fourth imaging optical systems are photoelectrically converted and an image signal is output. A fourth imaging device having a fourth solid-state imaging device;
The optical axes of the third imaging optical system and the fourth imaging optical system are parallel, and the third imaging device and the fourth imaging device are arranged apart from each other in a direction orthogonal to the optical axis, and The moving body according to claim 21 or 22, wherein the moving body is arranged so that a direction different from an imaging direction of the first imaging device and the second imaging device can be imaged.