WO2021176819A1

WO2021176819A1 - Processing device

Info

Publication number: WO2021176819A1
Application number: PCT/JP2020/048694
Authority: WO
Inventors: 真穂堀永; 直也多田; 裕史大塚
Original assignee: 日立Astemo株式会社
Priority date: 2020-03-06
Filing date: 2020-12-25
Publication date: 2021-09-10
Also published as: JPWO2021176819A1; DE112020005059T5; JP7250211B2

Abstract

This processing device can determine the necessity of controlling a control target even when the optical axis of a stereo camera shifts. This processing device, for processing a pair of captured images captured with a pair of cameras, performs processing for generating a first parallax image from the pair of captured images, recognizing a control target candidate from the first parallax image, acquiring a first parallax value inside of a candidate region where the control target candidate is present in the first parallax image, shifting the relative vertical position of the pair of captured images to generate a second parallax image, acquiring a second parallax value inside of the corresponding region of the second parallax image that corresponds to the candidate region of the first parallax image, and using the first parallax value and the second parallax value to determine whether or not to identify the control target candidate as the control target.

Description

Processing equipment

The present invention relates to, for example, a processing device that processes an captured image captured by a stereo camera mounted on a vehicle.

Conventionally, there has been known a technique for estimating which part of a road marking is imaged from a plurality of images captured as the vehicle moves (Patent Document 1).

JP-A-2010-107435

In a stereo camera, a relative vertical deviation occurs between the optical axis of the left camera and the optical axis of the right camera, and the stereo camera is repeatedly obliquely oblique toward the depth direction such as a flow guide zone (zebra zone). When the road surface on which the lines are painted is imaged, the area on which the road surface paint is written may be erroneously detected as a three-dimensional object. If the area where the road surface paint is written is falsely detected as a three-dimensional object, the safety system provided by the stereo camera such as AEB or ACC may be activated, and unnecessary warnings or brakes may be applied, giving the occupant a sense of discomfort. ..

The above-mentioned conventional method is for correcting the vertical deviation between the left and right cameras, and time is required for the correction process. If the vehicle travels on a road surface with a guide zone before the correction of the vertical deviation is completed, the guide zone may be erroneously detected as a three-dimensional object. Therefore, during the vertical deviation correction process, a mechanism such as stopping the system provided by the stereo camera is required.

However, if the system itself provided by the stereo camera is completely stopped, the safety system may not operate in the actual necessary situation, which may lead to a decrease in safety performance.

An object of the present invention is to differentiate a three-dimensional object from a non-three-dimensional object regardless of whether or not a vertical optical axis shift occurs between the left and right cameras of the stereo camera, and to make the stereo camera recognize it as a three-dimensional object. On the other hand, it is to provide a processing device for determining whether or not to control.

The processing device of the present invention that solves the above problems is a processing device that processes a pair of captured images captured by a pair of cameras, and generates a first parallax image from the pair of captured images. A unit, a control target candidate recognition unit that recognizes a control target candidate from the first parallax image, and a first discriminant value that acquires a first discriminant value in a candidate region in which the control target candidate exists in the first parallax image. The acquisition unit, the second disparity image generation unit that generates the second disparity image by shifting the relative vertical positions of the pair of captured images, and the second disparity corresponding to the candidate region of the first disparity image. Whether or not to certify the control target candidate as a control target by using the second parallax value acquisition unit that acquires the second parallax value in the corresponding region of the image and the first parallax value and the second parallax value. It is characterized by having a control target determination unit for determining.

According to the present invention, it is possible to determine whether or not a stereo camera recognizes a three-dimensional object as a control target. Further features relating to the present invention will become apparent from the description herein and the accompanying drawings. In addition, problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

The block diagram which shows the whole structure of the image pickup apparatus in 1st Embodiment. The flowchart explaining the control target determination process of the image pickup apparatus in 1st Embodiment. The figure which shows typically the left-right captured image and the parallax image. The figure which shows an example of the recognition area of a controlled object. The figure which shows the difference between the distant parallax and the near parallax in the left and right captured images. The figure explaining the reason why the parallax error differs depending on the inclination angle of the line imaged in an image. The figure explaining the parallax when the optical axis of the left and right cameras is relatively displaced in the vertical direction, and the parallax is imaged by the stereo camera. The figure which shows the relationship between the shift amount and the parallax average value. The figure explaining the example which makes the necessity judgment of a control target based on a statistical result. The flowchart explaining the control target determination process of the image pickup apparatus in 2nd Embodiment.

First, the reason why the stereo camera erroneously detects the headrace zone as a three-dimensional object will be described.
FIG. 3 is a diagram schematically showing a left and right captured image and a parallax image. FIG. 3A is a schematic view of a captured image of the front of the vehicle captured by the left camera, and FIG. 3B shows a schematic view of the captured image of the front of the vehicle captured by the right camera at the same time as the left camera. 3 (c) is a parallax image created by using the captured image of FIG. 3 (a) and the captured image of FIG. 3 (b). An intermediate region 322 is shown.

As shown in FIGS. 3A and 3B, the images captured by the left and right cameras capture the state in which the preceding vehicle 311 is traveling in front of the own vehicle. The own vehicle is traveling in the traveling lane 312, and guide zones 313 are provided on both sides of the traveling lane 312.

The stereo camera detects a three-dimensional object from the parallax image as shown in FIG. 3C and recognizes it as a control target candidate. In the absence of a three-dimensional object, the parallax is shown to gradually decrease as it shifts from the vicinity to the distance, and also in the parallax image as it gradually decreases as it shifts from the lower to the upper in the vertical direction. On the other hand, when there is a three-dimensional object, the parallax is the same in the vicinity and the distance, and the parallax image is also shown to have the same value along the vertical direction. Therefore, in the parallax image, it is determined that there is a three-dimensional object in a region having the same parallax along the vertical direction. In the parallax image shown in FIG. 3C, the preceding vehicle 311 which is actually a three-dimensional object is detected as a three-dimensional object 331, but a part of the diversion zone 313 which is not actually a three-dimensional object is also a three-dimensional object 332. Has been detected as.

FIG. 4 is a diagram showing an example of a recognition area of the control target candidate.
In the example shown in FIG. 4, the candidate area recognized as the control target candidate is shown as a rectangular frame 402. The position of the rectangular frame 402 indicating the candidate region is defined by the xy coordinates of the parallax image 401.

FIG. 5 is a diagram showing the difference between the distant parallax and the near parallax in the left and right captured images.
Two preceding

vehicles

511 and 521 are imaged on both the image 501 captured by the left camera and the image 502 captured by the right camera. Due to the characteristics of the stereo camera, as shown in FIG. 5, the parallax with respect to the distant preceding vehicle 521 is small, and the parallax with respect to the nearby preceding vehicle 511 is large.

FIG. 6 is a diagram for explaining the reason why the parallax error differs depending on the inclination angle of the line captured in the image. FIG. 6A is an image of a line extending straight along the traveling direction of the own vehicle, and a vertical line 611 extending straight along the vertical direction of the image is imaged. 6 (b) and 6 (c) are images of lines extending diagonally with respect to the traveling direction of the own vehicle, and the image of FIG. 6 (b) is inclined with respect to the lateral direction of the image. The diagonal line 612 with a large size is imaged, and the diagonal line 613 with a small inclination with respect to the lateral direction of the image is imaged in the image of FIG. 6 (c).

For example, when the lines shown in FIGS. 6 (a) to 6 (c) are imaged in a state where the optical axis of the left camera L is shifted upward from the optical axis of the right camera R, FIG. 6 (a) shows. The vertical lines 611 of the image 601 shown have the same positions P1 and P2 on the left and right when the image is taken by the left camera and the positions P2 when the image is taken by the right camera, and the deviation error is 0. On the other hand, in the diagonal line 612 of the image 602 shown in FIG. 6B, an error δ1 occurs in the left-right direction between the position P3 when the image is taken by the left camera and the position P4 when the image is taken by the right camera. .. Then, in the case of the diagonal line 613 of the image 603 shown in FIG. 6C, there is an error δ2 in the left-right direction between the position P5 when the image is taken by the left camera L and the position P6 when the image is taken by the right camera R. Occurs.

Comparing the error δ1 and the error δ2, the diagonal line 613 of the image 603 shown in FIG. 6 (c) has a larger inclination than the diagonal line 612 of the image 602 shown in FIG. However, the error δ2 is larger (δ1 <δ2). That is, when the optical axis of the left camera L and the optical axis of the right camera R are vertically displaced from each other in the vertical direction, the error increases as the inclination of the imaged line increases.

When the optical axis of the left camera L and the optical axis of the right camera R are completely parallel and do not deviate relatively in the vertical direction, when the image is taken with the left camera and when the image is taken with the right camera. The positions in the left-right direction are the same, and no discrepancy error occurs. The parallax error as shown in FIGS. 6 (b) and 6 (c) occurs only when the optical axes of the left and right cameras are relatively displaced in the vertical direction.

FIG. 7 is a diagram for explaining the parallax when the diversion zone is imaged by a stereo camera in which the optical axes of the left and right cameras are relatively displaced in the vertical direction. When the guide zone 702 is imaged from the stereo camera 701, the white line of the guide zone 702 is imaged so that the inclination is larger in the vicinity and gradually decreases as the distance is increased. Therefore, when the optical axes of the left and right cameras are relatively displaced in the vertical direction, the parallax error 721 to 723 in the left-right direction of the white line shifts from the vicinity to the distance as described with reference to FIG. Therefore, it gradually increases. On the other hand, the parallax 711 to 713 of the white line gradually decreases as the distance from the vicinity to the distance increases, as described with reference to FIG. 5, regardless of the deviation of the optical axis.

Therefore, when the optical axes of the left and right cameras are relatively offset in the vertical direction, the measured parallax in the vicinity is the sum of the relatively large correct parallax 711 and the relatively small parallax error 721, and the distant measurement. The parallax is the sum of a relatively small correct parallax 713 and a relatively large parallax error 723, and the measured parallax between near and far is the middle size correct parallax 712 and the middle size parallax error 722. And are added. Therefore, the measured parallax 703 is equal to each other in the vicinity, the distance, and the middle thereof, and there is a region in the parallax image in which the parallax is the same in the vertical direction, and there is a risk of erroneously detecting that there is a three-dimensional object.

In response to such a problem, the imaging device of the present embodiment differentiates a three-dimensional object from a non-three-dimensional object regardless of whether or not a vertical optical axis shift occurs between the left and right cameras of the stereo camera. A process of determining whether or not a stereo camera recognizes a three-dimensional object as a control target is performed.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
The imaging device of the present embodiment calculates the parallax average value when the pair of captured images captured by the pair of imaging units are not shifted in the vertical direction and the parallax average value when the pair of captured images are relatively shifted in the vertical direction, and shifts the images. It has a configuration in which the control necessity judgment of the control target is performed from the distribution shape of the amount and the parallax average value.

<First Embodiment>
FIG. 1 is a block diagram showing the overall configuration of the image pickup apparatus according to the first embodiment.
The image pickup device 100 is mounted on a vehicle such as an automobile, and images the front from the vehicle with a pair of left and right cameras, determines the presence or absence of a three-dimensional object based on parallax, or certifies the type of the three-dimensional object. It is a stereo camera that performs processing.

The image pickup device 100 includes a first image pickup unit 11 and a second image pickup section 12 which are a pair of cameras, and a processing device 1 which processes two captured images captured by the first image pickup section 11 and the second image pickup section 12. It has. In the present embodiment, the case where the processing device 1 is configured inside the imaging device 100, which is a stereo camera, will be described as an example, but the place where the processing device 1 is configured is not limited to the inside of the imaging device 100. Instead, it may be configured by an ECU or the like provided separately from the image pickup apparatus 100.

The first imaging unit 11 and the second imaging unit 12 are arranged at positions separated from each other in the vehicle interior of the vehicle in the vehicle width direction, and pass through the windshield to image an overlapping region in front of the vehicle. The first image pickup unit 11 and the second image pickup unit 12 are composed of an assembly in which, for example, an optical system component such as a lens and an image pickup element such as a CCD or CMOS are combined. The first imaging unit 11 and the second imaging unit 12 are adjusted so that their optical axes are parallel to each other and have the same height. The captured images captured by the first imaging unit 11 and the second imaging unit 12 are input to the processing device 1.

The processing device 1 is composed of hardware having, for example, a CPU and memory, and software installed and executed on the hardware. The processing device 1 has, as internal functions, a first image acquisition unit 13, a second image acquisition unit 14, a first parallax image generation unit 15, a control target candidate recognition unit 16, and a first parallax value acquisition unit 17. A second parallax image generation unit 18, a second parallax value acquisition unit 19, and a control target determination unit 20 are provided.

The first image acquisition unit 13 and the second image acquisition unit 14 acquire captured images that are simultaneously and periodically captured by the first imaging unit 11 and the second imaging unit 12. The first image acquisition unit 13 and the second image acquisition unit 14 cut out images of predetermined regions that overlap each other from a pair of captured images simultaneously captured by the first imaging unit 11 and the second imaging unit 12, respectively, and the first image. And get as a second image.

The first parallax image generation unit 15 generates a first parallax image using a pair of captured images acquired by the first image acquisition unit 13 and the second image acquisition unit 14. As a method for generating the first parallax image, a conventionally known method can be used. For example, with the first image as a reference, in the second image, a pixel array at the same height position in the vertical direction as the first image is scanned in the horizontal direction to find a coincidence point with the first image, and the first image and the second image are used. So-called stereo matching is performed, in which the amount of lateral deviation from the image is calculated as the parallax.

The control target candidate recognition unit 16 detects a three-dimensional object from the first parallax image generated by the first parallax image generation unit 15, and recognizes the three-dimensional object as a control target candidate. For example, a three-dimensional object exists in a region where the amount of change in parallax in the vertical direction of the image is small compared to the amount of change in parallax in the vertical direction of an image in which a flat surface continues without a three-dimensional object. Judge that you are doing. When a plurality of three-dimensional objects are detected from the first parallax image, the control target candidate recognition unit 16 recognizes each of them as a control target candidate. The control target candidate recognition unit 16 acquires the coordinate information of the candidate region in which the control target candidate is recognized in the first parallax image (for example, the

regions

331 and 332 in FIG. 3 and the region 402 in FIG. 4).

The first parallax value acquisition unit 17 acquires the first parallax value, which is the parallax calculation value in the candidate region where the control target candidate exists in the first parallax image. The first parallax value acquisition unit 17 executes a process of acquiring the first parallax value only when the control target candidate is recognized by the control target candidate recognition unit 16. The first parallax value acquisition unit 17 does not perform the process of acquiring the first parallax value when no control target candidate is recognized by the control target candidate recognition unit 16. For the first parallax value, which is the calculated parallax value, for example, the parallax average value in the candidate region can be used. The parallax average value in the candidate region is calculated from the parallax value and the number of parallax in the candidate region. The calculated parallax value is not limited to the average parallax value, but is the dispersion value of parallax, the maximum value of parallax, the minimum value of parallax, the difference between the maximum and minimum values of parallax, and the maximum and minimum values of parallax. The ratio of values, the average value of the maximum values of parallax, or the average value of the minimum values of parallax can be used.

The second parallax image generation unit 18 generates a plurality of second parallax images using the pair of captured images used when the first parallax image was generated. The second parallax image generation unit 18 uses a pair of shifted images cut out by shifting the relative vertical positions of the first image and the second image for each vertical coordinate or for each equally divided division. Generate a parallax image. The second parallax image generation unit 18 cuts out a pair of shifted images by shifting at least one position of the first image and the second image only in the upward direction, only in the downward direction, or in both the upper and lower directions.

The second parallax image generation unit 18 changes the amount of shift in the vertical direction to generate a plurality of pairs of shift images, and uses these to generate a plurality of second parallax images. That is, the second parallax image generation unit 18 shifts at least one position of the first image and the second image a plurality of times to generate a plurality of a pair of shift images, and generates a second parallax image from each of the pair of shift images. Generate.

The second parallax value acquisition unit 19 acquires the second parallax value, which is the calculated parallax value in the corresponding area, from the corresponding area of the second parallax image. The corresponding area of the second parallax image is set at the same position as the candidate area of the first parallax image. Since a plurality of second parallax images are generated, a corresponding region is set at the same position as the candidate region of the first parallax image for each second parallax image, and the second parallax value is obtained from each corresponding region. Further, when there are a plurality of candidate regions of the first parallax image, the corresponding regions of the second parallax image are set corresponding to each, and the second parallax value of each corresponding region is obtained.

As the second parallax value, for example, the parallax average value in the corresponding region can be used in the same manner as the first parallax value. The average parallax value in the corresponding area is calculated from the parallax value and the number of parallaxes in the corresponding area.
In addition, the parallax dispersion value, the maximum parallax value, the minimum parallax value, the difference between the maximum and minimum values of parallax, and the maximum and minimum values of parallax so that they can be compared as the same calculated value as the first parallax value. The average value of the ratio, the maximum value of parallax, or the average value of the minimum value of parallax can be used. The second parallax value is calculated for each of the plurality of second parallax images.

The control target determination unit 20 uses the first parallax image generated by the first parallax image generation unit 15 and the plurality of second parallax images generated by the second parallax image generation unit 18, and the necessity of the control target, that is, , It is determined whether or not the control target candidate detected as a three-dimensional object is recognized as a control target. The control target determination unit 20 determines the necessity of the control target based on the parallax calculation values calculated for each of the first parallax image and the plurality of second parallax images.

FIG. 2 is a flowchart illustrating a control target determination process of the image pickup apparatus according to the first embodiment.

In step S201, image acquisition is performed. Here, a pair of captured images simultaneously captured by the first imaging unit 11 and the second imaging unit 12 are acquired by the first image acquisition unit 13 and the second image acquisition unit 14. Then, in step S202, the first parallax image is generated. The first parallax image is generated by the first parallax image generation unit 15 using a pair of captured images acquired by the first image acquisition unit 13 and the second image acquisition unit 14.

In step S203, it is determined whether or not the control target candidate exists. Whether or not there is a control target candidate is determined depending on whether or not a three-dimensional object is detected from the first parallax image. Here, when at least one or more three-dimensional objects are detected, it is determined that the control target candidate exists in front of the own vehicle (YES in step S203), and the process proceeds to step S204.
On the other hand, if no three-dimensional object is detected, it is determined that there is no control target candidate in front of the own vehicle (NO in step S203), and the process returns to step S201. That is, in step S203, the three-dimensional object and the non-three-dimensional object are differentiated, and only the object recognized as the three-dimensional object is subjected to the process of determining whether or not it is to be controlled in step S204 or later.

In step S204, a process of shifting the image in the vertical direction is performed. Here, from the pair of captured images used when the first parallax image was generated, the first image and the second image are shifted by relative vertical coordinates or by equally divided divisions to obtain a pair of shifted images. The process of acquiring is performed.

In step S205, the parallax is calculated by stereo matching using the pair of shifted images, and the second parallax image is generated. In step S206, it is determined whether or not there are a plurality of parallax images. Then, when the number of the second parallax images is less than the preset number of two or more, it is determined that there are no plurality of parallax images (NO in step S206), and the process returns to step S204. Then, in step S204 and step S205, the vertical shift amount of the pair of images is changed to generate another pair of shift images having different shift amounts, and further another second parallax image is generated. Then, until it is determined in step S206 that there are a predetermined number or more of the second parallax images (YES in step S206), a process of generating a second parallax image from the pair of images in which the shift amount is changed is performed.

The amount of shift can be, for example, one pixel or a plurality of pixels. Further, the shifting direction may be, for example, at least one of the upper and lower sides of the second image with respect to the first image, and in the present embodiment, the second image is shifted both upward and lower with respect to the first image. Each of them generates a pair of shifted images, and a second parallax image is generated from each of the plurality of pairs of shifted images.

In step S207, a distribution of parallax average values is created. Here, the first parallax value of the candidate region in the first parallax image and the second parallax value of the corresponding region in the plurality of second parallax images are calculated, and an approximate straight line is obtained from the distribution of these parallax values.

FIG. 8 is a diagram showing the relationship between the amount of shift and the average parallax value. FIG. 8 (a) shows a state in which the parallax average value is substantially constant and the slope of the approximate straight line of the distribution is zero regardless of the amount of shift, and FIG. 8 (b) shows the state according to the amount of shift. It is a figure which shows the state which the parallax average value changes, and the slope of the approximate straight line of a distribution is equal to or more than a threshold.

In step S208, the slope of the approximate straight line of the distribution obtained in step S207 is compared with the preset threshold value. When the control target candidate is a three-dimensional object that is correctly recognized, the parallax value in the recognition area of the control target candidate does not change due to the vertical deviation, so that the distribution is an approximate straight line with zero slope. On the other hand, when the slope of the approximate straight line of the distribution is larger than the threshold value, it is determined that the parallax value of the corresponding region has changed due to the vertical deviation between the first image and the plurality of second images.

If the control target candidate recognized as a three-dimensional object in the first parallax image by the control target candidate recognition unit 16 is a true three-dimensional object such as a preceding vehicle, a bicycle, or a pedestrian, it is affected by the presence or absence of vertical deviation. The parallax value does not change between the candidate area to be controlled and the corresponding area. On the other hand, if the control target candidate recognized as a three-dimensional object in the first parallax image by the control target candidate recognition unit 16 is a non-three-dimensional object such as a headrace zone (zebra zone) on the road surface, the parallax value of the candidate region is used. The parallax value of the corresponding region has a characteristic that it changes in comparison with the presence or absence of vertical deviation.

In step S209, the control target candidate whose slope of the approximate straight line of the distribution is determined to be larger than the threshold value in the comparison result in step S208 is regarded as a control-unnecessary target, and a process of excluding it from the control target is executed. Then, the control target candidate that remains without being excluded from the control target candidates is determined to be a control target that requires control, and the control target information is output.

For example, as shown in FIG. 8A, when the parallax average value is a substantially constant value, that is, the parallax calculation value of the candidate region is irrespective of the relative vertical shift amount of the first image and the second image. When the parallax calculation values of the plurality of corresponding areas are almost the same, the control target candidate is recognized as a control target.
On the other hand, as shown in FIG. 8B, when the parallax average values are different from each other, that is, the area corresponding to the parallax average value of the candidate region according to the relative vertical shift amount of the first image and the second image. When the parallax average value of is increased or decreased, it is determined that the control target candidate is not a three-dimensional object but a erroneous detection of the headrace zone as a three-dimensional object, and processing is performed to exclude it from the control target.

An example of creating a distribution of the parallax average value in step S207 and determining whether to exclude the control target candidate from the control target based on the slope of the approximate straight line of the distribution has been described in step S208, but the present invention is limited to this. It's not something. For example, as another modification, instead of the slope of the approximate straight line of the distribution, the parallax dispersion value, the maximum parallax value, the minimum parallax value, the difference between the maximum and minimum parallax values, and the maximum and minimum parallax values. The ratio of values, the average value of the maximum values of parallax, or the average value of the minimum values of parallax can be used.

FIG. 9 is a diagram illustrating an example of determining the necessity of a controlled object based on statistical results.
In the example shown in FIG. 9, the average value of the deviations 1 to 6 which are the six types of shift images in the case of the positive detection and the false detection, and the average value (ave) and the dispersion value (sigma) of these deviation average values. , Minimum value (min), maximum value (max), difference between maximum value and minimum value (max-min), average value of minimum value (min / ave), and average value of maximum value (max / ave) Is shown, and it is also possible to judge the necessity of the control target by comparing these values with the threshold value.

The imaging device 100 of the present embodiment acquires a first image and a second image from a pair of captured images, generates a first parallax image using the first image and the second image, and a three-dimensional object from the first parallax image. Is detected and recognized as a control target candidate, and the parallax value in the candidate area where the control target candidate exists is acquired as the first parallax value. Then, from the pair of captured images obtained from the first image and the second image, a pair of shifted images in which the positional relationship in the vertical direction relative to the first image and the second image is shifted is acquired, and the pair of these shifted images is obtained. A second disparity image is generated from the shifted image, a corresponding area corresponding to the candidate area of the first disparity image is set in the second disparity image, and the disparity value in the corresponding area is acquired as the second disparity value.

Then, from the pair of captured images obtained from the first image and the second image, a pair of shifted images in which the shift amount is changed are acquired, a second parallax image is generated, a corresponding area is set, and a second parallax in the corresponding area is obtained. The value is acquired, and a plurality of second parallax values having different shift amounts are acquired. Then, it is determined from the distribution status of the first parallax value and the plurality of second parallax values whether or not the control target candidate erroneously detects the headrace zone as a three-dimensional object, and when it is determined to be erroneous detection. Is processed to be excluded from the control target. For example, when the first parallax value and the plurality of second parallax values are almost the same value, it is determined that the detection is positive, and the plurality of second parallax values change according to the amount of shift with respect to the first parallax value. If this is the case, it is judged to be a false positive.

According to the image pickup apparatus 100 of the present embodiment, it is possible to determine whether or not control of the control target is necessary even when the left and right cameras have optical axis deviations. Therefore, the headrace zone can be excluded from the control target as a three-dimensional object, and it is possible to prevent the safety system such as AEB and ACC from operating and applying unnecessary alarms and brakes. It is possible to prevent a person from feeling uncomfortable.

<Second Embodiment>
What is characteristic of this embodiment is that a three-dimensional object having a characteristic of a repeating pattern such as a fence or a fence is recognized as a control target. For example, even for a three-dimensional object having a characteristic of a repeating pattern such as a fence or a fence, the parallax value in the corresponding region of the second parallax image obtained in step S205 changes with the occurrence of vertical deviation of the pair of cameras. Have.

However, since fences and fences are control targets that need to be controlled by a safety system, processing is required to prevent fences and fences from being excluded from control targets. Hereinafter, the flow of processing for not excluding fences and fences from the control target will be described based on the flowchart of FIG. Except for step S210, the process is the same as the flowchart shown in FIG.

When it is determined in step S203 that the control target candidate exists, it is determined in step S210 whether or not the own vehicle speed is larger than the threshold value. It can be determined that the fence or fence is recognized as the control target when the fence or fence is located on the traveling path of the own vehicle, and it can be inferred that the own vehicle is traveling at a low speed. Therefore, only when the own vehicle speed is greater than the preset vehicle speed threshold value (YES in step S210), the process proceeds to step S204, and when the own vehicle speed on which the image pickup device 100 is mounted is equal to or less than the vehicle speed threshold value (in step S210). NO) does not generate a second parallax image.

According to the present invention, when the headrace zone is erroneously detected as a three-dimensional object due to the vertical deviation of the optical axis generated between the left and right cameras, it is determined that the light guide zone is not a three-dimensional object, and the safety system is prevented from operating. It is possible.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various designs are designed without departing from the spirit of the present invention described in the claims. You can make changes. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace other configurations with respect to a part of the configurations of each embodiment.

100 Imaging device 11 1st imaging unit 12 2nd imaging unit 13 1st image acquisition unit 14 2nd image acquisition unit 15 1st parallax image generation unit 16 Control target candidate recognition unit 17 1st parallax value acquisition unit 18 2nd parallax image Generation unit 19 Second parallax value acquisition unit 20 Control target judgment unit

Claims

A processing device that processes a pair of captured images captured by a pair of cameras.
A first parallax image generator that generates a first parallax image from the pair of captured images,
A control target candidate recognition unit that recognizes a control target candidate from the first parallax image,
The first parallax value acquisition unit that acquires the first parallax value in the candidate region where the control target candidate exists in the first parallax image, and the first parallax value acquisition unit.
A second parallax image generation unit that generates a second parallax image by shifting the relative vertical positions of the pair of captured images, and
A second parallax value acquisition unit that acquires a second parallax value in the corresponding region of the second parallax image corresponding to the candidate region of the first parallax image, and a second parallax value acquisition unit.
A control target determination unit that determines whether or not to certify the control target candidate as a control target using the first parallax value and the second parallax value.
A processing device characterized by having.
The first parallax image generation unit cuts out a first image and a second image from the pair of captured images, and generates the first parallax image using the first image and the second image.
The second parallax image generation unit is a pair cut out from the pair of captured images by shifting the relative vertical positions of the first image and the second image for each vertical coordinate or for each equally divided division. The processing apparatus according to claim 1, wherein the second parallax image is generated by using the shifted image.
The second parallax image generation unit is characterized in that at least one position of the first image and the second image is shifted only in the upward direction, only in the downward direction, or in both the upper and lower directions to cut out the pair of shifted images. The processing apparatus according to claim 2.
The second parallax image generation unit generates a plurality of pairs of shifted images by shifting at least one position of the first image and the second image a plurality of times, and the second parallax image is generated from each of the pair of shifted images. The processing apparatus according to claim 2, wherein an image is generated.
The control target determination unit obtains the parallax average value in the candidate region from the parallax value and the number of parallax in the candidate region of the first parallax image, and obtains the parallax value and the parallax in the corresponding region of the second parallax image. The parallax average value in the corresponding region is obtained from the number, and the inclination of the distribution between the parallax average value in the candidate region of the first parallax image and the parallax average value in the parallax average value in the corresponding region of the second parallax image is from the threshold value. The processing apparatus according to claim 2, wherein the control target candidate recognized by the control target candidate recognition unit is excluded from the control target when the size is large.
The control target determination unit obtains the parallax average value in the candidate region from the parallax value and the number of parallax in the candidate region of the first parallax image, and obtains the parallax value and the parallax in the corresponding region of the second parallax image. The parallax average value in the corresponding region is obtained from the number, and when the dispersion value of the distribution of the parallax average value is larger than the threshold value, the control target candidate recognized by the control target candidate recognition unit is excluded from the control target. The processing apparatus according to claim 2, wherein the processing apparatus is used.
The processing device according to claim 2, wherein the second parallax image generation unit does not generate a second parallax image when the traveling speed of the vehicle on which the processing device is mounted is equal to or less than a threshold value.