WO2023139675A1

WO2023139675A1 - Imaging device, parallax displacement correction method, and parallax displacement correction program

Info

Publication number: WO2023139675A1
Application number: PCT/JP2022/001734
Authority: WO
Inventors: 直也多田; 真穂堀永; 裕史大塚
Original assignee: 日立Astemo株式会社
Priority date: 2022-01-19
Filing date: 2022-01-19
Publication date: 2023-07-27
Also published as: DE112022005074T5; JPWO2023139675A1

Abstract

Provided is an imaging device that can detect a road surface position with high reliability and that can thereby detect parallax displacement with high accuracy. This imaging device is provided with: a first imaging unit for acquiring a first captured image; and a second imaging unit that is disposed at a position away by a fixed distance from the first imaging unit and that acquires a second captured image. A parallax calculation unit calculates parallaxes in various regions in the first captured image and in various regions in the second captured image. A region selection unit selects, from the first captured image or from the second captured image, regions having a threshold value of reliability or higher as the value of each of the parallaxes. A parallax displacement amount calculation unit makes a comparison between the values of the parallaxes in the respective regions selected by the region selection unit and ideal values of parallaxes to be acquired in the regions, and obtains parallax displacement amounts of the regions. A parallax displacement correction unit corrects parallax displacements in the first imaging unit and the second imaging unit using correction amounts determined in accordance with the parallax displacement amounts of the regions.

Description

Imaging device, parallax deviation correction method, and parallax deviation correction program

The present disclosure relates to an imaging device, a parallax deviation correction method, and a parallax deviation correction program.

A stereo camera is known as a device for recognizing objects in three dimensions. A stereo camera detects the parallax between multiple cameras based on trigonometry by using differences in how images are captured by multiple cameras placed at different positions, and uses the parallax to detect the distance and position from the cameras to an object. A stereo camera is mounted on a vehicle, for example, and used as a camera that constitutes a driving support system.

However, when the optical axis of the stereo camera deviates due to deterioration over time, it becomes impossible to calculate accurate parallax. As a result, the distance from the camera to the object cannot be measured correctly, increasing the possibility that the driving support system will not operate correctly. For this reason, various techniques for calibrating the optical axis (in other words, correcting the parallax deviation) have been proposed in order to accurately calculate the parallax (see, for example, Patent Document 1). Patent Literature 1 discloses a technique of calculating parallax between a position of a road surface near the vehicle (near road surface) and a position of a road surface far away from the vehicle (far road surface position), and correcting the parallax deviation.

JP-A-2017-44573

However, the device disclosed in Patent Document 1 preliminarily sets the road surface position used to calculate the parallax deviation, so there is a limit to the environment for correcting the parallax deviation. Also, since the texture of the road surface is monotonous, the parallax calculation can be unreliable. In particular, the reliability of parallax calculation may be lowered on a blurred road without white lines or various signs. Calculating the amount of parallax deviation using parallax with low reliability may lead to a decrease in the accuracy of correcting the parallax deviation and, in turn, a decrease in the accuracy of distance measurement.

The present disclosure proposes an imaging device, a parallax deviation correction method, and a parallax deviation correction program that can detect parallax deviation with high accuracy.

An imaging device according to one aspect of the present disclosure includes a first imaging unit that acquires a first captured image, a second imaging unit that is arranged at a certain distance from the first imaging unit and acquires a second captured image, a parallax calculation unit that calculates the parallax of each pixel of the first captured image and the second captured image, an area selection unit that selects an area having a reliability equal to or higher than a threshold as the parallax value from the first captured image or the second captured image, and selection by the area selection unit. A parallax shift amount calculation unit that compares a parallax value in the obtained area with an ideal parallax value that should be acquired in the area, and obtains a parallax shift amount of each area;

According to the present invention, it is possible to provide an imaging device, a parallax deviation correction method, and a parallax deviation correction program that can detect parallax deviation with high accuracy.

1 is a schematic diagram illustrating an overall configuration of a stereo camera 100 as an imaging device according to a first embodiment; FIG. 1B is a block diagram illustrating an example of a detailed configuration of the image processing device 2 of FIG. 1A; FIG. 4 is an explanatory diagram illustrating a method of computing a parallax D in a parallax computation unit 13; FIG. 4 is an explanatory diagram illustrating a method of computing a parallax D in a parallax computation unit 13; FIG. FIG. 11 is an explanatory diagram for explaining a method of selecting an area by an area selection unit 14; 4 is an explanatory diagram illustrating a method of computing a parallax D in a parallax computation unit 13; FIG. 4 is an explanatory diagram illustrating a method of computing a parallax D in a parallax computation unit 13; FIG. 4 is an explanatory diagram illustrating a method of calculating a parallax D in a parallax calculator 13 and a method of calculating a parallax shift amount ΔD in a parallax shift amount calculator 15; FIG. 4 is an explanatory diagram illustrating a method of calculating a parallax D in a parallax calculator 13 and a method of calculating a parallax shift amount ΔD in a parallax shift amount calculator 15; FIG. 4 is a flowchart illustrating a specific example of a procedure for calculating a parallax shift correction amount ΔC in the stereo camera 100 of the first embodiment; FIG. 12 is an explanatory diagram for explaining the operation of the area selection unit 14 in the stereo camera 100 of the second embodiment; 10 is a flowchart illustrating a specific example of a procedure for calculating a parallax shift correction amount ΔC in the stereo camera 100 of the second embodiment; FIG. 12 is an explanatory diagram for explaining the operation of the area selection unit 14 in the stereo camera 100 of the second embodiment; It is an explanatory view explaining operation of a 3rd embodiment. It is an explanatory view explaining operation of a 3rd embodiment. It is an explanatory view explaining operation of a 3rd embodiment.

The present embodiment will be described below with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be labeled with the same numbers. Although the attached drawings show embodiments and implementation examples in accordance with the principles of the present disclosure, they are for the purpose of understanding the present disclosure and are not used to interpret the present disclosure in a limited way. The description herein is merely exemplary and is not intended to limit the scope or application of this disclosure in any way.

In the present embodiment, the description is given in sufficient detail for those skilled in the art to implement the present disclosure, but it is necessary to understand that other implementations and forms are possible, and that changes in configuration and structure and replacement of various elements are possible without departing from the scope and spirit of the technical idea of the present disclosure. Therefore, the following description should not be construed as being limited to this.

[First embodiment]
The overall configuration of a stereo camera 100 as an imaging device according to the first embodiment will be described with reference to the schematic diagram of FIG. 1A. This stereo camera 100 is an in-vehicle stereo camera device mounted in a vehicle such as an automobile, and includes a stereo camera main body 1 , an image processing device 2 and an interface 5 . The stereo camera body 1, the image processing device 2, and the interface 5 are connected via a communication line such as a bus BU.

The stereo camera main body 1 is configured by arranging a plurality of, for example, two cameras, a first imaging unit 11 (left camera) and a second imaging unit 12 (right camera), spaced apart by a predetermined base line length B. The first image capturing unit 11 and the second image capturing unit 12 capture images in front of the vehicle from different positions by the base line length B to obtain images. The image processing device 2 can include, for example, a CPU 3 as an arithmetic device and a memory 4 as a storage device. Memory 4 may be RAM, ROM, hard disk drive (HDD), or a combination thereof. Stereo camera 100 can be connected to external device 200 via interface 5 .

The external device 200 may include various sensors for detecting the opening of the vehicle's accelerator (that is, throttle opening), brake operation amount (brake pedal operation amount), steering angle, vehicle speed, acceleration, temperature, humidity, etc., as well as an ECU (Electronic Control Unit) that controls various operations of the vehicle.

The memory 4 is a recording medium in which programs used for various processes in the image processing device 2 (parallax deviation correction program, etc.) and various types of information are stored. The CPU 3 performs predetermined arithmetic processing on signals received via an interface (I/F) 5 according to a control program stored in the memory 4, detects three-dimensional objects and road surfaces, and calculates the position, distance, direction, etc. of objects. The calculation result by the CPU 3 is output to the external device 200 via the interface 5 and used for judgment and control of various operations of the vehicle such as acceleration, braking and steering.

An example of the detailed configuration of the image processing device 2 will be described with reference to the block diagram of FIG. 1B. The image processing device 2 includes, for example, a parallax calculation unit 13, a region selection unit 14, a parallax shift amount calculation unit 15, and a parallax shift correction unit 16 realized by a program. The image processing device 2 calculates the distance Z from the stereo camera 100 to the object using the parallax D calculated by the parallax calculator 13 and corrected by the parallax shift corrector 16 , and supplies the calculation result to the external device 200 .

The parallax calculation unit 13 calculates the parallax D based on the image captured by the first imaging unit 11 and the image captured by the second imaging unit 12 . Specifically, the parallax calculation unit 13 sets, for example, an image captured by the first imaging unit 11 as a reference image, and extracts feature points having a change in gradation in the reference image. Next, the image captured by the other second imaging unit 12 is used as a reference image, and the reference image is searched for the position of the reference point in which the same subject as the feature point extracted in the reference image appears. Template matching such as SAD (Sum of Absolute Difference) can be used for the search. A parallax D is calculated as the difference between the position of the extracted feature point in the reference image and the position in the reference image. The calculated parallax D is calculated for each region containing the extracted feature points, and is temporarily stored in the memory 4 in association with the position of the region. That is, the parallax calculation unit 13 is configured to extract a plurality of feature points from a pair of standard image and reference image, and to calculate a parallax D in each of a plurality of regions including the plurality of extracted feature points.

The area selection unit 14 selects an area from which a highly reliable parallax D has been obtained, from among a plurality of areas from which data on the parallax D in the captured image has been obtained. Since a region where a highly reliable parallax D is obtained is selected and the parallax shift amount ΔD is calculated using the parallax D in the selected region, parallax shift correction can be performed with high accuracy. A method for determining the parallax D with high reliability will be described later. Note that the number of regions to be selected is irrelevant.

The parallax shift amount calculation unit 15 uses the values of the parallax D in the multiple regions selected by the region selection unit 14 to calculate the parallax shift amount ΔD in each region. The parallax shift amount ΔD is calculated for each region as the difference (D−Di) between the parallax D calculated for a certain region in the parallax calculation unit 13 and the parallax ideal value Di to be obtained in the region. Here, the “ideal parallax value Di” is determined by the height from the object of the first imaging unit 11 and the second imaging unit 12 and the angle of view of the camera that captures the area. A specific procedure for calculating the ideal parallax value Di will be described later.

The parallax shift correction unit 16 determines a parallax shift correction value ΔC in the captured image from a plurality of parallax shift amounts ΔD calculated for each of the plurality of selected regions.

Next, with reference to FIGS. 2A to 2C, a method for calculating the parallax D by the parallax calculator 13 and selecting a region having a highly reliable parallax D by the region selector 14 will be described. As shown in FIG. 2A, for example, an image of a road including a white line WL is captured by the first imaging unit 11 and the second imaging unit 12 to obtain a first captured image IM1 and a second captured image IM2. Here, the first captured image IM1 captured by the first imaging unit 11 is used as a reference image, and the second captured image IM2 captured by the second imaging unit 12 is used as a reference image. The parallax calculator 13 extracts a feature point 21 in the first captured image IM1, and extracts a reference point 21' having the same feature as the feature point 21 in the second captured image IM2. Then, the parallax calculator 13 performs SAD template matching between the feature point 21 and the reference point 21', and the similarity SM is calculated, for example, as shown in the graph of FIG. 2B. The horizontal axis of the graph in FIG. 2B indicates the horizontal coordinate of the second captured image IM2, and the vertical axis indicates the SAD value (similarity SM).

Regarding the feature point 21 in the first captured image IM1 (reference image), if the similarity SM is calculated at the position of the reference point 21' where the same object as the object captured in the feature point 21 is reflected in the second captured image IM2 (reference image), the similarity SM will be a small value. On the other hand, with respect to the feature point 21 of the first captured image IM1 (reference image), if the similarity SM is calculated at a position (a position other than the reference point) where an object that is not the same as the object captured in the feature point 21 is captured in the second captured image IM2 (reference image), the similarity SM becomes a high value. Here, the parallax D of the feature point 21 is the coordinate value on the horizontal axis of the position (first peak P1) where the similarity SM becomes the minimum value.

When the amount of protrusion of the first peak P1 (the amount of decrease in the similarity SM at the first peak P1) is k, it can be said that the larger the value of the amount of protrusion k, the larger the feature amount (for example, the difference in brightness) of the feature point 21 with respect to the adjacent area. The larger the luminance difference or the like with respect to the adjacent area, the larger the calculated protrusion amount k, and the higher the reliability of the parallax D calculated in that area. Therefore, the average value Av of the similarity SM is calculated in an area excluding the vicinity of the first peak P1 (the area indicated by the dashed line in the figure), and this is used as a reference value, and the difference between the average value Av (reference value) and the reliability SM of the minimum value of the first peak P1 is calculated as the protrusion amount k. The value of the protrusion amount k is calculated for a plurality of regions, and the region selection unit 14 selects a plurality of regions with high values of k. For example, as shown in FIG. 2C, a protrusion amount k is obtained for each of a plurality of areas (given with a unique area ID), the areas are sorted in descending order of the protrusion amount k, and a predetermined number of areas are selected by the area selection unit 14 in descending order of the protrusion amount k. As will be described later, a threshold THk can be set for the protrusion amount k, and the parallax shift correction operation can be stopped when the number of regions in which the protrusion amount k exceeds the threshold THk does not reach the minimum value Nreg.

Note that in FIGS. 2A and 2B, the protrusion amount k is calculated from the difference between the average value Av of the similarity SM (the value near the first peak P1 is excluded) and the lower limit value of the first peak P1, but as shown in FIGS. It may be calculated as the protrusion amount k.

Next, a method for calculating the parallax D in the parallax calculator 13 and a method for calculating the parallax shift amount ΔD in the parallax shift amount calculator 15 will be described with reference to FIG. 4A. Let f (mm) be the focal length of the lenses L of the first imaging unit 11 and the second imaging unit 12, δ (mm/pixel) be the pixel pitch of the imaging elements (e.g., CMOS sensors) of the first imaging unit 11 and the second imaging unit 12, J (pixel) be the position where the road surface is projected onto the imaging surface at a distance Z (mm) in the horizontal direction from the lens L, and h (mm) be the height of the lens L from the road surface.

[Number 1]
Z=(f×h)/(J×δ) (1)

In addition, the parallax D (pixel) at the distance Z (mm) satisfies the following equation (2), where B (mm) is the distance (base line length) between the first imaging unit 11 and the second imaging unit 12.

[Number 2]
D=(B×f)/(Z×δ) (2)

Substituting formula (1) into formula (2) gives the following formula (3).

[Number 3]
D=B×J/h (3)

In this way, the parallax D (pixels) of the road surface can be calculated from the baseline length B (mm) and height h of the stereo camera 100, and the coordinates J (pixels) of the road surface on the imaging plane. Since the base length B and the height h (mm) are values that can be specified when the first imaging unit 11 and the second imaging unit 12 are mounted on the vehicle, the parallax D (mm) can be uniquely specified by the coordinates J (pixel) of the road surface.

The ideal value Di of the parallax D of the road surface can be calculated based on the value of the base line length B and the value of the height h when the stereo camera 100 is mounted on the vehicle. Here, instead of obtaining the ideal parallax value Di on the road surface, as shown in FIG. 4B, for example, the ideal parallax value Di may be obtained for an object Ob (for example, a curbstone, a guardrail, etc.) whose height from the road surface is known. However, since the road surface is a feature that always exists in a general driving environment, in principle it is preferable to obtain the ideal parallax value Di on the road surface. In principle, the ideal parallax value Di is obtained on the road surface, but in special cases, it is also possible to obtain the ideal parallax value Di on an object other than the road surface.

In the parallax shift amount calculation unit 15, the parallax shift amount calculation unit 13 actually calculates the difference between the parallax D calculated according to the base line length B and height h at that time and the parallax ideal value Di calculated according to the base line length B and height h at the time of design as the parallax shift amount ΔD=D−Di. The parallax shift amount calculator 15 calculates the parallax shift amount ΔD for each of the plurality of road surface areas selected by the area selector 14 .

Next, a specific example of the procedure for calculating the parallax shift correction amount ΔC in the stereo camera 100 of the first embodiment will be described with reference to the flowchart of FIG.

First, the road surface on which the calculation of the parallax deviation correction amount ΔC is based is specified (step S101). Then, in the manner described above, the parallax D on the road surface is calculated by the parallax calculator 13 for each region including the feature point 21 (step S102), and the value of the amount of protrusion k is calculated for each region (step S103).

After the feature amount k is calculated for each area, the area selection unit 14 selects an area where the protrusion amount k is greater than the threshold THk (step S104). If the number of regions Nreg selected in step S104 is equal to or greater than the threshold THreg, the process proceeds to step S106, but if it is smaller than the threshold THreg, calculation of the parallax deviation correction amount ΔC and parallax deviation correction processing are canceled (step S105). This is because the parallax shift amount cannot be calculated with high accuracy if a certain number of regions in which the value of the protrusion amount k is equal to or greater than the threshold value THk cannot be obtained.

In step S106, the parallax shift amount calculator 15 calculates the ideal parallax value Di in each of the plurality of selected regions in the manner described above, and the difference between this ideal value Di and the actual parallax D is calculated as the parallax shift amount ΔD=D−Di (step S107). Then, the parallax shift correction unit 16 statistically processes the parallax shift amounts ΔD obtained in the plurality of regions, calculates the parallax shift amount ΔDfix for the entire image, and calculates the parallax shift correction amount ΔC according to the parallax shift amount ΔDfix (step S108).

As described above, according to the stereo camera 100 of the first embodiment, a region with a high protrusion amount k is selected, and the parallax shift amount ΔD is calculated based on the selected region. Therefore, it is possible to provide an imaging device that can detect parallax shift with high accuracy.

[Second embodiment]
Next, a stereo camera 100 according to a second embodiment will be described with reference to FIGS. 6A to 6C. Since the overall configuration of the stereo camera 100 of the second embodiment is the same as that of the first embodiment (FIGS. 1A and 1B), redundant description will be omitted.

In the above-described first embodiment, the area selection unit 14 calculates the degree of protrusion k (see FIG. 2B) of the similarity SM, and selects a plurality of areas to be used for calculating the parallax shift amount ΔD according to the degree of protrusion k. On the other hand, in the second embodiment, instead of obtaining a graph of the similarity SM, the area selection unit 14 determines the magnitude of the luminance difference between the attention area 41 and the corresponding area, for example, the adjacent area 42 adjacent to the attention area 41, as shown in FIG.

A specific example of the procedure for calculating the parallax shift correction amount ΔC in the stereo camera 100 of the second embodiment will be described with reference to FIG. 6B. First, a road surface is identified as a basis for calculation of the parallax shift correction amount ΔC (step S201). Then, the area selection unit 14 calculates the luminance difference between the specified attention area 41 on the road surface and the adjacent area 42 adjacent to the attention area 41 (step S202). After calculating the luminance difference for each region, the region selection unit 14 selects a region having a luminance difference greater than the threshold THin (step S203).

Subsequently, it is determined whether or not the number Nreg of regions selected in step S203 is greater than or equal to the threshold THreg' (step S205). If the determination is Yes, the process proceeds to step S206, but if the determination is No (smaller than the threshold THreg), the computation of the parallax deviation correction amount ΔC and the parallax deviation correction process are stopped.

In step S206, the ideal value Di of parallax in each of the plurality of selected regions is calculated in the manner described above, and the difference between this ideal value Di and the actual parallax D is calculated by the parallax shift amount calculator 15 as the parallax shift amount ΔD=D−Di (step S207). Then, the parallax shift amount ΔD obtained in a plurality of regions is statistically processed to calculate the parallax shift amount ΔDfix for the entire image, and the parallax shift correction amount ΔC is calculated according to the parallax shift amount ΔDfix (step S208).

It should be noted that, as shown in FIG. 6C, it is also possible to calculate the luminance difference between the region of interest 41 and a neighboring region 42' that is not adjacent to and is a little distant. The adjacent region 42 is a region that is located without a gap from the region of interest 41 , while the neighboring region 42 ′ is a region separated from the region of interest 41 by, for example, one region. If the edge portion of the attention area 41 (for example, the edge portion between the road surface and the white line) is blurred due to the resolution of the camera, the luminance difference between the attention area 41 and the adjacent area 42 may not be correctly calculated.

As described above, according to the stereo camera 100 of the second embodiment, the area used for calculating the parallax shift amount is selected according to the magnitude of the luminance difference, and the parallax shift amount ΔD is calculated based on the selected area. Therefore, it is possible to provide an imaging device that can detect the parallax shift with high precision, as in the first embodiment.

[Third embodiment]
Next, a stereo camera 100 according to a third embodiment will be described with reference to FIGS. 7 and 8. FIG. Since the overall configuration of the stereo camera 100 of the third embodiment is the same as that of the first embodiment (FIGS. 1A and 1B), redundant description will be omitted. The third embodiment differs from the above-described embodiments in the method of calculating the amount of parallax deviation in the parallax deviation correction unit 16 . Specifically, after area selection is performed by the area selection unit 14 in the same manner as in the above-described embodiment, the parallax shift correction unit 16 calculates a representative value (for example, the median value Median(J)) of the parallax shift amounts ΔD of a plurality of areas for each vertical coordinate J (1, 2, . . . m, . . . ), as shown in FIG.

Next, the average value of the calculated median values Median(1) to (m) is calculated as the average value ΔDav of the parallax deviation amounts ΔD. Using this parallax shift amount ΔDav as an input, the parallax shift correction amount ΔC is calculated in the parallax shift correction unit 16 . However, if the standard deviation SD of the median value Median(J) used to calculate the average value ΔDav is larger than the threshold, it is preferable not to correct the parallax shift in the captured image. This is because when the standard deviation SD is larger than the threshold, the reliability in the region containing the coordinates is judged to be small.

In the illustrated example of FIG. 8, one median value Median(j) is calculated for one coordinate J, but one median value Median(k~k+x) may be calculated collectively for a plurality of coordinates k~k+x. In particular, it is preferable to calculate one median value for a plurality of coordinates in the vertical direction on a road surface that is far from the vehicle (the road surface that appears in the upper part of the screen). The reason is that the road surface far from the vehicle is imaged by the first imaging unit 11 and the second imaging unit 12 with the amount of information per pixel compressed compared to the road surface in the vicinity (the road surface shown in the lower part of the screen). This leads to a decrease in accuracy of parallax shift correction. As described above, for a distant road surface, a decrease in accuracy can be prevented by calculating a median value of 1 for a plurality of coordinates.

Further, as shown in FIG. 9, the median value of the parallax deviation amount ΔD of all selected regions is calculated instead of each coordinate, and the median value is determined as the parallax deviation amount ΔD in the captured image, and can be used as the parallax deviation correction amount ΔC.

Although various embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Also, part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Moreover, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

100... stereo camera, 200... external device, 1... stereo camera body, 2... image processing device, 3... CPU, 4... memory, 5... interface, BU... bus, 11... first imaging unit, 12... second imaging unit, 13... parallax calculation unit, 14... area selection unit, 15... parallax shift amount calculation unit, 16... parallax shift correction unit.

Claims

a first imaging unit that acquires a first captured image;
a second imaging unit arranged at a position separated from the first imaging unit by a certain distance and acquiring a second captured image;
a parallax calculation unit that calculates a parallax in each region of the first captured image and the second captured image;
an area selection unit that selects an area having a reliability equal to or higher than a threshold value as the parallax value from the first captured image or the second captured image;
a parallax shift amount calculation unit that compares the parallax value in the region selected by the region selection unit and the ideal parallax value that should be acquired in the region, and obtains the parallax shift amount of each region;
An image pickup apparatus, comprising: a parallax shift correction section that corrects parallax shift in the first imaging section and the second imaging section by a correction amount determined according to the parallax shift amount of each region.
The parallax calculation unit performs template matching using the first captured image as a reference image and the second captured image as a reference image to calculate parallax,
2. The imaging apparatus according to claim 1, wherein the region selection unit selects the region based on a degree of protrusion, which is a difference between a minimum value of a peak of a similarity graph obtained in the template matching and a reference value.
The imaging apparatus according to claim 2, wherein the reference value is the average value of the similarities in an area excluding the vicinity of the peak.
The imaging device according to claim 2, wherein the reference value is a minimum value of a peak different from the peak.
The imaging apparatus according to claim 1, wherein the parallax shift correction by the parallax shift correction section is stopped when the number of regions selected by the region selection section is less than a predetermined number.
The imaging apparatus according to claim 1, wherein the parallax shift correction unit calculates a representative value of the parallax shift amount calculated in each region, and calculates the parallax shift correction amount according to the representative value.
2. The imaging apparatus according to claim 1, wherein the area selection unit selects the area based on the magnitude of a luminance difference between an attention area in the first captured image or the second captured image and an area corresponding to the attention area.
a step of acquiring a first captured image captured by a first imaging unit and a second captured image captured by a second imaging unit arranged at a position separated from the first imaging unit by a predetermined distance;
calculating a parallax in each region of the first captured image and the second captured image;
a step of selecting an area having a reliability equal to or higher than a threshold value as the parallax value from the first captured image or the second captured image;
a step of comparing the parallax value in the selected region and the ideal parallax value to be obtained in the region to determine the parallax shift amount of each region;
and correcting the parallax shift in the first imaging unit and the second imaging unit using a correction amount determined according to the parallax shift amount of each region.
In the step of calculating the parallax, the parallax is calculated by performing template matching using the first captured image as a reference image and the second captured image as a reference image,
9. The parallax shift correction method according to claim 8, wherein the step of selecting the region selects the region based on a degree of protrusion that is a difference between a minimum value of a peak of a similarity graph obtained in the template matching and a reference value.
The parallax deviation correction method according to claim 9, wherein the reference value is an average value of the similarities in an area excluding the vicinity of the peak.
The parallax deviation correction method according to claim 9, wherein the reference value is a minimum value of a peak different from the peak.
The parallax shift correction method according to claim 8, wherein the parallax shift correction is stopped when the number of regions selected by the step of selecting the regions is less than a predetermined number.
The parallax shift correction method according to claim 8, wherein the correction of the parallax shift calculates a representative value of the parallax shift amount calculated in each region, and calculates the parallax shift correction amount according to the representative value.
9. The parallax shift correction method according to claim 8, wherein the step of selecting the area selects the area based on the magnitude of a luminance difference between a region of interest in the first captured image or the second captured image and a region corresponding to the region of interest.
a step of acquiring a first captured image captured by a first imaging unit and a second captured image captured by a second imaging unit arranged at a position separated from the first imaging unit by a predetermined distance;
calculating a parallax in each region of the first captured image and the second captured image;
a step of selecting an area having a reliability equal to or higher than a threshold value as the parallax value from the first captured image or the second captured image;
a step of comparing the parallax value in the selected region and the ideal parallax value to be obtained in the region to determine the parallax shift amount of each region;
and correcting the parallax shift in the first imaging section and the second imaging section using a correction amount determined according to the parallax shift amount of each region.