JP2016173248A - Parallax value computation device, object recognition device, mobile instrument control system and parallax computation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a parallax value computation device that makes a correction by implementing appropriate processing corresponding to a cause of a parallax value error on a parallax value in a unit image area of a correction object, and further to provide an object recognition device, a mobile instrument control system and a parallax computation program.SOLUTION: A parallax image creation unit 132 includes: a parallax value calculation section 132C that calculates a parallax value among a plurality of acquired image data imaged by a plurality of imaging units for imaging the imaging areas for each unit image area; and a reliability determination section 132D. The parallax image creation unit 132 comprises: a reliability determination value change section 132E; a periphery complement processing section 132F; and an interframe complement processing section 132H, as parallax value correction means that corrects the parallax value of the unit image area satisfying at least one parallax value error condition among a plurality of mutually different parallax value error conditions, using correction processing corresponding to the at least one parallax value error condition among a plurality of different correction processing for each of the plurality of parallax value error conditions.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a parallax value calculation device, an object recognition device, a mobile device control system, and a parallax calculation program.

  Conventionally, a unit image in which a parallax value is not properly calculated after calculating a parallax value for each unit image area based on a positional deviation amount of an image portion that captures the same spot between a reference image captured by two imaging units and a comparison image A parallax value calculation device that corrects a parallax value of a region is known.

  For example, in Patent Document 1, a matching process is performed in which an image part obtained by capturing the same spot as the image part in the reference image is identified from the comparison image, and the image part in the comparison image identified by the matching process is included in the reference image. A stereo image processing apparatus that calculates the amount of positional deviation from the image portion as a parallax value is disclosed. After calculating the parallax value for each unit image area, this stereo image processing apparatus extracts a 3 × 3 unit image area group centered on the unit image area of interest, and sets the address and the parallax value of each unit image area. A majority voting filter means is provided that creates a histogram in which the corresponding frequencies are added and corrects the parallax value of the unit image area of interest in accordance with the maximum frequency parallax value extracted from the histogram.

  The stereo image processing apparatus disclosed in Patent Document 1 uses the parallax value of a unit image region (peripheral unit image region) located around the unit image region of interest (target unit image region) to generate the target unit image. This is a correction method of correcting the parallax value of the region. According to this correction method, if a correct parallax value is included in the peripheral unit image area, it is possible to appropriately correct the parallax value of the unit image area of interest. Therefore, defects that affect only the corresponding unit image area, such as pixel defects in the image sensor in the imaging unit and 1-bit soft error in which the bit state in the memory that stores the calculated parallax value is changed by radioactive particles, etc. If it is a parallax value error caused by the above, it can be said that this is an appropriate correction method.

  However, a parallax value error such as a parallax value not being calculated or an incorrect parallax value being calculated is not only caused by such a defect, but also affects, for example, a wide range of unit image area group parallax value errors. There are things that are caused by malfunctions. For such a problem, the parallax value cannot be appropriately corrected by the above-described correction method, but may be appropriately corrected by another correction method.

  In order to solve the above-described problem, the present invention provides a disparity including a disparity value calculating unit that calculates a disparity value between a plurality of captured image data captured by a plurality of image capturing units that capture an image capturing area for each unit image region. In the value calculation device, the disparity value of the unit image region that satisfies at least one disparity value error condition among the plurality of disparity value error conditions that are different from each other is calculated from the plurality of correction processes that differ for each of the plurality of disparity value error conditions. It is characterized by having a parallax value correcting means for correcting using a correction process corresponding to at least one parallax value error condition.

  According to the present invention, there is an excellent effect that the parallax value of the unit image area to be corrected can be corrected by an appropriate correction process corresponding to the cause of the parallax value error.

It is a functional block diagram in the parallax image generation part in the vehicle equipment control system which concerns on embodiment. It is a schematic diagram which shows schematic structure of the same vehicle equipment control system. It is a schematic diagram which shows schematic structure of the imaging unit and image analysis unit in the same vehicle equipment control system. (A) And (b) is explanatory drawing which each shows an example of the captured image each imaged with the two imaging parts in the imaging unit. It is explanatory drawing which shows the general ranging principle. It is a process block diagram for demonstrating the object recognition process of embodiment. (A) is explanatory drawing which shows an example of the parallax value distribution of a parallax image. (B) is explanatory drawing which shows the row parallax distribution map (V map) which shows the parallax value frequency distribution for every row | line | column of the parallax image of the same (a). It is explanatory drawing which shows eight surrounding pixels in 3 pixels x 3 pixels centering on the object pixel E. FIG.

Hereinafter, an embodiment in which a parallax value calculation device according to the present invention is applied to an object recognition device in a control system of an in-vehicle device (mobile device mounted) mounted on an automobile that is a moving body will be described.
FIG. 2 is a schematic diagram showing a schematic configuration of an in-vehicle device control system as a mobile device control system according to the present embodiment.
The vehicle-mounted device control system is configured to detect an object existing in front of the host vehicle from captured image data of a front region (imaging region) in the traveling direction of the host vehicle captured by the imaging unit 101 serving as an imaging unit mounted on the host vehicle 100 that is an automobile. And various in-vehicle devices are controlled using the recognition result.

  The in-vehicle device control system of the present embodiment is provided with an image pickup unit 101 that picks up an image of an area in the traveling direction of the traveling vehicle 100 as an image pickup area. For example, the imaging unit 101 is installed in the vicinity of a room mirror of the windshield 105 of the host vehicle 100. Various data including captured image data obtained by imaging by the imaging unit 101 is input to the image analysis unit 102. The image analysis unit 102 analyzes the data transmitted from the imaging unit 101 and recognizes recognition objects such as other vehicles in front of the host vehicle, pedestrians, and various obstacles. This recognition result is used for, for example, a notification process of notifying the driver of the host vehicle 100 of warning information using the display unit 103 or a speaker, or the moving direction of the host vehicle 100 is changed by the vehicle travel control unit 106. It is used for driving support control such as a steering device (steering) that is a device, an accelerator device that is a device that accelerates the host vehicle 100, and a brake device (braking device) that is a device that decelerates the host vehicle 100. To do. As the display unit 103, a liquid crystal display, a HUD (Head-Up Display), or the like can be used.

FIG. 3 is a schematic diagram illustrating a schematic configuration of the imaging unit 101 and the image analysis unit 102.
The imaging unit 101 is configured by a stereo camera including two imaging units 110A and 110B, and the two imaging units 110A and 110B are the same. The imaging units 110A and 110B respectively include imaging lenses 111A and 111B, sensor substrates 114A and 114B including image sensors 113A and 113B in which light receiving elements are two-dimensionally arranged, and analogs output from the sensor substrates 114A and 114B. It is composed of signal processing units 115A and 115B as data output means for generating and outputting captured image data obtained by converting an electrical signal (the amount of light received by each light receiving element on the image sensors 113A and 113B) into a digital electrical signal. ing. Note that the signal processing units 115A and 115B may be provided on the image sensors 113A and 113B. Luminance image data and parallax image data are output from the imaging unit 101 of the present embodiment. For example, the image data has a data amount of 12 bits per pixel and takes 0 to 4095 gradations (pixel values).

  In addition, the imaging unit 101 includes a processing hardware unit 120 as a parallax value calculation device including an image processing board or the like. The processing hardware unit 120 mainly performs processing that requires real-time performance on captured image data. Specifically, for example, a luminance correction process for correcting a luminance value (pixel value) by gamma correction or the like, a distortion correction process for correcting distortion of a captured image caused by an attachment position shift between the two imaging units 110A and 110B ( Parallelization of the left and right captured images), parallax calculation processing for executing parallax values by executing matching processing, and the like. Note that the luminance correction process and the distortion correction process are not necessarily performed.

  The processing hardware unit 120 of the present embodiment includes a luminance correction processing unit 124 serving as a pixel value correcting unit that performs luminance correction processing of luminance image data output from each of the imaging units 110A and 110B, and a pixel value that corrects image distortion. Disparity for calculating the parallax value of the corresponding image portion between the captured images captured by each of the imaging units 110A and 110B in order to obtain a parallax image from the luminance image data after the correction processing, the distortion correction processing unit 125 as a correction unit It is comprised by the calculating part 121 grade | etc.,.

  Here, the parallax value corresponds to the same point in the imaging region (the same location on the same object), with one of the captured images captured by the imaging units 110A and 110B as a reference image and the other as a comparison image. The positional deviation amount of the image portion (unit image region) on the comparison image with respect to the image portion (unit image region) on the reference image is calculated as the parallax value of the image portion. By using the principle of triangulation, the distance to the same point in the imaging region corresponding to the image portion can be calculated from the parallax value.

FIGS. 4A and 4B are explanatory diagrams illustrating examples of captured images captured by the two imaging units 110A and 110B, respectively.
In the example shown in FIGS. 4A and 4B, the same imaging region is imaged from different directions by the two imaging units 110A and 110B, and one of the two captured images obtained thereby (FIG. 4A ) Is a reference image, and the other (image shown in FIG. 4B) is a comparative image. Then, it is searched which image block in the comparison image corresponds to an image block including a certain unit image area in the reference image. For example, when searching for an image block in the comparison image corresponding to the image block Wa in the reference image, the horizontal position is changed while the vertical position is fixed at the same position as the vertical position of the image block Wa. The comparison image is searched in the order of Wb1 → Wb2 → Wb3. Then, while performing this search, the correlation operation between the reference image and the comparison image is performed, the image block having the highest correlation value is detected, and the corresponding points corresponding to the same point in the imaging region are compared with the reference image. A matching process that specifies both images is performed.

FIG. 5 is an explanatory view showing a general distance measuring principle.
When the image block Wb3 in the comparison image corresponding to the image block Wa in the reference image is specified by the matching process, the reference image and the corresponding point in each of the image blocks Wa and Wb3 corresponding to the same point in the imaging region are The amount of deviation from the comparative image is obtained as a parallax value. Specifically, as shown in FIG. 5, when the baseline direction position of the pixel of interest in the reference image is XR and the baseline direction position of the corresponding pixel in the comparison image is XL, the parallax value D is D = | XL-XR |.

In addition, by using the parallax value D obtained by the matching process, the distance (the distance from the base line to the measurement target point) to the same point (measurement target point) in the imaging region projected on the target pixel in the reference image ) Z can be calculated from the following equation (1). In the following equation (1), “F” is the focal length of the imaging units 110A and 110B, and “B” is the baseline length between the imaging units 110A and 110B.
Z = B × F / D (1)

  The image analysis unit 102 executes a storage unit 122 configured by a RAM, a ROM, or the like that stores luminance image data and parallax image data output from the imaging unit 101, and a computer program for performing recognition processing of an identification target. CPU (Central Processing Unit) 123. The CPU is responsible for the control of the image sensor controller of each of the imaging units 110A and 110B and the overall control of the processing hardware unit 120, and executes a recognition process for other vehicles, guardrails, and other various objects (recognition objects). Is read from the ROM, and various processes are executed with the luminance image data and parallax image data stored in the RAM as inputs, and the processing results are output to the outside. The recognition process result data output to the outside is used to notify warning information in the display unit 103 or the like, or the vehicle travel control unit 106 controls various vehicle control actuators (brake control, vehicle speed). Or used as input data for control).

FIG. 6 is a process block diagram for explaining the object recognition process of the present embodiment.
Two luminance image data output from the two imaging units 110 </ b> A and 110 </ b> B constituting the stereo camera are output to the image processing unit 131. The image processing unit 131 performs image processing such as luminance correction processing and distortion correction processing on the luminance image data, and includes a luminance correction processing unit 124 and a distortion correction processing unit 125 of the processing hardware unit 120. Is done.

  When the luminance image data is input, the image processing unit 131 executes the distortion correction processing by the distortion correction processing unit 125 after executing the luminance correction processing by the luminance correction processing unit 124. This distortion correction processing is performed by using luminance image data (compared with a reference image) output from each of the imaging units 110A and 110B based on the distortion of the optical system in the imaging units 110A and 110B and the relative positional relationship between the left and right imaging units 110A and 110B. Image) is converted into an ideal parallel stereo image obtained when two pinhole cameras are mounted in parallel.

  After performing the distortion correction processing in this way, next, the parallax image generation unit 132 configured by the parallax calculation unit 121 performs parallax image generation processing for generating parallax image data (parallax information). In the parallax image generation processing, first, the luminance image data of one imaging unit 110A of the two imaging units 110A and 110B is set as reference image data, and the luminance image data of the other imaging unit 110B is set as comparison image data, and these are used. The parallax values of the two are used to generate and output parallax image data. The parallax image data represents a parallax image in which pixel values corresponding to the parallax value d calculated for each image portion (unit image region) on the reference image data are represented as pixel values of the respective image portions. The unit image area in the present embodiment corresponds to one pixel on the luminance image data, but is not limited thereto.

  Specifically, the parallax image generation unit 132 defines an image block including one target pixel (unit image region) and surrounding pixels for a certain row of reference image data. Here, as the peripheral pixels that form the image block together with the target pixel, for example, seven pixels from the target pixel in the search direction (right direction of the image) are used, and an image of 1 pixel × 8 pixels in which the target pixel is located at the left end. Define the block. The definition method of the image block is not limited to this. For example, an image block of 3 pixels × 3 pixels centered on the target pixel may be used.

  On the other hand, in the same row in the comparison image data, an image block having the same size as the image block of the defined reference image data is shifted pixel by pixel in the horizontal line direction (X direction), and the pixel value of the image block defined in the reference image data ( A correlation value indicating a correlation between a feature value indicating the feature of (luminance value) and a feature value indicating the feature of the pixel value of each image block in the comparison image data is calculated. Then, based on the calculated correlation value, a matching process is performed to select an image block of the comparison image data that is most correlated with the image block of the reference image data among the image blocks in the comparison image data. Thereafter, a positional deviation amount between the target pixel of the image block of the reference image data and the corresponding pixel of the image block of the comparison image data selected by the matching process is calculated as the parallax value d. The parallax image data can be obtained by performing such processing for calculating the parallax value d for the entire area of the reference image data or a specific area.

  As the feature amount of the image block used for the matching processing, for example, each pixel value (luminance value) in the image block can be used. As the correlation value, for example, SSD (Sum of Squared Difference), ZSSD (Zero- Methods such as mean sum of squared difference (SAD), sum of absolute difference (SAD), and zero-mean sum of absolute difference (ZSAD) can be used. In the present embodiment, the absolute value of the difference between each pixel value (luminance value) in the image block of the reference image data and each pixel value (luminance value) in the image block of the comparison image data corresponding to each of these pixels. The matching process is performed using the SAD that is the sum of the correlation values as the correlation value. In this case, it can be said that the image block having the smallest SAD value is most correlated.

  After performing the parallax image generation processing in this way, the V map generation unit 133 next executes V map generation processing for generating a V map. Each pixel data included in the parallax image data is represented by a set (x, y, d) of an x-direction position, a y-direction position, and a parallax value d, which is represented by d on the X axis, y on the Y axis, XY two-dimensional histogram information is created by setting the frequency on the Z axis. This is called a V map.

  Specifically, the V map generation unit 133 calculates a parallax value frequency distribution for each row of the parallax image data generated by the parallax image generation unit 132. Explaining with a specific example, when parallax image data having a parallax value distribution as shown in FIG. 7A is input, the V map generation unit 133 performs line by row as shown in FIG. The disparity value frequency distribution is calculated and output. A two-dimensional orthogonal coordinate system in which the y-direction position on the parallax image (vertical direction position of the captured image) is taken on the Y-axis and the parallax value is taken on the X-axis from the information on the parallax value frequency distribution of each row obtained in this way. In addition, a V map in which frequencies are distributed can be obtained. This V map can also be expressed as an image in which pixels having pixel values corresponding to the frequency are distributed on the two-dimensional orthogonal coordinate system.

Next, in the present embodiment, road surface shape detection in which the road surface shape detection unit 134 detects the three-dimensional shape of the front road surface of the host vehicle 100 from the V map information (parallax histogram information) generated by the parallax image generation unit 132. Execute the process.
For example, an imaginary extension obtained by extending the front road surface of the own vehicle 100 relatively flat, that is, the front road surface of the own vehicle 100 is parallel to the road surface portion immediately below the own vehicle 100 in front of the own vehicle 100. When they coincide with the plane, in the lower part of the V map corresponding to the lower part of the image, the high-frequency points are distributed in a substantially straight line having an inclination such that the parallax value d decreases toward the upper part of the image. Pixels exhibiting such a distribution are pixels that are present at almost the same distance in each row on the parallax image, have the highest occupation ratio, and project a recognition object whose distance continuously increases toward the top of the image. It can be said that. Since such a pixel matches the characteristics of a pixel that displays the road surface, it can be estimated that the pixel displays the road surface.

  The road surface shape detection unit 134 linearly approximates a high-frequency point on the V map indicating the feature indicated by the parallax value corresponding to the road surface, that is, the feature that the parallax value becomes lower toward the upper side of the captured image. Process. Once the approximate straight line information is obtained, the road surface height table calculation unit 135 calculates the road surface height (relative height with respect to the road surface portion directly below the host vehicle) and forms a table. Perform the calculation process. From the approximate straight line information on the V map generated by the road surface shape detection unit 134, the distance to each road surface portion displayed in each row area (each position in the vertical direction of the image) on the captured image can be calculated. On the other hand, each surface part in the traveling direction of the virtual plane obtained by extending the road surface portion located directly below the traveling vehicle in front of the traveling direction of the vehicle so as to be parallel to the surface is displayed in which row area in the captured image. The virtual plane is represented by a straight line (reference straight line) on the V map. By comparing the approximate straight line output from the road surface shape detection unit 134 with the reference straight line, the height of each road surface portion ahead of the host vehicle can be obtained. In a simple manner, the height of the road surface portion existing in front of the host vehicle can be calculated from the Y-axis position on the approximate straight line output from the road surface shape detection unit 134 by a distance obtained from the corresponding parallax value. The road surface height table calculation unit 135 tabulates the height of each road surface portion obtained from the approximate straight line for the necessary parallax range.

  Next, the U map generation unit 136 will be described. The U map generation unit 136 performs U map generation processing for generating a U map. A set (x, y, d) of the x-direction position, the y-direction position, and the parallax value d in each pixel data included in the parallax image data is set to x on the X axis, d on the Y axis, and frequency on the Z axis. Thus, XY two-dimensional histogram information is created. This is called a U map. In the U map generation unit 133 of the present embodiment, a U map is created only for a point (x, y, d) of a parallax image whose height H from the road surface is in a predetermined height range (for example, 20 cm to 5 m). In this case, an object existing within the predetermined height range from the road surface can be appropriately extracted.

  For example, when there are guard rails on both the left and right sides of the road surface, and there are one preceding vehicle and one oncoming vehicle as the other vehicles, in the U map, the high frequency points corresponding to the left and right guard rails are It is distributed in a substantially straight line extending upward from the side toward the center. On the other hand, high-frequency points corresponding to other vehicles are distributed between the left and right guard rails in a line segment extending substantially parallel to the X-axis direction.

  Next, the isolated region detection unit 137 will be described. The isolated region detection unit 137 first performs smoothing processing of the U map from the information of the U map generated by the U map generation unit 136, and then performs binarization processing. After that, the coordinate with a value is labeled to detect an isolated area. Hereinafter, each processing will be described.

  The disparity value has dispersion due to calculation errors and the like, and the disparity value is not calculated for all pixels. Therefore, the actual U map includes noise. Therefore, a process for smoothing the U map is performed in order to remove noise and make it easy to identify the recognition target (object). In this smoothing process, a smoothing filter (for example, a simple average of 3 × 3 pixels) is applied to the frequency value in the same manner as the smoothing of the image. As a result, the frequency of points on the U map that can be considered as noise is reduced, and the point of the recognition object (object) becomes a higher group than the surroundings. As a result, it becomes easy to detect an isolated region in the subsequent processing.

  Next, from the information of the U map smoothed in this way, an isolated region having a higher frequency than the surroundings is detected on the U map. In this detection, the U map is first binarized. For this binarization processing, for example, an adaptive binarization method disclosed in Japanese Patent No. 4018310 can be used. Since each recognition object (object) has a difference in height, shape, contrast difference with the background, etc., the isolated region corresponding to each recognition object may have a large frequency value or a small one. is there. Therefore, there is a possibility that an isolated region that cannot be detected properly by binarization using a single threshold value may occur. In order to prevent this, it is preferable to use the adaptive binarization method described above. In binarization, a high frequency area is set to “1” (black), and a low frequency area is set to “0” (white).

  In this way, points having a value of “1” (black) in binarization processing (coordinates whose frequency value is higher than the binarization threshold) are labeled based on their connectivity, and one region with the same label is labeled as one. Detect as an isolated region.

  For each isolated area obtained in this way, the recognition target object projected in the isolated area calculated from the width (length in the X-axis direction on the U map) and the minimum parallax value d in the isolated area Using the distance z between the (object) and the host vehicle, the width W of the object displayed in the image area corresponding to the isolated area can be calculated. An isolated area in which the width W of the object is within a predetermined range is determined as an object candidate area.

Next, the parallax image corresponding region detection unit 138 will be described.
For the isolated area determined as the object candidate area by the isolated area detection unit 137, when a rectangular area inscribed by the isolated area is set, the width of this rectangular area (the length in the X-axis direction on the U map) is This corresponds to the width of the recognition object (object) corresponding to the isolated region. Further, the height of the set rectangular area corresponds to the depth (length in the traveling direction of the host vehicle) of the recognition object (object) corresponding to the isolated area. On the other hand, the height of the recognition object (object) corresponding to each isolated region is unknown at this stage. The corresponding region detection unit 138 of the parallax image detects a corresponding region on the parallax image corresponding to the isolated region in order to obtain the height of the object corresponding to the isolated region related to the object candidate region.

  Specifically, the corresponding region detection unit 138 of the parallax image is based on the information on the isolated region output from the isolated region detection unit 137, and the width of the isolated region, that is, the range in which the X-axis direction coordinates are from xmin to xmax (detection). With respect to (width), the parallax image is scanned in a predetermined range in the Y-axis direction, and the height of the rectangular area on the U map set in the isolated area, that is, the U map Y-axis direction coordinate (parallax value) is from dmin to dmax. Pixels having a parallax value in the range of are extracted as candidate pixels. The scanning range (Y-axis direction range of the parallax image) at this time is, for example, a range from a position below the parallax image by 1/6 to the road surface obtained from the maximum parallax dmax from the position below the parallax image. It can be.

  In the candidate pixel group extracted in this way, a horizontal line in which a predetermined number or more of candidate pixels exist in the parallax image X-axis direction with respect to the detection width is determined as an object candidate line. Next, when scanning is performed in the vertical direction and there are other object candidate lines with a predetermined density or more around the object candidate line of interest, the object candidate line of interest is determined as an object line. To do.

  The object area extraction unit 139 searches the object line determined in this way for each detection width corresponding to each isolated area, and detects the circumscribed rectangle of the object line group thus detected as the object area on the parallax image. Determine as.

Next, the object type classification unit 140 will be described.
From the height of the object area extracted by the object area extraction unit 139, the actual height of the recognition object (object) displayed in the image area corresponding to the object area can be calculated. Similarly, the actual width of the recognition object (object) displayed in the image area corresponding to the object area can be calculated from the width of the object area extracted by the object area extracting unit 139. The depth of the recognition target object (object) displayed in the image area corresponding to the object area can be calculated from the maximum parallax value dmax and the minimum parallax value dmin in the isolated area corresponding to the object area. .

  The object type classification unit 140 classifies the object type from information on the height, width, and depth of the object corresponding to the object area that can be calculated in this way. Specifically, compared to the table data for classifying the object type, whether the recognition object (object) in front of the host vehicle is a pedestrian, bicycle, small car, truck, etc. Etc. can be recognized separately.

Next, the three-dimensional position determining unit 141 will be described.
Since the three-dimensional position determination unit 141 also knows the distance to the object corresponding to the detected object area and the distance on the image between the image center of the parallax image and the center of the object area on the parallax image, the object Is determined.

Next, the guardrail detection unit 142 will be described.
Since the side walls and guard rails installed on the side of the road surface are generally within a range of 30 to 100 cm from the road surface, an area in the U map corresponding to this range is selected as the target range of the guard rail detection process. . Thereafter, the frequency of the U map is weighted with respect to the target range, Hough transform is performed to detect an approximate line, and a guardrail is detected from the approximate line.

Next, the parallax calculation process, which is a characteristic part of the present invention, will be described in detail.
FIG. 1 is a functional block diagram in the parallax image generation unit 132 configured by the parallax calculation unit 121 according to the present embodiment.
In the parallax image generation unit 132 according to the present embodiment, the reference image data and the comparison image data that have been subjected to the luminance correction processing and the distortion correction processing unit 125 by the image processing unit 131 are input to the flatness determination unit 132A.

  In the flatness determination unit 132A, for each pixel value of the reference image data and the comparison image data, is the difference between the pixel value (luminance value) of the pixel and the pixel values of surrounding pixels located in the vicinity thereof equal to or less than a specified value? The flatness indicating whether or not is calculated. Specifically, for example, as shown in FIG. 8, eight peripheral pixels A, B, C, D, F, G, and H in 3 pixels × 3 pixels centering on the target pixel E whose flatness is calculated. , I is used to calculate the flatness. At this time, for example, the results of calculating difference values between the pixel value of the target pixel E and the pixel values of the surrounding pixels A, B, C, D, F, G, H, and I are all equal to or less than a predetermined specified value. If the flatness is “1”, the flatness is “1”, otherwise the flatness is “0”. The flatness of each pixel calculated by the flatness determination unit 132A in this way is sent to the luminance noise removal unit 132B as a pixel value correction unit.

  The luminance noise removal unit 132B executes a noise removal process for removing noise in the reference image data and the comparison image data with a known noise removal filter in accordance with the flatness from the flatness determination unit 132A. Specifically, for a pixel having a flatness of “1”, a noise removal process for removing noise due to electromagnetic noise such as EMI (Electro-Magnetic Interference) is executed, and the flatness is “0”. Noise removal processing is not executed for a certain pixel. At this time, image processing such as edge emphasis processing may be performed on the pixels having a flatness of “0”. When the processing load of the flatness determination unit 132A is large, the flatness determination values for the reference image data and the comparison image data in the previous imaging frame are input to the luminance noise removal unit 132B with priority given to real-time processing. You may do it.

As a method for removing noise, for example, eight peripheral pixels A, B, C, D, F, G, and 3 pixels in 3 pixels × 3 pixels centered on the target pixel E to be flattened as shown in FIG. A method of correcting the pixel value of the target pixel E from the following equation (2) using H and I can be cited.
E = (A + 2 × B + C + 2 × D + 4 × E + 2 × F + G + 2 × H + I) ÷ 16 (2)

  The reference image data and the comparison image data that have been subjected to the noise removal processing by the luminance noise removal unit 132B in this way are sent to the parallax value calculation unit 132C. The parallax value calculation unit 132C performs the above-described matching process to calculate the parallax value of each pixel. That is, a matching process for specifying the image block Wb having the smallest SAD value from the comparison image data with respect to the entire image of the reference image data or a specific area with the image block Wa including the target pixel in the reference image data. Then, the positional deviation amount between the corresponding pixel of the image block Wb of the specified comparison image data and the target pixel of the reference image data is calculated as the parallax value d.

  As described above, the image block of this embodiment is defined by the image block Wa of 1 pixel × 8 pixels in which the target pixel is located at the left end, and the pixel in the comparison image at the same position as the target pixel is located at the left end. The image blocks Wb1, Wb2, Wb3,... Are shifted pixel by pixel in a predetermined search direction in order from the image block Wb1, and the SAD values of the image blocks Wb1, Wb2, Wb3,.

Table 1 below is a table showing an example of SAD values for the image blocks Wb1 to Wb8.

  In the example of Table 1, the image block Wb7 has the smallest SAD value, and the disparity value calculating unit 132C may calculate the disparity value d between the corresponding pixel of the image block Wb7 and the target pixel as it is. . However, when the difference between the SAD value of the image block Wb7 and the SAD value of another image block is a slight difference corresponding to the error, the image block Wb7 does not include the corresponding pixel corresponding to the target pixel. There is a high probability that no matching mistakes have occurred. In this case, it can be said that the reliability of the parallax value is low.

  Therefore, in this embodiment, in order to prevent a parallax value with low reliability from being calculated, the parallax value calculation unit 132C calculates the reliability of the calculated parallax value, and calculates the parallax value together with the reliability. And sent to the reliability determination unit 132D. Here, the reliability may be any value as long as it indicates the reliability of the parallax value, but in the present embodiment, the SAD value of the image block having the smallest SAD value and 2 for the image block. The comparison result with the SAD value of the image block shifted by the pixel is used as the reliability. Specifically, in the example of Table 1, “55” which is the SAD value of the image block Wb7 having the smallest SAD value and the SAD value of the image block Wb5 which is shifted by 2 pixels from the image block Wb. The difference value from “85” is used as the reliability. In this case, the reliability is “30”.

  When the reliability and the parallax value calculated in this way are received from the parallax value calculation unit 132C, the reliability determination unit 132D determines whether or not the reliability exceeds the reliability determination value. When the reliability determination value is “10”, since the reliability is “30” in the example of Table 1, the reliability determination value exceeds “10”. Therefore, in this case, it is determined that the reliability of the parallax value is high, and the parallax value is output as it is to the subsequent peripheral complement processing unit 132F. On the other hand, the parallax value whose reliability is equal to or lower than the reliability determination value has low reliability, and therefore, instead of outputting the parallax value to the peripheral complement processing unit 132F in the subsequent stage, a blank value or an error value is output.

  In this way, the disparity value output from the reliability determination unit 132D is obtained by excluding the disparity value with low reliability, but the disparity value determined to have low reliability includes a matching error. Also included are those that show appropriate parallax values that have not occurred. In particular, for an image portion that shows a subject portion that does not show a difference in luminance value, the pixel values (luminance values) in the image block included in the image portion are almost uniform and have no features. The SAD values of the image blocks Wb1 to Wb8 of FIG. In this case, even if the disparity value has a small degree of reliability (reliability indicated by the difference value of the SAD value), it is appropriate compared to the image portion in which a certain feature exists in the pixel value (luminance value) in the image block. There is a high possibility that a correct parallax value is calculated.

Table 2 below is a table showing other examples of SAD values for the image blocks Wb1 to Wb8. This example is an example of an image portion that projects a subject portion in which there is no difference in luminance value.

  In the example shown in Table 2, the SAD values of any of the image blocks Wb1 to Wb8 are close to those in the example of Table 1 described above. In this example, the image block Wb6 has the smallest SAD value, but the reliability is “52” which is the SAD value of the image block Wb6, and an image shifted by 2 pixels from the image block Wb. A difference value between “6” which is a difference value from “58” which is the SAD value of the block Wb4, or “60” which is a SAD value of the image block Wb8 which is shifted by 2 pixels from the image block Wb. It becomes a certain “8”. Regardless of which difference value is used as the reliability, since the reliability determination value “10” described above is not exceeded, the parallax value for the target pixel is output as a blank value or an error value. .

  Therefore, in the present embodiment, even when the reliability indicated by the difference in the SAD values is small, an image portion that is highly likely to have an appropriate parallax value calculated (an image that shows a subject portion that does not have a difference in luminance value) For (part), the reliability determination value is lowered so that the parallax value does not become a blank value or an error value. Specifically, the reliability determination value changing unit 132E determines whether the image portion is such an image portion by using the flatness calculated by the flatness determination portion 132A described above. For example, processing for lowering the reliability judgment value is performed.

  In detail, the reliability determination value changing unit 132E is based on the flatness received from the flatness determination unit 132A, and the flatness of all the pixels of the image block Wa including the target image is “1”. Determine whether the condition is met. At this time, if the pixel value flat condition is not satisfied, the reliability determination value used by the reliability determination unit 132D for the parallax value of the target image is set to “10” as usual. On the other hand, when the pixel value flat condition is satisfied, the reliability determination value used by the reliability determination unit 132D for the parallax value of the image of interest is set to “5”. Note that the pixel value flat condition is not limited to this, for example, a condition that a pixel having a flatness of “1” continues three times, for example, in an image block Wa including the target image, or an image block Wb in the comparison image. It is also possible to use a condition that utilizes the flatness of.

  As described above, in the present embodiment, when the reliability determination value changing unit 132E determines that the pixel value flatness condition is satisfied, the reliability determination value used in the reliability determination unit 132D is lowered. As in the example, the disparity value is also calculated for the disparity value having the reliability of “10” or less, and the disparity value that is the blank value or the error value can be complemented.

  Among the parallax values output from the reliability determination unit 132D in this way, there are still those indicating a blank value or an error value. There are various possible causes, one of which is caused by a hardware error. The hardware error referred to here is one that causes an erroneous parallax value locally, and that error can be detected.

  A specific example of the hardware error is, for example, a 1-bit soft error. A 1-bit soft error is a problem that a bit state in a memory is changed due to radioactive particles, and cannot be predicted when it occurs. In this embodiment, the signal processing units 115A and 115B perform checksum processing at the time of data input / output to / from the frame memory to check the occurrence of a 1-bit soft error. When the signal processing units 115A and 115B detect the occurrence of a 1-bit soft error, the detection information is sent to the processing hardware unit 120. In this embodiment, the hardware error detection unit 132G acquires detection information of 1-bit soft error sent from the signal processing units 115A and 115B of the imaging unit 101. As a result, it is possible to specify a pixel in which a 1-bit soft error has occurred.

  Further, as a specific example of the hardware error, for example, a pixel defect in which a light receiving element on the image sensors 113A and 113B of the imaging units 110A and 110B fails can be considered. In this case, the pixel value corresponding to the light receiving element always indicates zero or an error value. When detecting this pixel defect, the image analysis unit 102 outputs pixel defect information indicating a pixel defect error of the image sensors 113A and 113B to the processing hardware unit 120. This pixel defect information is acquired by the hardware error detection unit 132G, whereby the pixel in which the pixel defect has occurred can be specified.

The hardware error detection unit 132G that has detected such a hardware error sends the hardware error detection result to the peripheral complement processing unit 132F. The peripheral complement processing unit 132F performs peripheral parallax value correction processing for correcting the pixel specified from the hardware error detection result using the parallax value of the peripheral pixel located in the vicinity thereof. Specifically, for example, eight peripheral pixels A, B, C, D, F, G, H, and I in 3 pixels × 3 pixels centering on the target pixel E to be complemented as shown in FIG. And a method of correcting the pixel value of the target pixel E from the following equation (3).
E = (A + 2 × B + C + 2 × D + 2 × F + G + 2 × H + I) ÷ 12 (3)

  By the peripheral parallax value correction process in the peripheral complement processing unit 132F, the parallax value is supplemented for the pixel indicating the blank value or the error value due to the hardware error, or an inappropriate parallax value is set due to the hardware error. The parallax value is corrected for the indicated pixel.

  In this way, among the disparity values output from the peripheral complement processing unit 132F, there may still be a value indicating a blank value or an error value. The cause is the correction processing so far, that is, the luminance correction processing by the luminance correction processing unit 124, the distortion correction processing by the distortion correction processing unit 125, the noise removal processing by the luminance noise removal unit 132B, and the reliability judgment value changing unit 132E. This is the reason that the determination value reduction process for reducing the reliability determination value of the reliability determination unit 132D and the peripheral parallax value correction process performed by the peripheral complement processing unit 132F cannot be solved. Such causes include those caused by short-time errors that produce erroneous parallax values for only a short time. Specifically, for example, strong light such as a headlamp of an oncoming vehicle instantaneously enters the imaging unit 101 and the pixel values of the reference image and the comparison image are saturated.

  Therefore, in the present embodiment, in the inter-frame interpolation processing unit 132H, a normal parallax value captured before the imaging frame is calculated for a pixel in which a short-time parallax value error has occurred. A time difference correction process for correcting using the parallax value of the imaging frame is executed. Specifically, the simplest method is a method of complementing the parallax value of the past imaging frame at the same pixel position as the parallax value of the pixel in which the short-time parallax value error has occurred. However, when the host vehicle 100 is traveling, a point in the imaging region corresponding to a pixel in which a short-time parallax value error has occurred may correspond to another pixel in the past imaging frame. Therefore, the time difference correction process may be executed using the parallax value of another pixel position in the past imaging frame in consideration of the vehicle speed of the host vehicle 100, the time difference from the past imaging frame, and the like. .

  In the above description, the object recognition device using the parallax value calculation device according to the present invention is applied to the in-vehicle device control system. However, the present invention is not limited to this. For example, when an object recognition device using a parallax value calculation device according to the present invention is applied to FA (Factory Automation), a part flowing through a factory line is recognized by the object recognition device, and the flow is determined based on the recognition result. It can be used to control picking of incoming parts with a manipulator device or the like.

What has been described above is merely an example, and the present invention has a specific effect for each of the following modes.
(Aspect A)
A parallax value calculation unit 132C that calculates a parallax value between a plurality of captured image data such as reference image data and comparison image data captured by a plurality of imaging units 110A and 110B that capture an imaging region for each unit image region such as one pixel. In the parallax value calculation device including the parallax value calculation means such as the reliability determination unit 132D, the parallax value of the unit image area satisfying at least one parallax value error condition among a plurality of different parallax value error conditions is obtained. A reliability determination value changing unit 132E that corrects using a correction process corresponding to the at least one parallax value error condition among a plurality of correction processes that differ for each of the plurality of parallax value error conditions, a peripheral complement processing unit 132F, and inter-frame complement It has a parallax value correcting means such as a processing unit 132H.
According to this aspect, the parallax value (including an error value and a blank value) of the unit image region to be corrected is a value that satisfies at least one parallax value error condition among a plurality of different parallax value error conditions. is there. Therefore, if the error cause causing the parallax value error is set as a condition that can specify the parallax value error as these parallax value error conditions, the parallax value of the unit image area to be corrected can be distinguished for each error cause. In this aspect, there are a plurality of different correction processes for each of the plurality of parallax value error conditions, and for the parallax value of the unit image area to be corrected, correction corresponding to the parallax value error condition satisfied by the unit image area Corrections are made using the process. Therefore, the parallax value of the unit image area to be corrected can be corrected by an appropriate correction process corresponding to the cause of the error. The correction processing here is not only to correct the parallax value calculated by the parallax value calculating means to a more appropriate parallax value, but also to the parallax value of the unit image area for which the parallax value was not calculated by the parallax value calculating means. Including complementing.

(Aspect B)
In the aspect A, the plurality of correction processes is a peripheral parallax value correction process in which a parallax value of a unit image area to be corrected is corrected using a parallax value of a peripheral unit image area located around the unit image area. It is characterized by including.
According to this, a locally incorrect parallax value (including an error value and a blank value) can be corrected or supplemented with a parallax value estimated by a normal parallax value of the peripheral unit image area. If the disparity value of the peripheral unit image region is normal, the locally disparity value can be corrected appropriately.

(Aspect C)
In the aspect B, the plurality of parallax values include error detection means such as a hardware error detection unit 132G that detects a hardware error such as a 1-bit soft error or a pixel defect that causes an erroneous parallax value locally. The error condition includes a hardware error condition that the unit image area corresponds to the hardware error detected by the error detection unit, and the peripheral parallax value correction process is a correction process corresponding to the hardware error condition. It is characterized by that.
According to this, when a locally erroneous parallax value (including an error value and a blank value) is generated due to a hardware error, this can be corrected and complemented by the peripheral parallax value correction process. The local parallax value error caused by such a hardware error often has a normal parallax value in the peripheral unit image area. Therefore, it is possible to appropriately correct a parallax value that is locally erroneous due to a hardware error.

(Aspect D)
In any one of the aspects A to C, the disparity value calculating unit derives the reliability of the disparity value for each unit image area, and the derived reliability is greater than a predetermined reliability threshold. Is calculated as a parallax value of the unit image area, and a parallax value whose derived reliability is less than or equal to the predetermined reliability threshold value is not calculated as a parallax value of the unit image area, and the plurality of parallax value errors The condition is that a pixel value flat condition that a difference between a pixel value of a unit image area in at least one of the plurality of captured image data and a pixel value of a peripheral unit image area located around the unit image area is equal to or less than a specified value The plurality of correction processes include a threshold reduction process for reducing the predetermined reliability threshold as a correction process corresponding to the flat pixel value condition.
A disparity value with low reliability may cause a problem in subsequent processing, rather than a case where the disparity value is a blank value or an error value. Therefore, by not calculating a low-reliability parallax value whose derived reliability is equal to or less than a predetermined reliability threshold, it is possible to avoid causing problems in subsequent processing. However, an image portion that satisfies a pixel value flat condition that has a small difference in pixel values often shows an appropriate parallax value even when it is determined that the reliability is low. According to this aspect, for an image portion that satisfies the pixel value flatness condition with a small difference in pixel values, even if it is determined that the reliability is low as a result of the threshold reduction process for lowering the predetermined reliability threshold, The parallax value calculation means determines that the predetermined reliability threshold value is exceeded, and calculates the parallax value.

(Aspect E)
In any one of the aspects A to D, the plurality of correction processes may be performed such that the parallax value of the unit image area to be corrected is before and after the imaging timing of the plurality of captured image data for which the parallax value is calculated. Including a time difference correction process in which correction is performed using a parallax value between a plurality of captured image data captured at at least one of the times.
According to this, when an incorrect parallax value (including an error value and a blank value) is calculated for a short time, the parallax value is appropriately corrected or complemented by a parallax value based on captured image data having different imaging timings. can do.

(Aspect F)
In the aspect E, the plurality of parallax value error conditions include a short-time error condition that a unit image area causes an incorrect parallax value for a short time, and the time difference correction processing corresponds to the short-time error condition It is the correction process to perform.
According to this, when an instantaneous strong disturbance light is incident on the imaging means and an incorrect parallax value is generated for a short time, it can be appropriately corrected and supplemented by the time difference correction process.

(Aspect G)
In any one of the aspects A to F, pixel value correction means such as a luminance correction processing unit 124, a distortion correction processing unit 125, and a luminance noise removal unit 132B that correct pixel values in the plurality of captured image data are provided. The parallax value calculating means calculates a parallax value between a plurality of captured image data corrected by the pixel value correcting means for each unit image area.
According to this, a part of the cause of the parallax value error can be removed from the captured image data before the parallax value is calculated by the parallax value calculating means. Therefore, a more appropriate parallax value can be calculated.

(Aspect H)
Processing for calculating, for each unit image area, a parallax value between an imaging unit such as the imaging unit 101 including a plurality of imaging units 110A and 110B that captures an imaging area and a plurality of captured image data captured by the plurality of imaging units. An object recognition apparatus having a parallax value calculation device such as a hardware unit 120 and a recognition processing unit such as an image analysis unit 102 that recognizes a recognition target in an imaging region based on the parallax value calculated by the parallax value calculation device. The parallax value calculation device according to any one of the modes A to G is used as the parallax value calculation device.
According to this aspect, the recognition target object can be recognized based on the appropriately calculated parallax value, and a recognition process with few erroneous recognitions can be realized.

(Aspect I)
An object recognition device for recognizing recognition objects such as other vehicles, pedestrians, guardrails, etc. existing around the moving body such as the own vehicle 100, and a predetermined mounted on the moving body based on the recognition result of the object recognition apparatus In a mobile device control system comprising a mobile device control means such as a vehicle travel control unit 106 that controls the device, the object recognition device according to aspect H is used as the object recognition device. .
According to this aspect, it is possible to control the mobile unit mounted device based on recognition processing with few erroneous recognitions.

(Aspect J)
A parallax calculation program for causing a computer of a parallax value computing device that computes a parallax value between a plurality of captured image data captured by a plurality of imaging units that capture an imaging area to function as a parallax between the plurality of captured image data A parallax value calculating means for calculating a value for each unit image area, and a parallax value of the unit image area satisfying at least one parallax value error condition among a plurality of different parallax value error conditions from each other. The computer is caused to function as a parallax value correcting unit that performs correction using a correction process corresponding to the at least one parallax value error condition among a plurality of correction processes that differ for each condition.
According to this aspect, the parallax value (including an error value and a blank value) of the unit image region to be corrected is a value that satisfies at least one parallax value error condition among a plurality of different parallax value error conditions. is there. Therefore, if the error cause causing the parallax value error is set as a condition that can specify the parallax value error as these parallax value error conditions, the parallax value of the unit image area to be corrected can be distinguished for each error cause. In this aspect, there are a plurality of different correction processes for each of the plurality of parallax value error conditions, and for the parallax value of the unit image area to be corrected, correction corresponding to the parallax value error condition satisfied by the unit image area Corrections are made using the process. Therefore, the parallax value of the unit image area to be corrected can be corrected by an appropriate correction process corresponding to the cause of the error.
This program can be distributed or obtained in a state of being recorded on a recording medium such as a CD-ROM. It is also possible to distribute and obtain signals by placing this program and distributing or receiving signals transmitted by a predetermined transmission device via transmission media such as public telephone lines, dedicated lines, and other communication networks. Is possible. At the time of distribution, it is sufficient that at least a part of the computer program is transmitted in the transmission medium. That is, it is not necessary for all data constituting the computer program to exist on the transmission medium at one time. The signal carrying the program is a computer data signal embodied on a predetermined carrier wave including the computer program. Further, the transmission method for transmitting a computer program from a predetermined transmission device includes a case where data constituting the program is transmitted continuously and a case where it is transmitted intermittently.

DESCRIPTION OF SYMBOLS 100 Own vehicle 101 Imaging unit 102 Image analysis unit 103 Display unit 106 Vehicle travel control unit 110A, 110B Imaging part 113A, 113B Image sensor 115A, 115B Signal processing part 120 Processing hardware part 121 Parallax calculating part 122 Storage means 124 Brightness correction process Unit 125 correction processing unit 132 parallax image generation unit 132A flatness determination unit 132B luminance noise removal unit 132C parallax value calculation unit 132D reliability determination unit 132E reliability determination value change unit 132F peripheral complement processing unit 132G hardware error detection unit 132H frame Interpolation processing section

Japanese Patent No. 5239947

Claims (10)

In a parallax value calculation device including a parallax value calculation unit that calculates a parallax value between a plurality of captured image data captured by a plurality of imaging units that capture an imaging region for each unit image region,
A parallax value of a unit image area satisfying at least one parallax value error condition among a plurality of different parallax value error conditions is set as the at least one parallax value among a plurality of correction processes different for each of the plurality of parallax value error conditions. A parallax value computing device comprising parallax value correcting means for correcting using a correction process corresponding to an error condition. In the parallax value calculating apparatus according to claim 1,
The plurality of correction processes include a peripheral parallax value correction process for correcting a parallax value of a unit image area to be corrected using a parallax value of a peripheral unit image area positioned around the unit image area. A parallax value computing device. The parallax value calculation apparatus according to claim 2,
Having an error detection means for detecting a hardware error causing a locally incorrect parallax value;
The plurality of parallax value error conditions include a hardware error condition that is a unit image area corresponding to a hardware error detected by the error detection unit,
The peripheral parallax value correction process is a correction process corresponding to the hardware error condition. In the parallax value calculating device according to any one of claims 1 to 3,
The parallax value calculation means derives the reliability of the parallax value for each unit image area, calculates the parallax value for which the derived reliability exceeds a predetermined reliability threshold as the parallax value of the unit image area, A parallax value whose derived reliability is equal to or less than the predetermined reliability threshold value is not calculated as a parallax value of the unit image area,
The plurality of parallax value error conditions are such that a difference between a pixel value of a unit image area in at least one of the plurality of captured image data and a pixel value of a peripheral unit image area located around the unit image area is equal to or less than a specified value. Including the pixel value flatness condition,
The plurality of correction processes include a threshold value reduction process for reducing the predetermined reliability threshold value as a correction process corresponding to the pixel value flat condition. In the parallax value calculating device according to any one of claims 1 to 4,
The plurality of correction processes are a plurality of captured images obtained by capturing parallax values of a unit image area to be corrected at least before or after the imaging timing of the plurality of captured image data for which the parallax values are calculated. A parallax value calculation device comprising a time difference correction process for correcting using parallax values between data. In the parallax value calculating device according to claim 5,
The plurality of parallax value error conditions include a short-time error condition of being a unit image region that causes an erroneous parallax value for a short time,
The time difference correction process is a correction process corresponding to the short-time error condition. In the parallax value calculating device according to any one of claims 1 to 6,
Pixel value correcting means for correcting pixel values in the plurality of captured image data;
The parallax value calculation unit calculates a parallax value between a plurality of captured image data corrected by the pixel value correction unit for each unit image region. An imaging means comprising a plurality of imaging units for imaging the imaging region;
A parallax value calculation device that calculates a parallax value between a plurality of captured image data captured by the plurality of imaging units for each unit image area;
In an object recognition apparatus having a recognition processing means for recognizing a recognition object in an imaging region based on a parallax value calculated by the parallax value calculation apparatus.
An object recognition apparatus using the parallax value calculation apparatus according to claim 1 as the parallax value calculation apparatus. An object recognition device for recognizing a recognition object existing around a moving object;
In a mobile device control system comprising mobile device control means for controlling a predetermined device mounted on a mobile based on the recognition result of the object recognition device,
A mobile device control system using the object recognition device according to claim 8 as the object recognition device. A parallax calculation program for causing a computer of a parallax value calculation device that calculates a parallax value between a plurality of captured image data captured by a plurality of imaging units that capture an imaging region to function,
Parallax value calculating means for calculating a parallax value between the plurality of captured image data for each unit image area; and
A parallax value of a unit image area satisfying at least one parallax value error condition among a plurality of different parallax value error conditions is set as the at least one parallax value among a plurality of correction processes different for each of the plurality of parallax value error conditions. A parallax calculation program that causes the computer to function as parallax value correction means that corrects using a correction process corresponding to an error condition.

Images