JP6521796B2 - Stereo image processing device - Google Patents

Description

  The present invention relates to a stereo image processing apparatus that obtains parallax errors of a pair of left and right cameras mounted on a moving body and calibrates a parallax offset value.

  Conventionally, a three-dimensional measuring device that acquires three-dimensional distance information and recognizes the surrounding environment and the self position based on the distance information is known, and is mounted on a mobile object such as a vehicle or a helicopter and put to practical use There is. This type of three-dimensional measurement device finds the correlation between a pair of images of the same object captured by a pair of left and right cameras (stereo cameras), and uses camera parameters such as the attachment position of the stereo camera and focal length based on the parallax with respect to the same object. The distance to the object is calculated according to the principle of triangulation using.

  Therefore, in order to obtain stereo matching accuracy (reliable distance information), it is desirable that no positional deviation other than parallax be present in the stereo image. However, in actuality, the optical axis of the camera is shifted due to physical stress applied to the camera by screwing the camera at the time of attachment, etc. or distortion due to vibration or heat of the device equipped with the camera, etc. It may shift with time. If a parallax error occurs in the stereo camera, the parallax between the reference image and the comparison image does not correspond exactly to the actual distance of the object, and the accuracy of the three-dimensional position determination of the object by the stereo matching method decreases. The reliability of the distance information to the object is lost.

  In order to cope with this, for example, according to Patent Document 1 (Japanese Patent Laid-Open No. 2001-169310), when a stationary object such as a traffic light or a telephone pole fixed in front of the host vehicle is detected, the first time t0 The distance from the subject vehicle to the stationary object and the parallax at a second time when a predetermined time has elapsed from the first time t0 are detected, and further, the travel distance of the vehicle from time t0 to t1 is calculated. A technique is disclosed for obtaining a parallax error of a stereo camera based on parallax and a traveling distance, and calibrating a parallax offset value, which is an error caused by an inter-optical axis parallelism deviation of the stereo camera, using this parallax error.

JP 2001-169310 A

  However, with the technique disclosed in the above-mentioned document, the parallax offset value can not be calibrated when the stationary object is not detected, and there is a disadvantage that the opportunity to calibrate the parallax offset value can not be sufficiently ensured. .

  In particular, in recent vehicle control that generates a target travel path of a vehicle and performs various driving support such as lane keeping control (lane keeping assist) along the target travel path and automatic driving, white line recognition using a stereo camera The road shape ahead of the host vehicle is recognized, and the host vehicle is controlled to travel in the middle of the lane. Therefore, when there are few opportunities for calibration of the parallax offset value, the stereo matching accuracy is lowered, and there is a problem that it becomes difficult to make the vehicle travel continuously along the center of the lane in vehicle control.

  In view of the above circumstances, the present invention enables parallax error detection of a stereo camera without detecting a stationary still object, thereby increasing the opportunity for calibration of a parallax offset value and improving stereo matching accuracy. It is an object of the present invention to provide a stereo image processing apparatus that can

  A stereo image processing apparatus according to the present invention is mounted on a moving body, and a pair of left and right imaging means for imaging the front with the optical axis set parallel to the road surface, and a pair of images imaged by the pair of imaging means Vertical coordinate setting means for respectively setting an infinite distal vertical coordinate coincident with the optical axis, a near vertical coordinate and a far vertical coordinate located at a preset near road distance and a far road distance, and the pair of images A road surface parallax operation for extracting a characteristic portion on the road surface located at the near road surface distance to obtain a near road surface parallax, and extracting a characteristic portion on the road surface located at the distant road surface distance for obtaining a distant road surface parallax Means and a value obtained by multiplying the distance from the infinite distal vertical coordinate to the near vertical coordinate by the baseline length between the pair of imaging means divided by the near road surface parallax to obtain a road surface on the near road distance To the optical axis Near vertical height computing means for obtaining the near vertical height, and a value obtained by multiplying the distance from the infinite distal vertical coordinate to the far vertical coordinate by the baseline length between the pair of imaging means as the near vertical Based on the difference in optical axis parallelism of the pair of imaging means on the basis of the corrected far road surface parallax computing means for dividing by high to obtain the corrected far road surface parallax, and the difference between the corrected far road surface parallax and the far road surface parallax. And parallax error calculating means for obtaining a parallax error for calibrating the set parallax offset value.

  According to the present invention, the near road surface parallax is obtained based on the characteristic part on the road surface located at the near road surface distance ahead of the moving body, and the distant road surface parallax is obtained based on the characteristic part on the road surface located at the distant road surface distance. Also, the corrected distance road surface parallax is calculated using the distance from the infinity distal vertical coordinate to the distance vertical coordinate and the near vertical height, and from the difference between the corrected distance road surface parallax and the distance road surface parallax, the parallax is calculated Since the parallax error for calibrating the offset value is calculated, it is possible to detect the parallax error of the stereo camera without detecting the fixed stationary object, and the opportunity of the calibration of the parallax offset value increases, and the stereo matching Accuracy can be improved.

Functional block diagram of stereo image processing apparatus according to the first embodiment Same as functional block diagram of disparity correction processing unit Similarly, a flowchart showing a parallax offset calibration processing routine (part 1) Similarly, a flowchart showing a parallax offset calibration processing routine (part 2) It is an explanatory view showing the camera height in the near road surface distance and the camera height in the near road surface distance in the actual coordinates. The same characteristic chart showing the relationship between parallax and distance Similarly, the relationship between the road surface distance to each feature point between near and far and the parallax is shown, (a) is a plan view, (b) is a side view Similarly, (a) is an explanatory view of an image taken by the main camera, (b) is an explanatory view of an image taken by the sub camera, and (c) is a parallax on the road surface distance in the near and far distances taken by both cameras. Explanation Each explanatory drawing of the FIG. 8 equivalent by the same and another aspect Explanatory drawing which shows the camera height from the road surface which the near preceding vehicle rear wheel in real coordinates in the 2nd embodiment contacts to the ground, and the camera height from the road surface which a distant leading vehicle rear wheel contacts to the ground Similarly, (a) is an explanatory view of an image in which the rear wheels of the preceding vehicle are in contact with the road surface near the road distance, and (b) is an explanatory view of the image in which the rear wheels of the preceding vehicle are in contact with a road surface

  Hereinafter, an embodiment of the present invention will be described based on the drawings.

First Embodiment
The stereo image processing apparatus 1 shown in FIG. 1 is mounted on a mobile body traveling on a road surface, such as a vehicle represented by a car. In addition, below, the case where this stereo image processing apparatus 1 is mounted in the vehicle 20 as a moving body is illustrated and demonstrated.

  The stereo image processing apparatus 1 has an image processing unit 2 mainly composed of a computer, and an on-vehicle camera 5 is connected to the input side of the image processing unit 2 via A / D converters 3 and 4. There is.

  The on-vehicle camera 5 is a stereo camera including a main camera 5a and a sub-camera 5b each incorporating image sensors 6a and 6b (see FIG. 7) such as CCD and CMOS as imaging means, and this pair of cameras 5a , 5b are respectively mounted in front of the ceiling in the vehicle compartment at equal intervals on the right and left sides across the center in the vehicle width direction, and perform stereo imaging from different viewpoints of the environment outside the vehicle.

  The main camera 5a is disposed on the right side in the present embodiment, picks up a reference image (right image) necessary for performing stereo image processing, and the sub camera 5b disposed on the left side Capture the left image). The pair of captured left and right analog images are converted into digital images by the A / D converters 3 and 4 in a state where they are synchronized with each other, and transmitted to the image processing unit 2.

  The image processing unit 2 includes an image correction unit 11, a stereo image processing unit 12 as stereo image processing means, a parallax correction processing unit 13, a distance data storage unit 14, an image data storage unit 15, and an image recognition unit 16.

  The image correction unit 11 performs luminance correction, geometric conversion of an image, and the like on image data. Usually, there is an error in the attachment position of the pair of left and right cameras 5a and 5b, and the displacement resulting therefrom is generated in the left and right images. In order to correct this deviation, geometrical transformation such as rotation or translation of the image is performed using affine transformation or the like. By such image correction processing, reference image data is generated from the main camera 5a based on the image data, and comparison image data is generated based on the image data from the sub camera 5b.

The two image data are sent to the stereo image processing unit 12. The stereo image processing unit 12 is based on the reference image data and the comparison image data, regarding the captured image equivalent to one frame, using the principle of triangulation from the parallax dp for the same object, the distance to the object (distance data) D ,
D = f · L / dp (1)
Calculated from Here, f is the focal length, and L is the distance (baseline length) between the parallel optical axes of the left and right cameras 5a, 5b. Here, D is the distance from the position of the focal length f to the object.

  The parallax dp is a shift amount in the horizontal direction between the reference image data obtained by imaging the same object and the comparison image data, and is obtained by multiplying the parallax pixel number x by the pixel pitch ωi [mm / pix] on the image. (Dp = x · ωi). Then, the distance data is transmitted to the parallax correction processing unit 13.

  As shown in FIG. 2, the parallax correction processing unit 13 includes a parallax offset calibration unit 13 a and a position correction unit 13 b. The parallax offset calibration unit 13a sets a road surface distance on the near side (hereinafter referred to as “near road surface distance”) and a road surface distance on the far side (hereinafter referred to as “preceding near road surface distance”). Two feature parts that are common from the road surface image on the “distant road surface distance” Db are extracted, and the road surface parallax at the near road surface distance Da by stereo matching processing from each feature part (hereinafter, “near road surface parallax ) Md1, md2 and road surface parallax at a distant road surface distance Db (hereinafter referred to as "far road surface parallax") nd1, nd2 are determined. Then, the parallax error gd with respect to the far road surface parallax nd1 and nd2 average value is calculated based on the average value of the near road surface parallaxes md1 and md2, and the parallax error between the optical axes of both cameras 5a and 5b is calculated with this parallax error gd. The disparity offset value DP, which is preset, is calibrated in order to correct the resulting error.

  The position correction unit 13 b corrects the parallax dp set by the stereo image processing unit 12 with the parallax offset value DP, and obtains the distance D of the object based on the corrected parallax dp. Then, the data of the distance D to the object and the image data corresponding to the data of the distance D are stored in the distance data storage unit 14 and the image data storage unit 15, respectively.

  The image recognition unit 16 recognizes an object using the image data stored in the image data storage unit 15, and based on the distance data stored in the distance data storage unit 14, three-dimensional the object. Recognize a certain position. The three-dimensional position information of the object recognized by the image recognition unit 16 is appropriately read according to the purpose of use. For example, when the stereo image processing apparatus 1 is mounted on the vehicle 20, a dividing line dividing the left and right of the lane in which the vehicle 20 travels, a leading vehicle, a three-dimensional object such as a traffic light, and the like are objects to be recognized.

  The calibration process of the parallax offset value DP calculated by the above-described parallax offset calibration unit 13a will be described in more detail.

  As shown in FIG. 5, when the optical axis of each of the cameras 5a and 5b is parallel to the road surface, the vertical height (hereinafter referred to as "camera height") H from the road surface to the optical axis is It does not change even if it measures. Further, as shown in FIG. 6, the parallax is larger as it gets closer, smaller as it gets farther, and in inverse proportion to approach 0 at infinity.

  Therefore, the near camera height Ha which is the vertical height from the road surface to the light axis at the near road surface distance Da and the far camera height Hb which is the vertical height from the road surface to the light axis at the far surface distance Db are different. If so, it is estimated that a parallax error has occurred. At that time, since the far camera height Hb is more affected by the parallax error than the near camera height Ha, the far camera height Hb is calculated based on the near camera height Ha (correct value). The obtained parallax is corrected, and the parallax error gd for calibrating the parallax offset value DP is calculated from the difference between the corrected parallax and the parallax on the far side.

  The calibration of the parallax offset value DP performed by the parallax offset calibration unit 13a is performed according to a parallax offset calibration process routine shown in FIGS. 3 and 4. This routine is executed for each frame or for each predetermined frame on the images captured by the pair of left and right cameras 5a and 5b.

  When this routine is started, first, in step S1, disparity offset calibration start condition determination processing is executed. In the present embodiment, it is assumed that the camera height H does not change, so in the behavior of the vehicle 20, when any of pitching, yawing, and rolling is detected, the start condition is not satisfied. In addition, since the road needs to keep the road surface and the optical axis parallel, at least from the vehicle 20 to the distant road surface distance Db, the road needs to be a flat and straight road. Therefore, when a broken line indicating the start of the slope or the end of the slope is detected ahead, the start point of the curve is detected, or the road breaks before the far road surface distance Db in uphill driving ( In the case where the uphill road is completed), the start condition is not satisfied.

  Then, the process proceeds to step S2, and it is checked whether or not all of the start conditions are satisfied in the determination processing in step S1. If even one is not satisfied, it is determined that the start conditions are not satisfied, and the routine is exited. On the other hand, if all the start conditions are satisfied, it is determined that the disclosure condition is satisfied, and the process proceeds to step S3.

  When the process proceeds to step S3, one frame image captured by the pair of left and right cameras 5a and 5b is captured, and vertical coordinates at infinity distal are set. FIG. 8A shows an image (right image) captured by the main camera 5a, and FIG. 8B shows an image (left image) captured by the sub camera 5b. In both images, the lower left is the origin, the vertical direction is the J coordinate, the horizontal direction is the I coordinate, and the coordinates are represented by pixels.

  Also, in the figure, a straight road with one lane on one side is displayed as an example of the image. Here, the vehicle is left-handed, and the center line L1 after binarization processing by the brightness difference, the roadside division line L2 of the traveling lane, the roadside division line L3 of the opposite lane, and a plurality In the embodiment, the edges Pa, Pb, Qa, Qb of the two feature portions are displayed. This characteristic part is not particularly limited as long as a difference in luminance occurs between the road surface and an edge such as a tire mark, a dent, a puddle, or remaining snow formed on the road surface.

  Further, an infinite distal vertical coordinate (hereinafter referred to as "vertical coordinate") JV set as infinite distal on the J coordinate (vertical coordinate) is because the optical axis of each camera 5a, 5b is parallel to the road surface , Coincides with the horizon. In this case, the infinite distal end is set, for example, by detecting a point at which the center line L1 and the left and right roadside division lines L2 and L3 disappear on the optical axis.

  Then, the process proceeds to step S4, and it is checked whether or not the vertical coordinate JV is set. If the vertical coordinate JV can not be set, the routine is exited. If the vertical coordinate JV is set, the process proceeds to step S5.

  In step S5, the near J coordinate ja corresponding to the near road surface distance Da is set as the J coordinate, and in step S6 the far J coordinate jb corresponding to the distant road surface distance Db is set as the J coordinate. In this case, in the present embodiment, the near road surface distance Da is set to about 10 to 14 [m], and the far road surface distance Db is set to about 40 [m], but it is not limited thereto. An optimal value is set based on the type of vehicle camera 5 and the mounting state thereof.

  However, the camera height (optical axis height) H remains unchanged at any position, and the near camera height Ha and the far camera height Hb are set to the same fixed height. Both road surface distances Da and Db do not necessarily have to be accurately determined. The near J coordinate ja and the far J coordinate jb are set based on the depression angles θja and θjb set based on the road surface distances Da and Db, for example, as shown in FIG. 7B. The processes from step S3 to step S6 correspond to the vertical coordinate setting means of the present invention.

  Thereafter, the process proceeds to step S7, and the near road surface parallaxes md1 and md2 of the edges Pa and Qa of the two characteristic portions on the near J coordinate ja of the images captured by both cameras 5a and 5b are calculated from the following equation (2) In the following step S8, far road surface parallaxes nd1, nd2 of the edges Pb, Qb of the two feature parts on the far J coordinate jb are calculated from the following equation (3). The processes in steps S7 and S8 correspond to the road surface parallax calculating means of the present invention.

md1, md2 = f · L / Da (2)
nd1, nd2 = f · L / Db (3)
Further, in the present embodiment, the edges Pa, Qa and Pb, Qb at two different positions are determined on the J coordinates ja and jb, but the parallax is averaged from the edges of a plurality of feature portions. It is possible to derive the parallax by averaging the edges of three or more feature parts on each of the J coordinates ja and jb, since the parallax can be derived with higher accuracy.

Thereafter, the process proceeds to step S9, and the near camera height Ha [mm],
Ha = (L. (ja-JV) .ωj) / (ωi. (Md1 + md2) / 2) (3)
Calculated from Here, ωj is a pixel pitch [mm / pix] in the J coordinate direction, and ωi is a pixel pitch [mm / pix] in the I coordinate direction. The processing in this step corresponds to the near vertical height calculation means of the present invention.

Next, the process proceeds to step S10, where the far camera height Hb {mm] is
Hb = (L. (jb-JV) .. omega.j) / (. Omega.i. (Nd1 + nd2) / 2) (4)
Calculated from

  Thereafter, the process proceeds to step S11, the data JV, ja, jb, md1, md2, nd1, nd2, Ha, Hb obtained this time are stored in the memory, and the process proceeds to step S12. If the predetermined value is exceeded, the process proceeds to step S13. If the predetermined value is not reached, the routine is exited and data is sampled again.

  In step S13, the near-side linear approximation straight line is created by the least squares method based on the near road surface parallaxes md1, md2 and the near camera height Ha, and the far road surface parallax nd1, nd2 with the far camera Based on the height Hb, a far-side first-order approximate straight line is created by the least squares method, and the process proceeds to step S14. In step S14, it is checked whether or not both linear approximate straight lines, that is, the intercept and inclination have been obtained, and if the linear approximate curve can not be created, it is determined that the data is not valid, and the process branches to step S15. Discard the data of and exit the routine. On the other hand, when both the near side linear approximate straight line and the far side linear approximate curve are created, it is determined that the data is valid, and the process proceeds to step S16. The processing from step S11 to step S15 corresponds to the data determination means of the present invention.

  Then, in step S16, each data is divided into near road surface data (ja, md1, md2, Ha) and far road surface data (JV, jb, nd1, nd2, Hb) and stored in the memory.

  Thereafter, the process proceeds to step S17, and it is checked whether the number of data stored in the near road surface data and the distant road surface data is equal to or greater than a predetermined number. If the predetermined number is not reached, the routine is exited and data is accumulated again. . On the other hand, if the number is equal to or more than the predetermined number, the process proceeds to step S18, the parallax offset value calibration process is performed, and the routine is exited.

  In step S18, first, the frequency distribution of variation for the data ja, md1, md2, Ha stored in the near road surface data and the data JV, jb, nd1, nd2, Hb stored as the distant road surface data (Histogram) is created, and the highest frequency data ja, md1, md2, Ha, JV, jb, nd1, nd2, Hb are extracted.

  By the way, as shown in FIG. 7, the near road surface parallax md1 (md2) on the image sensors 6a and 6b is larger than the distant road surface parallax nd1 (nd2), and the influence of the parallax error gd is the far road surface parallax nd1 (nd2) Is larger than the near road surface parallax md1 (md2). Furthermore, when the camera heights Ha and Hb are calculated, the near camera height Ha is closer than the far camera height Hb, and the parallax is larger, so the accuracy is high, and the near camera height Ha is more true value ( The value is close to the camera height H).

Therefore, since the near camera height Ha can be estimated to be a more reliable value than the far camera height Hb, the near camera height Ha is substituted for the far camera height Hb of the above-mentioned equation (4). Distance road surface distance Db (corrected distance road surface parallax) Cd [pix],
Cd = (L (jb−JV) · ωj) / (ωi · Ha) (5)
Calculated from This process corresponds to the corrected far road surface parallax calculating means of the present invention.
Then, based on the corrected far road surface parallax Cd and the average value of the two far road surface parallaxes nd1 and nd2, the parallax error gd [pix] is
gd = Cd− (nd1 + nd2) / 2 (6)
Calculated from

  Then, the latest parallax offset value DP (n) is calibrated with the parallax error gd, and a new parallax offset value DP is set (DP ← DP (n) −gd). Then, the disparity offset value DP is sequentially updated. The processing here corresponds to the parallax error calculation means of the present invention. In the case of gd = 0, the current disparity offset value DP (n) indicates a correct value.

  In this embodiment, since the parallax offset value DP is sequentially calibrated by the parallax error gd, the parallax offset value DP, which changes every moment under the influence of weather, temperature, barometric pressure, aging deterioration, etc., can always be correctly calibrated. . At this time, in the present embodiment, the characteristic portion is extracted from the image obtained by imaging the road surface with the near road surface distance Da and the far road surface distance Db, and the parallax error gd is obtained based on the parallax. It is not necessary to sequentially detect a stationary object fixed to the point d, and the opportunity for calibration of the disparity offset value DP is increased, and stereo matching accuracy can be improved.

  By the way, in the above-described embodiment, the characteristic portion is intended for tire marks formed on the road surface, dents, water pools, residual snow, etc. which are formed artificially or spontaneously, for example, as shown in FIG. As in the right image shown in (a) and the left image shown in FIG. (B), the center line L1 and the roadside division line L2 that divide the traveling lane may be extracted as the characteristic part. Then, the inner edge Qr of the center line L1 extracted from the right image, the inner edge Pr of the roadside division line L2, the inner edge Q1 of the centerline L1 extracted from the left image, and the inner edge Pl of the roadside division line L2 It is also possible to detect from the luminance difference and obtain the near road surface parallaxes md1 and md2 on the near J coordinate ja and the distant road surface parallax nd1 and nd2 on the far J coordinate jb as shown in FIG. .

Second Embodiment
FIG. 11 shows a second embodiment of the present invention. In the first embodiment described above, the characteristic error formed on the road surface located at the near road surface distance Da and the road surface located at the far road surface distance Db is extracted to obtain the parallax error gd. On the other hand, in the present embodiment, when the left and right rear wheels of the near preceding vehicle 31 traveling immediately before are present on the near road surface distance Da, the road surface on which the left and right rear wheels are in contact is extracted as the feature portion. From the inner edges (or outer edges) Pa and Qa indicating the boundary between the left and right rear wheels and the road surface, near road surface parallaxes md1 and md2 as shown in FIG. 8C are obtained.

  Similarly, when the left and right rear wheels of the distant preceding vehicle 32 traveling immediately before are present on the distant road surface distance Db, the road surface on which the left and right rear wheels are in contact is extracted as a characteristic portion, and the inner edge (or outer edge) The far road surface parallaxes nd1 and nd2 shown in FIG. 8 (c) are obtained from Pb) and Qb. The distant preceding vehicle 32 does not have to be the same as the near preceding vehicle 31 described above, and even if it is a different vehicle, the distant road surface parallax nd1, nd2 can be obtained.

  Further, in FIG. 10, the distant preceding vehicle 32 is described in front of the near preceding vehicle 31 for convenience. However, when the on-vehicle camera 5 recognizes the distant preceding vehicle 32, the distant leading vehicle 32 There is no other vehicle such as the near preceding vehicle 31 etc. between 32 and 32, so the near road surface parallaxes md1, md2 and the far road surface parallaxes nd1, nd2 are detected at different timings.

  As described above, in the present embodiment, the left and right rear wheels of the near preceding vehicle 31 traveling on the near road surface distance Da and the far preceding vehicle 32 traveling on the far road surface distance Db are in contact with the road surface Is extracted as the feature portion, the parallax error gd can be obtained even when the feature portion is not formed on the road surface. Therefore, for example, even on a paved road which is newly laid and on which it is difficult to extract a characteristic portion, the parallax error gd can be easily obtained by detecting the leading vehicles 31 and 32.

1 ... stereo image processing device,
2 ... Image processing unit,
5 ... In-vehicle camera,
5a ... main camera,
5b ... sub camera,
6a, 6b ... image sensor,
11: Image correction unit
12: Stereo image processing unit,
13 ... disparity correction processing unit,
13a ... parallax offset calibration unit,
13b ... position correction unit,
14 ... distance data storage unit,
15: Image data storage unit,
16: Image recognition unit
20: Vehicle,
31 ... Near front car,
32 ... Far ahead car,
Cd ... corrected distant road surface parallax,
Da ... Near road distance,
Db ... distant road surface distance,
DP ... disparity offset value,
f ... focal length,
gd ... parallax error,
Ha ... Near camera height,
Hb ... distance camera height,
ja ... near J coordinates,
jb ... distant J coordinate,
JV ... infinite distal vertical coordinate,
L: Baseline length,
L1 ... center line,
L2 ... roadside section line,
L3 ... roadside section line,
md1, md2 ... near road surface parallax,
nd1, nd2 ... distant road surface parallax,
Pa, Pb, Qa, Qb ... edge,
Pl, Pr, Ql, Qr ... inner edge,
x ... number of parallax pixels,
θja, θjb...
ωi, ωj ... pixel pitch,

Claims (4)

A pair of left and right imaging means mounted on a moving body and imaging the front with the optical axis set parallel to the road surface;
In a pair of images captured by the pair of imaging means, an infinite distal vertical coordinate coinciding with the optical axis, a near vertical coordinate located at a near road surface distance and a preset near road surface distance, and a far vertical coordinate respectively Vertical coordinate setting means to be set;
A characteristic portion is extracted on the road surface located at the near road surface distance of the pair of images to obtain a near road surface parallax, and a characteristic portion is extracted on the road surface located at the distant road surface distance to obtain a distant road surface parallax Road surface parallax calculating means to be obtained;
A value obtained by multiplying the distance between the infinite distal vertical coordinate and the near vertical coordinate by the baseline length between the pair of imaging means is divided by the near road surface parallax to obtain the light from the road surface on the near road surface distance A near vertical height computing means for obtaining a near vertical height to the axis;
A correction distance road surface parallax calculating unit which obtains a corrected distance road surface parallax by dividing a value obtained by multiplying the distance from the infinite distal vertical coordinate to the distance vertical coordinate by the base length between the pair of imaging units by the near vertical height When,
Parallax error calculating means for obtaining a parallax error for calibrating a parallax offset value set based on an inter-optical axis parallelism deviation of the pair of imaging means based on a difference between the corrected far road surface parallax and the far road surface parallax; A stereo image processing apparatus comprising: At least a predetermined number of the near road surface parallax and the far road surface parallax obtained by the road surface parallax calculation means are sampled, the validity of the sampled data is judged, and when it is judged not to be valid, the data is discarded. The stereo image processing apparatus according to claim 1, further comprising data determination means for not executing the corrected far road surface parallax calculation means and the parallax error calculation means. The road surface parallax calculating means sets the near road surface parallax as an average value of the near road surface parallax calculated from at least two characteristic portions on the near road surface distance, and the far road surface parallax is at least 2 on the far road surface distance. The stereo image processing apparatus according to claim 1 or 2, characterized in that it is an average value of far road surface parallaxes obtained from the two feature parts. The road surface parallax calculating means obtains the near road surface parallax and the distant road surface parallax based on a common portion where a luminance difference occurs between the edge of the characteristic portion and the road surface from the pair of images. The stereo image processing apparatus in any one of -3.

Images