JP6322024B2 - Outside monitoring device - Google Patents

Description

  The present invention relates to an out-of-vehicle monitoring device that recognizes an out-of-vehicle environment based on an image captured by an in-vehicle camera.

  In recent years, in order to improve vehicle safety, driving support systems that actively support driving operations by drivers have been developed. As this type of driving support system, for example, the vehicle exterior environment is imaged by an in-vehicle camera, and a solid object such as a white line laid on the road surface or a preceding vehicle on the road surface is recognized from the captured image, and the recognition information is On the basis of this, there is known one that performs various driving support controls such as vehicle speed control and steering support control.

  By the way, in the outside monitoring apparatus adopted in such a driving support system, it is required that the camera accuracy, the image processing accuracy, and the like be extremely high. It is particularly important that the adjustment is performed with high accuracy. For example, in order to accurately and stably recognize white lines and the like when monitoring the outside of the vehicle in front of the vehicle, it is important that the optical axis of the camera is adjusted with high accuracy in the horizontal direction with respect to the longitudinal axis of the vehicle body. It becomes.

  On the other hand, for example, in Patent Document 1, detection processing of a simulated lane marking displayed on the road surface (specifically, two parallel straight lines similar to the traveling road dividing line) is performed at an inspection factory or the like. The simulated lane marking position is converted into coordinates on the real plane using the default camera parameters, and the deviation angle between the center line of the simulated lane marking on the real plane and the vertical axis of the vehicle local coordinate system is detected. Thus, a technique for calculating a camera parameter correction value is disclosed.

Japanese Patent No. 3600378

  However, the technique disclosed in Patent Document 1 described above has predetermined conditions such as the presence of left and right white lines (lane markings) on the road surface, and guaranteeing that these white lines are parallel to each other. Unless it is a certain inspection factory or the like, it is difficult to detect the deviation (tilt) of the imaging optical axis of the camera.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an out-of-vehicle monitoring device capable of detecting in real time a shift in the imaging optical axis under various traveling environments.

  An out-of-vehicle monitoring device according to an aspect of the present invention includes an imaging unit that images an environment outside a vehicle in front of the host vehicle, and a white line that divides the traveling path of the host vehicle based on an image captured by the imaging unit. A white line approximating means for approximating at a first time, and calculating a first lateral movement amount based on a constant of the approximate expression, based on the constant of the approximate expression, the amount of change in the lateral position of the vehicle with respect to the white line when the vehicle travels for a set time. A lateral movement amount calculating means and a change amount of the lateral position of the vehicle with respect to the white line when the vehicle has traveled for the set time are calculated as a second lateral movement amount based on the first order coefficient of the approximate expression. Light in the horizontal direction of the imaging means when the difference between the second lateral movement amount calculating means and a difference between the first lateral movement amount and the second lateral movement amount is greater than or equal to a preset threshold value Optical axis misalignment judgment hand that determines that an axis misalignment has occurred When, those having a.

  According to the outside monitoring apparatus of the present invention, it is possible to detect the deviation of the imaging optical axis in real time under various traveling environments.

1 is a schematic configuration diagram of a vehicle driving support system according to a first embodiment of the present invention. Explanatory drawing which shows the effective area cut out from a captured image as above As above, an explanatory diagram schematically showing an example of a captured image of the environment outside the vehicle As above, an explanatory diagram showing white line candidate points detected from the image of FIG. As above, an explanatory diagram showing an example of a change in luminance on the search line Explanatory drawing which shows the white line candidate point projected on real space same as the above (A) is explanatory drawing which shows the moving amount | distance of the own vehicle by driving | running | working, (b) is explanatory drawing which shows the relative moving amount | distance with respect to the own vehicle of the white line candidate point detected by the front frame. As above, an explanatory diagram showing a white line approximation line calculated using white line candidate points and a white line detection region used in the next frame Same as above, a flowchart showing a system stop determination routine Same as above, flowchart showing the optical axis misalignment determination subroutine in the horizontal direction The flowchart which shows the optical-axis shift | offset | difference determination subroutine to a perpendicular direction in connection with the 2nd Embodiment of this invention. Explanatory drawing which shows the calculation method of the distance from the own vehicle to a white line candidate point same as the above Explanatory drawing which shows the height calculation method of a white line candidate point on the basis of the own vehicle as above Same as above, explanatory diagram showing the slope state of the road surface relative to the host vehicle

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The drawings relate to the first embodiment of the present invention, FIG. 1 is a schematic configuration diagram of a vehicle driving support system, FIG. 2 is an explanatory diagram showing an effective area cut out from a captured image, and FIG. 3 is a captured image of an environment outside the vehicle FIG. 4 is an explanatory diagram schematically showing an example, FIG. 4 is an explanatory diagram showing white line candidate points detected from the image of FIG. 3, FIG. 5 is an explanatory diagram showing an example of a change in luminance on the search line, and FIG. FIG. 7A is an explanatory diagram showing the amount of movement of the host vehicle due to traveling, and FIG. 7B is a relative diagram of the white line candidate point detected in the previous frame relative to the host vehicle. FIG. 8 is an explanatory diagram showing a white line approximation line calculated using white line candidate points and a white line detection area used in the next frame, FIG. 9 is a flowchart showing a system stop determination routine, and FIG. The optical axis misalignment determination subroutine in the horizontal direction It is to flow chart.

  In FIG. 1, reference numeral 1 denotes a vehicle (own vehicle) such as an automobile, and a driving support system 2 is mounted on the vehicle 1. The driving support system 2 includes, for example, a stereo camera 3 as an imaging unit, a stereo image recognition device 4, a control unit 5, and the like. Of these configurations, in the present embodiment, an out-of-vehicle monitoring device for recognizing an out-of-vehicle environment ahead of the host vehicle is mainly configured by the stereo camera 3 and the stereo image recognition device 4.

  The host vehicle 1 is connected to a vehicle speed sensor 11 that detects the host vehicle speed V, a yaw rate sensor 12 that detects the yaw rate γ, a main switch 13 that performs ON / OFF switching of each function of the driving support control, and the like, and a steering wheel. A steering angle sensor 14 that detects the steering angle θst that is opposed to the steering shaft, an accelerator opening sensor 15 that detects an accelerator pedal depression amount (accelerator opening) θacc by the driver, and the like are provided.

  The stereo camera 3 is constituted by a set of CCD cameras using a solid-state imaging device such as a charge coupled device (CCD) as a stereo optical system. These left and right CCD cameras are each mounted at a certain distance in front of the ceiling in the vehicle interior, take a stereo image of an object outside the vehicle from different viewpoints, and output image data to the stereo image recognition device 4. In the following description, one image (for example, the right image) of the stereo images is referred to as a reference image, and the other image (for example, the left image) is referred to as a comparative image.

  Here, in the present embodiment, the stereo image recognition device 4 uses, for example, an area of a predetermined size as the effective area Ag among the image data (captured images) input from the stereo camera 3 as shown in FIG. For example, the following various image processing is performed on the cut-out and cut-out reference image and comparison image. Note that the effective area Ag is set, for example, in the approximate center of the captured image in the initial setting state.

  That is, the stereo image recognition apparatus 4 first divides the reference image into small areas of, for example, 4 × 4 pixels, finds a corresponding area by comparing the luminance or color pattern of each small area with the comparative image, Find the distance distribution over the entire image. Further, the stereo image recognition device 4 examines the luminance difference between each pixel on the reference image and an adjacent pixel, extracts those whose luminance difference exceeds a threshold value as an edge point, and extracts the extracted pixel (edge A distance image (a distribution image of edge points having distance information) is generated by giving distance information to (point). Then, the stereo image recognition device 4 performs a known grouping process on the generated distance image and compares it with a pre-stored three-dimensional frame (window), so that a white line and a side wall in front of the vehicle Recognize three-dimensional objects.

  Here, the white line to be recognized in the present embodiment is, for example, a single lane line or a multiple line (double line or the like) in which a line of sight guide line is provided inside the lane line, Lines that extend on the road and divide the vehicle lane are collectively referred to, and the form of each line includes a solid line, a broken line, and the like, and further includes a yellow line and the like. In the white line recognition of the present embodiment, even if the white line existing on the road is a double white line or the like, it is recognized by approximating it with a single approximate expression (a linear or curved approximate expression of the first or higher order). It shall be.

  By the way, the white line laid on the actual road has various variations as described above, and the form of the white line changes with the branching or merging of the traveling road. In addition, there are various noises such as road surface repair marks, water pools and snow on the road. Therefore, it is difficult to accurately recognize the white line in all scenes by uniform processing. Therefore, the stereo image recognition device 4 complementarily performs white line recognition using various methods without relying only on white line recognition using pattern matching based on the distance image as described above, and comprehensively recognizes these recognition results. Judgment to recognize the final white line.

  As one of such white line recognition, the stereo image recognition device 4 recognizes the left and right white lines based on a change in luminance in the horizontal direction on one image (for example, the reference image shown in FIG. 3) taken in front of the vehicle. I do.

  More specifically, for example, as shown in FIG. 4, the stereo image recognition device 4 sets left and right white line detection areas Ad (Adl, Adr) on the image, and horizontally in each white line detection area Ad. For a plurality of search lines 1 extending, a change in luminance is examined from the inside to the outside in the vehicle width direction for each search line l. Then, for example, as shown in FIG. 5, the stereo image recognition device 4 has a luminance differential that indicates that the luminance of the pixels on the outer side in the vehicle width direction is relatively higher than the luminance of the inner pixels and indicates the amount of change. A point (edge point) whose value is greater than or equal to the set threshold on the plus side is detected. Then, the stereo image recognition device 4 pours out the first edge point whose luminance changes more than a predetermined value from dark to bright on each search line l in each white line detection area Ad as a white line candidate point P, and in real space (See FIG. 6).

Further, for example, the stereo image recognition device 4 is based on the vehicle speed V of the host vehicle 1 and the yaw angle θ obtained from the yaw rate γ based on the relationship shown in FIG. The movement amounts Δx and Δz of the host vehicle 1 during the period until one frame is updated are calculated using the following equations (1) and (2).
Δx = V × sin θ (1)
Δz = V × cos θ (2)
Then, as shown in the following formulas (3) and (4), the stereo image recognition device 4 performs the own vehicle movement amount Δx, Δz with respect to the white line candidate point Pold (Xold, Zold ) detected before the previous frame. Then, the coordinates of the white line candidate point Ppre existing within a predetermined distance behind the host vehicle 1 are calculated by performing coordinate conversion to the vehicle fixed coordinate system (X, Z) in the current frame (FIG. 7). (See (b)).
Xpre = (Xold · Δx) × cos θ− (Zold · Δz) × sin θ (3)
Zpre = (Zold · Δx) × sin θ + (Zold · Δz) × cos θ (4)
Then, the stereo image recognition device 4 is based on a point sequence including the white line candidate point P detected in the current frame and the white line candidate point Ppre obtained by coordinate conversion of the white line candidate point Pold before the previous frame. The white line approximate lines Ll and Lr are defined by the approximate expression shown in the following expressions (5) and (6) using the least square method (see FIG. 8).

  Here, in the equations (5) and (6), the secondary coefficient A (Al, Ar) is a parameter indicating the curvature component of the white line, and the primary coefficient B (Bl, Br) is the lane yaw angle component (own vehicle) Is a parameter indicating a lane position (a horizontal position of the white line with respect to the host vehicle 1).

  Further, the stereo image recognition device 4 sets the white line detection areas Adl and Adr in the next frame by offsetting the left and right white line approximate lines Ll and Lr by ΔE respectively in the vehicle width direction inside and outside (see FIG. 8). .

  The stereo image recognition device 4 repeats the calculation of the white line approximation formula every set time in this way, so that the self-vehicle 1 corresponds to the white line when the vehicle 1 travels for the set time T0 (for example, T0 = several tens of seconds to several minutes). The lateral position change amount (first lateral movement amount Xc) of the own vehicle 1 is calculated based on the constant C of the white line approximation formula, and the lateral position of the own vehicle with respect to the white line when the own vehicle travels for the set time T0. Change amount (second lateral movement amount Xb) is calculated based on the primary coefficient B of the white line approximation formula. Then, the stereo image recognition device 4 compares the first lateral movement amount Xc and the second lateral movement amount Xb, and the lateral movement error Err that is the difference between them is equal to or greater than a preset threshold value Eth1. If it is, it is determined that the optical axis shift in the horizontal direction of the stereo camera 3 has occurred.

  Here, the stereo camera 3 of the present embodiment is configured such that the left and right cameras are integrally held via a stay or the like (not shown), and the optical axis between the left and right cameras is strictly adjusted in a factory or the like. Is. Therefore, in principle, no deviation occurs between the optical axes of the left and right cameras. Therefore, the optical axis shift assumed in the present embodiment is, for example, when the occupant or the like applies a strong impact to the stereo camera 3 or due to secular change of the vehicle body, etc. This is a case where the optical axis is displaced. That is, the optical axis shift of the stereo camera 3 in the present embodiment means that, for example, the optical axes of the left and right cameras are inclined in the horizontal direction at the same angle with respect to the longitudinal axis of the host vehicle 1.

  In the detection of the optical axis deviation of the stereo camera 3 in the present embodiment, the first and second lateral movement amounts Xc and Xb are, for example, average values of the lateral movement amounts calculated for the left and right white lines. Used. However, when only one of the left and right sides of the white line is detected, the lateral movement amount calculated for the detected white line is used as it is as the first and second lateral movement amounts Xc and Xb. It is done.

  Further, when the stereo image recognition device 4 determines that the horizontal optical axis shift has occurred in the stereo camera 3, the position of the effective area Ag cut out from the left and right captured images captured by the stereo camera 3 is determined as the first and first positions. 2 is corrected with a correction amount Δi corresponding to the difference Err between the lateral movement amounts Xc and Xb (see the broken line in FIG. 2). That is, the stereo image recognition device 4 corrects the optical axis shift of the stereo camera 3 by adjusting the cutout position.

  When the stereo image recognition device 4 determines that the horizontal optical axis shift has occurred in the stereo camera 3, the stereo image recognition device 4 determines that the vehicle control based on the vehicle exterior environment (travel environment information) recognized from the stereo image is stopped. The control unit 5 is instructed to stop various driving support controls. Further, when the stereo image recognition device 4 determines that the optical axis shift of the stereo camera 3 exceeds the range that can be corrected by adjusting the cut-out position, the stereo image recognition device 4 outputs a warning that the optical axis adjustment is performed in a dealer or the like.

  As described above, in the present embodiment, the stereo image recognition device 4 includes the white line approximation unit, the first lateral movement amount calculation unit, the second lateral movement amount calculation unit, the optical axis deviation determination unit, the effective area correction unit, and Each function as a control stop determination means is realized.

  The control unit 5 receives driving environment information in front of the host vehicle 1 recognized by the stereo image recognition device 4. Further, the vehicle speed V from the vehicle speed sensor 11, the yaw rate γ from the yaw rate sensor 12, and the like are input to the control unit 5 as travel information of the host vehicle 1, and operation input information by the driver from the main switch 13. An operation signal, a steering angle θst from the steering angle sensor 14, an accelerator opening θacc from the accelerator opening sensor 15, and the like are input.

  For example, when an execution of an ACC (Adaptive Cruise Control) function which is one of the functions of the driving support control is instructed through the operation of the main switch 13 by the driver, the control unit 5 recognizes the stereo image recognition device 4. The direction of the preceding vehicle is read, and it is identified whether or not the preceding vehicle to be followed is traveling on the own vehicle traveling path.

  As a result, when the preceding vehicle to be tracked is not detected, the constant speed traveling control for maintaining the vehicle speed V of the host vehicle 1 at the set vehicle speed set by the driver through the opening / closing control (engine output control) of the throttle valve 16. Execute.

  On the other hand, when a preceding vehicle that is a tracking target vehicle is detected and the vehicle speed of the preceding vehicle is equal to or lower than the set vehicle speed, the following traveling control is performed in which the following distance is converged to the target inter-vehicle distance. Executed. During this follow-up running control, the control unit 5 basically converges the inter-vehicle distance to the target inter-vehicle distance through the opening / closing control of the throttle valve 16 (engine output control). Furthermore, when it is determined that sufficient deceleration cannot be obtained only by controlling the throttle valve 16 due to sudden deceleration of the preceding vehicle, the control unit 5 controls the output hydraulic pressure from the active booster 17 (automatic braking intervention). Control) to converge the inter-vehicle distance to the target inter-vehicle distance.

  Further, when execution of the lane departure prevention function, which is one of the functions of the driving support control, is instructed through the operation of the main switch 13 by the driver, the control unit 5 will, for example, the left and right white lines that define the own vehicle traveling lane. A warning determination line is set based on (approximate white line recognized by the stereo image recognition device 4), and the own vehicle traveling path is estimated based on the vehicle speed V and the yaw rate γ of the own vehicle 1. Then, for example, when the control unit 5 determines that the host vehicle travel route crosses either the left or right alarm determination line within a set distance (for example, 10 to 16 [m]) ahead of the host vehicle, It is determined that the host vehicle 1 is likely to depart from the current host vehicle lane, and a lane departure warning is issued.

  Next, system stop determination based on the optical axis state of the stereo camera 3 will be described according to the system stop determination routine of FIG. This routine is repeatedly executed every set time in the stereo image recognition device 4. When the routine starts, the stereo image recognition device 4 first returns the driving support system 2 from the reset state in step S101. Check if it is immediately after. Here, as a scene where the driving support system 2 can be reset, for example, a case where the stereo camera 3 is replaced or repaired at a dealer or the like is assumed.

  If it is determined in step S101 that the driving support system 2 has just returned from the reset state, the stereo image recognition device 4 proceeds to step S102, and a horizontal optical axis shift occurs in the stereo camera 3. After setting the abnormality determination flag F indicating that there is “1”, the process proceeds to step S103. That is, when the driving support system 2 returns from the reset state, the stereo image recognition device 4 sets the abnormality determination flag F to “1” until it is verified that the optical axis shift of the stereo camera 3 has not occurred. Set to.

  On the other hand, if it is determined in step S101 that the driving support system 2 is not immediately after returning from the reset state, the stereo image recognition device 4 directly proceeds to step S103.

  When the process proceeds from step S101 or step S102 to step S103, the stereo image recognition device 4 performs optical axis misalignment determination of the stereo camera 3.

  This optical axis misalignment determination is executed, for example, according to the flowchart of the optical axis misalignment determination subroutine in the horizontal direction shown in FIG. 10, and when the subroutine starts, the stereo image recognition device 4 performs the above-mentioned in step S201. The first order coefficient B and the constant C of the white line approximate expression shown by the expressions (5) and (6) are acquired. That is, the stereo image recognition device 4 acquires white line inclination information with respect to the own vehicle 1 and lateral position information of the white line with respect to the own vehicle 1 as white line information defining the own vehicle travel lane.

  In subsequent step S202, the stereo image recognition device 4 determines whether or not the set time T0 has elapsed since the vehicle 1 started traveling (that is, whether or not the vehicle 1 has traveled the required sampling time T0). )

  If it is determined in step S202 that the set time T0 or more has not elapsed since the start of traveling of the host vehicle 1, the stereo image recognition device 4 directly exits the subroutine.

On the other hand, when it is determined in step S202 that the set time T0 or more has elapsed since the start of traveling of the host vehicle 1, the stereo image recognition device 4 proceeds to step S203, and based on the acquired constant C, the past set time A lateral movement amount (first lateral movement amount Xc) with respect to the white line associated with the traveling of the host vehicle 1 at T0 is calculated. That is, when the current vehicle position with respect to the white line is C2 and the own vehicle position with respect to the white line before the set time T0 with respect to the white line is C1, the stereo image recognition device 4 determines the first lateral movement amount Xc as follows: (7).
Xc = C2-C1 (7)
Then, when the process proceeds from step S203 to step S204, the stereo image recognition device 4 is based on the acquired primary coefficient B, and the amount of lateral movement relative to the white line accompanying the travel of the vehicle 1 at the past set time T0 (second The lateral movement amount Xb) is calculated. That is, when the current time is t2 and the time before the set time T0 is t1, the stereo image recognition device 4 calculates the second lateral movement amount Xb by the following equation (8).

In step S205, the stereo image recognition device 4 calculates the difference between the first lateral movement amount Xc and the second lateral movement amount Xb by the following equation (9). Calculated as Err.
Err = Xc−Xb (9)
Here, the lateral movement amount error Err is basically a value that becomes substantially zero when the optical axis direction of the stereo camera 3 is correctly adjusted.

  That is, in the white line approximation formulas shown in the above formulas (5) and (6), the constant C indicates the relative position to the white line on the side of the vehicle 1, so that the horizontal direction of the stereo camera 3 is It is a value that is not significantly affected by the optical axis shift. Accordingly, the first lateral movement amount Xc calculated using the constant C as a parameter is the actual lateral movement amount of the vehicle 1 even if a horizontal optical axis shift occurs in the stereo camera 3. On the other hand, it is possible to calculate without substantial error. On the other hand, the primary coefficient B is a coefficient indicating the inclination of the white line with respect to the own vehicle 1, and is a value greatly influenced by the horizontal optical axis shift of the stereo camera 3. Therefore, the second lateral movement amount Xb calculated using the primary coefficient B as a parameter is the actual lateral movement of the host vehicle 1 when the horizontal deviation of the optical axis occurs in the stereo camera 3. A value having an error with respect to the quantity is calculated. For these reasons, the lateral movement amount error Err calculated in the above equation (9) is substantially zero when there is no horizontal optical axis shift in the stereo camera 3, and an optical axis shift occurs. If it is, the absolute value becomes a value larger than a predetermined value.

  When the process proceeds from step S205 to step S206, the stereo image recognition device 4 checks whether or not the abnormality determination flag F is set to “1”.

  If it is determined in step S206 that the abnormality determination flag F is not set to “1” and is cleared to “0”, the stereo image recognition apparatus 4 proceeds to step S207, and the lateral movement error is determined. It is checked whether the absolute value | Err | is equal to or greater than a preset threshold value Eth1, and whether or not the state is maintained for a preset time T1 or more.

  In step S207, when it is determined that the absolute value | Err | of the lateral movement error is less than the threshold Eth1, or even if the absolute value | Err | of the lateral movement error is greater than or equal to the threshold, the set time T1 or more. If it is determined that it is not maintained, the stereo image recognition device 4 exits the subroutine as it is.

  On the other hand, if it is determined in step S207 that the state in which the absolute value | Err | of the lateral movement error is equal to or greater than the threshold Eth1 is maintained for the set time T1 or more, the stereo image recognition device 4 proceeds to step S208 and determines abnormality. After the flag F is set to “1”, the subroutine is exited.

  If it is determined in step S206 that the abnormality determination flag F is set to “1”, the stereo image recognition device 4 proceeds to step S209, and the absolute value | Err | of the lateral movement error is preset. It is checked whether the state is less than the threshold value Eth2 and the state is maintained for a preset time T2 or more. The threshold Eth2 and the set time T2 may be the same values as the thresholds Err1 and T1 described above, for example.

  In step S209, when it is determined that the absolute value | Err | of the lateral movement error is equal to or greater than the threshold Eth2, or even when the lateral movement error Err is less than the threshold Eth2, the set time T2 is not maintained. If it is determined that the stereo image recognition device 4 has left, the subroutine is exited.

  On the other hand, if it is determined in step S209 that the state in which the absolute value | Err | of the lateral movement error is less than the threshold Eth2 is maintained for the set time T2 or longer, the stereo image recognition device 4 proceeds to step S210 and determines whether there is an abnormality. After the flag F is cleared to “0”, the subroutine is exited.

  In the main routine shown in FIG. 9, when the process proceeds from step S103 to step S104, the stereo image recognition device 4 checks whether or not the abnormality determination flag F is set to “1”.

  If it is determined in step S104 that the abnormality determination flag F has been cleared to “0”, the stereo image recognition device 4 proceeds to step S105 and permits the control unit 5 and the like to execute the system. , Exit the routine. As a result, the control unit 5 or the like is permitted to execute vehicle control based on the environment outside the vehicle (traveling environment information) recognized by the stereo image. Various vehicle controls such as steering assist control can be appropriately implemented.

  On the other hand, if it is determined in step S104 that the abnormality determination flag F is set to “1”, the stereo image recognition device 4 proceeds to step S106, and the cause that the abnormality determination flag F is set to “1”. Is determined to be due to the reset of the driving support system 2.

  If it is determined in step S106 that the cause of the abnormality determination flag F being set to “1” is reset of the driving support system 2, the stereo image recognition device 4 proceeds to step S111.

  On the other hand, if it is determined in step S106 that the cause that the abnormality determination flag F is set to “1” is not the reset of the driving support system 2, the stereo image recognition device 4 proceeds to step S107, and the stereo camera 3 performs imaging. It is checked whether or not the effective area Ag cut out from the captured left and right captured images is in a correctable state. Here, the state where the effective area Ag can be corrected is, for example, that the effective area Ag does not reach the right end or the left end in the horizontal direction on the captured image, and the effective area Ag is corrected in the horizontal direction on the captured image by correction. A state where there is room for movement.

  If it is determined in step S107 that the effective area Ag is not correctable for the captured image, the stereo image recognition device 4 proceeds to step S110, and an optical axis shift has occurred in the stereo camera 3. After outputting an alarm for notifying the driver of the fact, the process proceeds to step S111.

  On the other hand, if it is determined in step S107 that the effective area Ag is correctable for the captured image, the stereo image recognition device 4 proceeds to step S108, and after the correction of the effective area Ag is performed, the set time is reached. It is checked whether T0 or more has elapsed.

  If it is determined in step S108 that the effective area Ag has been corrected in the past and the set time T0 or more has not yet elapsed since the effective area Ag has been corrected, the stereo image recognition device 4 Advances to step S111. That is, when the effective area Ag has been corrected in the past, it is difficult to determine whether or not the optical axis deviation has been eliminated by the correction until the set time T0 has elapsed in the above-described step S103. Therefore, the process proceeds to step S111 without newly correcting the effective area Ag.

  On the other hand, if it is determined in step S108 that the correction of the effective area Ag has not been performed in the past, or if the correction of the effective area Ag has been performed in the past and the correction time has exceeded the set time T0. If it is determined that the time has elapsed, the stereo image recognition device 4 proceeds to step S109, and for example, the cutout position of the effective area Ag on the captured image is set in the horizontal direction with the correction amount Δi corresponding to the lateral movement error Err. After the correction, the process proceeds to step S111. That is, for example, a map indicating the correction amount Δi corresponding to the lateral movement error Err is set in advance in the stereo image recognition device 4, and the stereo image recognition device 4 is obtained with reference to this map and the like. The effective area Ag is corrected with the correction amount Δi.

  When the process proceeds from step S106, step S108, step S109, or step S110 to step S111, the stereo image recognition device 4 instructs the control unit 5 and the like to stop the system, and then executes the routine. Exit. Thereby, in the control unit 5 or the like, various vehicle controls based on the outside information recognized from the stereo image are stopped.

  According to such an embodiment, when the vehicle 1 travels for the set time T0 by approximating the white line that divides the vehicle travel path based on the image captured by the stereo camera 3 with a first-order approximation equation or more. The amount of change in the lateral position of the vehicle 1 with respect to the white line (first lateral movement amount Xc) is calculated based on the constant C of the approximate expression, and the vehicle 1 with respect to the white line when the vehicle 1 travels for the set time T0. Is calculated based on the first order coefficient B of the approximate expression, and the first lateral movement amount Xc and the second lateral movement amount Xb are calculated. When the absolute value | Err | of the lateral movement error, which is the difference, is equal to or greater than the set threshold Eth1, imaging is performed by determining that the optical axis shift in the horizontal direction of the stereo camera 3 with respect to the host vehicle 1 has occurred. Realistic optical axis misalignment under various driving conditions It can be detected in the im.

That is, on the characteristic of the white line approximation formula recognized based on the captured image, the constant C indicating the relative position to the white line on the side of the host vehicle 1 is not significantly affected by the optical axis shift of the stereo camera 3. On the other hand, paying attention to the property that the primary coefficient B indicating the inclination of the white line with respect to the own vehicle 1 is greatly affected by the optical axis shift of the stereo camera 3, the first and second horizontal coefficients respectively calculated based on these characteristics When the absolute value | Err | of the difference between the direction movement amounts Xc and Xb (lateral movement error) is equal to or larger than the set threshold value Eth1, it is determined by determining that the optical axis deviation of the stereo camera 3 has occurred. The optical axis deviation can be detected in real time in an environment in which at least one of the left and right white lines that divide the vehicle travel path is detected.

  In this case, when it is determined that the optical axis deviation has occurred, the minor optical axis is corrected by correcting the effective area Ag on the captured image with the correction amount Δi corresponding to the lateral movement error Err. If there is a deviation, the optical axis can be corrected immediately in the vehicle 1 without mechanically adjusting the optical axis in a maintenance shop or the like.

  Next, FIGS. 11 to 14 relate to the second embodiment of the present invention, FIG. 11 is a flowchart showing an optical axis misalignment determination subroutine in the vertical direction, and FIG. 12 is a calculation of the distance from the own vehicle to the white line candidate point. FIG. 13 is an explanatory diagram showing a method for calculating the height of white line candidate points with reference to the own vehicle, and FIG. 14 is an explanatory diagram showing an inclined state of the road surface with reference to the own vehicle. Note that the present embodiment is obtained by adding optical axis misalignment determination in the vertical direction of the stereo camera 3 to the first embodiment described above.

  That is, in this embodiment, in the process of step S103 shown in the first embodiment, the optical axis shift in the vertical direction as well as the horizontal direction is determined. Although detailed description is omitted, when the optical axis shift in the vertical direction is determined in step S103, the stereo image recognition device 4 according to the present embodiment performs horizontal processing for the processes in and after step S104 shown in FIG. A process substantially similar to the process for the determination result for the optical axis shift in the direction is performed. However, the stereo image recognition device 4 outputs an abnormal alarm, stops the system, etc. based on an optical axis shift in at least one of the horizontal direction and the vertical direction in the processing of step S104 and the subsequent steps. If it is determined that the optical axis deviation has occurred in step S109, the stereo image recognition device 4 corrects the effective area Ag in the vertical direction in step S109.

  The optical axis misalignment determination in the vertical direction is executed, for example, according to the flowchart of the optical axis misalignment determination subroutine shown in FIG. 11, and when this subroutine starts, the stereo image recognition device 4 firstly performs step S301. An approximate expression representing the slope of the road surface is calculated. In other words, the stereo image recognition device 4 is, for example, in the process that the optical axes of the left and right cameras 31L and 31R constituting the stereo camera are adjusted without error with respect to the horizontal direction of the own vehicle 1. Calculate the slope.

  Specifically, the stereo image recognition device 4 first obtains the distance in the optical axis direction (distance z to the white line) for the white line candidate point Pn based on the parallax between the left and right cameras 31L and 31R. That is, for example, as shown in FIG. 12, the distance to the vehicle is z [mm], the distance between the cameras is L [mm], and from the lens optical systems 31La and 31Ra of the cameras 31L and 31R to the image sensors 31Lb and 31Rb. Is f [mm], the parallax (number of pixels) between the left and right cameras 31L and 31R with respect to the right camera 31R is d [pix], and the pixel pitch (horizontal) is Wi [mm / pix]. From the proportional relationship, for example, the following equation (10) is derived.

L: z = d · Wi: f (10)
Further, when the equation (10) is modified, the following equation (11) can be obtained.
z = (L · f) / (d · Wi) (11)
Also, for example, as shown in FIG. 13, the height from the ground surface of each camera 31L, 31R is CAHM [mm], and the coordinates of the imaging position of the white line candidate point Pn with reference to the optical axis jv are j [pix]. If the pixel pitch is Wj [mm / pix], for example, the following expression (12) is derived for the height y of the white line candidate point Pn.

(CAHM-y) / z = (Wi · (j−jv)) / f (12)
Furthermore, when the equation (12) is modified, the following equation (13) can be obtained.
y = ((Wi · (j−jv)) / f) · z + CAHM (13)
Therefore, the Z coordinate and the Y coordinate of the white line candidate point Pn can be calculated from the above equations (11) and (13).

The stereo image recognition device 4 performs such calculation for each white line candidate point on the distance image, and distributes the coordinates of each point on the ZY plane as shown in FIG. Approximate a straight line by the following equation (14).
Y = a · Z + b (14)
Thereby, in the stereo image recognition apparatus 4, the fluctuation | variation of the inclination a of the road and height b on the basis of the optical axis of the cameras 31L and 31R (in other words, the case where it is assumed that the own vehicle traveling road is a flat road) It is possible to detect the inclination and height fluctuation of the host vehicle (camera).

  When the process proceeds from step S301 to step S302, the stereo image recognition device 4 checks whether or not the set time T0 or more has elapsed since the vehicle 1 started to travel.

  If it is determined in step S302 that the set time T0 or more has not elapsed since the start of traveling of the host vehicle 1, the stereo image recognition device 4 directly exits the subroutine.

  On the other hand, if it is determined in step S302 that the set time T0 or more has elapsed since the start of traveling of the vehicle 1, the stereo image recognition device 4 proceeds to step S303, and the slope of the road surface in the calculated equation (14). An average value a_ave of the road surface inclination is calculated based on the history of the past set time T0 of the primary coefficient a indicating

  Then, when the process proceeds from step S303 to step S304, the stereo image recognition apparatus 4 has the abnormality determination flag F2 indicating that the optical axis shift in the vertical direction of the stereo camera 3 is set to “1”. Check for no.

  If it is determined in step S304 that the abnormality determination flag F2 is not set to “1” and is cleared to “0”, the stereo image recognition device 4 proceeds to step S305 and averages the road surface inclination. It is checked whether or not the absolute value | a_ave | of the value is greater than or equal to a preset threshold value ath1.

  If it is determined in step S305 that the absolute value | a_ave | of the average value of the slopes is less than the threshold value ath1, the stereo image recognition device 4 directly exits the subroutine.

  On the other hand, when it is determined in step S305 that the absolute value | a_ave | of the average value of the slopes is equal to or greater than the threshold value ath1, the stereo image recognition device 4 proceeds to step S306 and sets the abnormality determination flag F2 to “1”. Then exit the subroutine.

  If it is determined in step S304 that the abnormality determination flag F2 is set to “1”, the stereo image recognition device 4 proceeds to step S307, and the absolute value | a_ave | of the average value of the inclination is set in advance. It is checked whether it is less than the threshold value ath2. The threshold value ath2 may be the same value as the threshold value ath1 described above.

  If it is determined in step S307 that the absolute value | a_ave | of the average value of the slopes is equal to or greater than the threshold value ath2, the stereo image recognition device 4 directly exits the subroutine.

  On the other hand, when it is determined in step S307 that the absolute value | a_ave | of the average value of the slopes is less than the threshold value ath2, the stereo image recognition device 4 proceeds to step S308 and clears the abnormality determination flag F2 to “0”. Then exit the subroutine.

  According to such an embodiment, the same effects as those of the first embodiment described above can be obtained.

  In addition, this invention is not limited to each embodiment described above, A various deformation | transformation and change are possible, and they are also in the technical scope of this invention. For example, in each of the above-described embodiments, the optical axis deviation of the stereo camera has been described. However, particularly with respect to the optical axis deviation in the horizontal direction, even if the imaging means for monitoring the outside of the vehicle is a monocular camera, the monocular camera is concerned. Of course, the present invention can be applied to the case where the white line that divides the host vehicle travel path can be approximated by an approximate expression of the first order or higher based on the image captured in (1).

1 ... Own vehicle (own vehicle)
2 ... Driving support system 3 ... Stereo camera (imaging means)
4 ... Stereo image recognition device (white line approximation means, first lateral movement amount calculation means, second lateral movement amount calculation means, optical axis deviation determination means, effective area correction means, control stop determination means)
DESCRIPTION OF SYMBOLS 5 ... Control unit 11 ... Vehicle speed sensor 12 ... Yaw rate sensor 13 ... Main switch 14 ... Steering angle sensor 15 ... Accelerator opening sensor 16 ... Throttle valve 17 ... Active booster 31L, 31R ... Left and right camera 31La, 31Ra ... Lens optical system 31Lb , 31Rb ... Image sensor

Claims (3)

Imaging means for imaging the environment outside the vehicle in front of the vehicle;
White line approximating means for approximating a white line that divides the vehicle traveling path based on an image picked up by the image pickup means with a first-order or higher approximation formula;
The first lateral movement amount, which is the amount of change in the lateral position of the vehicle with respect to the white line when the vehicle travels for a set time , is expressed by the constant of the approximate expression at the present time and the constant of the approximate expression before the set time. a first lateral moving amount calculating means for de San based on the difference,
The second lateral movement amount, which is the amount of change in the lateral position of the vehicle relative to the white line when the host vehicle has traveled for the set time , is determined relative to the white line of the vehicle obtained from the primary coefficient of the approximate expression and the vehicle speed. a second lateral movement amount calculation means for output calculated based on the integrated value from the previous the set time of the transverse velocity component of the velocity to the present time,
When the difference between the first lateral movement amount and the second lateral movement amount is equal to or greater than a preset threshold value, it is determined that an optical axis deviation in the horizontal direction of the imaging unit has occurred. An outside-of-vehicle monitoring device comprising: an optical axis misalignment determining unit.   When the optical axis deviation determining means determines that the optical axis deviation has occurred, an effective area to be cut out from the image captured by the imaging means is the first lateral movement amount and the second lateral movement. The vehicle exterior monitoring device according to claim 1, further comprising an effective area correction unit configured to correct the amount according to a difference from the amount.   A control stop determination unit that determines stop of vehicle control based on captured image information when the optical axis shift determination unit determines that the optical axis shift has occurred. The exterior monitoring apparatus according to claim 1 or 2.

Images