JP5636493B2 - Image processing apparatus and image display apparatus - Google Patents

Description

  The present invention relates to an image processing device that processes a front image in a traveling direction of a vehicle by analysis and processing, and an image display device that displays a front image processed by the image processing device.

  In recent years, a technique has been proposed in which a surrounding image is picked up by a camera mounted on a vehicle and the surrounding state of the vehicle is detected by analyzing the picked-up image.

  For example, as described in Patent Document 1, a front image in the traveling direction of the vehicle is captured, white lines on both sides of the traveling lane displayed in the front image are recognized, and the shape of the white lines is approximated. The curvature of the traveling lane curve is calculated from the equation. And in this prior art, driving | operation is supported so that driving | running | working of the said vehicle on the present driving lane can be maintained by controlling the steering force added to a steering device according to the curvature of the curve of a driving lane.

JP 2008-305319 A ([0026] to [0030])

  For example, when the camera is attached to the vehicle by the user's own work, or when there is a possibility that the fixed position or posture of the camera moves, the imaging direction of the camera is likely to deviate from a specified direction. In this case, the shape of the white line in the front image differs due to an error in the imaging direction of the camera even for the same driving lane curve. Therefore, the above-described conventional technique can accurately calculate the curvature of the driving lane curve. It becomes difficult. For this reason, there has been a demand for a technique capable of displaying support that correctly corresponds to the curvature of the curve of the traveling lane only by analyzing the front image, even if there is a camera mounting error.

  The problem to be solved by the present invention includes the above-described problem as an example.

In order to solve the above problems, the invention according to claim 1 is based on the analysis and processing of the image content in the front image with respect to the front image obtained by imaging the front in the traveling direction of the vehicle by the imaging means mounted on the vehicle. An image processing apparatus that performs processing, wherein, in the front image , each of the two lane contour lines that form the contour of the traveling lane that is located on both the left and right sides of the traveling lane of the vehicle is extended in the traveling direction. Lane vanishing point detecting means for detecting a lane vanishing point that is an intersection on a line, and an average vanishing point located at an average position of each of the plurality of lane vanishing points detected in the past by the lane vanishing point detecting means, or the lane the vehicle lane vanishing point detected when the vehicle travels a substantially straight travel lane in the past by the vanishing point detecting means, and the vehicle axis point detecting means for detecting a vehicle axis point, the vehicle-axis direction A positional deviation detecting means for detecting a relative positional deviation of said detected lane vanishing point by the lane vanishing point detecting means with respect to the vehicle axis point detected by point detecting means, a display mode corresponding to the positional deviation And a front image processing means for processing the front image.

In order to solve the above-mentioned problem, the invention of claim 6 is processed by analyzing and processing the image content in the front image with respect to the front image obtained by imaging the front in the traveling direction of the vehicle by the imaging means mounted on the vehicle. An image processing apparatus for performing an outline of the traveling lane in the forward image, located at the center point of the image portion of the preceding vehicle in the traveling lane of the vehicle or on both the left and right sides of the traveling lane of the vehicle A traveling direction point detecting means for detecting one of the lane vanishing points, which is an intersection of the two lane contour lines on the extension line of the traveling direction, as a traveling direction point; Vehicle axis direction point detecting means for detecting a point located at an average position of the plurality of detected traveling direction points as a vehicle axis direction point, and the vehicle axis detected by the vehicle axis direction point detecting means Position deviation detecting means for detecting a relative position deviation of the traveling direction point detected by the traveling direction detecting means with respect to a direction point; and a front image processing means for processing the front image in a display mode corresponding to the position deviation. And having.
In order to solve the above-mentioned problem, the invention of claim 7 is processed by analyzing and processing the image content in the front image with respect to the front image obtained by imaging the front in the traveling direction of the vehicle by the imaging means mounted on the vehicle. An image processing method executed by a computing unit included in the image processing apparatus that performs the above-described processing, and in the front image, the center point of the image portion of the front vehicle in the traveling lane of the vehicle or both the left and right sides of the traveling lane of the vehicle A traveling direction point detecting step of detecting any one of the lane vanishing points, which are intersections of the two lane contour lines that are located on the extended line of the traveling direction and that form the contour of the traveling lane as a traveling direction point A vehicle axis direction point detecting step of detecting a point located at an average position of the plurality of traveling direction points detected in the past by the traveling direction point detecting step as a vehicle axis direction point; A position deviation detecting step for detecting a relative position deviation of the advancing direction point detected by the advancing direction detecting step with respect to the vehicle axial direction point detected by the biaxial direction point detecting step, and corresponding to the position deviation A front image processing step of processing the front image in a display mode.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a perspective view illustrating a configuration example of a vehicle equipped with a driving support system according to an embodiment including an image processing device and an image display device of the present invention. It is a block diagram which shows the system configuration | structure of the driving assistance system of embodiment. It is a figure which shows an example of the display of the vehicle window scenery seen from the cab, and the display corresponding to this. FIG. 4 is a diagram illustrating a display example when a front image captured by the front camera in the posture state illustrated in FIG. 3 is displayed on a display. It is a figure showing the example of a display at the time of displaying a front picture which imprinted a lane outline, a lane vanishing point, and an average vanishing point on a display. It is a figure showing an example of the vehicle mark containing the advancing direction mark. It is a figure showing the example of a superimposed display of a position deviation vector and a vehicle mark reflecting it. It is a figure showing the example of a display on the display of the front image which superimposed the vehicle mark on the image part of the front vehicle. It is a flowchart showing the control content which CPU of an imaging unit performs. It is a figure showing the example of a display on the display of the front image which superimposed the advancing direction mark other than the image part of the front vehicle. It is a figure showing the example of a superposition display of a vehicle mark at the time of applying a lane vanishing point to an advancing direction point, and applying an average vanishing point to a vehicle axis direction point. It is a figure showing the example of a superimposed display of the vehicle mark at the time of applying a lane vanishing point to the advancing direction point, and applying a front vehicle center point to a vehicle axial direction point. It is a figure showing the example of a display at the time of displaying the front image which copied the front vehicle center point and the average center point on a display. It is a figure showing the example of a superposition display of a vehicle mark at the time of applying a front vehicle center point to a direction point of movement, and applying an average center point to a vehicle axis direction point. It is a figure showing the superimposed display example of the vehicle mark at the time of applying a lane vanishing point to the advancing direction point, and applying an average center point to a vehicle axial direction point.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a perspective view showing a configuration example of a vehicle equipped with the driving support system of the present embodiment including the image processing device and the image display device of the present invention. In FIG. 1, the driving support system S is provided with a front camera 1 on the rear surface of the room mirror 101 on the ceiling inside the vehicle V. Note that the room mirror 101 is movable in the same mounting posture as a general one, and the front camera 1 can capture the front of the traveling direction of the vehicle V within the movable range of the posture. Fits in range. Further, within the movable range of the front camera 1 in the imaging direction Dc, the direction Dv of the vehicle axis passing through the center longitudinal direction of the vehicle is not excluded from the imaging range.

  FIG. 2 is a block diagram illustrating a hardware configuration example of the driving support system S. As illustrated in FIG. In FIG. 2, the driving support system S includes a front camera 1, an imaging unit 2, and a display 3.

  The front camera 1 corresponds to an image pickup unit, and uses, for example, a CCD image pickup device or the like to pick up a front image in the traveling direction of the vehicle and outputs a corresponding signal to a CPU (described later) of the imaging unit 2. Have In the example of the present embodiment, the front camera 1 continuously captures the front image at short time intervals, thereby capturing the front image in the form of a moving image.

  The display 3 corresponds to an image display device and a forward image display means, and is composed of, for example, an LCD panel and has a function of displaying a forward image based on an image signal input from a graphic controller (described later) of the imaging unit 2. .

  The imaging unit 2 corresponds to an image processing device, and includes a CPU 11, a storage device 12, a navigation device 13, a vehicle speed sensor 14, and a graphic controller 15.

The CPU 11 (corresponding to the calculation means) performs various calculations according to the operation of a predetermined program, and controls the entire driving support system by exchanging information with other units and outputting various control instructions. It has a function.

  The storage device 12 includes a ROM 12a, a RAM 12b, and a storage medium 12c. The ROM 12a is an information storage medium in which various processing programs and other necessary information are written in advance. The RAM 12b is an information storage medium on which information necessary for executing the various programs is written and read. The storage medium 12c is a non-volatile information storage medium such as a flash memory or a hard disk.

  The navigation device 13 corresponds to navigation means, uses a GPS sensor (not shown in particular) to measure the current location of the vehicle, obtains current location information, and obtains a predetermined route based on map information stored in advance. It has a function for searching and route guidance.

  The vehicle speed sensor 14 has a function of detecting the traveling speed of the host vehicle (in other words, a function of detecting traveling). Based on the detection signal of the vehicle speed sensor 14, the CPU 11 can identify whether the vehicle is in a running state or in a stopped state.

  The graphic controller 15 has a function of acquiring image data from a video RAM (not shown) and the navigation device 13 under the control of the CPU 11 and displaying an image signal based on the image data on the display 2.

  FIG. 3 is a diagram illustrating an example of a vehicle window landscape viewed from the cab and a display on the display 3 corresponding thereto. In the example shown in FIG. 3, the display 3 is arranged at the center position of the instrument panel 102, the right side in the figure corresponds to the driver's seat, and the steering wheel 103 is arranged in front thereof. .

  In the illustrated example, the vehicle V travels in a substantially straight traveling lane, and shows a state in which the vehicle V follows a front vehicle Vf that is in front of the traveling direction. The room mirror 101 is slightly inclined toward the driver. The front camera 1 provided on the back side (traveling direction side) of the room mirror 101 captures a forward image including the straight traveling lane and the forward vehicle Vf. At this time, as described above, the imaging range of the front camera 1 includes the vehicle axis direction Dv of the vehicle V without removing it. The display 3 displays a front image captured by the front camera 1.

  Here, as described above, when the mounting posture of the front camera 1 is movable together with the rearview mirror 101, and the imaging direction Dc is not in an arrangement relationship parallel to the vehicle axial direction Dv, the traveling lane is curved. However, it is difficult to accurately calculate the curvature of the curve only by analyzing the front image. This can also occur when the user attaches the front camera 1 at an arbitrary position or when it is attached manually. Therefore, in this embodiment, instead of accurately calculating the curvature of the curve of the traveling lane, a support display that correctly corresponds to the curvature of the curve to the extent that it can support the driver's driving is superimposed and displayed on the front image. Hereinafter, a method and a display example of such support display will be sequentially described.

  First, FIG. 4 shows a display example when a front image captured by the front camera 1 in the posture state shown in FIG. 3 is displayed on the display 3. In FIG. 4, the center position of the display area of the display 3, that is, the intersection position of the center lines Lv and Ll in the vertical direction and the horizontal direction in the figure is separated from the vehicle axis direction Dv. For this reason, the front image loses left-right symmetry with respect to the center line L1, and even if the actual driving lane changes to a right curve and a left curve with the same curvature, the front image is reflected with different curvatures (illustrated). (Omitted).

  On the other hand, in the front image captured when the vehicle travels in a straight traveling lane as in the illustrated example, the forward vehicle Vf is positioned substantially on the extension in the vehicle axial direction Dv. A lane vanishing point P1 corresponding to an infinitely far point of the traveling lane is also located on the extension of the vehicle axial direction Dv when the traveling lane is linear. Thus, the point in the position corresponding to the extension of the vehicle axial direction Dv in the front image is hereinafter referred to as a vehicle axial direction point.

  As a method for detecting the lane vanishing point P1, first, as shown in the figure, a white line applied to both sides of an actual traveling lane, a boundary line between a road surface and a curb, or an edge part of an asphalt road is displayed as a front image. And is recognized as two lane contour lines Lr forming the contour of the travel lane. Then, the intersection of the two lane outlines Lr on the extension line in the vehicle traveling direction is detected as the lane vanishing point P1 on the front image.

  Here, as shown in FIG. 5, for example, when the traveling lane is curved, the two lane outlines Lr are also recognized in accordance with the direction and curvature of the curve on the front image. Although not particularly illustrated, when the curvature of this curve changes midway, the two lane contours Lr are recognized with priority given to the curvature on the near side of the vehicle V. Then, an extension line of the curve is calculated with an approximate expression based on the curvature on the near side, and the position of the intersection is detected as the lane vanishing point P1.

  When the vehicle V has traveled for a long time in various lanes and detected and recorded many lane vanishing points P1, the average vanishing point P2 located at the average position of the lane vanishing points P1 is statistically It approaches on the extension of the vehicle axial direction Dv. That is, the average vanishing point P2 can be regarded as equivalent to the vehicle axial direction point described above.

  In the present embodiment, the vehicle mark Mv as shown in FIG. 6 is superimposed on the front vehicle Vf closest to the vehicle V in the front image and displayed on the display 3. In FIG. 6, the vehicle mark Mv is a geometric pattern configured so that its center position can be easily seen. The vehicle mark Mv includes a traveling direction mark Md that can rotate with respect to the center position. This advancing direction mark Md corresponds to a mark image, and in the example shown in the figure, it has a shape in which only the vicinity of each vertex is left out of each side of the equilateral triangle.

  Such a vehicle mark Mv is superimposed on the preceding vehicle Vf in a manner as shown in FIG. For example, as shown in FIG. 7A, when the traveling lane is a left curve, the CPU 11 of the imaging unit 2 recognizes the image portion of the forward vehicle Vf in the forward image and is positioned at the center position thereof. A vehicle center point P3 is detected. This forward vehicle center point P3 on the forward image is located on the traveling direction side of the vehicle V along the traveling lane with respect to the vehicle axial direction point (average vanishing point P2 in this example). In this way, the point corresponding to the traveling direction along the traveling lane of the vehicle V in the front image is hereinafter referred to as a traveling direction point.

  Then, the CPU 11 of the imaging unit 2 detects a relative position deviation vector X of the traveling direction point with respect to the vehicle axial direction point. That is, in this example, a relative position deviation vector of the forward vehicle center point P3 with respect to the average vanishing point P2 is detected. The direction and magnitude of the left-right direction component of the position deviation vector X can be considered to substantially correspond to the curve direction and curvature of the travel lane. In the present embodiment, a vehicle mark Mv including a traveling direction mark Md that is rotated in accordance with the direction and magnitude of the horizontal component of the position deviation vector X is generated, and this vehicle mark Mv is generated at the front vehicle center point P3. Overlay the position.

  Accordingly, in the case of the left curve shown in FIG. 7A, the vehicle mark Mv obtained by rotating the traveling direction mark Md counterclockwise is displayed in a superimposed manner. Further, in the case of the right curve shown in FIG. 7B, a vehicle mark Mv obtained by rotating the traveling direction mark Md in the clockwise direction is superimposed and displayed. The rotation angle of each traveling direction mark Md is proportional to the magnitude of the horizontal component of the position deviation vector X.

  FIG. 8 shows a display example on the display 3 of the forward image in which such a vehicle mark Mv is superimposed on the position of the forward vehicle center point P3. The driver can more clearly see the presence of the forward vehicle Vf by the vehicle mark Mv. Further, by steering the steering wheel 103 in accordance with the rotation angle of the traveling direction mark Md included in the vehicle mark Mv, steering corresponding to the curve direction and curvature in the actual traveling lane can be performed.

  FIG. 9 is a flowchart showing the control contents executed by the CPU 11 of the imaging unit 2 in order to realize the operation mode described above. This flow is called and started to be executed at an appropriate time interval, for example, while the front camera 1 is capturing a front image in the form of a moving image while the vehicle is running.

  In FIG. 9, first, in step S5, a front image is captured by the front camera 1.

  Next, the process proceeds to step S10, and the two lane contours Lr in the current travel lane are recognized in the front image captured in step S5. In addition, the procedure of this step S10 corresponds to a vehicle outline recognition means.

  Next, the process proceeds to step S15, and the lane vanishing point P1 in the current traveling lane is detected based on the two lane outlines Lr recognized in step S10. The procedure of step S15 corresponds to lane vanishing point detection means.

  Next, the process proceeds to step S20, and the lane vanishing point P1 detected in step S15 is additionally recorded so as to be accumulated in the storage medium 12c.

  Next, the process proceeds to step S25, and all lane vanishing points P1 recorded in the storage medium 12c are read out.

  Next, the process proceeds to step S30, and the average vanishing point P2 located at the average position of all the lane vanishing points P1 read out in step S25 is detected. Note that the procedure of step S30 corresponds to vehicle axial direction point detection means and average vanishing point detection means.

  Next, the process proceeds to step S35, and the image portion of the front vehicle Vf that is closest to the vehicle V on the same traveling lane as the vehicle V is recognized in the front image captured in step S5. In addition, the procedure of this step S35 corresponds to a forward vehicle recognition means.

  Next, the process proceeds to step S40, and the center point of the image portion of the forward vehicle Vf detected in step S35, that is, the forward vehicle center point P3 is detected. The procedure of step S40 corresponds to the traveling direction point detecting unit.

  Next, the process proceeds to step S45, and the relative position deviation vector X of the forward vehicle center point P3 detected in step S40 is detected with respect to the average vanishing point P2 detected in step S30. Note that the procedure of step S45 corresponds to a position deviation detecting means.

  Next, the process proceeds to step S50, where the left / right direction component of the position deviation vector X detected in step S45 is detected, and the vehicle mark Mv obtained by rotating the traveling direction mark Md in accordance with the direction and magnitude of the left / right direction component is detected. Generate.

  Next, the process proceeds to step S55, and the front image is processed so that the vehicle mark Mv generated in step S50 is superimposed on the position of the front vehicle center point P3 detected in step S40. Note that the procedure of step S50 and the procedure of step S55 correspond to the forward image processing means. Further, the CPU 11 transmits necessary data and commands to the graphic controller 15, and the actual processing is performed by the graphic controller 15.

  Next, the process proceeds to step S60, and the front image processed in step S55 is displayed on the display 3. The CPU 11 transmits a command to the graphic controller 15 and causes the graphic controller 15 to output image data of the processed front image to the display 3. Then, this flow ends.

  After Step S5, Steps S35 to S40 may be executed, then Steps S10 to S30 may be executed, and the processes after Step S45 may be executed.

As described above, in the imaging unit 2 provided in the driving support system S of the above-described embodiment, the front image obtained by capturing the front in the traveling direction of the vehicle V by the front camera 1 (corresponding to the imaging unit) mounted on the vehicle V. On the other hand, an imaging unit 2 (corresponding to an image processing device) that performs processing by analyzing and processing the image content in the front image, in the traveling direction along the traveling lane of the vehicle V in the front image. the traveling direction point detection means for detecting a traveling direction points in corresponding positions, said during front image, detecting a vehicle axial point at a position corresponding to the extension of the vehicle axis direction Dv of the vehicle V at the time Vehicle axis direction point detecting means for detecting a position deviation vector X (corresponding to a position deviation) relative to the traveling direction point with respect to the vehicle axis direction point Includes means exits, and a front image processing means for processing the front image in the display mode corresponding to the position deviation vector X.

  In this way, the position of the traveling direction point in the front image can be regarded as an approximate index position of the planned direction in which the vehicle V travels, and the position of the vehicle axial direction point in the front image is Can be regarded as an approximate index position in the direction in which is currently facing. Thus, the direction and magnitude of the relative position deviation vector X of the traveling direction point with respect to the vehicle axis direction point can be considered to substantially correspond to the curve direction and curvature of the traveling lane in which the vehicle V is traveling. . Therefore, by processing the front image in a display mode corresponding to the position deviation vector X, it is possible to reflect the support display that correctly corresponds to the curvature of the curve of the traveling lane. This image processing can be performed only by analyzing and processing the image content of the front image without being affected by the deviation between the imaging direction Dc of the front camera 1 and the vehicle axial direction Dv. As a result, even if there is a slight camera mounting error, it is possible to display support that correctly corresponds to the curvature of the curve of the traveling lane only by analyzing the front image.

  In addition to the above-described configuration, the imaging unit 2 included in the driving support system S further includes a forward vehicle recognition unit that recognizes an image portion of the forward vehicle Vf in the travel lane of the vehicle V in the forward image. The traveling direction point detecting means detects the traveling direction point in the image portion of the preceding vehicle Vf.

  In this way, since the position of the forward vehicle Vf positioned relatively to the traveling direction side with respect to the vehicle axial direction point in the forward image can be detected as the traveling direction point, the position deviation vector X can be detected almost correctly. .

  Further, in the imaging unit 2 provided in the driving support system S, in addition to the above-described configuration, the front image processing means further superimposes a traveling direction mark Md (corresponding to a seal image) in a display mode reflecting predetermined information. Then, the front image is processed.

  In this way, the driver can also recognize information that is usually difficult to see in the front image.

  In the display example of FIG. 8, the entire vehicle mark Mv including the traveling direction mark Md is superimposed on the position of the forward vehicle center point P3, that is, on the image portion of the forward vehicle Vf. In this case, the presence of the forward vehicle Vf can be clearly seen by the vehicle mark Mv, and the information by the progress mark Md can be recognized together with little movement of the viewpoint. However, the present invention is not limited to this. For example, as shown in FIG. 10 corresponding to FIG. 8, the vehicle mark Mv is disassembled into a part Mv1 including the traveling direction mark Md and a part Mv2 not including the traveling direction mark Md. May be displayed. In the illustrated example, a portion Mv2 that does not include the traveling direction mark Md is superimposed on the image portion of the forward vehicle Vf, and a portion Mv1 that includes the traveling direction mark Md is disposed and superimposed at a fixed position on the upper left of the front image. . In this case, the presence of the forward vehicle Vf can be clearly seen by the portion Mv2, and separately, the traveling direction mark Md included in the portion Mv1 can be clearly displayed at a fixed position away from the forward vehicle Vf.

  In addition, in the imaging unit 2 provided in the driving support system S, in addition to the above-described configuration, the front image processing unit reflects the position deviation as the predetermined information in the display mode of the traveling direction mark Md.

  In this way, steering corresponding to the direction and curvature of the curve in the actual driving lane is possible by steering the steering wheel 103 in accordance with the display mode of the traveling direction mark Md (the rotation angle in the above embodiment). It becomes.

  Further, the display 3 in the driving support system S of the above embodiment includes a front image display means for displaying the front image on which the traveling direction mark Md is superimposed by the front image processing means.

  In this way, it is possible to realize the display of the front image on which the traveling direction mark Md that supports the driving of the driver is superimposed.

Further, in the imaging unit 2 provided in the driving support system S, in addition to the above-described configuration, the contour of the traveling lane is formed 2 on the left and right sides of the traveling lane of the vehicle V in the front image. Lane outline recognizing means for recognizing the two lane outlines Lr, lane vanishing point detecting means for detecting an intersection of the two lane outlines Lr on the extension line in the traveling direction as a lane vanishing point P1 , The vehicle axis direction point detection means uses the lane vanishing point detection means to calculate the average vanishing point P2 detected when the vehicle V has traveled in a substantially straight traveling lane in the past as the vehicle axis. Detect as direction point.

  In this way, since the average vanishing point P2 at the position corresponding to the vehicle axis direction Dv in the front image can be detected as the vehicle axis direction point, the position deviation vector X can be detected almost correctly.

  In the above embodiment, the lane vanishing point P1 is detected in various traveling lanes, and the average vanishing point P2 at the average position is detected as the vehicle axial direction point, but the present invention is not limited to this. For example, when the navigation device 13 of the imaging unit 2 is used to detect that the vehicle V is traveling in a substantially straight traveling lane at that time, the lane disappearance detected in the front image captured by the front camera 1 is detected. You may detect the point P1 as a vehicle axial direction point.

  In this way, in the imaging unit 2 provided in the driving support system S, in addition to the basic configuration described above, in the front image, the contour of the traveling lane is located on both the left and right sides of the traveling lane of the vehicle V. Lane lane line recognizing means for recognizing the two lane line contours Lr forming the lane line, and a lane disappearance inspection for detecting, as a lane vanishing point P1, an intersection of the two lane line contours Lr on the extension line in the traveling direction. The lane vanishing point detecting means detects the lane vanishing point P1 in a front image captured by the front camera 1 when the vehicle V has traveled in a substantially straight traveling lane in the past. Then, the vehicle axis direction point detection means may detect the lane vanishing point P1 as the vehicle axis direction point.

  In this way, only the lane vanishing point P1 close to the vehicle axis direction Dv can be directly detected, and can be easily detected as it is as the vehicle axis direction point.

  In addition, in this way, in the imaging unit 2 provided in the driving support system S, in addition to the above-described configuration, the navigation device capable of acquiring current position information and map information where the vehicle V is located at that time. 13 (corresponding to navigation means), and the lane vanishing point detection means detects that the vehicle V is traveling on the substantially straight traveling lane at that time by the navigation device 13. You may detect the said lane vanishing point P1 in the front image which the front camera 1 imaged.

  In this way, a substantially straight traveling lane can be detected with high accuracy, and the corresponding lane vanishing point P1 can be accurately detected.

  In the above embodiment, the front vehicle center point P3 is applied to the traveling direction point and the average vanishing point P2 is applied to the vehicle axial direction point to detect the position deviation vector X. However, the present invention is not limited to this. . For example, as shown in FIG. 11 corresponding to FIG. 7A, the lane vanishing point P1 is applied to the traveling direction point and the average vanishing point P2 is applied to the vehicle axial direction point to detect the position deviation vector X. However, a support display that correctly corresponds to the curvature of the curve of the traveling lane can be reflected in the front image as in the above embodiment.

  In this way, in the imaging unit 2 provided in the driving support system S, in addition to the above-described configuration, in the front image, the contour of the traveling lane is defined on both the left and right sides of the traveling lane of the vehicle V. Lane lane line recognizing means for recognizing the two lane line contours Lr to be formed, and lane vanishing point detection for detecting the intersection point of each of the two lane contour lines Lr on the extension line in the traveling direction as the lane vanishing point P1 And the traveling direction point detecting unit may detect the lane vanishing point P1 as the traveling direction point.

  In this way, even when there is no forward vehicle, the position of the lane vanishing point P1 positioned relatively to the traveling direction side with respect to the vehicle axial direction point in the forward image can be detected as the traveling direction point. X can be detected almost correctly.

  Similarly, as shown in FIG. 12 corresponding to FIG. 7A, the lane vanishing point P1 is applied to the traveling direction point, and the forward vehicle center point P3 is applied to the vehicle axial direction point. Even when X is detected, a support display that correctly corresponds to the curvature of the curve of the traveling lane can be reflected in the front image as in the above embodiment.

  In addition, as shown in FIG. 13 corresponding to FIG. 5 described above, when the vehicle V has traveled for a long time in various lanes and a large number of front vehicle center points P3 are detected and recorded, The average center point P4 located at the average position of the point P3 is also statistically close to the extension in the vehicle axial direction Dv. That is, the average center point P4 can be regarded as equivalent to the vehicle axial direction point described above.

  Accordingly, for example, as shown in FIG. 14 corresponding to FIG. 7A, the forward vehicle center point P3 is applied to the traveling direction point, and the average center point P4 is applied to the vehicle axial direction point to thereby obtain the position deviation vector X. Even when detected, the support display that correctly corresponds to the curvature of the curve of the traveling lane can be reflected in the front image as in the above embodiment.

  Similarly, as shown in FIG. 15 corresponding to FIG. 7A, the lane vanishing point P1 is applied to the traveling direction point, the average center point P4 is applied to the vehicle axial direction point, and the position deviation vector X Even when detected, the support display that correctly corresponds to the curvature of the curve of the traveling lane can be reflected in the front image as in the above embodiment.

  In addition to those already described above, the methods according to the above-described embodiments and modifications may be used in appropriate combination.

1 Front camera (equivalent to imaging means)
2 Imaging unit (equivalent to image processing device)
3 Display (equivalent to image display device)
11 CPU
13 Navigation device (equivalent to navigation means)
15 Graphic controller Dc Imaging direction Dv Vehicle axial direction Lr Lane outline Mv Vehicle mark Md Traveling direction mark (corresponding to a mark image)
P1 Lane vanishing point P2 Average vanishing point P3 Vehicle center point P4 Average center point S Driving support system V Vehicle Vf Front vehicle X Position deviation vector (corresponding to position deviation)

Claims (7)

An image processing apparatus that performs processing by analyzing and processing image content in the front image with respect to a front image obtained by imaging the front in the traveling direction of the vehicle by an imaging unit mounted on the vehicle,
In the front image, a lane vanishing point that is an intersection of the two lane contour lines that are located on both the left and right sides of the travel lane of the vehicle and that forms the contour of the travel lane is detected on the extension line in the traveling direction. Lane vanishing point detecting means for
An average vanishing point located at an average position of each of the plurality of lane vanishing points detected in the past by the lane vanishing point detecting means, or the vehicle travels in a substantially straight traveling lane in the past by the lane vanishing point detecting means. Vehicle axis direction point detection means for detecting a lane vanishing point detected when the vehicle lane vanishing point is detected as a vehicle axis direction point;
Position deviation detecting means for detecting a relative position deviation of the lane vanishing point detected by the lane vanishing point with respect to the vehicle axis direction point detected by the vehicle axis direction point detecting means;
An image processing apparatus comprising: a front image processing unit that processes the front image in a display mode corresponding to the position deviation. The image processing apparatus according to claim 1.
It has navigation means that can acquire current position information and map information where the vehicle is located at that time,
The vehicle axis direction point detection means is configured to detect the lane disappearance based on a front image captured by the imaging means when the navigation means detects that the vehicle is currently traveling in the substantially straight traveling lane. An image processing apparatus for detecting a lane vanishing point detected by a point detection means as the vehicle axial direction point. The image processing apparatus according to claim 1 or 2,
The front image processing means processes the front image so as to superimpose a stamp image having a display mode reflecting predetermined information. The image processing apparatus according to claim 3.
The front image processing means reflects the positional deviation as the predetermined information in a display mode of the mark image. An image display device that displays the front image processed by the image processing device according to claim 4,
An image display apparatus comprising: a front image display unit that displays the front image on which the mark image is superimposed by the front image processing unit. An image processing apparatus that performs processing by analyzing and processing image content in the front image with respect to a front image obtained by imaging the front in the traveling direction of the vehicle by an imaging unit mounted on the vehicle,
In the front image, each of the two lane contour lines that form the contour of the traveling lane located at the center point of the image portion of the preceding vehicle in the traveling lane of the vehicle or on both the left and right sides of the traveling lane of the vehicle. Traveling direction point detecting means for detecting any of the lane vanishing points which are intersections on the extension line of the traveling direction as a traveling direction point;
Vehicle axis direction point detecting means for detecting, as a vehicle axis direction point, a point located at an average position of the plurality of traveling direction points detected in the past by the traveling direction point detecting means;
Position deviation detecting means for detecting a relative position deviation of the traveling direction point detected by the traveling direction detection means with respect to the vehicle axial direction point detected by the vehicle axial direction point detecting means;
An image processing apparatus comprising: a front image processing unit that processes the front image in a display mode corresponding to the position deviation. Image processing executed by arithmetic means included in an image processing apparatus that performs processing by analyzing and processing image contents in the front image with respect to a front image obtained by imaging the front in the traveling direction of the vehicle by an imaging unit mounted on the vehicle A method,
In the front image, each of the two lane contour lines that form the contour of the traveling lane located at the center point of the image portion of the preceding vehicle in the traveling lane of the vehicle or on both the left and right sides of the traveling lane of the vehicle. A traveling direction point detecting step of detecting any of the lane vanishing points that are intersections on the extension line of the traveling direction as a traveling direction point;
A vehicle axial direction point detecting step of detecting a point located at an average position of the plurality of traveling direction points detected in the past by the traveling direction point detecting step as a vehicle axial direction point;
A position deviation detecting step for detecting a relative position deviation of the traveling direction point detected by the traveling direction detection step with respect to the vehicle axial direction point detected by the vehicle axial direction point detecting step;
A front image processing step of processing the front image in a display mode corresponding to the position deviation.

Images