JPH10211849A - Vehicle rear visibility supporting system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make one's own vehicle so as to be safely and easily retractable by having a birds-eye image converted into a ground-surface coordinate system projected with traffic lane data secured from a rear monitor camera with this rear monitor camera as a visual point and one's own vehicle, both compositely displayed on a monitor of an image display part, respectively. SOLUTION: An image signal out of a camera 1 is preprocessed for smoothenizing processing or the like in a preprocessing part 3, and then it is processed for binary-coded processing in a binary-coded processing part 4. In a white line extracting part 5, a white line as a traffic lane is extracted from this processing result. In addition, a center line extracting part 6 extracts a center line in the extracted white line. Data of this center line is converted into the data of a ground-surface coordinate system projected with the camera 1 as a visual point, thereby forming a birds-eye image. This bird-eye image secured in a transparentness conversion part 7 is coordinate-converted, generating line image data corresponding to a display image plane of a birds-eye view display part 2a of an image display part, and further an image displayed with one's own vehicle is composed at the specified position of a monitor image. This composition conduces when it retreats.

Description

DETAILED DESCRIPTION OF THE INVENTION

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a rear view support system for a vehicle, and more particularly to a rear view support system for displaying a rear view of a vehicle on an image display unit in a vehicle cabin through an imaging device.

[0001]

2. Description of the Related Art FIG. 11 schematically shows a configuration of a conventionally known vehicle rear view support device, in which a camera 1 serving as a rear monitoring imaging unit is attached to an upper rear portion of a vehicle (truck). In addition, a monitor 2 serving as an image display unit is provided in front of the driver's seat 11.

In this vehicle rear view support device, a monitor 2 displays the rear view of the vehicle, and supports the driver so that the driver can drive backward without particularly turning around.

As a technique related to this, for example, Japanese Patent Application Laid-Open No. Hei 4-368241 discloses an apparatus that uses a camera and a monitor to photograph a blind spot behind and on a side of a commercial vehicle and displays the image to a driver. Has been proposed.

In this device, a monitor is installed around a console box so that a driver who is driving while looking ahead can look at a rear image without moving his / her gaze.

There is also a system that uses a camera and a monitor to display a driver as if looking at a rearview mirror. In this system, if a driver who is familiar with the rearview mirror displays the image of the camera as it is, it makes the driver feel uncomfortable without looking like a mirror, so a left-right inverted image (mirror image) as if looking in a mirror can be obtained Thus, the image of the rear view is displayed on the monitor to reduce the driver's discomfort.

[0006]

However, the device disclosed in Japanese Patent Application Laid-Open No. 4-368241 is merely an auxiliary device for confirming the rearward direction. It is very difficult to handle and back.

That is, since the monitor image when the driver retreats the vehicle is irrelevant to the retreat direction of the vehicle, relying only on the monitor image may cause an accident such as a collision with a rear object.

The first cause is that it is difficult to grasp the trajectory of the rear wheel from the monitor image. That is, in the case of a 10t-class heavy commercial vehicle (truck) as shown in FIG. 11, as shown in FIG.
Since the distance M (overhang) from the axis 14 of the (rear wheels 13a, 13b) to the position of the camera 1 increases, the rear wheel 1
Trajectories 15a and 15b passing through 3a and 13b and a trajectory 16 of the viewpoint position of the camera 1 greatly differ. for that reason,
It becomes difficult to predict the behavior of the vehicle from the monitoring image on the monitor 2.

[0009] The second cause is that it is difficult to grasp which direction the vehicle is facing. That is, when the camera 1 is installed at a position as shown in FIG. 11, the vehicle body of the own vehicle is not displayed on the monitor 2, and the vehicle does not move in a lane such as a white line drawn in a parking lot or the like. It is difficult to grasp whether such a positional relationship exists, and driving becomes difficult.

On the other hand, Japanese Patent Application Laid-Open No. Hei 7-249196 proposes a technique in which a camera is installed in a parking lot, the attitude of the vehicle is monitored, and information is provided to a position where it can be viewed from inside the vehicle. However, such a system is limited to a specific garage.

A third cause is that it is difficult to predict how much the steering wheel should be turned when backing.

On the other hand, an apparatus for assisting a garage in a parking lot or the like as described above is proposed in Japanese Patent Application Laid-Open No. 7-17328. In this apparatus, a plurality of wide-angle CCDs are provided.
Using a camera, inputting a range image with a laser range finder, generating a bird's-eye view (three-dimensional image) viewed from an oblique side by computer graphics, and using a head-up display for the display Includes the problem of large scale.

Therefore, the present invention provides a vehicle rear view support device having a rear monitoring image pickup section (camera) for obtaining a rear monitoring image of a vehicle and an image display section (monitor) for displaying the rear monitoring image. The object of the present invention is to display on a monitor a reverse image indicating a reverse course of the vehicle.

[0014]

In order to achieve the above object, a vehicle rear view support device according to the present invention comprises a lane extracting section for extracting lane data in a rear monitor image obtained from a rear monitor image pickup section. And a perspective transformation unit that converts the extracted lane data into data of a plane coordinate system based on the ground surface projected from the rear monitoring imaging unit as a viewpoint to generate a bird's-eye image, and a perspective transformation unit obtained from the perspective transformation unit. A line image generating unit that performs coordinate conversion on the bird's-eye image and generates line image data corresponding to the display screen of the image display unit, and converts the line image data into an image displaying the host vehicle at a specific position on the image display unit. And an image synthesizing unit for synthesizing.

That is, in the present invention, the lane extracting section extracts lane data in the rearward monitoring image obtained from the rearward monitoring image pickup section, and converts the extracted lane data (preferably, the lane data subjected to the thinning processing). The perspective transformation unit converts the image into projection image data of a plane coordinate system based on the ground surface projected from the rear monitoring imaging unit as a viewpoint (inverse transformation of normal perspective transformation).

The line image generator converts the projection image data into line image data that can be displayed corresponding to the display screen of the image display unit (monitor).

Further, the line image data and an image displaying the own vehicle at a specific position of the image display unit are screen-synthesized by the image synthesis unit and displayed on the image display unit.

As a result, a bird's-eye view image (an image viewed from directly above) of a lane drawn in a parking lot or the like at the time of retreat is displayed simultaneously on one screen of the image display unit. The driver can grasp the correlation between the lane such as the white line and the own vehicle as a bird's-eye view, and can easily guide a commercial vehicle or the like in a parking lot or the like.

In the present invention, another image display unit for displaying the unprocessed rear monitoring image obtained from the rear monitoring imaging unit may be provided separately from the bird's-eye image and the composite image of the own vehicle. .

Thus, it is possible to safely and easily retreat the vehicle while viewing the bird's-eye view display screen and the unprocessed monitor screen for determining whether an obstacle is present.

[0021]

FIG. 1 shows an embodiment of a vehicle rear view support apparatus according to the present invention. In this embodiment, a camera 1 serving as a rear monitoring and imaging unit is connected to an image processing unit 10. The output signal of the image processing unit 10 is
Is provided to the bird's-eye view display unit 2a.

The image processing unit 10 includes a pre-processing unit 3 for pre-processing an image signal output from the camera 1,
Binarization processing unit 4 for performing a binarization process on the output signal of
A white line extraction unit 5 for extracting a white line as a lane based on the processing result of the binarization processing unit 4, and a center line extraction unit 6 for extracting a center line of the white lines extracted by the white line extraction unit 5
And a perspective transformation unit 7 that converts the data of the center line into data of a ground plane coordinate system projected from the camera 1 as a viewpoint to generate a bird's-eye image, and coordinates-transforms the bird's-eye image obtained from the perspective transformation unit 7 Bird's-eye view display unit 2a as an image display unit
A line image generating unit 8 for generating line image data corresponding to the display screen, an image synthesizing unit 9 for synthesizing the line image data with an image displaying the own vehicle at a specific position on the display unit 2a, And 9 a parameter setting section PS for setting predetermined parameters.

The camera 1 is provided separately from the bird's eye view display section 2a.
The camera image display unit 2b for displaying the output image itself as it is without processing is additionally provided.

FIG. 2 is a flow chart showing an example of the operation algorithm of the vehicle rear support apparatus according to the present invention shown in FIG. Hereinafter, the operation of the embodiment shown in FIG. 1 will be described with reference to this flowchart.

First, in this flowchart, it is determined whether or not the shift position is at the back (retreat position) (step S1), and only when the shift position is at the back position, the following steps are executed.

In this case, a shift position detection signal is required by a shift position detection unit (not shown) such as a shift lever switch. However, step S1 is skipped and the following processing is continuously performed. You may.

If the shift position is in the back position (Yes in step S1), the white line is extracted in the image processing unit 10 using the conventional technique (step S2).

That is, the rear monitor image output from the camera 1 is first subjected to an appropriate preprocessing such as a smoothing process (a process of averaging the values of surrounding pixels to obtain the values of the pixels) in the preprocessing unit 3. Is performed, the binarization processing unit 4 performs a binarization process in which a value equal to or greater than a certain threshold is set to “1” and a value equal to or smaller than a certain threshold is set to “0”.

Further, the white line extracting unit 5 expands or reduces the image obtained by the binarization processing to remove noise due to fine projections, holes, grooves, and the like.
The white line WL is extracted on the monitor screen 20 as shown in FIG. 3 (step S3).

Then, the representative value of the position of the white line extracted in this way is obtained as coordinate data (step S
4), the center line CL of the white line WL is extracted by the white line center line extracting unit 6 as shown in FIG. This is performed using a well-known thinning process. In short, the thinning process is a process of narrowing the line width to a line figure without losing connectivity to a connected image, and is a process of thinning while maintaining the characteristics of the line.

At this time, in the case of shadows or rain, white lines may not be extracted in some cases, and it is necessary to appropriately adjust the threshold (setting parameter) for performing the binarization processing by the parameter setting unit PS.

The center line C of the white line WL thus determined
The relationship between L and the screen plane T of the display unit 2a is as shown in FIG. 5, and by performing the inverse transformation of the normal perspective transformation in the perspective transformation unit 7, as shown in FIG. The position of the center line CL of the white line on the plane can be obtained as a bird's-eye view.

That is, the perspective transformation unit 7 is provided with the center line extraction unit 6
, And performs perspective transformation when projecting this data from the camera position R to a plane T at a focal length f (see FIG. 5) from the camera position R (step S).
5).

Here, the operation principle of the perspective transformation unit 7 and its inverse transformation will be described in detail below. In FIG. 7, it is assumed that the camera 1 is located at a point R (0, 0, H) on the z-axis and monitors a white line on the ground surface (xy coordinate plane) at a look-down angle τ. The following shows how a white line appears on the monitor screen.

First, after the origin Q of the coordinate system (x, y, z) is translated to the camera position R, the angle τ about the x axis is
The coordinate system rotated by only the coordinate system of the camera viewpoint (x ",
y ", z").

Therefore, the point (x, y, z) of the coordinate system (x, y, z) is changed to the point (x ", y", z) of the coordinate system (x ", y", z ").
y ", z") is as follows.

(Equation 1)

Here, since the ground surface is z = 0, the above equation (1) becomes as follows.

(Equation 2)

Next, the screen plane T at the focal length f
FIG. 8 shows the result of the conversion for projecting the image.

That is, the point (x, y, z) on the center line of the white line on the ground surface (z = 0) of the coordinate system (x, y, z) viewed from the coordinate system (x ", y", z ") 0) are the coordinates (x ", y", z ") already transformed in the above equation (2), and projecting them on the screen plane T reduces the screen plane T by f / y". Two-dimensional coordinate system on T (α,
β) It is equivalent to projecting on a plane.

Therefore, the projection from the coordinate system (x ", y", z ") to the coordinate system (α, β) is represented by the following equation.

[0041]

(Equation 3)

By substituting equation (3) into equation (2), the final perspective transformation is expressed by the following equation.

(Equation 4)

If x and y are solved using this equation (4), the coordinates on the screen plane T are converted to the coordinates on the ground surface (the bird's-eye view coordinates of the center line of the white line) as shown in the following equation. (Reverse perspective transformation).

[0044]

(Equation 5)

Thereafter, returning to FIG. 2, in step S5, the line image generating unit 8 further converts the perspective transformed data into line image information to be displayed on the monitor screen, and sends the line image information to the bird's eye view synthesizing unit 9.

As described above, the coordinates (x, y) of the lane in the coordinate system x, y on the ground surface are obtained by the equation (5). Next, a line drawing is generated as shown in FIG. It is necessary to convert the coordinates of the monitor screen 20 to be displayed on the driver.

That is, first, the following conversion is performed by multiplying the coordinates (x, y) by appropriate coefficients Sx and Sy.

(Equation 6)

The coordinates (x ", y") may be converted into coordinates (x ^* , y ^* ) on the monitor screen 20 to be displayed on the driver.

The coordinates (x ^* ,
y ^* ), it is necessary to move the origin from the center S of the screen to the origin O of the monitor, and then rotate the coordinate axes x ″, y ″ by 90 °. Coordinates (x ",
y ”) is converted into coordinates (x ^* , y ^* ) as follows.

(Equation 7)

Therefore, the data (α, β) of the lane on the screen plane T captured by the camera 1 is displayed to the driver as shown in the following equation by substituting equation (5) into equation (7). (X ^* ,
y ^* ).

[0051]

X ^* = − Sy · H (βsinτ + fcosτ) / (− βcosτ + fsinτ) + x ^* _SIZE / 2 (Equation (8))

Y ^* = − Sy · H · α / (− βcosτ + fsinτ) + y ^* _SIZE / 2 (Equation 9)

The lane (white line) projected image data can be converted into line image data corresponding to the bird's eye view monitor screen 20 by using the above equations (8) and (9), and the ground as shown in FIG. A bird's eye view of the center line CL of the white line WL when viewed from above is generated (step S6).

Returning to FIG. 2, the image processing section 10 causes the bird's-eye view synthesizing section 9 to display the line image information generated as described above on the image of the own vehicle previously displayed at a predetermined position on the monitor screen. (Step S7).

That is, since the camera 1 is fixed to the own vehicle, if the position of the camera is known, the position of the vehicle can be specified. Therefore, the image of the own vehicle on a two-dimensional plane viewed from directly above. And the image of the white line can be combined on the monitor screen. This indicates that it is not necessary to generate a three-dimensional computer graphic image.

For example, the vehicle 30 on which the camera 1 is always fixedly installed is always displayed at a specific position as shown in FIG. 10, and the position of the center line CL of the white line is changed as the vehicle retreats. I do. Therefore, data viewed from directly above the host vehicle 30 may be displayed, or, if a dedicated monitor screen is used, the data may be directly drawn on the glass surface (front surface) of the monitor screen.

On the other hand, the camera image display unit 2b inputs the output image from the camera 1 without any processing, and if this image is displayed separately from the bird's-eye view display unit 2a,
The driver's eyes can directly check what is in the rear monitoring image.

[0057]

As described above, according to the vehicle rear view support apparatus according to the present invention, the lane data in the rear monitoring image obtained from the rear monitoring imaging unit is projected on the ground using the rear monitoring imaging unit as a viewpoint. The bird's-eye image is generated by converting the data into data of the upper surface coordinate system, the bird's-eye image is subjected to coordinate conversion to generate line image data corresponding to the display screen of the image display unit, and the own vehicle is moved to a specific position of the image display unit. Since the configuration is such that the image is synthesized with the displayed image, the following effects can be obtained.

As a display method for a large vehicle such as a commercial vehicle (a display method for a professional driver having a sense of vehicle), knowing the reference line will provide sufficient assistance when the vehicle retreats. Even if something is placed on the white line, fragmentary white line information can be obtained, and this point is also sufficient help. The system is limited to only the recognition of the white line on the ground surface, and the system can be algorithmically simplified without performing complicated object recognition.

Since the viewpoint is viewed from directly above, it is easy to grasp the relationship between the size of the vehicle and the distance between the white lines. There is no need to bother the head-mounted display as in the prior art, and it is not necessary to newly generate a three-dimensional computer graphic image. A clear bird's-eye view image can be obtained because images are taken with a camera from a high place like a commercial vehicle.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a vehicle rear view support device according to the present invention.

FIG. 2 is a flowchart illustrating the operation of the vehicle rear view support device according to the present invention.

FIG. 3 shows a vehicle rear view support device according to the present invention;
It is the figure which showed the example which extracted the white line after carrying out the value process.

FIG. 4 shows a vehicle rear view support device according to the present invention;
It is the figure which showed the example which extracted the center line by the thinning process from the white line after the value process.

FIG. 5 is a perspective view showing a relationship between a monitor screen plane and a white line in the vehicle rear view support device according to the present invention.

FIG. 6 is a bird's eye view showing a position of a white line center line on a ground plane in the vehicle rear view support device according to the present invention.

FIG. 7 is a perspective view (1) for explaining the operation principle of the perspective conversion unit used in the vehicle rear view support device according to the present invention.

FIG. 8 is a perspective view (2) illustrating the operation principle of the perspective conversion unit used in the vehicle rear view support device according to the present invention.

FIG. 9 is a graph showing coordinate transformation in a line image generation unit used in the vehicle rear view support device according to the present invention.

FIG. 10 is a bird's-eye view in which the center line of the white line and the host vehicle are combined and displayed on the monitor screen in the vehicle rear view support device according to the present invention.

FIG. 11 is a side view of a general vehicle equipped with a rear view support device.

FIG. 12 is a plan view showing an example of the field of view and the trajectory of the rear wheels when the vehicle equipped with the rear view support device is moved backward.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 rear monitoring imaging unit (camera) 2a bird's-eye view display unit 2b camera image display unit 3 pre-processing unit 4 binarization processing unit 5 white line extraction unit 6 white line center line extraction unit 7 perspective conversion unit 8 line image generation unit 9 bird's-eye view synthesis unit 10 Image processing unit 20 Monitor screen WL White line CL Center line of white line In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (3)

[Claims] 1. A rear view support device for a vehicle, comprising: a rearward monitoring imaging section for obtaining a rearward monitoring image of a vehicle; and an image display section for displaying the rearward monitoring image, wherein the rearward monitoring image obtained from the rearward monitoring imaging section is provided. A lane extracting unit for extracting lane data in an image, and a perspective for generating a bird's-eye view image by converting the extracted lane data into data of a plane coordinate system based on the ground surface projected from the rear monitoring imaging unit as a viewpoint. A conversion unit, a line image generation unit configured to perform coordinate conversion on the bird's-eye image obtained by the perspective conversion unit and generate line image data corresponding to a display screen of the image display unit, and display the line image data on the image display unit. An image synthesizing unit for synthesizing an image in which the own vehicle is displayed at a specific position of the vehicle. 2. The vehicle rear view support device according to claim 1, wherein the lane extracting unit performs a thinning process as the lane data. 3. An image display unit according to claim 1 or 2, further comprising an unprocessed rear monitoring image obtained from the rear monitoring imaging unit separately from the bird's-eye image and the composite image of the vehicle. A vehicle rear view support device characterized in that: