JP2002036987A - Vehicle surrounding monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle surrounding monitoring device capable of presenting a surrounding state of a vehicle to a driver so as to easily understand. SOLUTION: A spot light beam 13 is projected from a light projector 5 toward a road surface 11 of the backward of the vehicle on a reference straight line of x=0 at a projection light angle α, and an image of the backward of the vehicle including a bright point P at each projection angle a is photographed by a camera 3. A position of the bright point P in the photographed image at each projection angle α is detected, a road surface condition of the backward of the vehicle Δh (which corresponds to roughness on the road surface 11 or presence or absence of an obstacle 19 on the road surface 11) is computed based on this detection result, a range which has a problem for passing through a vehicle 1 is determined, an image for displaying this range is formed and displayed on a display.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery monitoring device for confirming a situation around a vehicle.

[0002]

2. Description of the Related Art As a conventional vehicle periphery monitoring device, for example, a device disclosed in Japanese Patent Application Laid-Open No. 9-39658 is known.

[0003] In this method, a plurality of infrared beams are emitted from a bumper of a vehicle toward a road surface behind the vehicle, and an image of the emission direction is captured by a camera installed at a rear upper end portion of the vehicle. Is displayed on the monitor in the cabin, so that the distance to the obstacle existing behind the vehicle can be ascertained from the change in the interval between the infrared beams.

[0004]

However, in such a conventional vehicle periphery monitoring device, a driver reads a change in the interval between a plurality of infrared beams, whereby the driver can read the change.
Since the presence, the shape, and the distance of the obstacle were estimated, it was sometimes difficult to grasp the situation around the vehicle. For example, if there is a curb behind the vehicle and the road surface immediately behind the curb is depressed, the distance between the infrared beams will increase when the vehicle passes over the curb. In this case, it is difficult for the driver to intuitively judge whether the spread is caused by moving the curb or by the depression behind the curb, and it is difficult for the driver to distinguish such spread. In such a case, it was difficult to grasp the situation around the vehicle.

[0005] The present invention has been made in view of the above,
An object of the present invention is to provide a vehicle periphery monitoring device capable of presenting a situation around a vehicle to a driver in an easily understandable manner.

[0006]

According to the first aspect of the present invention,
In order to solve the above problems, a light projecting means for projecting a spot light beam in a light projecting direction variably toward a road surface around a vehicle, and a light projecting direction of the spot light beam projected by the light projecting means, Light-projection direction changing means for changing the target light-projection position on the road surface of the light beam from a vicinity of the vehicle to a distance far from the vehicle on a predetermined reference straight line, and a road surface of the spot light beam for each light-projection direction. Image pickup means for picking up an image around the vehicle including the target light emitting position above, and light emitting position detection for detecting a light emitting position of a spot light beam in the image picked up by the image picking up means for each light emitting direction Means, and road surface condition calculating means for calculating the unevenness of the road surface on the reference straight line, based on the light projecting position of the spot light beam in the captured image with respect to each light projecting direction detected by the light projecting position detecting means. And said An image for processing a video imaged by the imaging means based on the road surface unevenness on the reference straight line calculated by the surface state calculation means to generate a display image indicating the road surface unevenness surrounding the vehicle. The gist of the present invention is to include a generation unit and a display unit for displaying an image generated by the image generation unit.

According to a second aspect of the present invention, in the first aspect of the present invention, the road surface condition calculating means projects the spot light beam in the picked-up image in each of the light projecting directions detected by the light projecting position detecting means. Road surface unevenness height calculating means for calculating the height of the road surface unevenness on the reference straight line, based on the light position, based on the road surface unevenness height on the reference straight line calculated by the road surface unevenness height calculating means Means for determining a range in which there is a problem with the passage of the vehicle, and the image generation means includes a vehicle-passing problem range determined by the vehicle-passing problem range determining means. Processing the video imaged by the imaging means,
The gist of the present invention is to generate a display image indicating a range in which there is a problem in passing a vehicle.

According to a third aspect of the present invention, in the first or second aspect, a plurality of parallel reference straight lines are provided as the reference straight lines.

[0009]

According to the first aspect of the present invention, the light projecting direction of the spot light beam on the road surface is changed such that the target light projecting position moves on the predetermined reference straight line from the vicinity of the vehicle to the distance of the vehicle. Projecting a spot light beam toward the road surface around the vehicle while changing, capturing an image around the vehicle including the target projection position on the road surface of the spot light beam for each projection direction,
Detects the projection position of the spot light beam in the captured image, and calculates the unevenness of the road surface on the reference straight line based on the projection position of the spot light beam in the captured image for each detected projection direction. Then, based on the calculated road surface unevenness on the reference straight line, the captured image is processed to generate a display image indicating the road surface unevenness around the vehicle, and the generated image is displayed. Thus, the situation around the vehicle can be presented to the driver in an easy-to-understand manner, and the driver can easily grasp the situation around the vehicle.

According to the second aspect of the present invention, the height of the unevenness of the road surface on the reference straight line is calculated based on the projection position of the spot light beam in the captured image in each projection direction. Based on the height of the road surface unevenness on the reference straight line, determine the problematic area for vehicle passing, process the captured image based on the determined vehicle passing problem area, and By generating the display image indicating the problematic range, the driver can easily grasp the problematic range in passing the vehicle.

According to a third aspect of the present invention, in order to solve the above problem, a plurality of reference straight lines are provided in parallel with each other.
The situation around the vehicle can be more accurately detected, and the driver can more reliably grasp the situation around the vehicle.

[0012]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 is a block diagram showing a configuration of a vehicle surroundings monitoring apparatus according to a first embodiment of the present invention.

The camera 3 (imaging means) shown in FIG. 1 captures an image behind the vehicle, and together with the light emitting device 5 (light emitting means), as shown in FIG. Attached to. The camera 3 and the light projecting device 5 are respectively arranged on the vehicle center line. In particular, the light projecting device 5 is actually buried inside the bumper and emits visible light toward the rear of the vehicle as described later. Light.

The camera 3 is, for example, a small CCD camera using a CCD as an image sensor, has a sufficient angle of view (horizontal / vertical), and picks up an image within a predetermined angle range (angle of view) behind the vehicle. I do. The angle φ in FIG. 2 is the vertical angle of view (the horizontal angle of view is not shown).

The light projecting device 5 has such a performance that a visible light can be converted into a spot light (concentrated light) like spot light beam like a laser light and projected in a target direction in a pinpoint manner. The wavelength of the light emitted from the light emitting device 5 may be a wavelength outside the visible region, such as infrared light. In this case, the camera 3 emits the light from the light emitting device 5 in addition to the normal image. It must be able to capture light of wavelengths.
Further, the light projecting device 5 has a mechanism capable of freely changing the direction in which light is projected. This mechanism is well known, and is realized by, for example, using a mirror or changing the direction of the light emitting device itself. In the example of FIG. 2, the light beam 13 is emitted from the light emitting device 5 toward the road surface 11 obliquely downward with respect to the horizontal direction, and a bright point is generated at a point P (light emitting position) indicated by an arrow. I have.

The camera 3 and the light projecting device 5 are connected to a control unit 7 for controlling the system. The control unit 7 is installed at an appropriate place inside the vehicle. Control unit 7
Controls the camera 3 to acquire an image captured by the camera 3, and on the other hand, controls the ON / OFF of the light projecting device 5 and the direction in which light is projected as a light projecting direction changing unit.
Further, the control unit 7 includes a projection position detecting unit,
As a road surface condition calculating means (road surface unevenness height calculating means, vehicle passing problem range determining means) and an image generating means, the data is processed to calculate the position of a bright spot on a straight line. It has a function of detecting a road surface condition behind the vehicle (a road surface unevenness condition) such as the presence or absence of an upper obstacle. Although not shown, the control unit 7 includes a CPU for performing various processes, a ROM for storing a control program,
And a RAM serving as a work area for control.

The control unit 7 is further connected with a display 9 (display means). The display 9 is installed, for example, near the driver's seat, and is used to display information generated by the control unit 7 and present it to the driver.

Here, the coordinate system of the vehicle 1 is set as shown in FIGS. This vehicle coordinate system is an orthogonal coordinate system including X, Y, and Z axes orthogonal to each other.
This is a right-handed system in which the center line of the vehicle 1 is set on the Y axis (the rear of the vehicle is positive). In this coordinate system, the coordinates of any point are
It is represented by (x, y, z). The coordinate origin O can be set arbitrarily, but here, for simplicity, the camera 3
Are (0,0, h1) and the coordinates of the light projecting device 5 are (0,0, h1).
0, h2). Here, h1 is the height from the reference plane 15 (the running plane of the vehicle 1) to the camera origin, and h2
Is the height from the reference plane 15 to the light emitting device origin.
3 and 4, a concave portion 17 of the road surface 11 (a region surrounded by a solid line in FIG. 3) is provided behind the vehicle 1.
It is assumed that there is an obstacle 19 having a height such as a wall (a range surrounded by a solid line in FIG. 3). Obstacle 19
Is located behind the concave portion 17 of the road surface 11. Note that FIG.
Is a plan view showing a positional relationship between the vehicle 1 and a coordinate system;
FIG. 4 is a cross-sectional view as seen from a plane where x = constant.

Next, a control operation of the vehicle periphery monitoring device will be described with reference to a control flowchart shown in FIG. The control flowchart shown in FIG. 5 is stored as a control program in the ROM of the control unit 7,
Performed by the PU.

First, in step S10, the control unit 7 controls the light projecting device 5 to diagonally lower the horizontal direction at an angle α (hereinafter, this angle is referred to as “light projecting angle”). The light beam 13 is projected toward the light source 11. As a result, the bright spot P
Appears. In FIG. 4, for simplicity,
A horizontal auxiliary line 21 parallel to the Y axis is drawn.

In step S20, the camera 3 is controlled to capture an image behind the vehicle, and the image captured by the camera 3 is input. This image includes the bright spot P.

In step S30, step S
At step 20, the position of the bright spot P in the image input is detected. This detection processing may be performed based on the wavelength (color) of the light beam to be projected, or within a very short time,
This may be performed by capturing two images, that is, the image when the light projecting device 5 is turned on and the image when the light projecting device 5 is turned off, and taking the difference between them.

Then, in step S40, step S
Based on the position of the luminescent spot P in the image detected at 30 and the lens characteristics, the mounting angle, and the mounting position of the camera 3, the angle θ (the straight line connecting the luminescent spot P and the camera 3 forms with the horizontal plane) Hereinafter, this angle is referred to as a “bright point detection angle”), and further, using the position of the camera 3, the position of the light projecting device 5, the light projection angle α, and the bright point detection angle θ, The coordinates are calculated, and further, from the coordinates of the luminescent point P (particularly, the coordinate value z on the Z axis), finally, a vertical distance Δh between the luminescent point P and the reference plane 15 (hereinafter referred to as “bright point vertical Distance). This bright point vertical distance Δh is determined by the road surface condition around the vehicle,
Specifically, it corresponds to the degree (depth / height) of the unevenness of the road surface 11 and the height of a point on the obstacle 19 on the road surface 11 (collectively, “road surface unevenness height”).

Then, in step S50, it is determined whether or not the above-described processing has been completed for all the projection angles α in a preset range. For example, the range in which the projection angle α is changed is set to 0 <α <(π / 2). If the above processing has not been completed for all the projection angles α (S50: NO), step S6.
0, and if the above processing has been completed for all the projection angles α (S50: YES), step S70
Proceed to.

In step S60, the light projection angle α is set (changed) to the next value, and the process returns to step S10. here,
It is efficient to change the projection angle α according to the resolution of the camera 3. In addition, the processing speed can be improved by gradually changing the light projection angle α in the vicinity of the vehicle 1 and changing the light projection angle α slightly in a place far from the vehicle 1. .

Thus, the light projection angle α is set to 0 <α <(π /
By changing the value within the range of 2) and performing the above-described processing for each value of α, the intersection of the x = constant plane with the reference plane 15 (hereinafter, this intersection is referred to as “x = It is possible to acquire information on the road surface condition (the unevenness of the road surface 11 and the presence / absence of the obstacle 19 on the road surface 11) behind the vehicle on the “constant reference straight line”.

That is, in step S70, from the information of the bright spot vertical distance Δh at all the projection angles α, x =
The information about the road surface condition behind the vehicle on a certain reference straight line is acquired.

For example, when the road surface condition on the x = constant reference straight line is as shown in FIG. 6, the relationship between the projection angle α and the bright spot vertical distance Δh is represented by a graph shown in FIG. Therefore, the control unit 7 can calculate that the road surface condition on the x = constant reference straight line is as shown in FIG. 8 (see thick line portions m and n).

Here, when the absolute value of the bright spot vertical distance Δh is relatively small, there is no problem in passing the vehicle 1, but when the absolute value is larger than a certain value, there is a problem in passing the vehicle 1. Whether or not a problem occurs during the passage of the vehicle 1 is determined by the minimum ground clearance of the vehicle 1, the structure of the vehicle 1, and the like.
The value of Δh that causes a problem in passing the vehicle 1 is set in advance as a threshold value and is stored in the control unit 7. In the range A shown in FIG. 8 due to the principle of the system, the bright spot P cannot exist (see FIG. 6).
It can be seen only that the bright spot vertical Δh is 0 or less. So, in these areas, to avoid danger,
It is treated as a range having a problem in passing the vehicle 1.

Then, in step S80, a range having a problem in passing the vehicle 1 is determined. Specifically, the range in which the luminescent spot vertical distance Δh is equal to or larger than the threshold value and the range in which the luminescent spot P cannot exist due to the principle of the system, as described above, are considered as problems in passing the vehicle 1. A certain range.

For example, assuming that FIG. 6 shows a road surface condition on the reference straight line at x = 0 in FIG. 3, the control unit 7 determines the range in which there is a problem in passing the vehicle 1 by the range in FIG. B and range C.

Then, in step S90, step S90
It is calculated which range in the image input in step S20 corresponds to the range obtained in step S20. For example, here, it is assumed that the range B and the range C in FIG. 8 extend in parallel to the X axis (the shaded portions B ′,
C ′), a range B ′ in FIG. 3 based on the lens characteristics, the mounting angle, and the mounting position of the camera 3.
And which range in the image the range C 'corresponds to. At this time, one image may be appropriately selected from a plurality of images captured for each projection angle α.

Then, in step S100, an image photographed by the camera 3 is processed based on the calculation result in step S90, and a region having a problem in passing the vehicle 1 in the image (for example, in FIG. Are generated for display, which clearly indicate the ranges B ′, C ′). For example, in the example shown in FIG.
Portions corresponding to the ranges B 'and C' in the video are represented by different oblique lines. It should be noted that a specific example of the image processing is not limited to this. Other methods, for example, filling or blinking a range having a problem in passing the vehicle 1 with a specific color (for example, red). And the like. Further, in order to compensate for the sense of distance that tends to be insufficient in the monitor image, a range closest to the vehicle 1 (in the example of FIG. 9, two ranges B ′ and C ′) among a plurality of ranges having a problem in passing the vehicle 1. Among them, the distance to the range B ′) closer to the vehicle may be displayed on the screen at the same time.

In step S110, the display image generated in step S100 is displayed on the display 9. Thereby, the road surface condition behind the vehicle is provided to the driver as a displayed image of a range in which there is a problem in passing the vehicle 1.

As a result, the effect of the first embodiment is that the road surface condition behind the vehicle (unevenness of the road surface 11 and the presence or absence of the obstacle 19 on the road surface 11) is detected, and the image behind the vehicle is processed. By generating an image for display showing the detection result (the road surface condition behind the vehicle) and displaying the generated image on the display 9, the road condition behind the vehicle can be presented to the driver in an easy-to-understand manner. Can easily grasp the road surface condition behind the vehicle.

At this time, the driver determines a range in which there is a problem in passing the vehicle by generating and displaying an image indicating a range in which there is a problem in passing the vehicle. The range can be easily grasped.

In addition to the display 9, an alarm sound generating device for generating an alarm sound is mounted in the cabin of the vehicle 1.
When the vehicle 1 approaches a range in which there is a problem in passing, a warning sound may be generated by a command from the control unit 7.

(Second Embodiment) The feature of the second embodiment is that in the first embodiment, the road surface condition only on the reference straight line of x = 0 (that is, on the vehicle center line) is determined. , X
The processing is performed on the assumption that the road surface condition is continuous in a direction parallel to the axis. On the other hand, as shown in FIG.
In addition to the road surface conditions on the reference straight line L1 at = 0, the road surface conditions on the reference straight line L2 at x = a and the reference straight line L3 at x = -a are also obtained (where a>
0). At this time, if the reference straight lines L2 and L3 are determined based on both sides (vehicle width) of the vehicle 1 as shown in FIG.

Here, not only the road surface conditions on the reference straight line L1 but also the road surface conditions on the reference straight lines L2 and L3 are the same as those in the first embodiment (steps S10 to S80 in the control flowchart shown in FIG. 5).
(See).

The means for projecting light on the reference straight lines L1, L2, L3 may be realized by shaking the light projecting device 5 shown in FIG. 1 not only vertically but also horizontally. (In this case, it has the same configuration as the vehicle periphery monitoring device according to the first embodiment.) Or, a dedicated light projecting device corresponding to each of the reference straight lines L1, L2, L3 is provided. May be realized by providing the light projecting device. Here, for convenience, the former configuration will be described as an example.

In this embodiment, the reference straight lines L1, L
The processing after obtaining the road surface condition on 2, L3 is as follows.

For example, in FIG. 10, a range 23 surrounded by a solid line is a concave portion of the road surface 11, and another range 25 surrounded by a solid line is an obstacle having a height such as a wall. . Also in this case, the obstacle 25 is located behind the concave portion 23 of the road surface 11, and both have a depth and a height that the vehicle 1 cannot pass.

In this case, the control unit 7 calculates the road surface condition on each of the reference straight lines L1, L2, L3 and obtains a result as shown in FIG. 11 (see a thick line portion in FIG. 11).

Then, the obtained reference straight lines L1, L2,
From the road surface condition on L3, for each of the reference straight lines L1, L2, L3, a range (in the example of FIG. 11, ranges C1, C2, C3) in which there is a problem in passing the vehicle 1 is determined, and a range of the sum of these (In the example of FIG. 11, the range C) is obtained.

Basically, the range C of the sum is
Assuming that the area extends parallel to the X axis, this area is defined as an area having a problem in passing the vehicle 1 in the image.
However, for example, as shown in the example of FIG. 11, a problem area is detected on a reference straight line L1 on a plane where y = constant (range C1
If the problem area is not detected on the reference straight line L2 (and vice versa), the extension parallel to the X-axis is made up to the intermediate position between both reference straight lines (see the broken line 27 in FIG. 10). I do.

In FIG. 11, the range C3 on the reference straight line L3 is a range in which the depth of the road surface concave portion cannot be measured because the bright spot P cannot exist. Is treated as a range having a problem in passing through the above as described above.

The presentation to the driver is performed in the same manner as in the first embodiment (step S1 in FIG. 5).
00 and step S110), a display image (see FIG. 12) is displayed in the captured image that clearly indicates a range in which the vehicle 1 has a problem in passing, and displayed on the display 9.

As a modification of the present embodiment, for example, as shown in FIG.
9 and a reference straight line L2 corresponding to both sides of the vehicle 1.
, L3, a range having a problem with vehicle passing is not detected, and a range having a problem with vehicle passing is detected on a reference straight line L1 corresponding to the vehicle center line.
A reference straight line L4 is set between the reference straight lines L1 and L2, and a reference straight line L5 is set between the reference straight lines L1 and L3.
5 By determining the upper road surface condition, it is possible to more accurately detect the range in which the vehicle 1 passes.

As a method of processing an image, a reference straight line L
1, L2, L3 (, L4, L5) The road surface condition is converted based on the lens characteristics, the mounting angle, and the mounting position of the camera 3 so as to match the image captured by the camera 3, and is superimposed on the image. By displaying them together, the stereoscopic effect can be emphasized in the display video (see FIG. 14).

As a result, a special effect relating to the second embodiment is that a plurality of reference straight lines L 1, L
By providing 2, L3 (, L4, L5), the road surface condition behind the vehicle can be detected more accurately, and the driver can more reliably grasp the road surface condition behind the vehicle.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a vehicle periphery monitoring device according to a first embodiment of the present invention.

FIG. 2 is a diagram for explaining a mounting position of a camera and a light projecting device with respect to a vehicle.

FIG. 3 is a plan view showing a positional relationship between a vehicle and a coordinate system.

FIG. 4 is an elevation view corresponding to FIG. 3 as viewed from the x = constant plane.

FIG. 5 is a control flowchart for explaining a control operation of the vehicle periphery monitoring device according to the first embodiment.

FIG. 6 is a diagram illustrating an example of a road surface condition on a reference straight line at x = 0.

7 is a graph showing a relationship between a projection angle α and a bright spot vertical distance Δh corresponding to the road surface condition shown in FIG. 6;

FIG. 8 is a diagram showing a calculation result of a road surface condition with respect to the example of FIG. 6;

FIG. 9 is a diagram illustrating an example of a display image corresponding to the example of FIG. 6;

FIG. 10 is a plan view for explaining the operation principle of the vehicle periphery monitoring device according to the second embodiment.

11 is a diagram illustrating a calculation result of a road surface condition with respect to the example of FIG. 10;

FIG. 12 is a diagram illustrating an example of a display image corresponding to the example of FIG. 10;

FIG. 13 is a plan view for explaining a modification of the second embodiment.

FIG. 14 is a diagram illustrating an example of a display image in which a three-dimensional effect is emphasized with respect to the example of FIG. 10;

[Explanation of symbols]

 Reference Signs List 1 vehicle 3 camera 5 light emitting device 7 control unit 9 display 11 road surface 13 spot light beam 17, 23 concave portion on road surface 19, 25, 29 obstacle on road surface A, B, B ', C, C' Problematic range L1, L2, L3, L4, L5 Reference straight line P Bright spot

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (reference) B60R 21/00 630 B60R 21/00 630D 1/00 1/00 A G08G 1/16 G08G 1/16 C H04N 7/18 H04N 7/18 J

Claims (3)

[Claims] 1. A light projecting means for projecting a spot light beam variably in a light projecting direction toward a road surface around a vehicle, and a light projecting direction of the spot light beam projected by the light projecting means is defined by a spot light beam. A light projecting direction changing means for changing the target light projecting position on the road surface so as to move from the vicinity of the vehicle to the far side of the vehicle on a predetermined reference straight line, for each light projecting direction, the spot light beam on the road surface Image capturing means for capturing an image of the periphery of the vehicle including the target light emitting position; light emitting position detecting means for detecting a light emitting position of a spot light beam in the image captured by the image capturing means for each light emitting direction; A road surface condition calculating unit that calculates a bumpy state of a road surface on the reference straight line based on a light projecting position of a spot light beam in a captured image with respect to each light projecting direction detected by the light projecting position detecting device; Road surface conditions Image generating means for processing an image taken by the image pickup means on the basis of the road surface unevenness condition on the reference straight line calculated by the output means to generate a display image indicating the road surface unevenness condition around the vehicle. And a display unit for displaying an image generated by the image generation unit. 2. The road surface condition calculating means, based on a light projecting position of a spot light beam in a captured image with respect to each light projecting direction detected by the light projecting position detecting means, irregularities of the road surface on the reference straight line Road surface unevenness height calculating means for calculating the height of the vehicle, and a vehicle for determining a range having a problem with the passage of the vehicle based on the road surface unevenness height on the reference straight line calculated by the road surface unevenness height calculating means Having a passing problem range determining means, wherein the image generating means processes an image taken by the imaging means based on the vehicle passing problem range determined by the vehicle passing problem range determining means, 2. The vehicle surroundings monitoring device according to claim 1, wherein a display image indicating a range having a problem in passing is generated. 3. The method according to claim 1, wherein a plurality of reference straight lines parallel to each other are provided as the reference straight lines.
Or the vehicle surrounding monitoring device according to item 2.