JP2002279596A - Vehicle surrounding monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately display an image around a crossing even when a crossing path is curved. SOLUTION: A vehicle surrounding monitoring device for picking up the image of surroundings of a vehicle and displaying the image on a display device installed in a room of the vehicle comprises a left-side monitoring camera 2 and a right-side monitoring camera 4 for picking up the sideway image of the vehicle on its side, a left-side rotating mechanism 3 and a right-side rotating mechanism 5 for changing the monitoring direction to the horizontal direction, and an ECU 6 for giving edge extracting processing to a region out of the sideway image, in proximity to the owned vehicle, to extract edge information showing a road figure, detecting a disappearance point showing a point where the edge information disappears and controlling the left-side rotating mechanism 3 and the right-side rotating mechanism 5 so that the position of the disappearance point is located at a central position in the horizontal direction of the sideway image to be picked up by the left-side monitoring camera 2 and the right-side monitoring camera 4.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle surroundings monitoring apparatus for capturing an image of the surroundings of a vehicle and displaying an image.

[0002]

2. Description of the Related Art Conventionally, a camera device having a visual field in a direction perpendicular to the traveling direction of a vehicle has been installed on the front side of the vehicle and on the left and right sides of the vehicle. 2. Description of the Related Art There has been known a vehicle surroundings monitoring device capable of displaying a situation on the front and left and right on a display monitor installed in a vehicle so that a driver can visually recognize the situation.

FIG. 7 shows a view range 104a, 10a and 10b of a camera device when a vehicle 101 equipped with the above-described vehicle surroundings monitoring device approaches an intersection 103 while traveling on a traveling road 102.
4b is shown. In this case, since the traveling path 102 and the intersection 103 are not orthogonal to each other, the intersection 103 is out of the visual field ranges 104a and 104b of the camera device, and there is a problem that an appropriate image cannot be visually recognized by the vehicle driver. Was.

[0004] In order to solve such a problem, a vehicle surroundings monitoring device disclosed in Japanese Patent Laid-Open Publication No. HEI 3-239,642 extracts a road white line by performing image processing on an image captured by a camera device. A vanishing point is detected from the extracted white line, and the rotation of the camera device is controlled so that the vanishing point is located at the center position in the horizontal direction of the camera device. As shown in FIG. 8, the vehicle surroundings monitoring apparatus that performs such processing detects the dotted lines in the figure as vanishing point positions 105a and 105b, and performs imaging with the vanishing point positions as the center positions in the horizontal direction. Thereby, the visual field ranges 104a, 10
4b can be directed to the intersection 103.
Was displayed on the display monitor.

[0005]

However, in the conventional vehicle surrounding monitoring apparatus, the detected vanishing point position 105
Since a and 105b are always the center positions of the field of view in the horizontal direction of the camera device, the vehicle driver should pay the most attention when the intersection 103 is curved as shown in FIG. The vicinity of the intersection deviates from the visual field ranges 104a and 104b of the camera device and cannot be displayed on the display monitor.

[0006] First, the white lines 106a, 106a shown in FIG.
Image processing is performed on the image including the 06b and the guardrail 107 to extract the white lines 106a and 106b, and an edge extracted image including the edge extraction curves 108a and 108b shown in FIG. 10B is created. Then, the edge extraction curve 108a,
108b is extended to the point where the extension line intersects to vanishing point 109
Detected as Then, the rotation of the camera device is controlled so that the vanishing point 109 is located at the center of the image captured by the camera device. By performing such control, the vehicle periphery monitoring device rotates the camera device so that the vanishing point 109 coincides with the central position 110 in the horizontal direction of the image.
As shown in (c), an image not including the vicinity of the intersection is displayed on the display monitor.

Accordingly, the present invention has been proposed in view of the above-described circumstances, and for example, a vehicle capable of reliably displaying an image near an intersection even when the intersection is curved. A surroundings monitoring device is provided.

[0008]

According to a first aspect of the present invention, there is provided a vehicle surroundings monitoring device for capturing an image of the surroundings of a vehicle and displaying the image on a display device disposed in a room of the vehicle. An imaging unit that captures a lateral image in a lateral direction; a driving unit that moves a monitoring direction of the imaging unit in a horizontal direction; and, among the side images captured by the imaging unit, a region near the own vehicle. Image processing means for performing edge extraction processing to extract edge information indicating a road shape and detecting a vanishing point indicating a point at which the edge information disappears; and capturing the vanishing point position determined by the image processing means, Control means for controlling the driving means so as to be located at the center position in the horizontal direction of the side image picked up by the means.

According to a second aspect of the present invention, in the vehicle surroundings monitoring apparatus according to the first aspect, the image processing means performs edge extraction processing to create a plurality of edge straight lines indicating a road shape, and a plurality of edge straight lines. The vanishing point is determined based on the straight line.

In the invention according to claim 3, claim 1 or 2
The image processing means according to claim 1, wherein the image processing means performs edge extraction processing on a road structure included in the side image captured by the imaging means and parallel to a road on which the vehicle travels. Extract information.

[0011]

According to the first aspect of the present invention, the vanishing point is detected by performing edge extraction processing on the area near the own vehicle in the side image taken by the imaging means, and the vanishing point position is determined. Since it is located at the center position in the horizontal direction of the imaging direction in which the image is taken by the imaging means, even when the end of the intersection where the vehicle is approaching is curved, a side image near the intersection is reliably taken. Then, it can be displayed on the display device.

According to the second aspect of the present invention, a plurality of edge straight lines indicating a road shape are created by performing edge extraction processing,
Since the vanishing point is calculated based on a plurality of edge straight lines, a vanishing point based on the road shape near the intersection where the vehicle is approaching can be obtained with high accuracy, and further, a side image near the intersection can be reliably captured. it can.

According to the third aspect of the present invention, a vanishing point is determined by performing edge extraction processing on a road structure parallel to the road on which the vehicle travels to extract edge information and determine a vanishing point. The element can also be used as edge information indicating the road shape, and a side image near the intersection can be reliably captured.

[0014]

Embodiments of the present invention will be described below with reference to the drawings.

The present invention is applied to, for example, a vehicle periphery monitoring device configured as shown in FIG.

[Configuration of Vehicle Surrounding Monitoring Apparatus] FIG. 1 shows the configuration of a vehicle surroundings monitoring apparatus to which the present invention is applied. The vehicle surrounding monitoring device includes a display monitor 1 disposed at a position in the vehicle that can be visually recognized by a driving vehicle, a left side monitoring camera 2 and a left side rotation mechanism 3 disposed on the left side in front of the vehicle.
And a right side surveillance camera 4 disposed on the right side in front of the vehicle
And a right rotation mechanism 5 and an ECU (electronic control unit) 6 for controlling the left rotation mechanism 3 and the right rotation mechanism 5.

In the following description, the left surveillance camera 2 and the right surveillance camera 4 are simply referred to as "surveillance cameras", and the left and right rotation mechanisms 3 and 5 are simply referred to as "surveillance cameras". This is called a “rotation mechanism”.

In this vehicle surroundings monitoring device, when the vehicle is running, the left side monitoring camera 2 images the left side of the vehicle as the monitoring direction, and the right side monitoring camera 4 images the right side of the vehicle as the monitoring direction. Then, the captured image is output to the display monitor 1, and the display monitor 1 displays the left image and the right image.

In this vehicle surroundings monitoring device, a side image taken by a monitoring camera is output to the ECU 6, and the ECU 6 outputs the side image.
U6 performs image processing and the like, outputs a rotation drive signal to the rotation mechanism based on the result of the image processing, controls the monitoring direction of the monitoring camera by rotating the rotation mechanism, and controls the field of view of the monitoring camera. Carry out vehicle surrounding monitoring control processing. The ECU 6 stores the camera control program therein, executes the vehicle surroundings monitoring control process, activates the camera control program, executes image processing on the side images stored in the internal memory, and performs rotation. Generate a drive signal.

FIG. 2 shows a processing procedure when the ECU 6 performs the vehicle periphery monitoring control process.

When the vehicle is traveling, for example, when it comes to an intersection, the ECU 6 starts the processing from step S1. In step S1, the ECU 6 inputs a side image captured by the monitoring camera, and proceeds to step S2.

In step S2, the ECU 6 performs image processing or the like on the side image input in step S1, extracts edge information indicating the road shape, and detects a vanishing point indicating a point where the edge information disappears. A vanishing point detection process is performed, and the process proceeds to step S3.

In step S3, the ECU 6 proceeds to step S4 when the vanishing point has been detected in the vanishing point detection process in step S2, and proceeds to step S6 when the vanishing point has not been detected. The details of the vanishing point detection processing will be described later.

In step S4, the ECU 6 calculates the rotation angle of the rotation mechanism so that the vanishing point position detected in step S2 coincides with the center position in the horizontal direction of the side image captured by the monitoring camera. Processing proceeds to

In step S5, the ECU 6 generates a rotation driving signal for rotating the monitoring camera by the rotation angle calculated in step S4 and outputs the rotation driving signal to the rotating mechanism, so that the monitoring direction of the monitoring camera is set in the horizontal direction. Then, the monitoring direction of the monitoring camera is changed, and the process proceeds to step S6.

In step S6, the side image taken by the surveillance camera is output to the display monitor 1, displayed, and the process is terminated.

FIG. 3 shows a processing procedure for performing the vanishing point detection process in step S2 in the vehicle surroundings monitoring control process.

In step S11 of FIG. 3, the surveillance camera transfers to the ECU 6 a partial side image 13 of only the area near the own vehicle from the side image including the white lines 11a and 11b and the guard rail 12 shown in FIG. I do. In response, the ECU 6 stores the input partial side image data in the internal image memory. Here, the own vehicle vicinity area is a lower area of the side image, and indicates an image area near the intersection when the vehicle approaches the intersection.

In the next step S12, the ECU 6
Performs edge extraction processing for differentiating the input partial side image data in, for example, the vertical direction in the image, extracts edge information indicating an edge portion of the partial side image, and proceeds to step S13.

In step S13, the ECU 6 performs a linear approximation using the edge information to generate a plurality of edge straight lines, and proceeds to step S14. At this time,
The ECU 6 divides the partial side image 13 into small areas, and performs a linear approximation using the edge information for each small area. Next, the ECU 6 converts each of the extracted edge straight lines into the partial side image 1
3 and organize them by combining them into a partial side image 1
3 is extracted as an entire edge straight line. This allows
The ECU 6, as shown in FIG.
Edge straight lines 14a, 1b extracted as edge components
An edge extraction image including 4b is created.

In step S14, the ECU 6 removes the edge information that was not linearly approximated as an edge straight line as a result of the linear approximation in step S13, and also removes the short edge straight line, and proceeds to step S15.

In step S15, the ECU 6 calculates intersections for all edge straight lines included in the partial side image 13 from which the noise component has been removed in step S14, and proceeds to step S16. In the example shown in FIG. 4, the edge straight line 14a and the edge straight line 14b
Is extended to calculate the position of the intersection.

In step S16, the ECU 6 removes, from the intersection calculated in step S15, an isolated point that is not clearly recognized as a vanishing point, and proceeds to step S17.

In step S17, the ECU 6 obtains the center point of each intersection that has not been removed in step S16, determines that point as the vanishing point 15, and ends the processing.

In the example shown in FIG. 4, step S15
Since only one intersection is calculated in step S16 and step S17, the processing proceeds without performing substantially any processing, and finally the edge straight lines 14a and 14
The intersection 15 obtained by extending b is determined as a vanishing point. On the other hand, since many edge components are detected in an actual image, there are a plurality of edge straight lines, and as a result, a plurality of intersections are calculated, so that the processes of steps S16 and S17 function effectively.

In the ECU 6 which performs such processing, FIG.
The white lines 11a and 11b and the guardrail 12 shown in FIG.
, Image processing is performed only on the partial side image 13 to extract an edge component (steps S12 and S13), and the edge straight lines 14a and 14 shown in FIG.
An edge extraction image including b is created. And ECU6
Then, the intersection of the extended straight lines is calculated by extending the edge straight line, noise is removed, and the center point of the remaining intersections is set to the vanishing point 15.
(Step S15 to Step S17).
Then, the ECU 6 controls the rotation of the surveillance camera so that the vanishing point 15 is located at the center position 16 in the horizontal direction of the image captured by the surveillance camera. With such control, E
CU6 is equivalent to A corresponding to the center position 16 from the vanishing point 15.
Only by rotating the surveillance camera, it is possible to take an image of the vicinity of the intersection even on an intersection having a curve as shown in FIG.

E for performing such vehicle surrounding monitoring control processing E
According to the vehicle surroundings monitoring device provided with the CU 6, an image portion taken near the intersection is extracted as a partial side image 13 from the side image captured by the monitoring camera, and the partial side image 1
3, the vanishing point 15 is calculated by extracting the edge information, so that, for example, even when the intersection is curved, the situation near the intersection can be reliably imaged by the monitoring camera. .

That is, according to this vehicle surroundings monitoring device, since the rotational drive of the monitoring camera is controlled using the partial side image 13 only near the intersection, the vehicle 101 is controlled as shown in FIG. Even on a road that approaches the intersection 103 from the approach road 102 and curves from the end of the intersection 103, the vanishing point position 16 is obtained from the road shape near the intersection, and the road as indicated by the dotted line A in the figure. The field of view 18 can be determined in the same manner as described above, and the situation around the intersection 103 can be reliably captured by the surveillance camera.

[Application Example to Another Side Image] In the above-described vehicle periphery monitoring control processing and vanishing point detection processing, an example in which white lines 11a and 11b on a road are extracted as edge components has been described. It is also possible to extract the edge of another installed object and rotate the surveillance camera.

That is, the ECU 6 operates, for example, as shown in FIG.
When the partial side image 13 includes only the left white line 11 and the guard rail 12, the image shown in FIG. Edge extraction is also performed on the guardrail installation portion 17 (step S12), and an edge straight line can be obtained. As a result, the ECU 6 obtains an edge straight line 1 obtained by extracting edges from the white line 11 as shown in FIG.
The vanishing point 15 is obtained by using three edge straight lines, 4d, an edge straight line 14e obtained by extracting edges from the guardrail 12, and an edge straight line 14f obtained by extracting edges from the guardrail installation portion 17.

The ECU 6 that performs such processing performs edge extraction not only on the white line 11 as a road structure but also on walls, guardrails, curbs, and the like parallel to the road to be imaged, and removes elements associated with the road. Is also used as edge information indicating the road shape, and the surveillance camera is driven to rotate.
The vanishing point 15 can be detected accurately.

The above embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and other than the present embodiment, various modifications may be made according to the design and the like within a range not departing from the technical idea according to the present invention. Can be changed.

Also, in the above-described embodiment, an example has been described in which the vehicle periphery monitoring control process is performed when the traveling vehicle approaches an intersection, but in other cases, the vehicle periphery monitoring control process is performed. Of course, it is also good.

Further, in the above-described embodiment, an example has been described in which the vehicle periphery monitoring control processing and the vanishing point detection processing are performed using the side images captured by the left side monitoring camera 2 or the right side monitoring camera 4. Of course, separate vehicle surrounding monitoring control processing and vanishing point detection processing may be performed for the left and right sides, and the left and right images may be divided and displayed on the display monitor 1.

Furthermore, in the above-described embodiment, an example has been described in which edge extraction processing and the like are performed on a white line, a guardrail, and a guardrail attachment portion. However, the present invention is not limited to this, and any road structure parallel to a road may be used. , Curbs and other objects.

Furthermore, in the above-described embodiment, a white line has been described as an example of a line marker, but this may be a yellow line, a continuous line or a broken line.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device to which the present invention is applied.

FIG. 2 is a flowchart illustrating a processing procedure when a vehicle periphery monitoring control process is performed by an ECU.

FIG. 3 is a flowchart illustrating a processing procedure when a vanishing point detection process is performed by an ECU.

FIG. 4A is a diagram illustrating that a partial side image is used from a side image captured by a surveillance camera;
(B) is a diagram for explaining extraction of an edge straight line from a white line of a partial side image, and (c) is a side image captured by a monitoring camera after a vanishing point is determined and a rotation mechanism is driven to rotate. It is a figure showing an image.

FIG. 5 is a diagram for explaining a visual field range of a road where an intersection curves as a result of performing a vehicle periphery monitoring control process and a vanishing point determination process by an ECU.

FIG. 6A is a diagram illustrating the use of a partial lateral image from another lateral image captured by a surveillance camera, and FIG. 6B is a diagram extracting an edge straight line from a white line of the partial lateral image. FIG.

FIG. 7 is a diagram showing a visual field range of a camera device when a vehicle equipped with a conventional vehicle surroundings monitoring device approaches an intersection while traveling on a traveling road.

FIG. 8 is a diagram for explaining an example of monitoring the surroundings of a vehicle by changing the field of view with a conventional vehicle surroundings monitoring device.

FIG. 9 shows a case where an intersection is curved.
FIG. 10 is a diagram for explaining a visual field range of a conventional vehicle surrounding monitoring device.

10A is a diagram illustrating an image captured by a conventional vehicle surroundings monitoring device, and FIG. 10B is a diagram illustrating a process of detecting a vanishing point using an edge extracted image created by the conventional vehicle surroundings monitoring device. FIG. 3C is a diagram showing an image when the visual field range is changed by a conventional vehicle surrounding monitoring device.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Display monitor 2 Left side monitoring camera 3 Left side rotation mechanism 4 Right side monitoring camera 5 Right side rotation mechanism 6 ECU 11 White line 12 Guard rail 13 Partial side image 14 Edge straight line 15 Vanishing point 16 Center position 17 Guard rail installation part

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) B60R 21/00 628 B60R 21/00 628F G06T 1/00 330 G06T 1/00 330A 7/60 200 7/60 200J H04N 7/18 H04N 7/18 JEF term (reference) 5B057 AA16 CA12 CA16 CB12 CB17 CE02 CE15 CF05 DA07 DA17 DB02 DC09 DC16 DC36 5C054 AA01 CF06 FC12 FC14 HA30 5H180 AA01 CC04 CC24 LL02 5L096 FA06 FA06 FA06 GA19

Claims (3)

[Claims] 1. A vehicle surroundings monitoring device that captures an image around a vehicle and displays the image on a display device disposed in a cabin of the vehicle, wherein an imaging unit captures a lateral image in a lateral direction of the vehicle. Driving means for moving the monitoring direction of the imaging means in the horizontal direction; and edge information indicating a road shape by performing edge extraction processing on the area near the own vehicle of the side image captured by the imaging means. Image processing means for extracting a vanishing point indicating a point at which the edge information disappears, and the vanishing point position determined by the image processing means is set at the center in the horizontal direction of a side image taken by the imaging means. And a control means for controlling the driving means so as to set the position to a position. 2. The image processing device according to claim 1, wherein the image processing unit performs edge extraction processing to create a plurality of edge straight lines indicating a road shape, and determines a vanishing point based on the plurality of edge straight lines. Vehicle surrounding monitoring device. 3. The image processing means extracts edge information by performing edge extraction processing on a road structure included in a side image taken by the imaging means and parallel to a road on which a vehicle travels. The vehicle surroundings monitoring device according to claim 1 or 2, wherein: