JP2002120676A - Surroundings monitor system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the number of installed monitoring cameras when the latest position of an arbitrary object in shot images picked up by the monitoring cameras is triangulated. SOLUTION: With the use of a plurality of shot images picked up by a single monitoring camera 11 on a time series basis, the latest position of an object is triangulated. The number of installed monitoring cameras 11 is reduced below that in a conventional example that uses a plurality of monitoring cameras to detect the latest position of an object from stereograms.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a peripheral monitoring device for detecting an object around a motor vehicle.

[0002]

2. Description of the Related Art There is a method of capturing a stereogram image and detecting a positional relationship with an arbitrary object and a separation distance by triangulation based on the captured image.

In this method, for example, as shown in FIG.
a and 2b are installed, the object 3 is imaged by the two monitoring cameras 2a and 2b, and a pair of images 4a and 4b as shown in FIG.
Get. Then, after performing the image recognition processing and the feature extraction processing from each of the images 4a and 4b to recognize the object 3, the positional deviation between the images 4a and 4b of the object 3 is detected, and the positional deviation is detected. The position and the separation distance of the object 3 are calculated by triangulation based on the above information.

According to this method, surveillance cameras 2a, 2b
, It is possible to recognize not only the position of the object 3 from the viewpoint but also the distance from the viewpoint, and it is possible to monitor the periphery of the vehicle.

[0005]

In the conventional method of calculating the position and the separation distance of the object 3 based on the stereogram images 4a and 4b, at least one pair of the monitoring cameras 2 is used.
Since a and 2b were required, there was a problem that the installation cost was high.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a peripheral monitoring device capable of detecting the position and the separation distance of an object only by using a single monitoring camera.

[0007]

Means for Solving the Problems In order to solve the above problems,
According to the first aspect of the present invention, a single surveillance camera that captures an image of the periphery of a vehicle, a first sensor that detects a vehicle speed or a position of the vehicle, and a second sensor that detects a direction of the vehicle.
Sensor, storage means for accumulating a captured image captured by the monitoring camera at regular intervals, time-series changes in the captured image stored in the storage means, the vehicle speed or position of the car and A control unit that detects a relative position between the object and the vehicle based on the direction and according to a triangulation method, and outputs a result of the detection as result information.

According to a second aspect of the present invention, in the peripheral monitoring device according to the first aspect, the result information is based on a distance distribution in the captured image obtained according to the triangulation method. This is a planar image in which the positional relationship between a car and the object is developed in a plane, and further includes a display device that displays the planar image.

According to a third aspect of the present invention, in the peripheral monitoring device according to the first or second aspect, the result information is obtained when the vehicle and the object are close to each other within a predetermined distance. This is alarm information for outputting an alarm, and further includes an alarm output device for outputting the alarm information.

According to a fourth aspect of the present invention, in the peripheral monitoring device according to the second aspect, the control unit is configured to perform the control based on the distance distribution in the captured image obtained according to the triangulation method. The three-dimensional shape of the object is detected, and the three-dimensional shape of the object is drawn on the two-dimensional image.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS <Structure> FIG. 1 is a diagram showing a periphery monitoring device according to an embodiment of the present invention. This peripheral monitoring device is different from the conventional example shown in FIGS. 12 and 13 in that a plurality of captured images are used, and a separation distance to an object is determined based on a positional shift of the object in the captured images. Similarly, in this embodiment, a single surveillance camera 11
Using a plurality of captured images taken in chronological order in, the distance from the car to the object is calculated based on the displacement of the object between these captured images, in this regard, The periphery monitoring device of this embodiment is different from the conventional example.

More specifically, as shown in FIG. 1, the periphery monitoring device detects the current position of the vehicle 10 based on a vehicle speed sensor (first sensor) 12 and a steering angle sensor (second sensor) 13. At the same time, the position of an arbitrary object 14 (FIGS. 2 to 5) captured by a single surveillance camera 11 and the distance from the vehicle 10 are detected, and the vehicle 10 and the object 14 are detected.
Is displayed on a predetermined display device 15 as viewed from above.

As the single surveillance camera 11, a general surveillance camera having a CCD image pickup device is used. For example, the single surveillance camera 11 is installed at the left end of a front bumper of the automobile 10 so as to capture a scene in front. FIG. 3 shows a captured image Pc1 of the monitoring camera 11 in the situation shown in FIG.
Surveillance camera 11 when the situation changes from the situation of FIG.
5 are shown in FIG. The captured images Pc1 and Pc2 captured by the monitoring camera 11 are provided to an image processing device 16, where image recognition processing and feature extraction processing (hereinafter, collectively referred to as “image extraction”) are performed. The area of the object 14 with respect to the background in the captured images Pc1 and Pc2 is specified.
The information on the captured images Pc1 and Pc2 and the area of the object 14 specified by the image processing device 16 is stored in a volatile storage unit (RAM: storage unit) 18 connected via the main bus 17.
And given to the CPU (control unit) 19. still,
The storage of information in the RAM 18 is stored so as to be repeatedly accumulated at a fixed minute time interval, whereby information on the plurality of captured images Pc1 and Pc2 and the area of the object 14 is stored in the RAM 18 in an accumulative manner. Will be done.

The CPU 19 has a nonvolatile storage unit (ROM)
21 operates according to a software program stored in advance in the vehicle speed sensor 1.
2 and a vehicle position detection function for detecting the position and traveling speed of the vehicle 10 based on signals given from the steering angle sensor 13 and the captured images Pc1 and Pc stored in the RAM 18.
The three-dimensional shape of the object 14 is detected using triangulation based on c2 and a plurality of pieces of information about the area of the object 14 and the position and speed of the vehicle 10 detected by the vehicle position detection function. A three-dimensional shape detection function, a positional relationship detection function for detecting the positional relationship of the object 14 with respect to the vehicle 10 by using the triangulation method, and a separation distance of the object 14 from the vehicle 10 also by using the triangulation method Distance detection function for detecting the three-dimensional shape of the object 14 obtained by the three-dimensional shape detection function, the positional relationship of the object 14 obtained by the positional relationship detection function, and the object 14 obtained by the separation distance detection function A plan view conversion function for converting the positional relationship between the vehicle 10 and the object 14 into a plan view based on the distance to the vehicle 10 and the position of the vehicle 10 obtained by the vehicle position detection function; And a plan view output function of outputting a plan view converted generated by the conversion function at a predetermined drawing processor 22,
It has a notifying function of notifying the presence of the object 14 through a predetermined alarm output device 23 and a speaker 24 based on the relative positional relationship between the object 14 and the automobile 10.

The drawing processing device 22 converts the plan view given from the CPU 19 into a flat image at a predetermined resolution.
It has a function of transmitting the image to the display device 15 at a predetermined frame rate through a predetermined display I / F (interface) device 25.

The CPU 19 has a notification function, in particular, for determining whether or not the object 14 is within a certain distance from the vehicle 10 based on the detection result of the separation distance detection function. It has a function of giving a command to the warning output device 23 to output a warning from the speaker 24 when it is determined that the vehicle is within a certain distance from the speaker 10.

<Operation> The operation of the peripheral monitoring device having the above configuration will be described.

First, the surveillance camera 11 captures a scene in front of the automobile 10 in time series as, for example, captured images Pc1 and Pc2 (FIGS. 3 and 5), and an image is extracted in the image processing device 16, The region of the object 14 with respect to the background in the captured images Pc1 and Pc2 is specified. At this time, the information on the captured images Pc1 and Pc2 and the area of the specified target object 14 is accumulated in the RAM 18 at every fixed minute time.
The CPU 19 controls the object 1 stored and stored in the RAM 18.
Based on the region 4 and the plurality of captured images Pc1 and Pc2, the three-dimensional shape of the target object 14 is detected using triangulation.

Further, the CPU 19 detects the current position and the vehicle speed of the vehicle 10 based on signals given from the vehicle speed sensor 12 and the steering angle sensor 13.

At the same time, the CPU 19 uses the triangulation method based on a plurality of pieces of information about the area of the object 14 stored in the RAM 18 and information on the current position and the vehicle speed of the automobile 10 to determine the object 14. Of the vehicle 10 with respect to the automobile 10 and the separation distance are detected.

Then, the CPU 19 determines the vehicle 10 and the vehicle 10 based on the three-dimensional shape of the object 14, the position of the object 14, the distance to the object 14, the current position of the vehicle 10, and the vehicle speed obtained as described above. The positional relationship with the object 14 is converted into a plan view.

The converted plan view is the drawing processing device 2
The drawing processing device 22 converts the plan view into a flat image at a predetermined resolution and sends the image to the display device 15 at a predetermined frame rate through a predetermined display I / F (interface) device 25. I do. As a result, a flat image showing the automobile 10 and the object 14 is displayed on the display device 15.

The notifying function of the CPU 19 determines whether or not the object 14 is within a certain distance from the car 10 based on the detection result of the separation distance detecting function. When it is determined that the object 14 is within a certain distance from the automobile 10, a command is issued to the alarm output device 23 to output an alarm from the speaker 24.
The alarm output device 23, based on such a command,
A sound is output from the speaker 24 indicating that the object 0 is approaching the object 14.

As described above, even if only a single surveillance camera 11 is used, the object 1 can be obtained by utilizing the parallax due to the time difference.
4 relative to the car 10 (direction and separation distance)
Can be detected, compared to the conventional example in which two or more surveillance cameras 11 are used to detect the relative position (direction and separation distance) of the object 3 by a stereogram. 11 can be reduced in number. Therefore, it is possible to provide a peripheral monitoring device that requires less installation cost.

FIGS. 6 to 9 show examples of using the peripheral monitoring device. Here, FIG. 7 shows a plane image Pc3 of the surveillance camera 11 in the situation shown in FIG. 6, and FIG. 9 shows a plane image Pc4 of the surveillance camera 11 in the situation shown in FIG. As shown in FIGS. 7 and 9, the three-dimensional shape and the current position of the object 14 are detected, and the planar image P
Since c3 and Pc4 can be displayed on the display device 15, real planar images Pc3 and Pc4 can be displayed on the display device 15.

In the above embodiment, an example was described in which the surveillance camera 11 was installed at the left end of the front bumper of the automobile 10. May be performed. As an example of this, FIG. 11 shows a plane image Pc5 of the surveillance camera 11 in the situation shown in FIG. In this case, as shown in FIG. 10, when the vehicle 10 changes lanes to the left as shown in FIG. 10, the monitoring camera 11 monitoring the rear of the vehicle 10 captures the rear vehicle as the object 14 and the rear vehicle (object 14). Is displayed on the display device 15 as a plane image Pc5, indicating that the lane is changing to the left, so that the safety can be easily confirmed. In such a case, the detection accuracy is lower than in the case where the relative position of the stationary target object 14 is determined because the target object 14 is moving. The judgment may be made by increasing the pitch.

In place of the vehicle speed sensor 12, a car navigation device for detecting the current position of the vehicle may be used.

Further, in the above embodiment, after performing the image recognition process and the feature extraction process to specify the object 14,
Although the relative position of the object 14 and its three-dimensional shape were detected by obtaining the distance distribution by triangulation, the image recognition processing and the feature extraction processing were omitted, and the captured images Pc1 and Pc2 captured by the monitoring camera 11 were omitted. The distance distribution is obtained by triangulation for all the pixels of the object 1, and the object 1 is determined only by this distance distribution.
The relative position of No. 4 and its three-dimensional shape may be determined.

[0029]

According to the first aspect of the present invention, a relative position of an object is detected by triangulation using a plurality of images taken in time series by a single monitoring camera. Therefore, compared to the conventional example where the relative position of the object is detected by a stereogram using a plurality of surveillance cameras,
The number of surveillance cameras to be installed can be reduced, and thus installation costs can be reduced.

According to the second aspect of the present invention, the positional relationship between the vehicle and the object is easily displayed as a planar image on the display device based on the relative position of the object detected using a single surveillance camera. it can.

According to the third aspect of the present invention, an alarm can be easily issued based on the relative position of an object detected by using a single surveillance camera when the vehicle approaches an obstacle as the object. It is possible to emit.

According to the fourth aspect of the present invention, the three-dimensional shape of the object can be detected, and a planar image can be displayed on the display device based on the three-dimensional shape. There is an effect that an image can be displayed on a display device.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a periphery monitoring device according to one embodiment of the present invention.

FIG. 2 is a plan view showing an example of a relative relationship between an automobile and an object.

FIG. 3 is a diagram illustrating an example of a captured image in the state of FIG. 2;

FIG. 4 is a plan view showing another example of the relative relationship between the automobile and the object.

FIG. 5 is a diagram showing another example of the captured image in the state of FIG. 4;

FIG. 6 is a plan view showing another example of the relative relationship between the automobile and the object.

FIG. 7 is a diagram illustrating an example of a planar image displayed on a display device.

FIG. 8 is a plan view showing another example of the relative relationship between the automobile and the object.

FIG. 9 is a diagram illustrating another example of the planar image displayed on the display device.

FIG. 10 is a plan view showing another example of the relative relationship between the vehicle and the object in a modified example.

FIG. 11 is a diagram showing another example of the planar image displayed on the display device in the modification.

FIG. 12 is a diagram showing an example in which a conventional peripheral monitoring device is used.

FIG. 13 is a diagram showing a captured image of a conventional stereogram.

[Explanation of symbols]

 REFERENCE SIGNS LIST 10 automobile 11 surveillance camera 12 vehicle speed sensor 13 steering angle sensor 14 object 15 display device 16 image processing device RAM 18 RAM CPU 19 CPU 22 drawing processing device 23 alarm output device 24 speaker

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) B60R 21/00 626 B60R 21/00 626G 628 628C 1/00 1/00 A G01B 11/00 G01B 11/00 H // G08G 1/16 G08G 1/16 C (72) Inventor Jun Okada 1-7-10 Kikuzumi, Minami-ku, Nagoya-shi, Aichi Harness Research Institute, Inc. (72) Inventor Yozo Nishiura Nagoya-shi, Aichi 1-7-10 Kikuzumi, Minami-ku Harness Research Institute, Inc. F-term (reference) 2F065 AA06 AA42 AA53 CC11 FF04 FF09 FF36 JJ03 JJ26 QQ24 SS09 UU05 5H180 CC04 LL01 LL04 LL07

Claims (4)

[Claims] 1. A single surveillance camera for imaging the periphery of a car; a first sensor for detecting a speed or a position of the car; a second sensor for detecting a direction of the car; A storage unit that stores the captured image at regular intervals; a time-series change in the captured image stored in the storage unit; and a vehicle speed or a position and the direction of the vehicle. A peripheral monitoring device comprising: a control unit that detects a relative position between the target object and the vehicle according to a surveying method and outputs a result of the detection as result information. 2. The surroundings monitoring device according to claim 1, wherein the result information is a distance between the vehicle and the object based on a distance distribution in the captured image obtained according to the triangulation method. A peripheral monitoring device which is a planar image in which a positional relationship is developed on a plane, and further comprising a display device for displaying the planar image. 3. The peripheral monitoring device according to claim 1, wherein the result information is alarm information that outputs an alarm when the vehicle and the object are close to each other within a predetermined distance. A peripheral monitoring device further comprising an alarm output device that outputs the alarm information. 4. The peripheral monitoring device according to claim 2, wherein the control unit detects a three-dimensional shape of the target object based on a distance distribution in the captured image obtained according to the triangulation method. And a peripheral monitoring device that draws the three-dimensional shape of the object on the planar image.