JP2002202136A - Observation apparatus for state in circumference of vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an observation apparatus by which the state (especially positional information on other vehicles or the like) at the front or the rear (including the oblique front or the oblique rear) of a vehicle is observed by a simple processing operation, at low costs and precisely. SOLUTION: One imaging means 2 which images the image at the front or the rear of an own vehicle 1 is installed. A control unit 3 which stores map data used to convert the position coordinates of a specimen (another vehicle or the like) on the image into coordinates on the surface of the ground around the own vehicle 1 which judges the position on the image of the specimen and which converts the position into the coordinates on the surface of the ground by the map data so as to generate positional information on the surface of the ground on the specimen is installed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for observing the surroundings of a vehicle which is used, for example, for monitoring other vehicles or obstacles in front of or behind a vehicle, or following a preceding vehicle or monitoring the distance between vehicles.

[0002]

2. Description of the Related Art In order to monitor other vehicles or obstacles in front of or behind a vehicle, to control running following a preceding vehicle, or to notify a driver of danger information, a situation around the vehicle (particularly, a vehicle existing around the vehicle). As a device for observing position information of other vehicles or obstacles), a distance measuring device (that is, a radar) using laser, millimeter wave, infrared ray, or the like has been developed. For example, it has already been put into practical use in a four-wheeled vehicle. It is standard on some models. There is also known a camera in which two cameras, such as CCDs, are installed as stereo cameras in a vehicle and position information of a preceding vehicle or the like is generated based on the principle of triangulation.

[0003]

However, in a distance measuring apparatus using a laser or the like, a laser or the like scans and irradiates a predetermined detection area, and if the information of the reflected wave obtained by the processing is not processed in a complicated manner, the object to be measured may be damaged. Since the shape and size of the detected object are not known, the type determination and the position determination of the detected object can be performed instantaneously, and as a result, the situation around the vehicle (particularly, the situation at a relatively short distance of about several tens of meters) can be determined. It is difficult to obtain desired information in real time, and there is a limit to improving responsiveness and reliability while reducing costs. In addition, when a stereo camera is used, it is necessary to mount two cameras so as to realize a predetermined positional relationship and a predetermined mounting angle with high positioning accuracy, thereby increasing the cost,
Baseline length (the length that becomes the standard for triangulation;
(Specifically, if the distance between the two cameras) or the mounting angle of the cameras is slightly shifted, the detection accuracy of the position information of the preceding vehicle or the like is likely to be greatly reduced because the principle of triangulation is used. There is a problem that. Therefore, the present invention is based on the situation (particularly, the position information of the detected object or the position information of the detected object) in front of or behind the vehicle (including diagonally forward and diagonally rearward) from the image of one imaging unit attached to the vehicle. It is an object of the present invention to provide a surroundings observation device capable of accurately observing dangerous information and the like by simple processing.

[0004]

A surroundings observation apparatus according to a first aspect of the present invention includes an image pickup means for picking up an image in front of and / or behind a vehicle, and coordinates of the position of an object on the image. A storage means for storing map data for converting to coordinates on the ground surface with the vehicle as the center, and an object to be detected in the image is extracted, and a position of the object to be detected on the image is extracted. And determining means for converting the position into coordinates on the ground surface using the map data to generate position information on the horizontal plane of the object to be detected. here,
The “position information of the detected object on the horizontal plane” includes not only current position information (ie, current position information) but also past position information, past and current position information (ie, time of position information). (Trajectory information that is a target change) may be included.

[0005] A surrounding situation observation apparatus according to a second invention of this application is an image pickup means for picking up an image in front of and / or behind a vehicle, and a driving or driving of the vehicle based on the position of an object to be detected on the image. Storage means for storing map data for deriving information on control (for example, information on the degree of risk, information on whether or not lane change is possible, etc.), and extracting an object in the image, And a processing means for determining the position on the image and deriving the information from the position based on the map data. Here, the “position of the object to be detected on the image” includes not only the current position (ie, the current position), but also the past position, and the past and present positions (ie, the temporal change of the position). (Trajectory).

The imaging means is installed so that its axis crosses the road surface in front of or behind the vehicle at an angle, and the angle between the axis and a direction parallel to the ground surface is inversely proportional to the detection distance. It is preferable that the setting is made. With this configuration, it is easy to obtain an image that can be widely seen in the lane and the roadside zone adjacent to the own lane, and the determination of the position information and the like based on the map data becomes more accurate. It is preferable that the field of view, the direction of the axis, and the installation position of the imaging unit are set such that the position at infinity on the axis is located above the center position in the image to be captured. If the position at infinity is an image (for example, an image as shown in FIG. 3A) that is more preferably about 20 to 40% lower than the upper end of the image, for example, the other vehicle to be observed is high. Even large heavy trucks can be imaged as a whole to reliably discriminate, maximize the resolution of position determination, and minimize useless image data such as surrounding scenery. it can.

[0007]

An embodiment of the present invention will be described below with reference to the drawings. As shown in FIG. 1 (a) or (b), for example, as shown in FIG. 1 (a) or (b), an ambient condition observation device according to this embodiment includes an imaging unit 2 (for example, a CCD camera) attached to a vehicle 1 (inside or outside of the vehicle), and this imaging unit And a control unit 3 that determines a surrounding situation of the vehicle 1 (in this case, in particular, position information of another vehicle) based on the image data obtained in Step 2. The imaging unit 2 is not limited to a CCD camera as long as it can obtain image data (at least light and dark data), and may be a device that can obtain a color image or a black and white image. Further, the imaging means 2 may have a zoom function. Further, the imaging means 2 is preferably installed at a position somewhat higher than the ground surface. For example, as shown in FIG. 1 (c), its axis 2a obliquely intersects the road surface in front of or behind the own vehicle 1. And an angle α between the axis 2a and a direction parallel to the ground surface is set so as to be inversely proportional to the detection distance.

That is, when a rear image is taken, for example, as shown in FIG. 3 (a), it is installed at an installation position and an installation angle obliquely looking down on the rear road surface so that an image as shown in FIG. When capturing a distant rear image, the viewing angle is narrowed and the angle α is set relatively small, and when capturing a relatively close rear image, the viewing angle is increased and the viewing angle is increased. The angle α is set relatively large. In other words, in order to obtain an image (for example, an image as shown in FIG. 3A) that can be widely seen in the lane and the roadside zone adjacent to the own lane, the left and right visual fields and the upper and lower visual fields of the imaging unit 2 are necessary and sufficient. Needless to say, the image pickup means 2 is preferably installed diagonally downward from a position somewhat higher than the ground surface. More specifically, when capturing a rear image, for example, as shown in FIG. 1 (c), it may be attached to the lower surface of the roof spoiler. When an image in front is taken, the image pickup means 2 may be attached to the front part of the vehicle as shown in FIGS. 1A and 1C, for example. Since it is not possible, in order to improve the visibility, it is preferable to attach it to, for example, the upper part of the front window, the front end of the roof upper surface, the front end of the ceiling in the passenger compartment, the position of a room mirror (rearview mirror), and the like.

Further, the resolution of position determination is increased by increasing the amount of movement of an object such as another vehicle on the screen as much as possible, and useless images of surrounding scenery (sky (sky), distant mountains, etc.) are obtained. From the viewpoint of reducing the data, for example, as shown in FIG. 3A, the position at infinity on the axis 2a of the imaging means 2 (the position where the extension line such as the white line on the road surface intersects one point on the image). Above the center position in the image that is always taken (preferably 20 to
It is preferable that the field of view of the imaging means 2, its axial direction, or its installation height be set so as to be located at about 40% below). Note that the inventors have found that the position at the infinity is approximately 30% below the upper end of the image (for example, a state as shown in FIG. 3A), which is the most preferable. . When such an image is obtained, for example, even if the other vehicle to be observed is a large truck with a height, the entire vehicle can be imaged and reliably identified, and the resolution of the position determination is maximized. Useless image data such as surrounding scenery can be minimized.

Next, although not shown, the control unit 3 includes a microcomputer (hereinafter, referred to as a microcomputer) for processing image data and a non-volatile memory (storage means) for storing various data. . The data stored in the non-volatile memory includes, in addition to data for extracting an object to be detected, coordinates of the position of the object on the image captured by the imaging unit 2, centered on the own vehicle. Map data (see FIG. 5) for conversion into coordinates on the ground surface are also registered. FIG.
FIG. 3 is a diagram for explaining the principle of the map data.
On the image as shown in FIG.
(A) In the case of (a), a change in the position of the center of gravity of another vehicle traveling behind the host vehicle (an example of an actual measurement result).
FIG.

In this case, the actual position of the other vehicle (vehicle position) in the two-dimensional coordinate system parallel to the ground surface is 200 cm laterally away from the center of the own vehicle as shown by a white point in FIG. Course 650c backward from own car
The vehicle is moving relative to the vehicle in the same direction as the vehicle from a position about m apart to a position about 300 cm away. On the other hand, the position (pixel position) on the image picked up by the image pick-up means 2 is, as shown by a black point in FIG. (About 118 pixels in the direction) and a relatively lower right position (about 140 pixels in the left and right direction,
(About 9 pixels).

As described above, the actual position of the detected object in the two-dimensional coordinate system parallel to the ground surface (hereinafter referred to as the ground surface coordinate system) and the position of the detected object on the image set as shown in FIG. The position of the detected object does not correspond to the direction of the change (that is, even if the position of the detected object actually moves relative to the own vehicle in the same direction, the position change on the image is naturally oblique as the distance from the center of the own vehicle increases). However, the individual position data all correspond uniquely, and the position on the image also changes considerably with respect to the actual position change. Therefore, for example, as shown in FIG. 5, such a correspondence relationship of the position data is previously obtained theoretically or experimentally for a predetermined observation range, and a data table or a function expression representing such a correspondence relationship is obtained. Is stored as the above-described map data, the position on the image can be very easily and accurately converted to the actual position on the ground surface coordinate system using the map data. In FIG. 5, reference numerals Y1 to Y7 indicate positions in the ground surface coordinate system, and reference numerals y1 to y7 indicate positions Y1 to Y7.
To Y7, respectively.

Next, when the microcomputer of the control unit 3 is activated as an observation device, its operation program and the like are set so as to repeatedly execute at least the processing shown in FIG. That is, first, in step S1, the imaging means 2 is driven to sample image data. Next, in steps S2 and S3,
A well-known edge detection and contour extraction for determining a contour of a candidate to be detected by detecting a point (that is, an edge) at which a change in brightness or the like is performed. Next, in step S4, for example, the contour data of the detected object candidate is compared with appropriate stored contour data of the detected object (such as contour data of various vehicles), or the relative speed is analyzed from past observation results. By doing so, it is determined whether or not the object corresponds to an appropriate detection object (for example, various vehicles) registered in advance. In other words, simply performing edge detection and contour extraction may cause a signboard on the road shoulder or a spot on the road surface to be grasped as an object to be detected. Here, it is determined whether or not the detection target corresponds to the appropriate detection target. Then, in the branching process of step S5, if it is a proper detection target, the process proceeds to the next step S6, and if it is not a proper target, the process of one sequence is ended.

Next, in steps S6 and S7, the position of the center of gravity of the detected object on the image is determined from the contour data of the detected object determined to be appropriate. In step S8, based on the above-described map data, the coordinates of the position of the center of gravity in the above-described ground surface coordinate system are obtained from the position of the center of gravity on the image of the detected object. Next, in step S9,
The position information of the detected object obtained in step S8 (or
From the position information similarly obtained in the past observation), the distance of the detected object from the vehicle amount (distance in the front-rear direction and the left-right direction) or the relative speed of the detected object is calculated. Is completed. Note that, when there are a plurality of detected proper targets, the processes of steps S6 to S9 are naturally executed for each.

The position information and the like finally obtained in steps S8 and S9 may be, for example, an inter-vehicle distance monitoring control for outputting a warning to prevent a rear-end collision based on the relative distance and the relative speed, or a predetermined value for a preceding vehicle. This can be effectively used for advanced control functions of a vehicle such as a preceding vehicle follow-up control that automatically controls the vehicle to follow the vehicle while maintaining the following distance. In particular, when the position information of the other vehicle behind can be obtained based on the image behind the own vehicle as shown in FIG. 3A, the inter-vehicle distance to the following vehicle traveling in the same lane as the own vehicle. Monitoring can be performed, and danger information or the like can be notified to a vehicle traveling in an adjacent side lane and obliquely behind the vehicle. For example, FIG.
As shown in (a), when there is another vehicle 11 running on a relatively short distance (attention zone) on the side of the own vehicle, the lane side of the other vehicle 11 (in this case, the overtaking lane) side)
Alerts the driver of his / her vehicle with a warning lamp or the like that the vehicle needs to be careful to change lanes, and there is another vehicle on the side of the vehicle that runs a very short distance (danger zone). In such a case, a control such as notifying the driver of the own vehicle with a warning lamp or the like or suppressing a steering operation can be performed to prevent the own vehicle from changing lanes to the lane side of the other vehicle. When such advanced functions are realized, it is possible to reliably prevent contact with a vehicle running diagonally behind the host vehicle when changing lanes. When an alarm or warning is issued to the driver as described above, not only the current position of the detection target of another vehicle or the like, but also past position information of the other vehicle or the past and current positions of the other vehicle or the like. The degree of danger or the like may be determined from the information (that is, trajectory information that is a temporal change in the position information), and an alarm or the like may be output based on the determination result. That is, the apparatus of the present invention not only observes instantaneous position information of a detection target, but also observes a temporal change in the position of the detection target, and determines a degree of risk or the like based on the observation result. May be provided.

As described above, according to the apparatus of the present embodiment, only the image data obtained from a single camera is used, and the situation around the front or rear (including diagonally forward or diagonally rear) of the vehicle (particularly, Danger information or caution information based on the position information of the detected object or the position information of the detected object) can be accurately and almost in real time observed by simple processing. Therefore, advanced functions for improving vehicle safety, ease of driving operation, and the like can be realized inexpensively and effectively. In addition, the inventors have carried out a driving experiment on a highway by actually mounting the above-described imaging means and the like on a vehicle, and have obtained a sufficiently accurate ground plane coordinate system from position data on images of other vehicles and the like. We have confirmed that location information can be obtained almost in real time.

The present invention is not limited to the above embodiment, but may have various aspects and modifications. For example, the imaging unit may be configured to install both a unit that captures an image in front of the vehicle and a unit that captures an image in the rear, and observe both front and rear states of the vehicle. Further, even if the imaging directions are the same, the number of imaging means is not necessarily limited to one. For example, two cameras are installed rearward on both left and right sides of the vehicle, and similar processing is performed on the images of the respective cameras. Alternatively, it may be possible to observe an extremely wide area in the left-right direction. However, in consideration of cost, it is preferable that one imaging unit is installed at the center in the left-right direction of the vehicle for each of the rear and front. In particular, if the position, direction, etc. of the imaging means are set as in the above-described embodiment, 1
This is because a wide range of observations is practically possible with only the stand.

Further, in the above-described embodiment, the mode in which the position data on the image is converted into the position data in the ground plane coordinate system, and the degree of danger is determined using the converted position information is exemplified. However, it is not limited to this. For example, map data in which information such as a degree of danger, a matter to be noted by the driver, or a matter to be executed by the control means of the vehicle is set and stored in advance corresponding to an area defined on the image. Every
The risk level information or the like may be directly derived from the map data and the position data of the detected object on the image (data of a temporal change in position). For example, FIG.
As shown in (a), regarding the area where another vehicle 11 is traveling on an adjacent lane, as shown in the figure, a “safe zone”, a “caution zone”, and a “danger zone” are information corresponding to the area on the image. When the other vehicle behind is set in the “safety zone”, information notification or the like indicating that it is not dangerous even if the lane is changed is executed (or no information may be issued). When the vehicle is located in the "danger zone", a warning is issued, and when the vehicle is located in the "danger zone", a control or the like for suppressing the lane change is executed. In such a mode, the coordinate conversion processing of the position data is not required, so that the processing becomes simpler,
Responsiveness is further improved. Note that in this case, theoretically, when the road surface is curved and when it is a straight road surface,
Each of the map data is set and registered separately or, for example, only the map data in the case of a straight road is set and registered, and when the road is curved, the map data is set according to the curvature or the like. Need to be modified and used. However, on an expressway where the above-mentioned advanced functions (inter-vehicle distance monitoring function, etc.) are mainly used, since the curvature of the road surface is small, even if the road surface is curved, the same map data (linear The above-described embodiment may be implemented using map data for a road surface.
3A and FIG. 3B are plotted based on an actual rear image obtained as a result of a driving experiment on an expressway (the angle α of the imaging unit is 20 degrees).
3 (a) is a rear image while traveling on a curved road, and FIG. 3 (b) is a rear image while traveling on a curved road surface. Only images near infinity are different. (For example, in the range of about 3 m to 30 m behind the own vehicle), the positional relationship of the detected object on the image is almost the same as in the case of a straight road, and the same depending on the required determination accuracy. This is because there is a possibility that map data can be used.

Further, the size of the field of view (including the state of zooming, etc.) and the installation position (height, etc.) of the imaging means, or the direction of its axis (center line of the field of view) are set (for example, the angle α described above).
) May be fixed to a fixed value, but may be switched to a plurality of types (for example, for short-distance measurement and long-distance measurement) (a configuration for switching by human power, or a configuration for automatically switching by providing a driving unit) ). However, in this case, theoretically, the above-mentioned map data (map data in the case of directly deriving risk information or the like from a position or a change in position on an image) is separately stored corresponding to each setting. It is necessary to switch between these map data.

[0020]

According to the apparatus for observing the surroundings of a vehicle according to the present invention, only the image data obtained from one camera can be used, in front of or behind the vehicle (including obliquely forward or obliquely backward).
(Particularly, the position information of the detected object or the danger information and the caution information based on the position information of the detected object) can be accurately and almost observed in real time by simple processing. Therefore, advanced functions (inter-vehicle distance monitoring function and the like) for improving the safety of the vehicle and the ease of the driving operation can be realized inexpensively and effectively.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration of a vehicle surroundings observation device.

FIG. 2 is a flowchart showing processing contents of the apparatus.

FIG. 3 is a diagram illustrating an example of an image behind a vehicle.

FIG. 4 is a diagram illustrating the principle of map data.

FIG. 5 is a diagram illustrating an example of the contents of map data.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 vehicle 2 imaging means 3 control unit (storage means, processing means) 11 other vehicle (detected object)

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (reference) G01B 11/00 G01B 11/00 HF term (reference) 2F029 AA02 AB01 AC02 AC09 AC16 2F065 AA03 AA12 AA17 AA51 BB15 CC00 CC11 CC40 FF04 FF26 GG09 JJ03 JJ26 MM03 MM07 PP01 QQ00 QQ01 QQ24 QQ32 SS02 SS09 SS13 UU02 UU05

Claims (4)

[Claims] An image pickup means for picking up an image in front of and / or behind a vehicle, and for converting coordinates of a position of an object on the image into coordinates on the ground surface centered on the vehicle. Storage means for storing map data, extracting an object in the image, determining the position of the object on the image, further converting the position to coordinates on the ground surface by the map data. Processing means for generating position information of the object to be detected on the horizontal plane by conversion. 2. An image pickup means for picking up an image in front of and / or behind a vehicle, and map data for deriving driving or control information of the vehicle from a position of an object to be detected on the image. A storage unit for storing, and a processing unit for extracting an object in the image, determining a position of the object on the image, and further deriving the information from the position using the map data. Distance measuring device characterized by the above-mentioned. 3. The imaging means is installed such that its axis obliquely intersects a road surface in front of or behind the vehicle,
The distance measuring apparatus according to claim 1, wherein an angle between the axis and a direction parallel to the ground surface is set so as to be inversely proportional to a detection distance. 4. The field of view, the direction of the axis, and the installation position of the imaging means are set such that the position at infinity on the axis is located above the center position in the image to be captured. The distance measuring apparatus according to claim 3, wherein: