JP2005216255A - Driving support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow drivers to immediately grasp the sense of distance between a vehicle and objects around it, to quickly reflect it to the driving operations. <P>SOLUTION: The driving support system, which is a system to support drivers driving, has a photographing means 1 for photographing the surroundings of a vehicle, an image processing means 5 for correcting the distortions and converting the viewpoint for images photographed by the photographing means 1, and a distance measuring means 2 for measuring the distances between a vehicle and objects around it. In addition, the system has a course-predicting means 6 for obtaining predicted courses for a vehicle, based on the driving condition information from each sensor 3 that detects driving situations of the vehicle, and an information superimposing means 7 for superimposing, at least either the distance information acquired by the distance measurement means 2, or the predicted course information, acquired by the course prediction means 6 on an image processed by the image processing means 5. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a driving support system that supports driving of a driver.

  In recent years, research on ITS (Intelligent Transport Systems) has been actively conducted in order to solve various problems and improve the efficiency of road traffic. In this context, researches on automatic driving and driving assistance related to AVCSS (Advanced Vehicle Control and Safety Systems), which are expected to be essential solutions for traffic accidents and traffic jams, are also being actively conducted.

  By the way, in driving a car, there are blind spots that the driver cannot see directly from the driver's seat and parts that are difficult to see, and there is a limit to indirectly supplementing with a mirror. In addition, even in the surrounding traffic situation, it is difficult for the driver to obtain visual information for determining driving at an intersection with poor visibility, and this also causes an accident.

Thus, several systems have been developed that support drivers by using a camera to display images around the vehicle on a monitor inside the vehicle. For example, there are a back monitor that supports a rear view when parking or the like, a side view monitor that supports a side view such as when turning left or parallel, and a blind corner monitor that supports a view at an intersection with poor visibility. In addition, there is a study of an All Around View monitor system that provides a virtual plan view image from above the vehicle by installing eight cameras on the vehicle (see Non-Patent Document 1).
Sugano and Takagi: Development of the peripheral monitoring system, JSPS Spring Conference, 20003144, No.9-03, pp.13-16 (2003)

  In addition, the back monitor and the side view monitor described above not only provide images around the vehicle, but also provide a system that provides information such as a predicted course and a vehicle width by combining them.

  However, in the conventional system that supports the driver, even if the vehicle and the objects around the vehicle are captured in the image, the sense of distance is difficult to understand, and the viewpoint position is not accurate, narrow road, intersection with poor visibility, It could not be immediately reflected in driving operations when parking, etc., and was not always sufficient in terms of driving assistance.

  The present invention has been proposed in view of the above, and it is possible to immediately grasp the distance between the vehicle and the objects around the vehicle, and the position of the viewpoint is also accurate. It aims at providing the driving assistance system which can be reflected in driving operation.

  In order to achieve the above object, an invention according to claim 1 is a driving support system for supporting driving of a driver, wherein imaging means for imaging a vehicle periphery, image distortion correction and viewpoint conversion from the imaging means are performed. Vehicle prediction based on driving status information and vehicle current position information from image processing means to be performed, distance measuring means for measuring the distance between the vehicle and objects around the vehicle, and various sensors for detecting the driving status of the vehicle A route predicting unit for obtaining a route, and an information superimposing unit for superimposing at least one of the distance information from the distance measuring unit and the predicted route information from the route predicting unit on an image from the image processing unit. Yes.

  According to a second aspect of the present invention, in addition to the configuration of the first aspect of the present invention, a viewpoint selecting unit that selects a viewpoint position of viewpoint conversion performed by the image processing unit according to a driver's operation is provided. It is characterized by having.

  In addition to the configuration of the invention described in claim 1 or 2, the invention described in claim 3 is based on the predicted course information and distance information between the vehicle and the vehicle surrounding object. It is characterized in that it comprises a warning means for determining whether or not a dangerous situation occurs and issuing a warning if a dangerous situation occurs.

  According to a fourth aspect of the present invention, in addition to the configuration of the first aspect of the present invention, the various sensors are at least a steering angle sensor and a speed sensor. Yes.

  In the present invention, after distortion correction and viewpoint conversion of the image around the vehicle, the distance information between the vehicle and the vehicle surrounding object and the predicted course information are superimposed on the image. The distance can be immediately grasped and the position of the viewpoint can be accurately determined, which can be immediately reflected in the driving operation at a narrow road, an intersection with poor visibility, or at the time of parking. Therefore, more accurate support can be provided to the driver.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of the driving support system of the present invention. In the figure, a driving support system of the present invention is a system that supports driving of a driver, and includes an image pickup means 1 for picking up an image of the periphery of the vehicle, and an image processing means 5 for correcting distortion of the image from the image pickup means 1 and for viewpoint conversion. And a route measuring means 2 for measuring the distance between the vehicle and the vehicle peripheral object, and a route predicting means 6 for obtaining a predicted course of the vehicle based on driving situation information from various sensors 3 for detecting the driving situation of the vehicle. And information superimposing means 7 for superimposing at least one of the distance information from the distance measuring means 2 and the predicted course information from the course predicting means 6 on the image from the image processing means 5. Then, the information superimposing means 7 projects the image on the monitor 101 in the vehicle and displays it on the driver.

  Further, the viewpoint selecting unit 4 is connected to the image processing unit 5 so that the viewpoint position of the viewpoint conversion performed by the image processing unit 5 can be selected according to the operation of the driver.

  Further, a warning means 8 is provided, and it is determined whether or not a dangerous situation occurs for the driver based on the predicted course information obtained by the information superimposing means 7 and the distance information between the vehicle and the vehicle surrounding object. If a dangerous situation occurs, a warning is issued through the speaker 102 or the operation instruction display unit 102. Next, a more specific configuration example will be described with reference to FIGS.

  FIG. 2 is a diagram showing a specific configuration example of the driving support system of the present invention, and FIG. 3 is a diagram showing a vehicle mounting position of the measuring device. The image processing means 5, the course prediction means 6, the information superimposing means 7 and the warning means 8 are mainly configured by a CPU (ECU) that operates in accordance with the software according to the present invention. In the configuration example of FIG. It is shown as an ECU 100 for use.

  In the configuration example of FIG. 2, a CCD camera 11 is used as the imaging unit 1.

  As the distance measuring means 2, a laser range finder (LRF) 21 is used, and an ultrasonic sensor 22 is used as necessary. In addition to this, the distance measuring means 2 includes a laser radar or the like, but each has its own characteristics, may be adopted in consideration of characteristics and cost, and may be used depending on the vehicle speed.

  Here, as the various sensors 3, a steering angle sensor 31 and a speed sensor 32 are used, and a yaw sensor 33 that detects a speed at which the vehicle rotates is used as necessary. Moreover, you may make it use GPS34 which detects a vehicle position, and map information (map data) together. This map information is stored in the hard disk 35.

  The CCD camera 11, the LRF 21, the ultrasonic sensor 22, the steering angle sensor 31, the speed sensor 32, the yaw sensor 33, the GPS 34, and the hard disk 35 are connected to the visual navigation ECU 100 and detected data via an interface (not shown). Or the stored data is output to the ECU 100 for visual navigation. The visual navigation ECU 100 performs various arithmetic processes based on the data and based on the viewpoint position selected by the driver using the image selection switch 41 connected as the viewpoint selecting means 4, and monitors the processing results. 101, the speaker 102, and the operation instruction display unit 103. The driver can use the image selection switch 41 to arbitrarily select a viewpoint from the driver's eye position, a viewpoint from the rear center of the vehicle, a viewpoint from which the vehicle is viewed from above, a viewpoint from which the vehicle is viewed from the side, and the like. Can be selected as the viewpoint.

  The operation instruction display unit 103 is a display unit for providing an optimum operation to the driver, and includes four LEDs 103a, 103b, 103c, and 103d. When the visual navigation ECU 100 determines that it is necessary to steer in the right direction with respect to the predicted course of the vehicle, the LED 103a is lit, and when the required steering angle is small, the inside of the LED 103a is lit. As the size increases, the LED 103a lights up to the outside. Further, when it is determined that it is necessary to steer leftward, the LED 103b is lit. When it is determined that the speed needs to be increased, the LED 103c is lit. When it is determined that the speed needs to be decreased, the LED 103d is lit. It is supposed to be.

  The CCD camera 11 uses three 150,000 color CCD cameras with 250,000 pixels using a special lens with a horizontal angle of view of 170 degrees. As shown in FIG. And it is attached to a total of three places of two side parts. The front center CCD camera 11 is installed at a height of about 90 cm and an elevation angle of about 45 degrees, and the two CCD cameras 11 and 11 at the side are about 1.1 m in height and are almost directly below the side mirror. Was installed.

  As the LRF 21, an LCK 221 made by SICK (scan angle: 180 degrees, resolution: 0.5 degrees, distance measurement range: 30 m) is used. As shown in FIG. Was placed at a position of about 50 cm.

  Further, on the road on which the vehicle 110 travels, as shown in FIG. 3, three color cones B1, B2, and B3 arranged at intervals of 2 m on one side are arranged on both sides to be an obstacle.

  In the driving support system configured as described above, the visual navigation ECU 100 performs various arithmetic processing and image processing according to the present invention and outputs the results to the monitor 101. The processing contents will be described in order.

  First, the front of the vehicle as seen from the driver's seat is as shown in Fig. 4. As can be seen from this photograph, the color cones B1 and B1 ahead of the driver can only see the head, and the installation surface is not visible. , It is a blind spot. Also, the color cones B2, B2, B3, and B3 in front of the driver cannot be seen by the driver.

  FIG. 5 shows direct images captured by the three CCD cameras 11. Among the three photographs, the upper is an image of the front center, the lower left is the left side, and the lower right is an image by the CCD camera 11 on the right side. Since the photograph at this stage is an ultra-wide-angle lens, it can be seen that the image is distorted. Even in the image as it is, since the own vehicle is reflected in the image, the positional relationship with respect to the object can be grasped to some extent, but it is difficult for the driver to respond instantaneously by looking at this image. .

  FIG. 6 shows an image in which distortion caused by the lens is corrected. Among the three photographs, the upper is an image of the front center, the lower left is the left side, and the lower right is an image by the CCD camera 11 on the right side. In this image, a sense of distance can be seen in the front image, and in the left and right images, the directionality with the object can be grasped.

  FIG. 7 shows an image in which the viewpoint is further moved over the vehicle. Among the three photographs, the left is the front center, the center is the left side, and the right is the image by the CCD camera 11 on the right side. The viewpoint selection above the vehicle is performed by the driver using the image selection switch 41. Although the area of the corrected image is narrow and the surroundings are not shown, the image has a good sense of distance and directionality.

  In FIG. 8, the distance graph of the result measured by LRF21 is shown. The positions of the two color cones B1 and B1 in front of the vehicle are measured as P1 and P1, and the positions of the two color cones B2 and B2 placed on the front side of the vehicle are measured as P2 and P2, respectively. Understand.

  Further, in FIG. 9, the distance measurement data obtained by performing the coordinate conversion of FIG. 8 and the value of the predicted course (the white line portion in the figure) are superimposed on the distortion corrected image of FIG. Similarly, in FIG. 10, the distance measurement data and the predicted course value of FIG. 8 are superimposed on the image viewed from above in FIG. In both cases, the positional relationship of the object can be grasped instantaneously. The viewpoint position can be selected according to the driver's preference.

  As can be seen from the screens of FIGS. 6 to 10 described above, it is possible to instantly grasp information on the blind spot, a part that is difficult to visually recognize, and a part that becomes a driving obstacle.

  The visual navigation ECU 100 displays the recommended steering and speed based on the predicted course, the current steering angle, and the speed, and if necessary, the map information of the yaw sensor 33, the GPS 34, and the hard disk 35. 103, the speaker 102, the monitor 101, etc., are presented to the driver to avoid contact between the vehicle and the obstacle.

  Moreover, the visual navigation ECU 100 can predict the amount of movement of the vehicle, and can call attention even when there is a risk of involving a vehicle peripheral object due to an inner wheel difference, an outer wheel difference, or the like after the vehicle has moved. .

  FIG. 11 is a diagram showing a flow of processing in the configuration example of FIG. As shown in the figure, the visual navigation ECU 100 performs distortion correction on camera images at a plurality of positions and performs viewpoint conversion (perspective conversion). On the other hand, on the basis of distance measurement data from the LRF, data on nearby obstacles is accumulated, and on the basis of measurement data on the steering angle, speed, and yaw rate, vehicle movement amount calculation and course estimation are performed. Then, according to the nearby obstacle data and the estimated route, the intersection is detected. Subsequently, based on the viewpoint conversion, nearby obstacle data, intersection data, etc., the images are superimposed and displayed on the monitor. In addition, based on the intersection data, a warning recommendation and warning that the vehicle intersects with an obstacle is performed via a speaker. Further, the recommended steering and speed in that case are calculated, and the driver is instructed via the operation instruction display unit and the monitor.

  As described above, in the present invention, after correcting the distortion of the image around the vehicle and converting the viewpoint, the distance information between the vehicle and the object around the vehicle and the predicted course information are superimposed on the image. The position becomes clear, and it becomes possible to immediately grasp the distance between the vehicle and the object around the vehicle, the blind spot of the driver, the part that is difficult to visually recognize, and the part that becomes a driving obstacle. Therefore, it can be immediately reflected in driving operations at narrow roads, intersections with poor visibility, parking, etc., and more accurate support can be provided to the driver.

  In that case, since the recommended steering and speed are calculated and the driver is instructed, the driver can be supported more accurately.

  In the above description, the number of CCD cameras, LRFs, etc., the installation location, etc. have been exemplified, but the number and installation location of these measuring devices and sensors are not limited and may be arbitrarily changed as necessary. Can do.

It is a block diagram which shows the structure of the driving assistance system of this invention. It is a figure which shows the specific structural example of the driving assistance system of this invention. It is a figure which shows the vehicle mounting position of a measuring apparatus. It is a figure which shows the mode of the vehicle front seen from the driver's seat. It is a figure which shows the direct image imaged with three CCD cameras. It is a figure which shows the image which correct | amended the distortion by a lens. It is a figure which shows the image which moved the viewpoint to the vehicle sky. It is a figure which shows the distance graph of the result measured by LRF. It is a figure which superimposes the ranging data which performed the coordinate transformation of FIG. 8, and the value of a prediction course on the distortion correction image of FIG. It is a figure which superimposes the ranging data of FIG. 8, and the value of a prediction course with respect to the image seen from the sky of FIG. It is a figure which shows the flow of the process in the structural example of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging means 2 Distance measuring means 3 Various sensors 4 Viewpoint selection means 5 Image processing means 6 Course prediction means 7 Information superimposing means 8 Warning means 11 CCD camera 21 Laser range finder (LRF)
22 Ultrasonic sensor 31 Steering angle sensor 32 Speed sensor 33 Yaw sensor 35 Hard disk 41 Image selection switch 100 Visual navigation ECU
101 Monitor 102 Speaker 103 Operation Instruction Display Unit 110 Vehicle B1, B2, B3 Color Cone

Claims (4)

In the driving support system that supports the driving of the driver,
Imaging means for imaging the periphery of the vehicle;
Image processing means for performing distortion correction and viewpoint conversion of the image from the imaging means;
Distance measuring means for measuring a distance between the vehicle and an object around the vehicle;
A course prediction means for obtaining a predicted course of the vehicle based on driving situation information from various sensors for detecting the driving situation of the vehicle;
Information superimposing means for superimposing at least one of the distance information from the distance measuring means and the predicted course information from the course predicting means on the image from the image processing means;
A driving support system comprising:   The driving support system according to claim 1, further comprising: a viewpoint selecting unit that selects a viewpoint position of viewpoint conversion performed by the image processing unit according to a driver's operation.   Based on the predicted course information and the distance information between the vehicle and surrounding objects, it is determined whether or not a dangerous situation occurs for the driver, and a warning is issued if a dangerous situation occurs. The driving support system according to claim 1, comprising means. The driving support system according to claim 1, wherein the various sensors are at least a steering angle sensor and a speed sensor.