JP2009026003A - Operation support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an operation support system for making it easy to understand the correspondence of a scene which is actually viewed by a driver and a scene on an image. <P>SOLUTION: The display/non-display of a front vehicle in a panorama image is switched according to the input of a switching instruction by a display switching device 25 which inputs the switching instruction of the display/non-display of a front vehicle which travels in the same traveling direction as that of its own vehicle. Thus, it is possible to display a front vehicle. Therefore, it is possible for a driver to easily understand the correspondence of a scene which is actually viewed by the driver from the front glass of a vehicle and a scene on the panorama image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a driving support system that supports driving of a driver.

  As a driving support system that supports the driving of the driver, when the host vehicle runs while rubbing against other vehicles, the in-vehicle camera mounted on the host vehicle captures the traveling direction of the host vehicle and the host vehicle The vehicle mounted on the other vehicle (oncoming vehicle) traveling toward the vehicle acquires an image obtained by capturing the traveling direction of the other vehicle through inter-vehicle communication, and is mounted on the host vehicle. A system that informs the driver of whether or not the host vehicle can rub against the other vehicle by recognizing a common object from the image captured by the camera and the image acquired from the other vehicle and displaying their positional relationship. (See, for example, Patent Document 1).

In addition, in the host vehicle, an image of the traveling direction of the host vehicle captured by the on-board camera mounted on the host vehicle and another vehicle captured by the on-board camera mounted on the other vehicle traveling in the traveling direction of the host vehicle. There is also a system in which an image of the traveling direction of the vehicle is analyzed, a pedestrian is detected to determine a risk level, and traveling control is performed according to the determined risk level (see, for example, Patent Document 2).
JP 2005-327250 A JP 2006-151114 A

  By the way, when the host vehicle travels on a road with multiple lanes, the range that the driver must pay attention to, for example, a pedestrian crossing the road is wide, but the lane in which the host vehicle is traveling When another vehicle is traveling in the lane adjacent to the vehicle, there is a blind spot that is behind the other vehicle, and there is a problem that the driver cannot pay sufficient attention to the pedestrian. Such a problem cannot be solved by the techniques described in Patent Documents 1 and 2, and means for solving the above problem is desired.

  Therefore, the applicant of the present application acquires an image of the traveling direction of the host vehicle taken by the in-vehicle camera of the host vehicle when the host vehicle travels on a road with multiple lanes, and what is the lane in which the host vehicle is traveling. Acquire images of the traveling direction of the host vehicle taken by the in-vehicle camera of another vehicle traveling in different lanes, create a panoramic image by combining the acquired images of the traveling direction of the host vehicle, and create the panorama image Has applied for a driving support system for displaying the panoramic image on the display device (Japanese Patent Application No. 2007-135449).

  With this driving support system, the driver can appropriately pay attention to a pedestrian crossing the road. However, in this driving support system, the vehicle ahead cannot be displayed on the display device, so it is difficult for the driver to intuitively understand the correspondence between the scene actually seen from the windshield of the vehicle and the scene on the image. There is.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a driving support system that makes it easy to understand the correspondence between a scene that the driver actually sees and a scene on the image. is there.

  According to the first aspect of the present invention, the control unit acquires an image of the traveling direction of the host vehicle captured by the host vehicle side capturing unit and travels in a lane different from the lane in which the host vehicle is traveling. Progress of the host vehicle imaged by the other vehicle side imaging unit mounted on the other vehicle by causing the own vehicle side communication unit to perform inter-vehicle communication with the other vehicle side communication unit mounted on the other vehicle Direction images are acquired, and the acquired plurality of images of the traveling direction of the host vehicle are displayed on the display means.

This not only displays the image of the traveling direction of the host vehicle captured by the host vehicle-side photographing means mounted on the host vehicle, but also a lane (for example, an adjacent lane) different from the lane in which the host vehicle is traveling. By displaying the image of the traveling direction of the host vehicle captured by the other-vehicle-side photographing unit mounted on the other vehicle traveling on the vehicle, the traveling direction image of the host vehicle can be displayed at a wide angle. As a result, when the host vehicle travels on a multi-lane road, the driver can appropriately pay attention to a pedestrian crossing the road, for example, and can appropriately support the driver's driving. . In particular, when there is a traffic jam, the effect is great because the pedestrian is expected to jump out from behind the other vehicle.

  Further, the control means displays / hides the preceding vehicle in a plurality of images in response to an input of a switching instruction by the display switching means for inputting a display / non-display switching instruction for the preceding vehicle traveling in the same traveling direction as the host vehicle. Switch the display. As a result, the vehicle in front can be displayed, and the driver can easily understand the correspondence between the scene actually seen from the windshield of the vehicle and the scene on the image.

  According to the second aspect of the present invention, the control means communicates with the own vehicle side communication means between the forward vehicles traveling in the same lane as the lane in which the own vehicle is traveling. When the vehicle state data of the preceding vehicle is acquired and the preceding vehicle is displayed in a plurality of images, information based on the acquired vehicle state data is displayed on a plurality of screens. As a result, the driver can grasp the vehicle state of the preceding vehicle.

  According to the invention described in claim 3, the control means acquires the current position of the host vehicle acquired by the host vehicle side current position specifying means, and between the other vehicle side communication means mounted on the other vehicle. To obtain the current position of the other vehicle specified by the other vehicle side current position specifying means mounted on the other vehicle by causing the own vehicle side communication means to perform inter-vehicle communication, and the acquired current position of the own vehicle and the other The relative position between the host vehicle and the other vehicle is recognized from the relationship with the current position of the vehicle, and a plurality of images in the traveling direction of the host vehicle are reduced or enlarged according to the recognition result and displayed on the display means. As a result, the image of the traveling direction of the own vehicle captured by the own vehicle side photographing means and the image of the traveling direction of the own vehicle photographed by the other vehicle side photographing means are reduced or enlarged so as to harmonize perspective, for example. By doing so, it is possible to display the plurality of images without a sense of incongruity with an appropriate perspective.

  According to the invention described in claim 4, the control means creates a panoramic image by continuously synthesizing a plurality of images in the traveling direction of the host vehicle in the horizontal direction, and displays the created panoramic image on the display means. Let As a result, the driver can appropriately pay attention to the pedestrian, for example, by displaying a panoramic image that is very close to the actual situation.

  According to the fifth aspect of the present invention, the image analysis means analyzes the image of the traveling direction of the own vehicle acquired by the control means, and the control means causes the image analysis means to analyze the image of the traveling direction of the own vehicle. Then, it is determined whether or not there is a moving body in the image, and if it is determined that the moving body exists, moving body information representing the moving body is displayed on the display means. As a result, not only the captured image is displayed but also the moving body is highlighted and displayed as moving body information so that the driver can grasp the presence of the moving body. Can be surely grasped.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows the overall configuration of the driving support system. The driving support system 1 includes a driving support control device 2, an in-vehicle camera 3 (own vehicle side photographing means in the present invention), an in-vehicle navigation device 4 (own vehicle side current position specifying means in the present invention), a display device 5 (present The display means and the notification means referred to in the invention) and the travel control device 6 (the travel control means referred to in the present invention) are provided.

  The driving support control apparatus 2 includes a control unit 7 (control unit in the present invention) that controls the entire system, a short-range wireless communication unit 8 (in-vehicle side communication unit in the present invention) that performs short-range wireless communication, an in-vehicle camera. An image input unit 9 for inputting an image captured by the vehicle 3, a current position input unit 10 for inputting a current position specified by the in-vehicle navigation device 4, a display control unit 11 for controlling the display operation of the display device 5, and a travel control device 6. A travel control unit 12 that controls a travel control operation, an image storage unit 13 that stores images, an image analysis unit 14 that analyzes images (image analysis means in the present invention), and a millimeter wave radar 15 are configured. .

  The in-vehicle camera 3 is constituted by a camera having, for example, a fisheye lens that realizes a photographing viewing angle within a predetermined range. In addition, not only a single camera but the structure which implement | achieves the imaging | photography viewing angle of a predetermined range with a some camera may be sufficient. The short-range wireless communication unit 8 forms a short-range wireless communication area within a predetermined range, and when another vehicle 21 traveling around the host vehicle enters the short-range wireless communication area, the short-range wireless communication unit 8 is mounted on the other vehicle 21. A short-range wireless communication line is connected to the wireless communication unit 22 (another vehicle-side communication means in the present invention). The short-range wireless communication unit 8 preferably uses light or electromagnetic waves with high directivity in order to avoid interference.

  When the short-range wireless communication unit 8 connects the short-range wireless communication line to the short-range wireless communication unit 22 mounted on the other vehicle 21, the control unit 7 sends a transmission request signal from the short-range wireless communication unit 8. An image that is transmitted to the other vehicle 21 and photographed from the other vehicle 21 by the in-vehicle camera 23 (another vehicle-side imaging means in the present invention) mounted on the other vehicle 21 and the in-vehicle navigation mounted on the other vehicle 21 The short distance wireless communication unit 8 receives the current position specified by the device 24 (the other vehicle side current position specifying means in the present invention).

  The display device 5 is installed in, for example, the center console, and displays a display screen when a display control signal is input from the control unit 7 via the display control unit 12. The display switching device 25 inputs a display / non-display switching instruction for a forward vehicle traveling in the same traveling direction as the host vehicle, and a switching instruction corresponding to the input is output to the display device 5.

  The travel control device 6 includes an engine control device 16 that controls the engine, a brake control device 17 that controls the brake, a power steering control device 18 that controls the power steering, an automatic transmission control device 19 that controls the automatic transmission, and the like. When a travel control signal is input from the control unit 7 via the travel control unit 12, one of the devices is controlled to control the travel of the host vehicle.

  In addition, the driving support control device includes a storage device (not shown), and the storage device stores drawing image data (hereinafter referred to as 3D data) of the preceding vehicle to be displayed on a panoramic image to be described later. This 3D data is provided for each vehicle type assigned by each automobile manufacturer.

  The basic operation of the driving support system configured as described above is described in detail in Japanese Patent Application No. 2007-135449 (paragraph numbers 0021 to 0030 and FIGS. 2 to 8) previously filed by the present applicant. Here, the outline is explained here. In the driving support control device 2, the control unit 7 determines whether another vehicle is traveling around the own vehicle based on the detection result by the millimeter wave radar 15, and determines that the other vehicle is traveling. Then, it is determined whether or not these other vehicles are within the short-range wireless communication area of the short-range wireless communication unit 8. Next, when it is determined that the other vehicle is within the short-range wireless communication area of the short-range wireless communication unit 8, the transmission request signal is transmitted from the short-range wireless communication unit 8 to the other vehicle until the predetermined time elapses. It is determined whether an image taken by an in-vehicle camera mounted on a vehicle and a current position specified by an in-vehicle navigation device mounted on the other vehicle are received.

  When the control unit 7 determines that the image and the current position have been received from another vehicle before the predetermined time has elapsed, the control unit 7 stores the received image in the image storage unit 13. Next, the control unit 7 calculates the relationship between the current position of the own vehicle specified by the in-vehicle navigation device 4 of the own vehicle and the current position of the other vehicle received from the other vehicle, and calculates the relative position between the own vehicle and the other vehicle. The panorama image is created by combining the image captured by the in-vehicle camera 3 of the vehicle and the image received from the other vehicle. In this case, the control part 7 gives continuity in the location where an image overlaps by reducing or enlarging an applicable image on the basis of the distance from the own vehicle.

  Next, the control unit 7 performs image analysis on the panoramic image created in this way by the image analysis unit 14, determines whether or not a moving body (for example, a pedestrian) exists in the panoramic image, and the moving body exists. When it is determined to do so, the moving direction and moving speed of the moving body are specified. Next, the control unit 7 causes the display device 5 to display a panoramic image. In this case, if a moving body exists in the panoramic image, the control unit 7 displays moving body information (a pedestrian in the present embodiment) representing the moving body.

  In this way, the control unit 7 can display not only the image taken by the in-vehicle camera 3 of the own vehicle but also the image taken by the in-vehicle camera of the other vehicle. Pedestrians that cannot be photographed can be displayed. The display device 5 may be a windshield display that is superimposed on the actual scenery, and the control unit 7 may display an image in a range that does not hinder the driving of the driver.

  Next, characteristic operations of the driving support system will be described with reference to FIGS. In the driving support control device 2, when the control unit 7 displays a panoramic image as shown in FIG. 2 on the display device 5, the control unit 7 moves in the same traveling direction as that of the host vehicle in response to an input of a switching instruction from the display switching device 25. It is possible to switch between display and non-display of the forward vehicles VC1, VC2, and VC3 that travel toward the vehicle. This display / non-display switching is performed by inputting the display switching device 25.

  As shown in FIG. 2, the driver switches the display switching device 25 corresponding to the circled numbers 1, 2, and 3 assigned to each of the forward vehicles VC1, VC2, and VC3 displayed on the panoramic image. Press the input button of number 1, 2, 3, ... 9 above once, and then press the input button that displays "OK" to display the front vehicle corresponding to the pressed number Can be made. Each number on the display switching device 25 is displayed shining if the preceding vehicle corresponding to that number is displayed on the panoramic image. In addition, by pressing the input button of the number displayed shining once and then pressing the input button displayed as “decision”, the display of the vehicle ahead corresponding to the pressed number is hidden. .

  In this way, by inputting a display / non-display switching instruction for the preceding vehicle from the display switching device 25, it is possible to switch the display / non-display of the preceding vehicle in the panoramic image. As a result, the vehicle in front can be displayed, and the driver can easily understand the correspondence between the scene actually seen from the windshield of the vehicle and the scene on the image.

  In addition, as shown in FIG. 2, you may display the outline about the non-displaying front vehicle VC2 with a dotted line. Further, when a touch panel is employed for the display device 5, the forward vehicle VC2 that is not displayed may be displayed by directly touching the forward vehicle displayed with the dotted line.

  FIG. 3 shows a display switching process for switching display / non-display of the preceding vehicle. First, in step S1 of FIG. 3, in the driving assistance control device 2, the control unit 7 searches for a forward vehicle traveling in the same traveling direction as the own vehicle based on the detection result by the millimeter wave radar 15, and the step In S <b> 2, it is determined from the search result whether there is a forward vehicle existing in the short-range wireless communication area of the short-range wireless communication unit 8. If an affirmative determination is made in step S2, the process proceeds to step S3. If a negative determination is made, the process proceeds to step S1, and step S1 and subsequent steps are repeated.

  In step S <b> 3, the vehicle ahead requesting communication is requested to transmit vehicle state data (corresponding to vehicle information) of the vehicle ahead and image information taken by an onboard camera mounted on the vehicle ahead. In step S4, it is determined whether vehicle state data and image information or only image information has been received as information from the preceding vehicle. If an affirmative determination is made in step S4, the process proceeds to step S5. If a negative determination is made, the process proceeds to step S3, and step S3 is repeated.

  Here, the vehicle state data of the preceding vehicle will be described. The vehicle status data includes the vehicle type (vehicle type and body color assigned by each automobile manufacturer), travel position (current position), vehicle speed, travel direction, presence / absence of blinker operation, and direction of the blinker operation when there is an operation. , The presence / absence of a brake operation, the lighting state of the interior lighting, and various data of the traveling lane position. These data are mainly obtained from an in-vehicle navigation device mounted on the preceding vehicle.

  In step S <b> 5 of FIG. 3, image information is stored in the image storage unit 13 among the received information from the preceding vehicle. When vehicle state data is also received, it is temporarily stored in a RAM (not shown) in the control unit 7. In step S6, it is determined whether there is vehicle state data for the preceding vehicle. If an affirmative determination is made in step S6, the process proceeds to step S7. If a negative determination is made, the process proceeds to step S8.

  In step S7, the relationship between the current position of the host vehicle identified by the in-vehicle navigation device 4 of the host vehicle and the current position of the preceding vehicle received from the preceding vehicle is calculated to calculate the relative position between the host vehicle and the other vehicle. In step S8, a panoramic image is created and displayed, or the display is updated. In step S <b> 9, it is determined whether or not there is an input from the display switching device 25 to display the vehicle ahead. Here, if an affirmative determination is made, the forward vehicle display process of step S10 is executed. If a negative determination is made, the process proceeds to step S1, and the above-described process is repeated.

  In step S11 of FIG. 4, 3D data corresponding to the vehicle type is extracted with reference to the vehicle type included in the vehicle state data received from the preceding vehicle. In step S12, the presence / absence of the blinker operation (the direction of the blinker operation when there is an operation), the presence / absence of the brake operation, and the lighting state of the interior lighting are reflected in the 3D data in the vehicle state data.

  In step S13, 3D data is superimposed and displayed on the panoramic image in consideration of the traveling state of the host vehicle and the preceding vehicle. Specifically, as shown in FIG. 6, the difference in travel position / direction between the preceding vehicle and the host vehicle is reflected in the display position, size, and orientation of the 3D data. The coordinates (x1, y1, z1) in FIG. 6 are the coordinates of the relative position on the panorama image when the coordinates (x0, y0, z0) of the host vehicle are used as the origin, and are reflected based on the coordinates of the relative position. Let

  In step S14, the inter-vehicle distance between the preceding vehicle and the host vehicle and the vehicle speed of the preceding vehicle are displayed superimposed on the 3D data. Thereby, as shown in FIG. 5, the panoramic image in which the preceding vehicles VC1, VC2, and VC3 are displayed is displayed. The DP in FIG. 5 displays the vehicle speed XX [km / h] and the inter-vehicle distance YY [m] of the preceding vehicle VC2. The display of the vehicle speed and the inter-vehicle distance may be displayed only when the driver designates the display. Further, in FIG. 5, the front vehicle VC2 shows a state in which the illumination in the passenger compartment is turned on by brightening the color of the rear window RW. As a result, the driver can grasp the vehicle state of the preceding vehicle.

  As described above, according to the present embodiment, the driving support system 1 acquires an image of the traveling direction of the host vehicle captured by the in-vehicle camera 3 of the host vehicle, and the lane in which the host vehicle is traveling. Acquiring an image of the traveling direction of the host vehicle captured by the in-vehicle camera 23 of the other vehicle 21 traveling in a different lane, and combining the acquired plurality of images of the traveling direction of the host vehicle to create a panoramic image; Since the created panoramic image is configured to be displayed, an image of the traveling direction of the host vehicle can be displayed at a wide angle. As a result, when the host vehicle travels on a multi-lane road, the driver walks. The driver can be appropriately paid attention and can appropriately support the driving of the driver.

  Furthermore, the display / non-display of the preceding vehicle in the panoramic image is switched in response to the input of the switching instruction by the display switching device 25 that inputs the display / non-display switching instruction of the preceding vehicle traveling in the same traveling direction as the host vehicle. I made it. As a result, the vehicle ahead can be displayed, and the driver can easily understand the correspondence between the scene actually seen from the windshield of the vehicle and the scene on the panoramic image.

  In addition, since the current position of the own vehicle and the current position of the other vehicle are recognized, and a plurality of images of the traveling direction of the own vehicle are reduced or enlarged according to the recognition result, the vehicle is mounted on the vehicle. By displaying the image of the traveling direction of the own vehicle captured by the camera 3 and the image of the traveling direction of the own vehicle captured by the in-vehicle camera of the other vehicle, for example, by reducing or enlarging the images so as to harmonize perspective. Can be displayed without a sense of incongruity with an appropriate perspective.

  Also, when the image of the traveling direction of the host vehicle is image-analyzed and it is determined that there is a moving body in the image, the moving body information representing the moving body is displayed. For example, the driver can be made sure of the presence of a pedestrian.

It is a functional block diagram which shows one Embodiment of this invention. It is the figure which showed the front vehicles VC1, VC2, and VC3 displayed on the display switching device 25 and a panoramic image. It is a flowchart for demonstrating the display switching process which switches the display / non-display of a front vehicle. It is a flowchart for demonstrating a front vehicle display process. It is the figure which showed the panoramic image on which forward vehicles VC1, VC2, and VC3 were displayed. It is a figure for demonstrating that the difference of the driving | running | working position and direction with the own vehicle is reflected in the display position, magnitude | size, and direction of a front vehicle.

Explanation of symbols

  In the drawing, 1 is a driving support system, 3 is an in-vehicle camera (own vehicle side photographing means), 4 is an in-vehicle navigation device (own vehicle side current position specifying means), 5 is a display device (display means, notifying means), and 6 is A travel control device (travel control means), 7 is a control section (control means), 8 is a short-range wireless communication section (own vehicle side communication means), 14 is an image analysis section (image analysis means), and 22 is short-range wireless communication. Unit (other vehicle side communication means), 23 is an in-vehicle camera (other vehicle side photographing means), 24 is an in-vehicle navigation device (other vehicle side current position specifying means), and 25 is a display switching device (display switching means).

Claims (5)

Own vehicle side photographing means for photographing the traveling direction of the own vehicle;
Own vehicle side communication means for performing inter-vehicle communication with other vehicle side communication means mounted on another vehicle;
Other vehicle side communication means mounted on another vehicle traveling in a lane different from the lane in which the host vehicle is traveling, while acquiring an image of the traveling direction of the host vehicle captured by the host vehicle side photographing means The vehicle-side communication means performs vehicle-to-vehicle communication between the vehicle and the vehicle-side shooting means mounted on the other vehicle to acquire an image of the traveling direction of the vehicle, and the acquired vehicle Control means for displaying a plurality of images in the traveling direction on the display means;
Display switching means for inputting a display / non-display switching instruction of a preceding vehicle traveling in the same traveling direction as the host vehicle,
The control means switches between display and non-display of the preceding vehicle in the plurality of images in response to an input of a switching instruction by the display switching means. In the driving support system according to claim 1,
The control means causes the own vehicle side communication means to perform inter-vehicle communication with a preceding vehicle traveling in the same lane as the traveling lane of the preceding vehicle. When the vehicle state data of the vehicle is acquired and the preceding vehicle is displayed in the plurality of images, information based on the acquired vehicle state data is displayed on the plurality of screens in an overlapping manner. . In the driving support system according to claim 1 or 2,
A vehicle-side current position specifying means for specifying the current position of the host vehicle;
The control means acquires the current position of the own vehicle acquired by the own vehicle side current position specifying means, and communicates with the own vehicle side communication means between the other vehicle side communication means mounted on the other vehicle. Obtain the current position of the other vehicle specified by the other vehicle side current position specifying means mounted on the other vehicle through communication, and from the relationship between the acquired current position of the own vehicle and the current position of the other vehicle A driving support system characterized by recognizing a relative position between the host vehicle and another vehicle and displaying on the display means a plurality of images of the traveling direction of the host vehicle according to the recognition result. In the driving support system according to any one of claims 1 to 3,
The control means creates a panoramic image by continuously synthesizing a plurality of images in the traveling direction of the host vehicle in the horizontal direction, and displays the created panoramic image on the display means. . In the driving support system according to any one of claims 1 to 4,
Image analysis means for analyzing the image of the traveling direction of the host vehicle acquired by the control means,
The control means causes the image analysis means to analyze an image of the traveling direction of the host vehicle to determine whether or not a moving body exists in the image, and when determining that the moving body exists, A driving support system, wherein moving body information representing the moving body is displayed on the display means.

Images