JP2018149869A - Drive assisting device and drive assisting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology that enables a driver who looks at a display screen showing prediction route information to easily grasp a situation in the advancing direction of the vehicle.SOLUTION: A drive assisting device displays a predicted route of a vehicle such that the predicted route is superposed on a photographic image taken by a camera used for photographing the periphery of the vehicle. The drive assisting device comprises: a prediction face generating part that generates a pair of prediction faces, left and right, showing the predicted route stereoscopically; a detecting part that detects an object overlapping the prediction faces from the photographic image; a setting part that sets a transmissivity for the prediction faces; and a combined image generating part that generates a combined image in which the photographic image and the pair of prediction faces are combined on the basis of the transmissivity set by the setting part. The setting part alters the transmissivity according to the positional relation of the prediction faces and the object in a superposition area where the prediction faces and the object are superposed in the combined image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a driving support device and a driving support method.

  There is known a driving support device that displays an image captured by a vehicle-mounted camera on a vehicle-mounted monitor and supports vehicle driving. In addition to the image captured by the in-vehicle camera, the driving support device displays a guide line indicating the vehicle width line superimposed on the monitor screen so as to extend to the rear of the vehicle, and moves the direction of the guide line in accordance with the steering wheel operation. The driver can easily recognize the direction. Conventionally, the guide line indicating the width line of the vehicle, the guide line on the inner side where the vehicle bends, that is, the side that is likely to come into contact with another vehicle, is displayed as a curved surface that rises from the ground to the bumper position. It is trying to make it easier to determine whether the bumper that touches the obstacle is in contact with the obstacle. Such a technique is disclosed in Patent Document 1, for example.

JP 2004-147083 A

  If the driver cannot immediately grasp the vehicle's direction of travel by looking at the guide displayed on a three-dimensional curved surface that rises from the guide line on the ground to the bumper position, the driver misrecognizes the state of the vehicle's direction of travel. there's a possibility that. If the driver misrecognizes the situation in the traveling direction of the vehicle, the driving vehicle may collide with a pedestrian or another vehicle.

  In view of the above problems, an object of the present invention is to provide a technique that allows a driver who has seen a display screen on which predicted course information is displayed to easily grasp the situation in the traveling direction of a vehicle.

  The driving support device of the present invention is a driving support device that displays the predicted course of the vehicle on a screen by superimposing the predicted course of the vehicle on a captured image taken by a camera that captures the periphery of the vehicle, and includes a left and right that three-dimensionally show the predicted course. Set by a prediction plane generation unit that generates a pair of prediction planes, a detection unit that detects an object that overlaps the prediction plane from the captured image, a setting unit that sets the transmittance of the prediction plane, and the setting unit A composite image generation unit that generates a composite image in which the captured image and the pair of prediction planes are combined based on the transmitted transmittance, and the setting unit includes the prediction plane in the composite image. In the overlapping region where the object overlaps, the transmittance is changed according to the positional relationship between the prediction plane and the object (first configuration).

  In the driving support device having the first configuration, in the overlapping region, the object is positioned on the opposite side of the first region with respect to the first region located between the pair of prediction surfaces and the prediction surface. And the setting unit compares the transmittance of the portion of the prediction plane that overlaps the first region with the transmittance of the portion of the prediction plane that overlaps the second region. It is preferable that the configuration be increased (second configuration).

  The driving support device having the second configuration preferably has a configuration (third configuration) further including a notification processing unit that performs processing for notifying the detection of the first region.

  In the driving support device of the third configuration, the notification processing unit may be configured to change a notification mode according to a distance from the vehicle in the first region (fourth configuration).

  In the driving support apparatus having any one of the second to fourth configurations, the setting unit changes the transmittance of a portion of the prediction plane that overlaps the second region according to the type of the object ( (5th structure).

  In the driving support apparatus having any one of the first to fifth configurations, the setting unit is configured to set the transmittance of the predicted surface in a plurality of types based on the curvature of the predicted surface (sixth configuration). ).

  In the driving support apparatus having any one of the first to fifth configurations, the setting unit is configured to set the transmittance of the prediction plane in a plurality of types based on the distance from the vehicle (seventh configuration). ).

  In the driving support device having any one of the first to seventh configurations, the prediction plane generation unit generates the pair of prediction planes based on a plurality of the shot images shot at different times (eighth). It is preferable that

  The driving support method of the present invention is a driving support method for displaying the predicted course of the vehicle on a screen by superimposing the predicted course of the vehicle on a captured image taken by a camera that captures the periphery of the vehicle, and is a left-right display that shows the predicted course in three dimensions. A prediction surface generation step for generating a pair of prediction surfaces, a detection step for detecting an object overlapping the prediction surface from the captured image, a setting step for setting the transmittance of the prediction surface, and a setting step A composite image generation step of generating a composite image in which the captured image and the pair of prediction planes are combined based on the transmitted transmittance, and the setting step includes the prediction plane in the composite image In the overlapping region where the object overlaps, the transmittance is changed according to a positional relationship between the prediction plane and the object (a ninth configuration).

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the technique which the driver | operator who looked at the display screen on which prediction course information is displayed can grasp | ascertain the condition of the advancing direction of a vehicle easily.

The figure which shows the structure of the driving assistance system which concerns on embodiment of this invention. Schematic diagram showing the composite image generated by the composite image generator Schematic diagram for explaining the processing performed by the notification processing unit Flow chart showing an operation example of the driving support device Schematic diagram for explaining a first modification of the driving support device Schematic diagram for explaining a second modification of the driving support device Schematic diagram for explaining a third modification of the driving support device

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following description, the direction in which the vehicle travels straight and is directed from the driver's seat to the steering wheel is referred to as “front direction”. Further, a direction in which the vehicle travels straight and is directed from the steering wheel to the driver's seat is referred to as “rearward direction”. Also, the direction from the right side to the left side of the driver who faces the forward direction and is the direction perpendicular to the straight traveling direction of the vehicle and the vertical line is referred to as “left direction”. Further, the direction from the left side to the right side of the driver who faces the forward direction and is the direction perpendicular to the straight traveling direction of the vehicle and the vertical line is referred to as “right direction”.

<1. Driving support system>
FIG. 1 is a diagram showing a configuration of a driving support system SYS1 according to the embodiment of the present invention. The driving support system SYS1 includes a driving support device 1, a camera 2, a display device 3, and an audio output device 4. The driving assistance apparatus 1 displays the vehicle on the screen with the predicted course of the vehicle superimposed on the captured image captured by the camera 2 that captures the periphery of the vehicle. In the present embodiment, the driving support device 1 is a device that supports back parking of the vehicle. Details of the driving support device 1 will be described later.

  In the present embodiment, the camera 2 is configured by a back camera that is provided at the rear end of the vehicle and captures the rear of the vehicle. However, the camera 2 is not limited to the back camera. For example, when the driving assistance device 1 is not limited to the back parking assistance, the camera 2 may include a front camera that photographs the front of the vehicle. The camera 2 outputs the captured image to the driving support device 1. In the following, “photographed image” described without particular notice means an image photographed by the camera 2.

  The display device 3 includes a display composed of liquid crystal or the like having an operation function such as a touch panel. The display device 3 displays an image output from the driving support device 1. The display device 3 is provided at a position where a vehicle occupant (typically a driver) can visually recognize the display screen of the display. The display device 3 is installed on, for example, an instrument panel of a vehicle.

  The audio output device 4 includes a device for outputting audio such as a speaker and an amplifier. The audio output device 4 acquires audio information from the driving support device 1 and outputs audio into the vehicle interior. The audio output device 4 may be configured to be built in the display device 3 or may be configured to be provided separately from the display device 3.

<2. Driving assistance device>
The driving support device 1 is configured by a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory). The computer processes and transmits / receives information based on a program stored in the ROM. In the present embodiment, the driving support device 1 includes an acquisition unit 11, a prediction plane generation unit 12, a detection unit 13, a setting unit 14, a composite image generation unit 15, a notification processing unit 16, and an image output unit. 17 and an audio output unit 18. The functions of these units 11 to 17 included in the driving support device 1 are realized by the CPU performing arithmetic processing according to a program.

  The acquisition unit 11 acquires analog or digital photographed images from the camera 2 continuously in a predetermined cycle (for example, 1/30 second cycle). If the acquired captured image is analog, the acquisition unit 11 converts the analog captured image into a digital captured image (A / D conversion). The acquisition unit 11 outputs the acquired captured image or the acquired and converted captured image to the prediction plane generation unit 12. One captured image output from the acquisition unit 11 becomes one frame image.

  The prediction plane generation unit 12 generates a pair of left and right prediction planes that three-dimensionally indicate the predicted course of the vehicle. The detection unit 13 detects an object that overlaps the prediction plane from the captured image. The setting unit 14 sets the transmittance of the prediction plane. The composite image generation unit 15 generates a composite image in which the captured image and the pair of prediction planes are combined based on the transmittance set by the setting unit 14.

  FIG. 2 is a schematic diagram illustrating the composite image CP generated by the composite image generation unit 15. Details of the prediction plane generation unit 12, the detection unit 13, the setting unit 14, and the composite image generation unit 15 will be described with reference to FIG. A vehicle 5 indicated by a broken line in FIG. 2 is a vehicle on which the driving support device 1, the camera 2, the display device 3, and the audio output device 4 are mounted. In FIG. 2, the vehicle 5 is shown for convenience in order to facilitate understanding of the invention, and does not show a state reflected in the composite image CP. Moreover, the arrow shown with a broken line in FIG. 2 is shown for convenience, in order to make an understanding of invention easy, and shows the advancing direction of the vehicle 5 typically. The arrow also does not indicate the state shown in the composite image CP.

  In the present embodiment, the predicted course of the vehicle 5 is a predicted course when the vehicle 5 performs the back. Corresponding to the presence of a pair of left and right rear wheels of the vehicle 5, the prediction plane generation unit 12 generates a pair of left and right prediction planes 20a and 20b as shown in FIG. The pair of prediction surfaces 20a and 20b is generated so as to extend rearward of the vehicle.

  Specifically, the prediction plane generation unit 12 estimates the moving direction of the vehicle and the amount of movement per unit time from two captured images taken at different times while the vehicle 5 is moving. Specifically, the prediction plane generation unit 12 estimates the movement direction of the vehicle 5 and the movement amount per unit time from optical flows (motion vectors) of feature points located on the road surface in the two frame images. The feature points are, for example, white line edges drawn on the road surface, cracks on the road surface, stains on the road surface, gravel on the road surface, and the like.

  It should be noted that a feature point having a unique moving direction is preferably excluded from a feature point sample used in the estimation process. As a result, for example, feature points derived from moving objects can be excluded from feature point samples used in the estimation process, so that the direction of movement of the vehicle 5 using the optical flow and the estimation accuracy of the amount of movement per unit time can be improved. improves. The timing of exclusion may be, for example, after calculating the optical flow of each feature point and before performing the estimation process using the calculation result. For example, the optical flow of each feature point is calculated and the calculation result is calculated. It may be after performing an estimation process using. When the timing of exclusion is the latter, the excluded result is reflected in the estimation process after the next time, and the accuracy of the estimation process after the next time is improved. The determination of whether or not the moving direction is singular is, for example, obtained by averaging the moving directions of all feature points and determining that the moving direction is singular when the deviation from the average exceeds a predetermined allowable level. That's fine.

  The predicted surface generation unit 12 generates a predicted route of the vehicle 5 drawn on the road surface of the captured image based on the estimated movement direction of the vehicle 5. The predicted course line is a line indicating a predicted position through which the rear wheel of the vehicle 5 passes, for example. A pair of left and right predicted course lines are generated. And the prediction surface production | generation part 12 expands each left and right prediction course line to the perpendicular direction, and produces | generates a pair of prediction surface 20a, 20b which shows the prediction course of the vehicle 5 in three dimensions. The amount of expansion in the vertical direction is not particularly limited, but is preferably an amount that exceeds at least the bumper of the vehicle. Thereby, possibility that the collision with the vehicle which is easy to occur at the time of parking will occur can be reduced.

  In the present embodiment, as described above, the predicted surface generation unit 12 generates a pair of predicted surfaces 20a and 20b based on a plurality of captured images captured at different times. For this reason, in this Embodiment, a prediction surface can be produced | generated only by image processing, without acquiring vehicle speed information, steering angle information, etc. of a steering. However, this is an exemplification, and the prediction plane generation unit 12 acquires, for example, information acquired by the vehicle speed sensor and the steering angle sensor via a CAN bus or the like, and generates a pair of prediction planes based on these information. The structure to produce | generate may be sufficient.

  The pair of prediction planes 20a and 20b may overlap with the object 21 shown in the captured image. In FIG. 2, a vehicle is shown as an example of the object 21. The object 21 may be other than a vehicle, such as a person, a pole, or a building wall. In the present embodiment, the detection unit 13 detects the object 21 by pattern matching processing using image information regarding the object stored in advance in a storage unit (not shown). The detection part 13 should just detect the object 21 which overlaps with the prediction surfaces 20a and 20b in a synthesized image. For this reason, it is preferable that the detection unit 13 detects the object 21 limited to the region occupied by the pair of prediction planes 20a and 20b or the region and its periphery in the captured image. As a result, necessary information can be acquired efficiently.

  The setting unit 14 can set the transmittance of the prediction surfaces 20a and 20b in the range of 0% to 100%. Here, the “transmittance” is a ratio at which the colors and lines of the captured image that overlap the prediction planes 20a and 20b transmit the colors of the prediction planes 20a and 20b. When the transmittances of the prediction surfaces 20a and 20b are increased, the colors of the prediction surfaces 20a and 20b are displayed lightly, and the captured image passes through the prediction surfaces 20a and 20b. For example, when the transmittance of the prediction planes 20a and 20b is 50%, the colors of the prediction planes 20a and 20b are displayed lightly, and the captured image is displayed through the prediction planes 20a and 20b displayed lightly. That is, the prediction surfaces 20a and 20b are translucent. Also, assuming that the transmittance of the prediction planes 20a and 20b is 100%, the colors of the prediction planes 20a and 20b are not displayed, and only the captured images are displayed. Further, when the transmittance of the prediction planes 20a and 20b is 0%, the colors of the prediction planes 20a and 20b are displayed darkly, and a captured image overlapping the prediction planes 20a and 20b is not displayed.

  In the present embodiment, the setting unit 14 basically sets the transmittance of the prediction planes 20a and 20b to the standard transmittance. The standard transmittance is stored in a storage unit (not shown). The standard transmittance is set to 20%, for example, and the captured image overlapping the prediction surfaces 20a and 20b is displayed lightly through the prediction surfaces 20a and 20b. However, the setting unit 14 may change the transmittance of the specific areas of the prediction planes 20a and 20b from the standard transmittance under a predetermined condition.

  In the present embodiment, only one value is prepared as the standard transmittance of the prediction surfaces 20a and 20b. However, this is merely an example, and a plurality of values may be prepared as the standard transmittances of the prediction planes 20a and 20b, and a plurality of values may be used depending on the situation. For example, when the predicted course of the vehicle 5 is bent, the standard transmittance of the predicted surface inside the traveling direction of the vehicle 5 and the standard transmittance of the predicted surface outside the traveling direction may be set to different values. Good. In this case, taking into account that a dangerous situation is likely to occur on the inner side in the traveling direction, the predicted surface on the inner side in the traveling direction may be set to have a larger transmittance than the predicted surface on the outer side in the traveling direction. In the example shown in FIG. 2, the prediction plane 20a is a prediction plane on the inner side in the traveling direction, and the prediction plane 20b is a prediction plane on the outer side in the traveling direction. For example, the standard transmittance of the predicted surface 20a on the inner side in the traveling direction may be 40%, and the standard transmittance of the predicted surface 20b on the outer side in the traveling direction may be 20%.

  The setting unit 14 changes the transmittance of the prediction surfaces 20a and 20b according to the positional relationship between the prediction surfaces 20a and 20b and the object 21 in the overlapping region 22 where the prediction surfaces 20a and 20b and the object 21 overlap in the composite image CP. To do. According to this, for example, it is possible to display in an easy-to-understand manner to the driver that part or all of the object 21 is an obstacle and exists on the predicted course of the vehicle 5. In the example illustrated in FIG. 2, only the prediction surface 20 b on the outer side in the traveling direction among the two prediction surfaces 20 a and 20 b has the overlapping region 22 that overlaps the object 21. Then, in a part of the prediction surface 20b on the outer side in the traveling direction, there is a portion where the transmittance is set to be different from the standard transmittance. However, when there is an object that overlaps the prediction surface 20a on the inner side in the traveling direction, the transmittance is also changed and set on the prediction surface 20a as well as the prediction surface 20b on the outer side in the traveling direction.

  Specifically, in the overlap region 22, the object 21 is positioned between the first region 21a located between the pair of prediction surfaces 20a and 20b and the first region 21a on the opposite side of the prediction regions 20a and 20b. In the case of having two regions 21b, the setting unit 14 compares the transmittance of the portion overlapping the first region 21a of the prediction surfaces 20a and 20b with the transmittance of the portion overlapping the second region 21b of the prediction surfaces 20a and 20b. Make it bigger. That is, the transmittance of the predicted surfaces 20a and 20b that overlap a part of the object 21 existing in the predicted path of the vehicle 5 is determined from the transmittance of the predicted surfaces 20a and 20b that overlap a part of the object 21 existing outside the predicted path. Also make it bigger. According to this configuration, the driver can clearly recognize the portion of the object 21 protruding inside the prediction planes 20a and 20b by looking at the composite image CP. That is, the driver can immediately determine whether there is an obstacle on the predicted course and can avoid a collision with the obstacle.

  Note that in the overlapping region 22, the object 21 may have only the first region 21a. That is, it is a case where all of the object 21 exists in the predicted course. In the present embodiment, in this case, the setting unit 14 increases the transmittance of the portion overlapping the first region 21a of the prediction surfaces 20a and 20b as compared with the transmittance of other regions of the prediction surfaces 20a and 20b. To do. Moreover, in the superimposition area | region 22, the case where the object 21 has only the 2nd area | region 21b may also arise. That is, it is a case where all of the object 21 exists outside the predicted course. In the present embodiment, in this case, the setting unit 14 sets the transmittance of the prediction planes 20a and 20b to a uniform value in all regions. Further, in the overlapping region 22, the object 21 may have a third region 21c located on the prediction planes 20a and 20b. That is, this is a case where a part of the object 21 exists at the boundary between the predicted course and the predicted course. In the example shown in FIG. 2, the object 21 has a third region 21c. In this Embodiment, the setting part 14 makes the transmittance | permeability of the part which overlaps the 3rd area | region 21c of the prediction surfaces 20a and 20b the same as the transmittance | permeability of the part which overlaps the 1st area | region 21a of the prediction surfaces 20a and 20b.

  In the example illustrated in FIG. 2, the setting unit 14 sets the transmittance of the portion overlapping the first region 21a of the prediction surface 20b to 100%. That is, in the part which overlaps with the 1st area | region 21a of the prediction surface 20b, the color of the prediction surface 20b is not displayed, and only the color and line of the 1st area | region 21a of the object 21a are displayed as a result. The setting unit 14 sets the transmittance of the portion overlapping the second region 21b of the prediction surface 20b to the standard transmittance (20% in this example). For this reason, the second region 21b of the object 21 is displayed lightly through the prediction surface 20b. In addition, the transmittance | permeability of the part which overlaps with the 1st area | region 21a of the prediction surface 20b may be other than 100%. Further, the transmittance of the portion overlapping the second region 21b of the prediction surface 20b may be a transmittance other than the standard transmittance, for example, a transmittance larger than the standard transmittance (for example, 50%).

  In the example shown in FIG. 2, the first area 21 a and the second area 21 b of the object 21 can be distinguished at a glance, and the first area 21 a of the object 21 is conspicuous. Since the space between the pair of prediction planes 20a and 20b corresponds to the predicted course of the vehicle 5, in the example shown in FIG. It is possible to immediately recognize that an obstacle exists in the predicted course.

  In the present embodiment, the setting unit 14 sets the first distance from the predetermined position of the vehicle 5 to the prediction planes 20a and 20b in the real space and the predetermined position of the vehicle 5 for each pixel position in the overlapping region 22. To the second distance from the object 21 is calculated. The setting unit 14 compares the calculation results for each pixel position in the overlapping region 22. The setting unit 14 assigns the first region 21a to the pixel position where the first distance is greater than the second distance. That is, the first area 21a is assigned to the pixel position of the object image existing in the predicted course. The setting unit 14 assigns the second region 21b to the pixel position where the first distance is smaller than the second distance. That is, the second region 21b is assigned to the pixel position of the object image that exists outside the predicted course. The setting unit 14 assigns the third region 21c to pixel positions having the same first distance and second distance. That is, the third region 21c is assigned to the pixel position of the object image that exists at the boundary between the predicted course and the predicted course.

  The coordinates of each pixel in the captured image have a one-to-one correspondence with the coordinates (so-called world coordinates) for the vehicle in the actual space. For this reason, by converting the coordinate value of the superimposed region 22 detected on the captured image into world coordinates, the distance in real space from the predetermined position of the vehicle 5 can be calculated for each pixel position of the object 21. Further, as described above, the prediction planes 20a and 20b are obtained by extending the progress prediction line drawn on the road surface in the captured image in the vertical direction. The distance in the real space to each point on the progress prediction line can be calculated by the coordinate conversion described above, and the vertical lengths of the prediction planes 20a and 20b can be converted to the length of the real space. For this reason, the distance in the real space from the predetermined position of the vehicle 5 can be calculated for each pixel position of the prediction planes 20a and 20b. The predetermined position of the vehicle 5 may be, for example, the center of the rear wheel shaft that extends in the left-right direction of the vehicle 5.

The composite image generation unit 15 combines the images of the prediction surfaces 20a and 20b and the captured image so that the transmittances of the prediction surfaces 20a and 20b become the transmittance set by the setting unit 14. In the present embodiment, the composite image generation unit 15 performs alpha blending on a portion where the prediction planes 20a and 20b overlap the captured image. Specifically, the transmittance is α (%),
Each pixel value in the synthesized image is calculated by the following formula: pixel value after synthesis = pixel value of prediction plane × (1−α / 100) + pixel value of captured image × α / 100. The composite image generation unit 15 generates a composite image CP based on the calculated pixel values. Note that the pixel value on the prediction plane is a preset pixel value, and the pixel value is stored in the storage unit. However, the setting of the pixel value on the prediction plane may be configured to be changed as appropriate.

  In the example shown in FIG. 2, the transmittance α is set to 100% for the pixel position overlapping the first region 21a and the third region 21c of the prediction surface 20b on the outer side in the traveling direction. For the remaining pixel positions of the prediction surface 20b on the outer side in the traveling direction and all the pixel positions on the prediction surface 20a on the inner side in the traveling direction, the transmittance α is set to the standard transmittance (transmittance 20%).

  The notification processing unit 16 performs a process for notifying the detection of the first region 21a. In the present embodiment, when the first area 21a is detected, the notification processing unit 16 performs a process for notifying the driver of the fact by screen display and sound. As a result, the driver can accurately recognize that a dangerous state has occurred, and the probability of an accident can be reduced. In the present embodiment, the notification processing unit 16 performs the notification process even when the third region 21c is detected.

  FIG. 3 is a schematic diagram for explaining processing performed by the notification processing unit 16. As illustrated in FIG. 3, the notification processing unit 16 performs image processing for displaying a frame 23 that surrounds the first region 21 a. The frame 23 is preferably composed of a conspicuous color such as red, for example, so that the driver can recognize that a dangerous state has occurred. In the example shown in FIG. 3, the shape of the frame 23 is a rectangular shape. However, the shape of the frame may be other shapes such as a circular shape. In this embodiment, the frame 23 is displayed to notify the detection of the first area 21a. However, the first area 21a is detected using a graphic other than the frame, such as an arrow display. May be notified. In some cases, the detection of the first region 21a may be notified by using a character informing the danger.

  Moreover, the alerting | reporting process part 16 reads the audio | voice data memorize | stored in the memory | storage part not shown, when the 1st area | region 21a is detected. In the present embodiment, the notification processing unit 16 is configured to perform processing relating to image processing and sound, but this is an example, and may be configured to perform only one of these two processings. . In some cases, the notification processing unit 16 may not be provided.

  The notification processing unit 16 may change the notification form according to the distance from the vehicle 5 in the first region 21a. For example, the notification processing unit 16 calculates the shortest distance from a predetermined position of the vehicle 5 to the first region 21a. The notification processing unit 16 compares the shortest distance with at least one preset threshold value and changes the notification mode. Thus, the driver can accurately recognize the level of the dangerous state, and can appropriately perform the danger avoidance action.

  An example of changing the notification mode is to change the color of the frame 23, for example. For example, the color of the frame 23 may be changed to green, yellow, red, or the like as the distance from the vehicle 5 to the first region 21a becomes shorter. Moreover, as a change of an alerting | reporting form, changing the magnitude of an audio | voice or changing a timbre is mentioned. For example, the loudspeaker sound may be gradually increased as the distance from the vehicle 5 to the first region 21a becomes shorter.

  The image output unit 17 outputs the composite image CP generated by the composite image generation unit 15 or the image data of the image subjected to the notification process on the composite image CP to the display device 3. The display device 3 displays an image corresponding to the input image data on the screen. The driver can confirm the state of the traveling direction of the vehicle in a three-dimensional manner by looking at the images with the prediction surfaces 20a and 20b displayed on the display device 3. Therefore, even a driver who is not good at parking can cleanly park in the parking frame. Moreover, since it is possible to grasp at a glance whether there is an obstacle in the predicted course of the vehicle 5 by changing the transmittance of the predicted surfaces 20a and 20b, the possibility of an accident occurring at the time of parking is reduced. Can do.

  The audio output unit 18 outputs the audio data to the audio output device 4 when the audio data is input from the notification processing unit 16. The audio output device 4 outputs audio corresponding to the input audio data. The driver can perform a danger avoidance action by the warning sound output by the audio output device 4.

<3. Driving support processing>
FIG. 4 is a flowchart illustrating an operation example of the driving support device 1. When the operation start event occurs, the driving support device 1 starts the operation of the flowchart shown in FIG. As the operation start event, for example, the position of the shift lever of the vehicle 5 is switched to the reverse range.

  First, a predicted surface generation process is performed by the predicted surface generation unit 12 (step S1). The prediction plane generation process is a process of generating a pair of left and right prediction planes 20a and 20b that three-dimensionally show the predicted course of the vehicle 5. In the present embodiment, the pair of prediction planes 20a and 20b is generated using image processing of a captured image. Next, the detection process by the detection part 13 is performed (step S2). The detection process is a process of detecting the object 21 that overlaps the prediction planes 20a and 20b. Next, a setting process by the setting unit 14 is executed. The setting process is a process for setting the transmittance of the prediction planes 20a and 20b.

  Specifically, the setting unit 14 confirms whether or not the overlapping region 22 where the prediction planes 20a and 20b and the object 21 overlap is generated in the composite image CP based on the detection processing result of the detection unit 13 ( Step S3). When there is no overlapping region 22 (Yes in Step S3), the setting unit 14 performs normal setting in which the transmittances of the prediction planes 20a and 20b are uniformly set to the standard transmittance (20% in this example) (Step S4).

  When the setting unit 14 sets the transmittance of the prediction surfaces 20a and 20b, the composite image generation unit 15 performs a process of combining the pair of prediction surfaces 20a and 20b with the captured image (step S5). When there is no overlapping region 22, it is determined that there is no state in which the object 21 exists as an obstacle on the predicted course of the vehicle 5, and thus the process by the notification processing unit 16 is not executed.

  When the composite image CP is generated by the composite image generation unit 15, the image output unit 17 outputs the image data of the composite image CP to the display device 3 (step S6). Thereafter, the driving assistance device 1 checks whether or not an operation end event has occurred (step S7). As the operation end event, for example, the position of the shift lever of the vehicle is switched from the reverse range to another range. If no operation end event has occurred (No in step S7), the process returns to step S1. Thereby, the process based on the new frame image is repeated. On the other hand, if the operation end event has occurred (Yes in step S7), the driving support device 1 ends the operation of the flowchart shown in FIG.

  On the other hand, when there is the overlapping region 22 (No in step S3), the setting unit 14 checks whether or not the object 21 has only the second region 21b in the overlapping region 22 (step S8). The 2nd field 21b is a field located on the opposite side of the 1st field 21a located between a pair of prediction fields 20a and 20b on the basis of prediction fields 20a and 20b. When the object 21 is only the second region 21b (Yes in Step S8), the parking support device 1 performs the operations after Step S4.

  In the overlapping region 22, when the object 21 has at least one of the first region 21a and the third region 21c (No in step S8), the setting unit 14 sets the transmittance of the prediction surfaces 20a and 20b as a special setting. (Step S9). The first region 21a is a region located between the pair of prediction planes 20a and 20b. The third region 21c is a region where the object 21 is located on the prediction planes 20a and 20b. The setting unit 14 sets the transmittance of the prediction surfaces 20a and 20b to 100% for the first region 21a and the third region 21c. The setting unit 14 sets the standard transmittance for the other areas of the prediction planes 20a and 20b.

  When the setting unit 14 sets the transmittances of the prediction surfaces 20a and 20b, the composite image generation unit 15 performs processing for combining the pair of prediction surfaces 20a and 20b with the captured image (step S10). The synthesizing process includes a process for adding the frame 23 by the notification processing unit 16.

  When an image obtained by performing notification processing on the composite image is generated, the image output unit 17 outputs the generated image data to the display device 3. Further, the voice output unit 18 outputs a voice to the voice output device 4 according to a command from the notification processing unit 16 (step S11). Thereafter, the driving assistance device 1 checks whether or not an operation end event has occurred (step S7). Since the subsequent processing has been described above, the description thereof will be omitted.

  Instead of performing step S7, the driving support device 1 constantly monitors whether or not an operation end event has occurred, and immediately ends the operation of the flowchart if an operation end event occurs. Also good. In this case, when the processes of step S6 and step 11 are completed, the process may return to step S1.

<4. Modification>
Various technical features disclosed in the present specification can be variously modified within the scope of the technical creation in addition to the above-described embodiment. In addition, a plurality of embodiments and modifications shown in the present specification may be appropriately combined and implemented within a possible range.

(4-1. First modification)
The setting unit 14 may change the transmittance of a portion overlapping the second region 21b of the prediction surfaces 20a and 20b according to the type of the object 21. According to this, the appearance of the portion that exists outside the predicted course of the vehicle 5 and overlaps the predicted planes 20 a and 20 b can be made different depending on the type of the object 21. For example, since the size and shape of the object 21 are different between a vehicle and a person, the configuration of this modification is effective.

  This modification will be further described with reference to the example shown in FIG. FIG. 5 is a schematic diagram for explaining a first modification of the driving support device 1. In the example shown in FIG. 5, two objects 21A and 21B that overlap with the prediction plane 20b are detected in the captured image. The first object 21A is a vehicle, and the second object 21B is a person. The prediction surface 20b and the first object 21A have a first overlapping region 22A. In the first overlapping region 22A, the first object 21A includes a first region 21Aa located between the pair of prediction surfaces 20a and 20b, and a second region located on the opposite side of the first region 21a with respect to the prediction surface 20b. 21Ab and a third region 21Ac located on the prediction surface 20b. The prediction surface 20b and the second object 21B have a second overlapping region 22B. In the second overlapping region 22B, the second object 21B includes a first region 21Ba located between the pair of prediction surfaces 20a and 20b, and a second region located on the opposite side of the first region 21Ba with respect to the prediction surface 20b. 21Bb and a third region 21Bc located on the prediction surface 20b.

  In the example illustrated in FIG. 5, for example, the setting unit 14 determines the type of an object based on information stored in advance in the storage unit for the objects 21 </ b> A and 21 </ b> B recognized by the pattern matching process. When there are a plurality of discriminated object types, the setting unit 14 sets the transmittance corresponding to the object type according to the classification information stored in advance. In this example, the setting unit 14 treats the first object 21A and the second object 21B as different types of objects.

  The setting unit 14 changes the transmittance of the portion overlapping the second regions 21Ab and 21Bb of the prediction surface 20b between the first object 21A and the second object 21B. Specifically, the setting unit 14 sets the transmittance of the portion overlapping the second region 21Bb of the prediction surface 20b (the transmittance on the person 21B side) to the transmittance of the portion overlapping the second region 21Ab of the prediction surface 20b (the vehicle 21A). Side transmittance).

  More specifically, the setting unit 14 sets the transmittance of the portion (vehicle 21A side) overlapping the second region 21Ab of the prediction surface 20b to the standard transmittance (20% in this example). The setting unit 14 sets the transmittance of the portion overlapping the second region 21Bb (the person 21B side) of the prediction surface 20b to a transmittance (50% in this example) that is larger than the standard transmittance. The setting unit 14 sets the transmittance of the portion of the prediction surface 20b that overlaps the first regions 21Aa and 21Ba and the third regions 21Ac and 21Bc to 100%. Further, the setting unit 14 sets the transmittance of the remaining part of the prediction surface 20b as the standard transmittance. Further, in this example, the setting unit 14 sets all the regions to the standard transmittance for the prediction plane 20a that does not have a region overlapping the object in the captured image.

  In the configuration of the present modification, the second object 21B (person) is easier to see than the first object 21A (vehicle) in the portion that exists outside the predicted course of the vehicle 5 and overlaps the prediction planes 20a and 20b. . For this reason, the driver can easily grasp the danger in the traveling direction of the vehicle 5 at a glance at the composite image CP.

  In the present modification, a person and a vehicle are shown as types of the object 21, but this is an example. For example, there may be more than two types of objects. Moreover, as a kind of object, it is good also as a grouping form, for example, a moving object and a stationary object. When the moving object and the stationary object are separated, the transmittance of the portion overlapping the second region of the prediction surfaces 20a and 20b may be configured to be larger on the moving object side than on the stationary object side. The moving object includes, for example, a person and a vehicle. Stationary objects include, for example, poles and building walls.

(4-2. Second Modification)
The setting unit 14 may set the transmittances of the prediction surfaces 20a and 20b in a plurality of types based on the curvatures of the prediction surfaces 20a and 20b. According to this, the driver can easily grasp the shapes of the prediction surfaces 20a and 20b superimposed on the captured image.

  This modification will be further described with reference to the example shown in FIG. FIG. 6 is a schematic diagram for explaining a second modification of the driving support device 1. In the example illustrated in FIG. 6, the object 21 that overlaps the prediction plane 20b is detected in the captured image. The object 21 is positioned on the prediction surface 20b, a first region 21a located between the pair of prediction surfaces 20a and 20b, a second region 21b located on the opposite side of the first region 21a with respect to the prediction surface 20b. And a third region 21c.

  The setting unit 14 sets the transmittance of the portion overlapping the first region 21a and the third region 21c of the prediction surface 20b to 100%. The setting unit 20b sets the transmittance of the remaining part of the prediction surface 20b and the prediction surface 20a based on the curvatures of the prediction surfaces 20a and 20b. In this example, the setting unit 14 sets the transmittance by comparing the magnitude with a threshold value relating to the curvature stored in advance in the storage unit. The setting unit 14 sets the transmittance of a portion having a curvature greater than or equal to the threshold smaller than that of a portion having a curvature smaller than the threshold. That is, the transmittance of the portion having a large curvature is made smaller than the others. In the example shown in FIG. 6, the transmittance on the near side of the vehicle 5 is set to 40%, and the transmittance on the back side of the vehicle 5 is set to 20% (standard transmittance). According to this, it is possible to easily imagine a portion having a large curvature of the prediction surfaces 20a and 20b as a wall.

  In addition, in this modification, although it is set as the structure which divides | segments the transmittance | permeability of the prediction surfaces 20a and 20b into two types based on the curvature of the prediction surfaces 20a and 20b, this is an illustration and may be divided into three or more types. . Moreover, you may change how to change the transmittance | permeability of the prediction surfaces 20a and 20b based on a curvature with one side and the other of a pair of prediction surfaces 20a and 20b.

(4-3. Third modification)
The setting unit 14 may set the transmittances of the prediction surfaces 20a and 20b in a plurality of types based on the distance from the vehicle 5. According to this, the appearance of the object that overlaps the prediction plane changes depending on the position that the driver should be particularly careful and the position that is not so, and the driver can easily grasp the situation of the traveling direction of the vehicle 5. Become.

  This modification will be further described with reference to the example shown in FIG. FIG. 7 is a schematic diagram for explaining a third modification of the driving support device 1. In the example illustrated in FIG. 7, an object 21 that overlaps the prediction surface 20b is detected in the captured image. The object 21 is positioned on the prediction surface 20b, a first region 21a located between the pair of prediction surfaces 20a and 20b, a second region 21b located on the opposite side of the first region 21a with respect to the prediction surface 20b. And a third region 21c.

  The setting unit 14 sets the transmittance of the portion overlapping the first region 21a and the third region 21c of the prediction surface 20b to 100%. The setting unit 20b sets the transmittance of the remaining part of the prediction surface 20b and the prediction surface 20a based on the distance from the vehicle 5. In this example, the setting unit 14 sets the transmittance on the side closer to the vehicle 5 to a larger transmittance than that on the far side. Specifically, the setting unit 14 changes the transmittance in four steps from the side closer to the vehicle 5 toward the far side. For example, the transmittance is set to 80%, 60%, 40%, and 20% from the side closer to the vehicle 5. According to this modification, as the distance from the vehicle 5 is shorter, an object positioned outside the predicted course of the vehicle 5 can be clearly recognized through the prediction planes 20a and 20b.

  In this modification, the transmittances of the prediction surfaces 20a and 20b change in a stepwise manner, but this is an example, and the transmittances of the prediction surfaces 20a and 20b change continuously. Good. Further, in the case of a configuration in which the transmittances of the prediction planes 20a and 20b change stepwise, the configuration may change in a number other than four steps.

(4-4. Others)
In the embodiment described above, it has been described that various functions are realized in software by the arithmetic processing of the CPU according to the program. However, at least a part of these functions is performed by an electrical hardware circuit. It may be realized. As the hardware, for example, an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA) may be used.

DESCRIPTION OF SYMBOLS 1 Driving assistance apparatus 2 Camera 5 Vehicle 12 Prediction surface production | generation part 13 Detection part 14 Setting part 15 Composite image generation part 16 Notification processing part 20a, 20b Prediction surface 21 Object 21a 1st area | region 21b 2nd area | region 22 Superimposition area | region CP Composite image

Claims (9)

A driving support device that displays on a screen a predicted image of the vehicle superimposed on a captured image captured by a camera that captures the periphery of the vehicle,
A prediction plane generation unit that generates a pair of left and right prediction planes that three-dimensionally represent the predicted course;
A detection unit for detecting an object overlapping the prediction plane from the captured image;
A setting unit for setting the transmittance of the predicted surface;
A composite image generation unit that generates a composite image in which the captured image and the pair of prediction planes are combined based on the transmittance set by the setting unit;
With
The setting unit is a driving support device that changes the transmittance according to a positional relationship between the prediction plane and the object in an overlapping region where the prediction plane and the object overlap in the composite image. In the overlap region, when the object has a first region located between the pair of prediction planes and a second region located on the opposite side of the first region with respect to the prediction plane,
2. The operation according to claim 1, wherein the setting unit increases the transmittance of a portion overlapping the first region of the prediction surface as compared to the transmittance of a portion overlapping the second region of the prediction surface. Support device.   The driving support device according to claim 2, further comprising a notification processing unit that performs processing for notifying detection of the first region.   The driving support device according to claim 3, wherein the notification processing unit changes a notification form according to a distance from the vehicle in the first region.   The driving support device according to any one of claims 2 to 4, wherein the setting unit changes the transmittance of a portion overlapping the second region of the prediction plane according to the type of the object.   The driving support device according to any one of claims 1 to 5, wherein the setting unit sets the transmittance of the prediction surface in a plurality of types based on the curvature of the prediction surface.   The driving support device according to any one of claims 1 to 5, wherein the setting unit sets the transmittance of the prediction plane in a plurality of types based on a distance from the vehicle.   The driving support device according to any one of claims 1 to 7, wherein the predicted surface generation unit generates the pair of predicted surfaces based on a plurality of the captured images captured at different times. A driving support method for displaying on a screen a predicted route of the vehicle superimposed on a captured image captured by a camera that captures the periphery of the vehicle,
A prediction plane generating step for generating a pair of left and right prediction planes that three-dimensionally show the predicted course;
A detection step of detecting an object overlapping the prediction plane from the captured image;
A setting step for setting the transmittance of the predicted surface;
A composite image generation step of generating a composite image in which the captured image and the pair of prediction planes are combined based on the transmittance set by the setting step;
With
The driving support method, wherein the setting step changes the transmittance according to a positional relationship between the prediction surface and the object in an overlapping region where the prediction surface and the object overlap in the composite image.