JP7452479B2 - display control device - Google Patents

Description

The present invention relates to a display control device that displays captured images of the surroundings of a vehicle.

As a vehicle driving support technology, a technology has been proposed in which a camera mounted on the vehicle images the surroundings of the vehicle and the captured image data, which is the result of the image capturing, is displayed on a display unit (see Patent Document 1 below). ).

JP 2018-6943 Publication

In the case of vehicles such as passenger cars, the corners are generally designed to prevent the body (for example, the overhang of the vehicle) from swinging out beyond the side of the vehicle just before the turn when turning. There is. On the other hand, in the case of commercial vehicles such as trucks, priority is given to load capacity, etc., and the above measures are not taken.

Therefore, the present invention has been made in view of these points, and an object of the present invention is to allow a driver to easily grasp the overhang of the vehicle and the possibility of contact with an obstacle when the vehicle turns. .

In one aspect of the present invention, image generation generates a bird's-eye view image including the vehicle from images taken around the vehicle by a plurality of imaging devices provided at a position higher than the line of sight of a driver of the vehicle. a trajectory estimating section that estimates a traveling trajectory of the vehicle when the vehicle turns; and a trajectory estimating section that estimates a traveling trajectory of the vehicle when the vehicle turns, and a superimposing and displaying of the traveling trajectory estimated by the trajectory estimating section on the bird's-eye image generated by the image generating section. A display control device is provided, which includes a display control unit that causes a display to be displayed on a portion of the screen.

Furthermore, the display control unit may display the trajectory of an inner wheel difference, an outer wheel difference, and a rear overhang of the vehicle superimposed on the bird's-eye view image that occurs when the vehicle turns, and display the superimposed trajectory on the display unit.

Further, the display control unit may change a display area of the bird's-eye view image on the display unit depending on a direction in which the vehicle turns.

Further, the display control unit displays a predetermined line superimposed on the bird's-eye view image along the traveling direction of the vehicle, and emphasizes a portion of the travel trajectory that occurs when the vehicle turns that conflicts with the line. It may also be displayed.

Further, the display control unit may set the line on the side surface at a predetermined distance from a reference line passing between the center of the front wheels and the center of the rear wheels of the vehicle along the traveling direction immediately before the vehicle turns. It is also possible to display a line along the line.

According to the present invention, when the vehicle turns, the driver can easily grasp the possibility of contact between the overhang of the vehicle and an obstacle.

FIG. 1 is a schematic diagram for explaining the configuration of a display system 1 according to one embodiment. FIG. 3 is a schematic diagram for explaining the position of an imaging device. FIG. 2 is a schematic diagram for explaining an example of a bird's-eye view image. 3 is a schematic diagram for explaining a bird's-eye view image I displayed on a display unit 7. FIG. FIG. 3 is a schematic diagram for explaining an example of changing a display area of a bird's-eye view image. FIG. 2 is a schematic diagram for explaining an example of highlighted display in a bird's-eye view image.

<Display system overview>
The configuration of a display system according to one embodiment will be described with reference to FIG. 1.

FIG. 1 is a schematic diagram for explaining the configuration of a display system 1 according to one embodiment. The display system 1 is an around-view monitor system that is mounted on a vehicle and displays a bird's-eye view of the surroundings of the vehicle on a vehicle-mounted monitor. The vehicle is, for example, a mounted vehicle having a loading platform, and specifically a truck. As shown in FIG. 1, the display system 1 includes a plurality of imaging devices 3a, 3b, 3c, and 3d, a sensor group 5, a display section 7, and a display control device 10.

The imaging devices 3a, 3b, 3c, and 3d are, for example, cameras and take images of the surroundings of the vehicle. The imaging devices 3a, 3b, 3c, and 3d are provided at different positions on the vehicle. The imaging device 3a images the front of the vehicle, the imaging device 3b images the left side of the vehicle, the imaging device 3c images the right side of the vehicle, and the imaging device 3d images the rear of the vehicle. By using the four imaging devices 3a, 3b, 3c, and 3d, which are wide-angle cameras, it is possible to image the entire surroundings of the vehicle. The imaging devices 3a, 3b, 3c, and 3d output the captured images to the display control device 10.
The four image pickups 3a, 3b, 3c, and 3d are provided at a position higher than the line of sight of the driver (for example, at a position higher than the mirror of the vehicle). Note that only three of the four imaging devices may be provided at a position higher than the driver's line of sight.

FIG. 2 is a schematic diagram for explaining the position of the imaging device. FIG. 2 shows an imaging device 3a that is a front camera, an imaging device 3b that is a side camera, and an imaging device 3d that is a rear camera. The imaging devices 3a, 3b, and 3d are located at a position higher than the line of sight E of the driver of the vehicle V. Note that the imaging device 3c, which is not shown in FIG. 2, is provided at a position symmetrical to the imaging device 3b. When the imaging device 3a etc. is located at a position higher than the driver's line of sight E, the blind spot (of the line of sight E) shown in FIG. (lower area) can also be appropriately imaged. Furthermore, since the imaging device 3a and the like capture images diagonally below from a high position, it becomes possible to recognize and image an object three-dimensionally. Furthermore, it becomes possible to image a wide range of surroundings.

Sensor group 5 detects the state of the vehicle. The sensor group 5 includes, for example, a sensor that detects the amount of rotation of the steering wheel, and is capable of detecting the steering angle of the vehicle. The sensor group 5 outputs the detected results to the display control device 10.

The display unit 7 is, for example, a display provided in the vehicle, and displays various information regarding the vehicle and the like. The display unit 7 displays a bird's-eye view image based on the captured images captured by the imaging devices 3a, 3b, 3c, and 3d. The bird's-eye view image is an image that shows the situation when looking down on the vehicle and its surroundings from above the vehicle. A bird's-eye view image is generally obtained by two-dimensionally converting a three-dimensional view captured by an imaging device using the road surface as a reference, and then combining captured images from a plurality of imaging devices.

FIG. 3 is a schematic diagram for explaining an example of a bird's-eye view image. The part surrounded by a frame 90 in FIG. 3 is the bird's-eye view image I displayed on the display unit 7. The bird's-eye image I includes the vehicle and surrounding objects. The objects here are persons A1 and A2 located around the vehicle V and obstacles B1, B2, and B3. As described above, since the imaging devices 3a to 3d are positioned higher than the driver's line of sight, the people A1 and A2 and the obstacles B1, B2, and B3 are displayed three-dimensionally. Therefore, the driver can easily identify the persons A1 and A2 and the obstacles B1, B2, and B3 in the bird's-eye image I. If the persons A1 and A2 are identified, an alarm may be issued.

The display control device 10 controls the operation of the display system 1. The display control device 10 causes the display unit 7 to display a bird's-eye view image while the vehicle is running. Although the details will be described later, the display control device 10 superimposes the traveling trajectory of the vehicle when the vehicle turns on a bird's-eye view image, and causes the display unit 7 to display the superimposed trajectory. Thereby, by viewing the bird's-eye view image displayed on the display unit 7, the driver can easily understand visually whether or not there is a possibility that the vehicle will come into contact with surrounding obstacles when turning.

<Detailed configuration of display control device>
The detailed configuration of the display control device 10 will be explained with reference to FIG. 1.

The display control device 10 includes a storage section 20 and a control section 30, as shown in FIG.
The storage unit 20 includes, for example, a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 20 stores programs and various data for the control unit 30 to execute.

The control unit 30 is, for example, a CPU (Central Processing Unit). The control unit 30 functions as an image acquisition unit 33, an image generation unit 34, a trajectory estimation unit 35, and a display control unit 36 by executing a program stored in the storage unit 20.

The image acquisition unit 33 acquires an image of the surroundings of the vehicle. Specifically, the image acquisition unit 33 acquires captured images captured by the imaging devices 3a, 3b, 3c, and 3d around the vehicle. For example, the image acquisition unit 33 acquires images taken by the imaging devices 3a, 3b, 3c, and 3d at predetermined intervals while the vehicle is running.

The image generation unit 34 generates a bird's-eye view image showing the running vehicle and its surroundings. The image generation unit 34 generates a bird's-eye view image from the captured image acquired by the image acquisition unit 33. Specifically, the image generation unit 34 generates a bird's-eye view image including the vehicle by combining the captured images captured by the imaging devices 3a, 3b, 3c, and 3d. As described above, since the imaging devices 3a, 3b, 3c, and 3d are located at positions higher than the driver's line of sight, it is possible to generate a bird's-eye view image that includes obstacles located in blind spots. In addition, it is possible to generate a wide bird's-eye view image that extends far into the distance.

The trajectory estimation unit 35 estimates the traveling trajectory of the vehicle when the vehicle turns. For example, the trajectory estimation unit 35 estimates the trajectory of a rear overhang (hereinafter referred to as a rear overhang) when the vehicle turns. When the vehicle is a truck, the trajectory estimation unit 35 estimates the trajectory of the rear overhang at the rear end of the loading platform. Note that the trajectory estimation unit 35 may estimate the trajectory of the overhang on the front side of the vehicle. The trajectory estimation unit 35 can estimate the trajectory of the rear overhang using the detection results of the sensor group 5 (for example, the steering angle of the vehicle, etc.). Note that the trajectory of the rear overhang is calculated using a known calculation method, so a detailed explanation will be omitted.
The trajectory estimating unit 35 also estimates, as the traveling trajectory, a trajectory of the inner wheel difference and the outer wheel difference according to the steering angle of the vehicle. For example, the trajectory estimation unit 35 estimates the trajectory of the inner wheel difference and the outer wheel difference when the vehicle turns.

The display control unit 36 controls the display by the display unit 7. The display control unit 36 causes the display unit 7 to display the bird's-eye view image generated by the image generation unit 34. For example, the display control unit 36 causes the display unit 7 to display a bird's-eye view image when the vehicle turns right or left, when the vehicle travels backward, or when the vehicle travels at low speed.

The display control unit 36 superimposes the travel trajectory of the vehicle on the bird's-eye view image and displays it on the display unit 7. That is, the display control unit 36 superimposes the travel trajectory estimated by the trajectory estimation unit 35 on the bird's-eye view image generated by the image generation unit 34, and causes the display unit 7 to display the superimposed travel trajectory. For example, when the vehicle turns, the display control unit 36 causes the display unit 7 to display the trajectory of the inner wheel difference, the outer wheel difference, and the rear overhang superimposed on the bird's-eye view image.

FIG. 4 is a schematic diagram for explaining the bird's-eye view image I displayed on the display section 7. As shown in FIG. The part surrounded by a frame 90 in FIG. 4 is the bird's-eye view image I displayed on the display unit 7. This bird's-eye view image I is displayed on the display unit 7 when the vehicle V starts turning right at an intersection. Here, three overhang trajectories L1, L2, and L3 estimated by the trajectory estimation section 35 are superimposed on the bird's-eye view image I generated by the image generation section 34. The overhang loci L1 and L2 are the loci of the rear overhang, and the overhang locus L3 is the locus of the front overhang. In FIG. 4, the overhang trajectories L1, L2, and L3 are shown as dashed-dotted lines, but the display mode is not limited to this, and may be, for example, a solid line. If the overhang trajectories L1, L2, and L3 intersect with surrounding obstacles, the driver can determine that there is a possibility of contacting the obstacles if the driver continues to drive. Note that the trajectory of the inner wheel difference and the outer wheel difference of the vehicle may also be displayed superimposed on the bird's-eye image I.
In FIG. 4, the overhang trajectory L3 is the trajectory of the left front end of the vehicle V, but when the vehicle turns left, the overhang trajectory L3 becomes the trajectory of the right front end of the vehicle V. Note that the display control unit 36 may display only the overhang locus L1 and L2 on the rear side of the vehicle and not display the overhang locus L3 on the front side of the vehicle.

The display control unit 36 may change the display area of the bird's-eye view image on the display unit 7. The display control unit 36 changes the display area of the bird's-eye view image when the vehicle turns. For example, the display control unit 36 changes the display area of the bird's-eye view image on the display unit 7 depending on the direction in which the vehicle is turning.

FIG. 5 is a schematic diagram for explaining an example of changing the display area of a bird's-eye view image. Before the vehicle V enters the intersection, the display control unit 36 displays a bird's-eye view image I1 as shown in FIG. 5(a). The bird's-eye view image I1 does not include an overhang trajectory. On the other hand, when the vehicle V starts turning right at an intersection, the display control unit 36 displays a bird's-eye view image I2 in which the overhang trajectories L1, L2, and L3 are superimposed, as shown in FIG. 5(b). At this time, the display control unit 36 makes the display area of the bird's-eye image I2 wider than the display area of the bird's-eye image I1. That is, the frame 90 of the bird's-eye view image I2 is larger than the frame 90 of the bird's-eye view image I1. Specifically, the display control unit 36 displays a bird's-eye view image that widens the right turn destination (that is, the right side). This makes it easier for the driver to determine in advance whether there is a possibility of contact with an obstacle at the right turn destination.
In addition, although the display example when the vehicle V turns right was described above, when the vehicle V turns left, a bird's-eye view image that widely displays the left turn destination is displayed.

The display control unit 36 may perform highlighted display in the bird's-eye view image to help the driver understand the possibility that the overhang of the vehicle will come into contact with an obstacle. For example, the display control unit 36 superimposes and displays a predetermined determination line on a bird's-eye view image along the traveling direction of the vehicle, and displays a predetermined determination line in a travel trajectory (for example, a rear overhang trajectory) that occurs when the vehicle turns. Highlight the conflicting parts. Here, the display control unit 36 sets a determination line along the side surface at a position a predetermined distance away from a reference line that passes between the center of the front wheels and the center of the rear wheels of the vehicle along the direction of travel, immediately before the vehicle turns. Display the line.

FIG. 6 is a schematic diagram for explaining an example of highlighted display in a bird's-eye view image. Here, it is assumed that a vehicle V turns right at an intersection, and overhang trajectories L1, L2, L3 and a determination line T are superimposed on a bird's-eye view image I. The determination line T is displayed at a position a predetermined distance away from the left side surface of the vehicle V making a right turn so as to be parallel to the left side surface of the vehicle V. The display control unit 36 highlights the portions of the overhang trajectories L1, L2, and L3 that protrude to the left of the determination line T. Here, a portion of the overhang locus L1 that protrudes to the left of the determination line T is highlighted with a solid line. Note that the highlighted display may be performed using different line colors. Thereby, the driver can easily grasp the timing at which the vehicle protrudes from the determination line T when turning right. Note that when the vehicle turns left, the determination line T is displayed at a position a predetermined distance away from the right side of the vehicle V.

In the above description, the determination line T is displayed at a predetermined distance from the side of the vehicle, but the present invention is not limited thereto. For example, the determination line T may be displayed at the same position as the side of the vehicle. Furthermore, the determination line T may be displayed at the boundary between the lane in which the vehicle travels and the sidewalk.

<Effects of this embodiment>
The display control device 10 of the embodiment described above generates a bird's-eye view image including the vehicle from a captured image of the surroundings of the vehicle. The display control device 10 also estimates the travel trajectory of the vehicle when turning. Then, the display control device 10 displays the generated bird's-eye image I on the display unit 7 with the traveling trajectory of the vehicle superimposed thereon.
This makes it easier for the driver to visually determine whether or not there is a possibility that the overhang of the vehicle will come into contact with an obstacle when turning by viewing the bird's-eye image on which the travel trajectory during the turn is superimposed. As a result, the driver can steer the vehicle to avoid contact with obstacles.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. be. For example, all or part of the device can be functionally or physically distributed and integrated into arbitrary units. In addition, new embodiments created by arbitrary combinations of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiment resulting from the combination have the effects of the original embodiment.

3a, 3b, 3c, 3d Imaging device 7 Display section 10 Display control device 34 Image generation section 35 Trajectory estimation section 36 Display control section I Bird's eye image L1, L2, L3 Overhang trajectory T Judgment line V Vehicle

Claims (3)

an image generation unit that generates a bird's-eye view image including the vehicle from captured images of the surroundings of the vehicle by a plurality of imaging devices provided at a position higher than the line of sight of a driver of the vehicle;
a trajectory estimation unit that estimates a travel trajectory of the vehicle when the vehicle turns;
a display control unit that superimposes the travel trajectory estimated by the trajectory estimation unit on the bird's-eye image generated by the image generation unit and displays the same on a display unit;
Equipped with
The display control section includes:
displaying a predetermined line superimposed on the bird's-eye view image along the traveling direction of the vehicle;
A display control device that highlights a portion of the travel trajectory that occurs when the vehicle turns that conflicts with the line . The display control unit causes the display unit to display a superimposed trajectory of an inner wheel difference, an outer wheel difference, and a rear overhang of the vehicle on the bird's-eye view image that occurs when the vehicle turns.
The display control device according to claim 1. The display control unit changes a display area of the bird's-eye view image on the display unit according to a direction in which the vehicle turns.
The display control device according to claim 2.

Images