JP7477015B1 - Display control device and display control method - Google Patents

Abstract

[Problem] Displaying a marker at an appropriate timing. [Solution] A display control device 30 has a reception unit 321 that receives an operation to move the vehicle in reverse, an acquisition unit 322 that acquires an image of the vehicle in the reverse direction, and a display control unit 323 that causes a display unit 20 to display a superimposed image in which a marker indicating the height of the vehicle is superimposed on the captured image at a first time when a first period of time has elapsed since the time when the reception unit 321 received the operation, and that causes a captured image without the superimposed marker to be displayed from the second time onward if the vehicle is moving between the first time and a second time when a second period of time has elapsed. [Selected Figure] Figure 1

Description

The present invention relates to a display control device and a display control method.

The driving assistance device in Patent Document 1 displays a marker indicating the vehicle width superimposed on the image behind the vehicle when the vehicle backs up to park.

JP 2001-6097 A

Conventional driving assistance devices always display a marker when the vehicle is reversing. As a result, even when parking in a position where the vehicle is unlikely to come into contact with an obstacle, the marker may be displayed in the rear view image, which can be annoying to the driver, or the driver may lose attention to the marker and miss the marker even when there is a possibility that the vehicle may come into contact with an obstacle.

Therefore, the present invention was made in consideration of these points, and aims to display a marker at the appropriate timing.

The display control device according to the first aspect of the present invention includes a reception unit that receives an operation to move the vehicle in reverse, an acquisition unit that acquires an image of the vehicle in the reverse direction, and a display control unit that causes a display unit to display a superimposed image in which a marker indicating the height of the vehicle is superimposed on the image at a first time when a first hour has elapsed since the time when the reception unit received the operation, and causes the display unit to display the image without the superimposed marker from the second time onwards if the vehicle is moving between the first time and a second time when a second hour has elapsed.

If the vehicle is stopped between the first time and the second time, the display control unit may continue to display the superimposed image after the second time.

The acquisition unit may acquire obstacle information indicating whether or not there is an obstacle in the backward direction of the vehicle, and the display control unit may display the superimposed image at the first time if the obstacle information acquired by the acquisition unit at the first time indicates that there is an obstacle.

The display control unit may display the superimposed image with the marker superimposed in an area of the captured image that corresponds to the outline of the upper rear end of the vehicle.

The acquisition unit may acquire the captured image captured using a wide-angle lens, and the display control unit may display the superimposed image on which the marker corresponding to the distortion aberration of the wide-angle lens is superimposed.

The acquisition unit may acquire the captured image captured using a fisheye lens, and the display control unit may display the superimposed image on which the marker corresponding to the barrel aberration of the fisheye lens is superimposed.

The display control unit may display the superimposed image in which the marker having a length corresponding to the vehicle width is superimposed.

The display control method according to the second aspect of the present invention includes a receiving step executed by a processor to receive an operation to move the vehicle in reverse, an acquisition step to acquire an image of the vehicle in the reverse direction, and a display control step to display, on a display unit, a superimposed image in which a marker indicating the height of the vehicle is superimposed on the image at a first time when a first hour has elapsed since the time when the operation was received in the reception step, and, if the vehicle is moving between the first time and a second time when a second hour has elapsed, to display the image without the superimposed marker after the second time.

The present invention has the effect of displaying a marker at an appropriate time.

1 is a diagram for explaining an overview of a vehicle S according to an embodiment of the present invention; FIG. 13 is a diagram for explaining a superimposed image. 1 shows a superimposed image in which a marker M is superimposed on an image captured using a fisheye lens. 10 is a diagram showing an example of a processing sequence in the display control device 30. FIG.

<Overview of Vehicle S>
Fig. 1 is a diagram for explaining an overview of a vehicle S according to this embodiment. The vehicle S shown in Fig. 1 includes a shift lever 10, a camera 11, a speed sensor 12, an obstacle sensor 13, a display unit 20, and a display control device 30. The vehicle S has a function of displaying a superimposed image in which markers indicating the height and width of the vehicle S are superimposed on an image captured in the backward direction of the vehicle S when the vehicle S reverses.

The shift lever 10 is a lever with which the driver operates the transmission. The camera 11 is provided at the upper rear of the vehicle S, and is an imaging device for generating captured images of the vehicle S in the backward direction using a wide-angle lens or a fisheye lens. The speed sensor 12 is a sensor for detecting the speed of the vehicle S. The obstacle sensor 13 is a sensor for detecting obstacles in the backward direction of the vehicle S, and is, for example, a sonar sensor or an ultrasonic sensor. Obstacles include obstacles that may come into contact with the top end of the vehicle S, such as eaves. The display unit 20 is a display for displaying captured images or superimposed images.

When the vehicle S reverses, the display control device 30 executes a process of displaying on the display unit 20 a superimposed image in which markers indicating the height and width of the vehicle S are superimposed on an image captured by the camera 11 in the reverse direction of the vehicle S. The display control device 30 may have a housing including electronic components, or may be a printed circuit board on which electronic components are mounted. Details of the display control device 30 will be described later.

Figure 2 is a diagram for explaining the superimposed image. In Figure 2, vehicle S, area P captured by camera 11, and marker M indicating height H and width W of vehicle S are shown. Both ends of marker M indicate the position of the side of vehicle S. Distance D is the distance between camera 11 and area P, and is, for example, 3 meters from the rear of vehicle S. Display control device 30 superimposes marker M on an area corresponding to the outline of the upper rear edge of vehicle S in the captured image of area P, and displays it on display unit 20.

If the marker M is always displayed when the vehicle S is reversing, the driver of the vehicle S may feel annoyed because the marker M is displayed even when there is no possibility that the vehicle S will come into contact with an obstacle. In addition, by always displaying the marker M, the driver's attention to the marker M may be reduced, and the driver may overlook the marker M even when there is a possibility that the vehicle S will come into contact with an obstacle.

Therefore, if the vehicle S continues to reverse during a predetermined time (e.g., 5 seconds) from the time when the superimposed image with the marker M is displayed, the display control device 30 makes the superimposed image invisible and displays the captured image. By operating in this manner, if the vehicle S continues to reverse, it is possible to determine that there are no obstacles in the vicinity that the vehicle S may come into contact with and that the driver considers the marker M unnecessary, and to make the marker M invisible. As a result, the display of the marker M allows the driver to reverse while being aware of obstacles behind the vehicle, and, if the driver considers that the display of the marker M is unnecessary, the marker M is hidden, so that the driver does not feel bothered by the marker M.
The configuration and operation of the display control device 30 will be described in detail below.

<Configuration of display control device 30>
The display control device 30 includes a storage unit 31 and a control unit 32. The control unit 32 includes a reception unit 321, an acquisition unit 322, and a display control unit 323.

The storage unit 31 has a storage medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), or an SSD (Solid State Drive). The storage unit 31 stores a program executed by the control unit 32. The storage unit 31 stores various information for the control unit 32 to display the captured image or superimposed image on the display unit 20.

The control unit 32 is, for example, a processor such as a CPU (Central Processing Unit) or an ECU (Electronic Control Unit). The control unit 32 executes a program stored in the storage unit 31 to function as a reception unit 321, an acquisition unit 322, and a display control unit 323. The control unit 32 may be configured with one processor, or may be configured with multiple processors or a combination of one or more processors and an electronic circuit.
The configuration of each unit realized by the control unit 32 will be described below.

The reception unit 321 receives an operation to move the vehicle S in reverse. For example, the reception unit 321 receives that the driver has operated the shift lever 10 to select a gear for moving the vehicle S in reverse (so-called reverse gear).

The acquisition unit 322 acquires an image of the vehicle S in the backward direction from the camera 11. For example, the acquisition unit 322 acquires an image of the area P shown in FIG. 2 captured by the camera 11 using a wide-angle lens or a fisheye lens every time a certain period of time elapses. The acquisition unit 322 acquires the vehicle speed of the vehicle S from the speed sensor 12 every time a certain period of time elapses. The acquisition unit 322 acquires obstacle information from the obstacle sensor 13 indicating whether or not there is an obstacle in the backward direction of the vehicle S. For example, the acquisition unit 322 acquires a notification indicating that the obstacle sensor 13 has detected an obstacle as the obstacle information.

The display control unit 323 acquires a captured image from the acquisition unit 322 every time a certain period of time has elapsed, and causes the display unit 20 to display the captured image or a superimposed image in which the marker M is superimposed on the captured image. The display control unit 323 causes the display unit 20 to display a superimposed image in which the marker M indicating the height of the vehicle S is superimposed on the captured image at a first time when a first time has elapsed since the time when the acceptance unit 321 accepted an operation to reverse the vehicle S. The first time is the time from when the driver operates the shift lever 10 to when the driver checks for safety to the left rear and right rear and starts reverse the vehicle S, and is, for example, 15 seconds.

For example, the display control unit 323 displays the captured image at the time when the receiving unit 321 receives an operation to reverse the vehicle S, and displays a superimposed image at the first time if the obstacle information acquired by the acquisition unit 322 at the first time indicates the presence of an obstacle. That is, the display control unit 323 displays a superimposed image at the first time if the obstacle sensor 13 detects an obstacle at the first time, and continues to display the captured image if the obstacle sensor 13 does not detect an obstacle at the first time. By the display control unit 323 operating in this manner, the marker M can be displayed only when there is an obstacle behind the vehicle S. As a result, the driver can be more easily aware of the presence of an obstacle behind the vehicle S.

If the vehicle S is moving between the first time and the second time when the second time has elapsed, the display control unit 323 displays the captured image without the marker M superimposed thereon from the second time onwards. The second time is the time from when the driver checks the superimposed image to when he or she decides whether or not to reverse the vehicle S, and is, for example, 5 seconds. For example, if the vehicle speed acquired by the acquisition unit 322 at the second time continues to be greater than 0 for a predetermined time or more, the display control unit 323 determines that the vehicle S is moving, and switches the superimposed image to the captured image at the second time. The predetermined time is stored in the storage unit 31 and is a fixed value equal to or less than the second time. By operating in this manner, the display control unit 323 can hide the marker M when the driver determines that there is no possibility of contact with an obstacle and starts reverse the vehicle S.

On the other hand, for example, if the vehicle S is stopped between the first time and the second time, the display control unit 323 continues to display the superimposed image after the second time. For example, if the vehicle speed acquired by the acquisition unit 322 at the second time is greater than 0 for less than a predetermined time or the vehicle speed remains 0 for the second time, the display control unit 323 determines that the vehicle S is stopped and continues to display the superimposed image after the second time. By operating in this manner, the display control unit 323 can continue to display the marker M if the driver stops the vehicle S after determining that there is a possibility of contact with an obstacle. Also, the driver can continue to display the marker M by stopping the vehicle S.

If the vehicle S is a large vehicle such as a truck, there is a possibility that the vehicle S may come into contact with an obstacle installed at a high place such as a eaves or a signboard when backing up. Therefore, the display control unit 323 displays a superimposed image in which a marker M of a length corresponding to the vehicle width of the vehicle S is superimposed in an area corresponding to the outline of the upper rear end of the vehicle S in the captured image.

For example, when the area of the captured image is the area P shown in FIG. 2, the display control unit 323 identifies the area of the upper end of the outline of the rear of the vehicle S when the vehicle S has retreated to the position of the area P (i.e., when the vehicle S has retreated a distance D). The display control unit 323 superimposes the marker M on the area of the upper end of the identified outline in the captured image (i.e., the area of the marker M in the area P shown in FIG. 2). By operating in this manner, the display control unit 323 can indicate whether or not there is a possibility of contacting an obstacle installed at a high place. Furthermore, since both ends of the marker M indicate the positions of the sides of the vehicle S, the length of the marker M corresponds to the vehicle width of the vehicle S. Therefore, by superimposing the marker M, the driver can determine whether or not an obstacle will come into contact with the side of the vehicle S without superimposing the marker M on the area corresponding to the outline of the side of the vehicle S, and therefore the area in which the marker M is displayed can be made smaller.

When the camera 11 captures an image using a wide-angle lens, the display control unit 323 displays a superimposed image on which a marker corresponding to the distortion aberration of the wide-angle lens is superimposed, and when the camera 11 captures an image using a fisheye lens, the display control unit 323 displays a superimposed image on which a marker corresponding to the barrel aberration of the fisheye lens is superimposed. For example, the display control unit 323 creates an image in which the marker M shown in FIG. 2 is distorted by referring to the distortion aberration or barrel aberration stored in the memory unit 31, and superimposes the image on the captured image.

Figure 3 shows a superimposed image in which the marker M is superimposed on an image captured using a fisheye lens. Figure 3 shows area B showing the rear of the vehicle S, area G showing the floor surface in the reverse direction of the vehicle S, area L and area R showing the side walls in the reverse direction of the vehicle S, area F showing the front wall in the reverse direction of the vehicle S, the eaves E, and the marker M. As shown in Figure 3, the display control unit 323 creates an image in which the marker M corresponding to the width of the vehicle S is distorted, and superimposes it on the captured image in an area corresponding to the outline of the upper end of the rear of the vehicle S. By the display control unit 323 operating in this manner, the marker M distorted according to the aberration of the lens can be superimposed. As a result, the driver can easily grasp the distance between the obstacle and the vehicle S.

<Processing sequence in the display control device 30>
Fig. 4 is a diagram showing an example of a processing sequence in the display control device 30. The processing sequence shown in Fig. 4 shows an operation of displaying a captured image or a superimposed image on the display unit 20 when the vehicle S moves backward.

First, the reception unit 321 receives an operation for moving the vehicle S in reverse (S11). If the first time has not elapsed since the reception unit 321 received the operation (NO in S12), the display control device 30 repeats the process of checking whether the first time has elapsed (processing in S12).

If the first time has elapsed (YES in S12), the display control unit 323 executes a process of displaying a superimposed image or a captured image on the display unit 20 at the first time when the first time has elapsed since the time when the reception unit 321 received the operation (i.e., the processes from S13 to S16). For example, the display control unit 323 acquires a captured image of the vehicle S in the backward direction acquired by the acquisition unit 322 (S13). Next, if the acquisition unit 322 has not acquired obstacle information indicating that there is an obstacle in the backward direction of the vehicle S at the first time, the display control unit 323 determines that there is no obstacle (NO in S14), and displays the captured image acquired by the acquisition unit 322 on the display unit 20 after the first time (S15). On the other hand, if the acquisition unit 322 acquires obstacle information at the first time, the display control unit 323 determines that there is an obstacle (YES in S14), and displays a superimposed image in which the marker M is superimposed on the captured image on the display unit 20 after the first time (S16).

If the second time has not passed since the first time when the first time has passed (NO in S17), the display control device 30 repeats the operations from S13 to S16. That is, the display control unit 323 repeats the process of determining whether or not there is an obstacle at the second time and displaying the captured image or the superimposed image on the display unit 20. If the second time has passed (YES in S17), the display control unit 323 determines whether or not the vehicle S was stopped at the second time (S18). If the vehicle S was stopped at the second time (YES in S18), the display control unit 323 continues to display the superimposed image from the second time when the second time has passed since the first time (S19). On the other hand, if the vehicle S is not stopped at the second time (NO in S18), the display control unit 323 displays the captured image at the second time (S15).

<First Modification>
In the above description, an example has been given of an operation in which whether or not the vehicle S has moved at the second time and whether or not the vehicle S has stopped at the second time are determined using the same predetermined time, but this is not limiting. The display control unit 323 may determine whether or not the vehicle S has moved using a first predetermined time, and may determine whether or not the vehicle S has stopped using a second predetermined time that is different from the first predetermined time.

<Second Modification>
In the above description, an example has been given of an operation in which the display control unit 323 displays a captured image or a superimposed image on the display unit 20 when the reception unit 321 receives an operation to move the vehicle S in reverse, but the present invention is not limited to this. The display control unit 323 may also display a captured image or a superimposed image on the display unit 20 when the reception unit 321 receives an operation of a foot brake (not shown) provided on the vehicle S.

<Third Modification>
In the above description, an example of the operation of determining whether the vehicle S is moving or stopped at the second time based on the vehicle speed detected by the speed sensor 12 has been described, but the present invention is not limited to this. The display control unit 323 may determine whether the vehicle S is moving or stopped based on the time when the driver operated the foot brake, which is received by the receiving unit 321. The acquisition unit 322 may also obtain the vehicle speed from another ECU (not shown) that has acquired the vehicle speed detected by the speed sensor 12, and determine whether the vehicle S is moving or stopped.

<Effects of the display control device 30>
As described above, the display control device 30 has a reception unit 321 that receives an operation to cause the vehicle S to reverse, an acquisition unit 322 that acquires an image of the vehicle in the reverse direction, and a display control unit 323 that causes the display unit 20 to display a superimposed image in which a marker M indicating the height of the vehicle S is superimposed on the image at a first time when a first hour has elapsed since the time when the reception unit 321 received the operation, and that causes the display unit 20 to display an image without the marker M superimposed on the image if the vehicle S is moving between the first time and a second time when a second hour has elapsed.

By configuring the display control device 30 in this manner, after the superimposed image with the marker M superimposed thereon is displayed on the display unit 20, if the vehicle S is moving, the marker M can be hidden. As a result, if the driver performs an action that does not require checking the area of the marker M because there is no obstacle to come into contact with when reversing, the marker M can be hidden, so that the marker M can be displayed at an appropriate timing.

Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination also has the effect of the original embodiment.

REFERENCE SIGNS LIST 10 Shift lever 11 Camera 12 Speed sensor 13 Obstacle sensor 20 Display unit 30 Display control device 31 Storage unit 32 Control unit 321 Reception unit 322 Acquisition unit 323 Display control unit

Claims (8)

A reception unit that receives an operation to move the vehicle backward;
An acquisition unit that acquires an image of the vehicle in a backward direction;
a display control unit that causes a display unit to display a superimposed image in which a marker indicating a height of the vehicle is superimposed on the captured image at a first time when a first hour has elapsed since the time when the reception unit received the operation, and, if the vehicle is moving during a period from the first time to a second time when a second hour has elapsed, causes a display control unit to display the captured image without the superimposed marker after the second time;
A display control device having the following. the display control unit, when the vehicle is stopped between the first time point and the second time point, continues displaying the superimposed image even after the second time point.
The display control device according to claim 1 . the acquisition unit acquires obstacle information indicating whether or not there is an obstacle in a backward direction of the vehicle,
the display control unit causes the superimposed image to be displayed at the first time when the obstacle information acquired by the acquisition unit at the first time indicates the presence of the obstacle.
The display control device according to claim 1 . the display control unit displays the superimposed image in which the marker is superimposed in an area corresponding to an outline of an upper end of a rear portion of the vehicle in the captured image.
The display control device according to claim 1 . The acquisition unit acquires the captured image captured using a wide-angle lens,
the display control unit displays the superimposed image on which the marker corresponding to the distortion aberration of the wide-angle lens is superimposed.
The display control device according to claim 4. The acquisition unit acquires the captured image captured using a fisheye lens,
the display control unit displays the superimposed image on which the marker corresponding to the barrel aberration of the fisheye lens is superimposed.
The display control device according to claim 4. The display control unit displays the superimposed image on which the marker having a length corresponding to a vehicle width of the vehicle is superimposed.
The display control device according to claim 1 . The processor executes
a receiving step of receiving an operation to move the vehicle in reverse;
an acquisition step of acquiring an image of the vehicle in a backward direction;
a display control step of displaying, on a display unit, a superimposed image in which a marker indicating the height of the vehicle is superimposed on the captured image at a first time when a first period of time has elapsed since the time when the operation is accepted in the accepting step, and, if the vehicle is moving during a period of time from the first time to a second time when a second period of time has elapsed, displaying the captured image without the superimposed marker after the second time;
A display control method comprising:

Images