JP2022133849A - Display control device, display control method, and program - Google Patents

Abstract

To provide a display control device, a method, and a program, which can appropriately call attention.SOLUTION: A display control device includes: an image data acquisition section 121 that acquires image data of an image obtained by capturing the outside of an own vehicle; an object detection section 122 that detects an object included in the image; a distance measurement data acquisition section 125 that acquires distance measurement data on distance from the own vehicle to the object; a risk degree setting section 127 that sets a degree of risk to the object according to the size of the object when the distance is a threshold or less; a highlighting processing section 128 that performs highlighting processing according to the degree of risk on the image data; and an output section 129 that outputs a highlighted image obtained by performing the highlighting processing.SELECTED DRAWING: Figure 2

Description

The present invention relates to a display control device, a display control method and a program.

Patent Literature 1 discloses a perimeter monitoring device having a distance detection unit that detects the distance from a vehicle to an object. The perimeter monitoring device of Patent Document 1 has a camera that captures an image of an object from an imaging unit provided in a vehicle. The perimeter monitoring device superimposes the color of the object on the color corresponding to the height of the object.

Japanese Patent Laid-Open No. 2002-201001 discloses a vehicle surroundings visual recognition device that includes obstacle detection means for detecting a distance between a vehicle and an obstacle, and an image capturing section for capturing an image of a landscape in the outward direction of the vehicle. When an obstacle is detected, the apparatus of Patent Document 2 displays a figure in the imaged image so that the direction in which the obstacle is located can be recognized. This device changes the display color or size of the figure according to the distance to the obstacle.

JP 2016-97843 A Japanese Patent Application Laid-Open No. 2006-311222

In such a device, it is desirable to appropriately call attention to the driver. For example, if multiple obstacles are involved, it is preferable to call attention to the more dangerous objects. However, when a plurality of objects exist around the vehicle, changing the display color according to the distance or height may make the display complicated. For example, if an object with a low degree of danger is emphasized, it becomes impossible to effectively draw attention to an object with a high degree of danger.

In view of the above problems, an object of the present invention is to provide a display control device, a display control method, and a program that can effectively call attention to the user by appropriately setting the degree of risk.

A display control device according to the present embodiment includes an image data acquisition unit that acquires image data of an image of an image of the outside of a vehicle; an object detection unit that detects an object included in the image; a distance measurement data acquisition unit that acquires distance measurement data relating to a distance to, a risk setting unit that sets a risk level for the object according to the size of the object when the distance is equal to or less than a threshold; A highlighting processing unit that performs highlighting processing on image data according to the degree of risk, and an output unit that outputs a highlighting image subjected to the highlighting processing are provided.

A display control method according to the present embodiment includes the steps of acquiring image data of an image of the exterior of a vehicle, detecting an object included in the image, and measuring a distance from the vehicle to the object. obtaining distance data; setting a risk level to the object according to the size of the object when the distance is equal to or less than a threshold; and a step of outputting the highlighted image on which the highlighting process has been performed.

A program according to the present embodiment includes the steps of acquiring image data of an image of the exterior of a vehicle, detecting an object included in the image, and ranging data relating to the distance from the vehicle to the object. setting a risk level for the object according to the size of the object when the distance is equal to or less than a threshold; and performing highlighting processing according to the risk level on image data. and a step of outputting the highlighted image on which the highlighting process has been performed are executed by a computer.

Advantageous Effects of Invention According to the present invention, it is possible to provide a display control device, a display control method, and a program that can effectively call attention to a user by appropriately setting the degree of risk.

1 is a diagram showing a vehicle according to an embodiment; FIG. It is a figure which shows the display control apparatus concerning embodiment. It is a color bar showing a display color according to the degree of risk. 4 is a flow chart showing a display control method according to the first embodiment; 4 is a schematic diagram for explaining display control according to the first embodiment; FIG. 4 is a schematic diagram for explaining display control according to the first embodiment; FIG. 4 is a schematic diagram for explaining display control according to the first embodiment; FIG. 9 is a flow chart showing a display control method according to the second embodiment; FIG. 10 is a schematic diagram for explaining display control according to the second embodiment; FIG. FIG. 10 is a schematic diagram for explaining another display control method;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described through embodiments of the invention, but the invention according to the scope of claims is not limited to the following embodiments. Moreover, not all the configurations described in the embodiments are essential as means for solving the problems. For clarity of explanation, the following descriptions and drawings are omitted and simplified as appropriate. In each drawing, the same elements are denoted by the same reference numerals, and redundant description is omitted as necessary.

<Embodiment 1>
FIG. 1 is a diagram showing a vehicle 1 according to Embodiment 1. FIG. A vehicle 1 has a sensor unit 2 and a display control device 100 . In the following description, in order to distinguish the vehicle 1 from surrounding vehicles, the vehicle 1 may be referred to as the subject vehicle and the surrounding vehicles may be referred to as peripheral vehicles.

The sensor unit 2 has at least one imaging device and images the exterior of the own vehicle. The sensor unit 2 outputs image data captured by the imaging device to the display control device 100 . The sensor unit 2 has a ranging sensor that detects obstacles. The sensor unit 2 outputs ranging data measured by the ranging sensor to the display control device 100 .

The display control device 100 can be installed at any position on the vehicle 1 . The display control device 100 can be connected to a CAN (Controller Area Network). The display control device 100 performs image processing for detecting objects (obstacles) in the image captured by the sensor unit 2 . The display control device 100 performs highlighting to call the user's attention. For example, the display control device 100 gives an emphasis color such as red to the object included in the image. This makes it easier for a user such as a driver to recognize the presence of the object and avoid contact. The display control device 100 will be described later.

Hereinafter, the term "image" also means "image data representing an image" as a processing target in information processing. Similarly, hereinafter, the term "image" also means "image data representing an image" as a processing target in information processing. Note that the images and videos may be moving images or continuous still images.

FIG. 2 is a block diagram showing the configuration of the display control device 100 and the display control system 10 having the display control device 100 according to the first embodiment. The display control system 10 has a sensor unit 2 , a display section 30 , an ECU (Electronic Control Unit) 40 , and a display control device 100 . The display control device 100 is communicably connected to each of the sensor unit 2, the display section 30, and the ECU 40. FIG.

The display unit 30 is, for example, a display. The display unit 30 displays images under the control of the display control device 100 . Note that the display unit 30 may include a speaker. In this case, the display unit 30 may output audio from the speaker. Moreover, when the display control device 100 is mounted in the vehicle 1 , the display unit 30 may be provided inside the vehicle 1 at a position where the driver of the vehicle can visually recognize the vehicle while driving. In this case, the display unit 30 may be implemented by the interface unit 108, which will be described later. The display unit 30 may be a monitor of a car navigation system. Moreover, the display unit 30 does not have to be an in-vehicle device permanently installed in the vehicle 1 . The display unit 30 may be configured by, for example, a display of an information terminal such as a smartphone or tablet terminal owned by the user. Furthermore, part of the display control processing may be performed by a smartphone having a display unit or the like.

The sensor unit 2 includes a front camera 21F and a rear camera 21R. When the vehicle 1 moves forward, the forward camera 21F captures an image forward of the vehicle 1 . The rear camera 21R captures an image behind the vehicle 1 when the vehicle 1 is backing up. The front camera 21F and the rear camera 21R generate image data of 30 frames per second (30 fps), for example, and supply the generated image data to the display control device 100 every 1/30th of a second. The shooting data is, for example, H.264. 264 or H.264. It may be generated using a scheme such as H.265. Alternatively, the sensor unit 2 may be an omnidirectional camera that captures images of the entire circumference of the vehicle 1 .

The sensor unit 2 has a distance sensor 22 for detecting the distance to objects around the vehicle 1 . A distance sensor 22 detects the distance from the vehicle to an object. The ranging sensor 22 is, for example, a LIDAR (Laser Imaging Detection and Ranging). Alternatively, the ranging sensor 22 may be an infrared camera, stereo camera, millimeter wave radar, or the like. Furthermore, the ranging sensor 22 may be configured by combining these sensors. Also, the distance measuring sensor 22 is not limited to a single physical device. For example, the distance measurement sensor 22 may be composed of LIDARs arranged at a plurality of locations on the vehicle 1 .

The distance measurement sensor 22 supplies the display control device 100 with distance measurement data regarding the distance from the own vehicle to the object. Furthermore, the ranging sensor 22 may detect the direction as well as the distance to the obstacle. That is, the ranging data may be associated with the distance to the object and the direction of the object. Here, the direction of the object is the direction with respect to the own vehicle. Further, the front, rear, left, and right directions of the object may be indicated by azimuth angles, and the vertical directions may be indicated by elevation angles.

The ECU 40 is included in the vehicle 1 and is part of a configuration that controls the vehicle 1 and the like. The display control device 100 and the ECU 40 are communicably connected via an in-vehicle network such as CAN.

The display control device 100 includes a control unit 102, a storage unit 104, a communication unit 106, and an interface unit 108 (IF: Interface). The display control device 100 includes a travel information acquisition unit 110, an image data acquisition unit 121, an object detection unit 122, a distance measurement data acquisition unit 125, a risk setting unit 127, a highlighting processing unit 128, and an output unit. 129. The travel information acquisition unit 110 includes a traveling direction acquisition unit 111 and a speed information acquisition unit 113 .

The control unit 102 is a processor such as a CPU (Central Processing Unit), for example. The control unit 102 has a function as an arithmetic device that performs control processing, arithmetic processing, and the like. The storage unit 104 is, for example, a storage device such as memory or hard disk. The storage unit 104 is, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 104 has a function of storing a control program, a calculation program, and the like executed by the control unit 102 or as a function of the control unit 102 . The storage unit 104 also has a function of temporarily storing processing data and the like.

The communication unit 106 performs processing necessary for communicating with the sensor unit 2 and the display unit 30 . Also, the communication unit 106 may perform processing necessary for communicating with a CAN (not shown). Communication unit 106 may include a communication port. The interface unit 108 is, for example, a user interface (UI). The interface unit 108 has an input device such as a keyboard, touch panel, or mouse, and an output device such as a speaker.

Therefore, the display control device 100 has a function as a computer. In addition, the display control device 100 executes a program stored in the storage unit 104 by the processor of the control unit 102, so that the traveling information acquisition unit 110, the image data acquisition unit 121, the object detection unit 122, the distance measurement data acquisition unit 125, a risk setting unit 127, a highlighting processing unit 128, and an output unit 129. Further, each component of the display control device 100 is not limited to being realized by software based on a program, and may be realized by any combination of hardware and software. Further, each component of the display control device 100 may be implemented using a user-programmable integrated circuit such as an FPGA (field-programmable gate array), a microcomputer, or an SOC (System on a chip). . In this case, this integrated circuit may be used to implement a program composed of the above components. This also applies to other embodiments described later.

The travel information acquisition unit 110 acquires travel information regarding travel of the vehicle 1 . The travel information acquisition unit 110 acquires various types of information from the ECU 40 . The travel information acquisition unit 110 includes a travel direction acquisition unit 111 and a speed information acquisition unit 113 .

The traveling direction acquisition unit 111 acquires information indicating the traveling direction of the vehicle 1 . For example, the traveling direction acquisition unit 111 detects whether the vehicle 1 is moving forward or backward based on the shift position of the vehicle 1 . When the shift position of the vehicle 1 becomes the reverse (R) position, it is detected that the vehicle 1 is moving backward. Alternatively, when the shift lever of the vehicle 1 is in the drive (D) position or the like, it is detected that the vehicle 1 is moving forward.

Further, the traveling direction acquisition unit 111 may acquire steering angle information as the traveling direction. The traveling direction acquisition unit 111 acquires steering angle information indicating the steering angle of the wheels of the vehicle 1 by acquiring a signal from CAN or the like. The steering angle information includes information indicating the steering direction, such as right or left, in addition to information indicating the steering angle.

The speed information acquisition unit 113 acquires speed information indicating the speed of the vehicle 1 . For example, the speed information acquiring section 113 detects the vehicle speed based on the vehicle speed pulse from the ECU 40 . The speed information acquisition unit 113 acquires speed information indicating a vehicle speed of 40 kilometers per hour, for example. Alternatively, the speed information may be graded information such as low speed, medium speed, high speed, and so on.

The image data acquisition unit 121 acquires image data captured by the front camera 21F and the rear camera 21R. Note that the image data acquisition unit 121 acquires the image data according to the traveling direction acquired by the traveling direction acquisition unit 111 . Images to be acquired may be switched. For example, when the vehicle 1 is moving forward, the image data acquisition unit 121 acquires the front image of the front camera 21F. When the vehicle 1 is backing up, the image data acquisition unit 121 acquires the rear image of the rear camera 21R. Of course, the image data acquisition unit 121 may acquire both the front image and the rear image. In this case, the display control device 100 may switch the image to be processed according to the traveling direction.

The object detection unit 122 detects objects included in the image. The object detection unit 122 recognizes an object in the image by performing image analysis on the image data. For example, the object detection unit 122 can perform object recognition by comparing a pre-registered pattern with an image. Also, when a plurality of objects are included in the image, the object detection unit 122 detects each object. Furthermore, the object detection unit 122 may specify the position coordinates and size of the object within the image.

Here, the objects are surrounding vehicles, walls, wheel chocks, viaducts, bicycles, and the like. Objects may also include people such as pedestrians. Note that a known method can be used for the object detection processing in the object detection unit 122, so detailed description thereof will be omitted.

The ranging data acquisition unit 125 acquires ranging data regarding the distance to the object. Also, if the image includes images of a plurality of objects, the distance measurement data acquisition unit 125 obtains the distance to each object. That is, based on the distance measurement data from the distance measurement sensor 22, the distance from the own vehicle to each object is calculated.

A risk setting unit 127 sets a risk to an object. Also, when the image includes images of a plurality of objects, the risk setting unit 127 sets the risk for each object. That is, a value indicating the degree of danger is attached to each object.

Specifically, the risk setting unit 127 sets the risk according to the possibility of the vehicle coming into contact with an object and the effect of contact. The risk setting unit 127 sets the risk according to the distance to the object. The risk level setting unit 127 expresses the value of the risk level, for example, in 10 levels, with 10 being the highest risk level and 0 being the lowest risk level. As for the risk value, the closer the object is, the higher the risk is, and the farther the object is, the lower the risk. In addition, the risk level setting unit 127 may set the risk level to 0 for an object with an extremely low possibility of contact. You may do so.

Alternatively, the risk setting unit 127 sets the risk according to the size of the object. The risk level setting unit 127 increases the risk level for larger-sized objects and decreases the risk level for smaller-sized objects. For example, the risk setting unit 127 can calculate the size of the object according to the size of the object image in the image and the distance to the object. Processing in the risk setting unit 127 will be described later. Here, the size of the object can be one or both of the size in the horizontal direction (horizontal direction) and the size in the vertical direction (vertical direction).

The highlighting processing unit 128 performs highlighting processing for emphasizing the object in the image according to the degree of risk. For example, the highlighting processing unit 128 superimposes a highlighting color such as red on the object in the image for an object with a high degree of danger. The highlighting processing unit 128 superimposes a non-emphasized color such as blue on the object in the image for a low-risk object. The highlighting processing unit 128 does not perform highlighting processing on non-dangerous objects, that is, objects that are unlikely to come into contact with the host vehicle. The highlighting processing unit 128 superimposes on the object in the image a color that emphasizes the higher the risk of the object, in order to display a warning to the user.

The degree of risk can be represented by a color bar of color scheme distribution as shown in FIG. The higher the degree of danger, the more red, that is, the longer wavelength side, and the lower the danger, the more blue, that is, the shorter wavelength side. The highlighting processing unit 128 superimposes a color corresponding to the degree of danger on the object in the image. For example, in the image, objects with high risk are superimposed in red, and objects with low risk are superimposed in blue. Green is superimposed on the object in the image for objects of intermediate risk. Then, the highlighting processing unit 128 generates a superimposed image in which a color corresponding to the degree of danger is superimposed on the object in the image. The output unit 129 outputs the image data so that the superimposed image is displayed on the display unit 30 as the highlighted image.

By changing the superimposed color according to the degree of danger in this way, it is possible to appropriately call the user's attention. That is, the higher the degree of danger, the more emphasized the object is displayed, so that the user can easily recognize the object with the higher degree of danger. A warning display can be performed by superimposing an emphasis color on an object in an image. A user can drive with more attention to high-risk objects than to low-risk objects. It is possible to appropriately prompt the user's attention to the object. Therefore, it can contribute to safer driving.

The highlighting is not limited to the color scheme distribution shown in FIG. For example, the degree of danger may be indicated by shading. For example, the same color may be applied to the image of the object, and only the shade may be changed according to the degree of danger. The highlighting processing unit 128 may perform highlighting processing by changing the color tone, shade, brightness, transparency, etc. of the object image. Further, the highlighting processing unit 128 may perform highlighting processing by adding a contour line to the object. In this case, the outline of an object with a higher degree of danger is made thicker. In this manner, the highlighting processing unit 128 performs highlighting processing such that the higher the degree of risk, the more the risk is emphasized.

Examples of risk setting processing and highlighting processing will be described with reference to FIGS. 4 to 7. FIG. FIG. 4 is a flowchart showing display control processing according to this embodiment. 5 to 7 are diagrams for explaining the degree of danger when reversing and its highlighting process. 5 to 7 show examples in which the vehicle 1 is parked in the parking space 300 in reverse. 5 to 7 show, from top to bottom, a superimposed image, a color scheme according to the degree of risk, a camera image, and a top view. As the vehicle 1 moves backward at low speed, the states change in the order of FIGS. 5, 6 and 7. FIG.

A rear wall 301 is arranged behind the parking space 300, and side walls 302 are arranged on the left and right. That is, three sides of the parking space 300 are surrounded by the rear wall 301 and the side walls 302 . Then, the vehicle retreats into the parking space 300 partitioned by the rear wall 301 and the side wall 302 . Wheel stoppers 304 are arranged on the road surface of the parking space 300 . The rear camera 21R is imaging the parking space 300. FIG. Distance sensor 22 detects the distance between rear wall 301 and side wall 302 as the distance to the object.

First, the image data acquisition unit 121 acquires image data of an image captured by the rear camera 21R (S401). The object detection unit 122 detects an object included in the image (S402). For example, the object detection unit 122 detects each of the rear wall 301, the side wall 302, and the wheel stop 304 included in the image. Distance measurement data measured by the distance measurement sensor 22 is acquired (S403). Thereby, the distances from the vehicle 1 to the rear wall 301, the side wall 302, and the wheel stop 304 are calculated. The risk setting unit 127 detects the size of the object based on the distance measurement data and the image data (S404). As a result, the distance and size of each object are detected.

Next, the risk setting unit 127 determines whether or not the distance from the vehicle to the object is equal to or less than the threshold distance (S405). The risk setting unit 127 compares, for example, the distance to the closest object with the threshold distance. When multiple objects are included in the image, the risk setting unit 127 determines whether or not the distance to the closest object is equal to or less than the threshold distance. Here, the distance to the rear wall 301 is compared with the threshold distance.

If the distance to the object is not equal to or less than the threshold distance (NO in S405), the risk setting unit 127 sets the risk according to the distance (S406). That is, the closer the distance to the object is, the higher the risk setting unit 127 increases the risk of the object.

If the distance to the object is equal to or less than the threshold distance (YES in S405), the risk setting unit 127 sets the risk according to the size of the object (S407). In other words, the larger the object, the higher the risk setting unit 127 sets the risk.

Then, the highlighting processing unit 128 performs highlighting processing according to the degree of risk (S408). That is, the highlighting processing unit 128 generates a superimposed image by superimposing a red color on an object with a high degree of danger in the image. The output unit 129 outputs the superimposed image subjected to the highlighting process to the display unit 30 (S409). The process is repeated while the vehicle 1 is backing up. As a result, the superimposed images shown in FIGS. 5 to 7 are displayed.

For example, in Figures 5 and 6, the distance to the rear wall 301 is greater than the threshold distance. Therefore, as in S406, the risk setting unit 127 sets the risk according to the distance to the object. As shown in FIG. 5, when the distance to the rear wall 301 is long, the degree of danger is low. Then, as the vehicle 1 moves backward, the vehicle 1 approaches the rear wall 301 and the side walls 302 . Therefore, the degree of danger gradually increases. In the state shown in FIG. 6, since the vehicle 1 is closer to the rear wall 301 and side walls 302 than in the state shown in FIG. Therefore, as the vehicle 1 retreats from the positions shown in FIGS. 5 to 6, the color of the superimposed image changes.

On the other hand, in FIG. 7, the distance to the rear wall 301 is less than or equal to the threshold distance. Therefore, as in S407, the risk setting unit 127 sets the risk according to the size of the object. Since the rear wall 301 and side wall 302 are larger than the predetermined size, the highest risk is set. In the superimposed image of FIG. 7, red color indicating the highest degree of danger is superimposed on the rear wall 301 and side walls 302 .

In this way, when the distance to the rear wall 301 becomes equal to or less than the threshold distance, the risk setting unit 127 sets the risk according to the size of the object. By doing so, the user can easily recognize a large object with a high degree of danger. For example, when the degree of danger changes only according to the distance, the same degree of danger is set regardless of the size of the object. Therefore, it is difficult to call attention to a large object at a distance from the vehicle 1 . In contrast, according to the present embodiment, when the object is closer than the threshold distance, the risk level setting unit 127 sets the risk level according to the size of the object. When the vehicle 1 comes into contact with a large object, the effect is great. In the present embodiment, the danger level of a large object can be set high, so it is possible to call attention to a large object that is distant from the vehicle 1 .

In this way, when the vehicle 1 approaches the object to the threshold distance, the risk setting unit 127 sets the risk according to the size of the object. Therefore, the user can appropriately recognize a large object with a high degree of danger. The driver, who is the user, can slow down and park appropriately. Thereby, driving can be assisted effectively. Further, when the vehicle 1 is separated from the object by the threshold distance or more, the risk level setting unit 127 sets the risk level according to the distance to the object. Therefore, since the degree of danger changes according to the approach to the object, the user can recognize the distance to the object. By doing so, it is possible to appropriately set the degree of risk according to the situation, so that it is possible to effectively call attention to the user.

Note that in step S407, if the size of the object closer than the threshold distance is smaller than a predetermined size, the risk level setting unit 127 may set according to the distance, as in step S405. That is, as the distance becomes shorter, the risk setting unit 127 increases the risk of small-sized objects. In other words, if the size of an object that is closer than the threshold distance is smaller than a predetermined size, the degree of risk may be medium. Furthermore, the size of the object may be divided into three or more stages, and the degree of danger may be set in multiple stages.

Further, the risk setting unit 127 may exclude objects smaller than a predetermined size from being compared with the threshold distance in S405. In other words, the risk setting unit 127 compares the distance to the object closest to the vehicle 1 among objects having a certain size or larger with the threshold distance. Specifically, the wheel chock 304 is Since the object is smaller than the predetermined size, it is not compared in step S404.

Furthermore, the degree of danger may be set to 0 for objects that are unlikely or unlikely to come into contact with the vehicle 1 . For example, the risk setting unit 127 calculates height information indicating the height of the object as the size of the object. If the height of the object is less than the ground clearance of the vehicle 1, the possibility of contacting the vehicle 1 is small. For example, the height of the wheel chock 304 is lower than the minimum ground clearance of the vehicle 1, so the degree of danger is small. Therefore, the wheel stopper 304 may be excluded from the targets of the comparison processing in step S404.

Note that the threshold distance may be determined according to the size of the vehicle 1 . For example, a 4t truck or the like has a long vehicle length of about 8m. Therefore, the threshold distance from the object to the vehicle is about 4m. Further, for example, in a small/ordinary passenger car or the like, the overall length is as short as about 4.70 m or less. Thus, the threshold distance from the object to the vehicle can be about 2m. Alternatively, if the depth of the parking space 300 is approximately 10% of the total length of the vehicle, the threshold distance may be less than the total length of the vehicle. Also, the threshold distance may be determined according to the width of the vehicle 1 .

<Embodiment 2>
In Embodiment 2, the risk setting unit 127 performs risk setting processing according to travel information. More specifically, the setting process performed by the risk setting unit 127 is switched according to the traveling direction. For example, while the vehicle 1 is moving forward, the risk level setting unit 127 performs the first setting process. While the vehicle 1 is backing up, the risk setting unit 127 performs a second setting process different from the first setting process. Note that the configurations of the vehicle 1 and the display control system 10 are the same as those shown in FIGS.

FIG. 8 is a flowchart showing display control processing in this embodiment. First, the travel information acquisition unit 110 acquires travel information from the ECU 40 (S801). That is, the traveling direction acquisition unit 111 acquires the traveling direction. Furthermore, the speed information acquisition unit 113 may acquire the speed information.

The image data acquisition unit 121 acquires the image data of the image captured by the rear camera 21R (S802). The object detection unit 122 detects an object included in the image (S803). Distance measurement data measured by the distance measurement sensor 22 is acquired (S804). The risk setting unit 127 detects the size of the object based on the distance measurement data and the image data (S805). The processing of steps S802 to S805 is basically the same as the processing of steps S401 to S404 in the first embodiment, so description thereof will be omitted. However, step S802 is different from step S401 in that the image data acquisition unit 121 acquires the image data of the front image from the front camera 21F and the rear image from the rear camera 21R.

Next, it is determined whether or not the vehicle 1 is moving forward (S806). If the vehicle 1 is moving forward (YES in S806), the risk setting unit 127 sets the risk by the first setting process (S807). The first setting process will be described later.

If the vehicle 1 is not moving forward (NO in S806), the risk level setting unit 127 sets the risk level by the second setting process (S808). Note that the second setting process is different from the first setting process. When the vehicle 1 is in reverse, the risk level setting unit 127 sets the risk level by a second setting process different from the first setting process. For the second setting process, for example, the technique shown in the first embodiment can be used. That is, in the second setting process, the risk setting unit 127 sets the risk according to the size and distance of the object.

Then, the highlighting processing unit 128 performs highlighting processing according to the degree of risk (S809). That is, the highlighting processing unit 128 generates a superimposed image by superimposing a red color on an object with a high degree of danger in the image. The output unit 129 outputs the superimposed image subjected to the highlighting process to the display unit 30 (S810).

An example of the first setting process will be described below with reference to FIG. FIG. 9 shows a camera image of the road 310 in front of the vehicle and a superimposed image obtained by performing highlighting processing on the camera image. A viaduct 311 is provided in front of the vehicle. That is, the vehicle 1 is traveling on the road 310 passing under the viaduct 311 .

Here, it is assumed that the vehicle 1 has a height that prevents it from passing under the viaduct 311 . For example, assume that the elevated bridge 311 is 1.9 m above the ground and the vehicle height is higher than 1.9 m.

The object detection unit 122 detects the elevated bridge 311 ahead in the traveling direction as an object. The risk setting unit 127 detects the ground clearance of the viaduct 311 . That is, the clearance height from the road to the viaduct 311 is detected. The risk setting unit 127 compares the height of the vehicle 1 and the ground clearance of the viaduct 311 . Then, the risk setting unit 127 sets the risk according to the comparison result.

When the height of the vehicle 1 is such that the vehicle 1 cannot pass through the viaduct 311, the risk setting unit 127 sets the viaduct 311 to a high risk. The highlighting processing unit 128 performs highlighting according to the degree of danger. The highlighting processing unit 128 superimposes a red highlighting color corresponding to a high degree of risk as a warning display 312 on the viaduct 311 in the image. Since there is a high possibility that the vehicle 1 will come into contact with the viaduct 311, the display unit 30 can display a warning 312 to the user. Although the warning display 312 in the drawing is superimposed only on the viaduct 311 in the upper part in the traveling direction, it may be superimposed on the entire surface of the viaduct 311 .

In general, the vehicle speed during forward movement is faster than the vehicle speed during reverse movement. For example, the vehicle 1 travels at a relatively low speed while in reverse because of parking in the garage. When reversing for parking, the vehicle speed slows down, so the user is more likely to notice the height restriction. On the other hand, since the vehicle speed increases while moving forward, the user may overlook the height limit. Therefore, while moving forward, the risk setting unit 127 detects the height of the object and compares the vehicle height with the height of the object. By switching the risk level setting process according to the traveling direction in this way, the risk level can be appropriately set. It is possible to more appropriately call attention to the user who is the driver.

In the above description, an example of passing through the road 310 under the elevated bridge 311 has been described, but the processing of this embodiment can be used for other than the elevated bridge 311 as well. For example, parking lots such as multilevel parking lots, indoor parking lots, and mechanical parking lots may have height restrictions. When entering a parking lot with such height restrictions, the processing of the present embodiment can be applied. The risk level setting unit 127 sets the risk level for the object according to the underpass height limit of the viaduct 311 and the height inside the building. The risk setting unit 127 compares the height of the object with the vehicle height of the vehicle 1 to determine whether the vehicle 1 can pass under the object. The risk setting unit 127 sets the risk according to the determination result.

A margin may be provided for the comparison result between the vehicle height of the vehicle 1 and the clearance height of the object. For example, in the above description, when the vehicle height of the vehicle 1 is higher than the height limit under the girder of the viaduct 311, the warning display is performed. If so, a warning may be displayed. The risk level setting unit 127 may increase the risk level when the difference between the vehicle height of the vehicle 1 and the ground clearance of the object is equal to or less than a predetermined value. Specifically, it is possible to take a height margin in consideration of steps on the road surface and the like.

Note that in the first setting process, the degree of risk may be set according to the distance to the object. For example, weighting may be performed so that the closer the distance, the higher the risk. Furthermore, if the distance to the object is longer than the threshold distance, the degree of risk may be set according to the distance as in the first embodiment. Then, the heights may be compared at the timing when the distance to the object becomes equal to or less than the threshold distance. By doing so, it is possible to more appropriately call attention to the user. For example, when the distance is long, the accuracy of height detection is low.

Also, the risk setting unit 127 may set the risk based on the speed information. For example, the risk setting unit 127 may adjust the timing of warning display according to the speed information and the distance to the object. When the vehicle 1 is traveling at high speed, the time from detection of the object to arrival at the viaduct 311 is shortened. Therefore, it is preferable to display a warning to the user as early as possible. It is preferable that the faster the speed, the faster the timing of displaying the highlight color. A warning can be displayed at an appropriate timing.

Note that the risk setting unit 127 may measure at least one of the height of the object from the road surface and the position of the object in the horizontal direction. The risk setting unit 127 determines whether there is a possibility that the object will collide with the vehicle 1 based on the measurement result. The risk setting unit 127 sets the risk to the object according to the determination result. This processing will be described with reference to FIG. Here, an example of parking forward or backward in the parking space 320 will be described.

Wheel stoppers 324 are provided in the parking space 320 . Further, a peripheral vehicle 321 is parked on the side of the parking space 320 . In addition, peripheral vehicles 322 are parked at the back of the parking space 320 . The surrounding vehicles 321 and 322 are closer to the vehicle 1 than the threshold distance. The size in the left-right direction (horizontal direction) is approximately the same as that of the vehicle 1 . Furthermore, there is a pedestrian 325 far from the parking space 320 . Note that the distance to the pedestrian 325 is longer than the threshold distance.

The degree-of-risk setting unit 127 measures the lateral positions of the surrounding vehicles 321 and 322 based on the distance measurement data. The degree-of-risk setting unit 127 determines whether there is a possibility that the vehicle 1 will collide with the surrounding vehicles 321 and 322 based on the measurement result of the lateral position. The risk setting unit 127 sets the risk of the surrounding vehicles 321 and 322 according to the determination result.

For example, since the surrounding vehicle 321 is right next to the parking space 320, the risk setting unit 127 determines that there is a possibility of collision. Since the surrounding vehicle 322 is ahead of the parking space 320 in the traveling direction, the risk setting unit 127 determines that there is a possibility of collision. Therefore, the risk setting unit 127 sets a high risk for the surrounding vehicles 321 and 322 . The front in the traveling direction is the rear of the vehicle 1 when the vehicle is moving backward, and the front of the vehicle 1 when the vehicle is moving forward.

In the superimposed image, warning displays 331 and 332 are superimposed on surrounding vehicles 321 and 322, respectively. That is, emphasized colors such as red are superimposed on the surrounding vehicles 321 and 322 as the warning displays 331 and 332 .

The pedestrian 325 is at a position away from the front of the vehicle 1 in the left-right direction (horizontal direction). The degree-of-risk setting unit 127 measures the position of the pedestrian 325 in the left-right direction based on the distance measurement data. The degree-of-risk setting unit 127 determines whether there is a possibility that the vehicle 1 will collide with the pedestrian 325 based on the measurement result of the lateral position. The risk level setting unit 127 sets the risk level of the pedestrian 325 according to the determination result. The risk setting unit 127 determines that the possibility of colliding with the pedestrian 325 is extremely low. A low risk level is set for the pedestrian 325 . The risk level setting unit 127 sets the risk level of the pedestrian 325 to 0, for example. Therefore, the pedestrian 325 is not displayed with a warning in the superimposed image.

In this manner, the risk level setting unit 127 may set the risk level according to the horizontal position and size (horizontal width) of the object. Further, it may be determined whether there is a possibility of collision with the vehicle 1 according to the steering angle, the width of the object, and the position in the left-right direction (lateral direction). Then, when the possibility of contact with the vehicle 1 is low, the risk level setting unit 127 may set the risk level to be low. Also, if there is no possibility of contact, the risk setting unit 127 may set the risk to zero. Note that when determining whether or not to collide with the vehicle 1 based on the position in the left-right direction, the steering angle information of the vehicle 1 may be taken into consideration. In other words, the risk setting unit 127 may determine whether or not there is an object ahead in the traveling direction, for example, in the front left or the front right, depending on the steering angle.

The risk setting unit 127 measures the height of the wheel stopper 324 based on the distance measurement data. The risk setting unit 127 determines whether or not the vehicle will collide with the wheel stopper 324 based on the height measurement result. The risk setting unit 127 sets the risk to the object according to the determination result. Wheel chock 324 is lower than the ground clearance of vehicle 1 . Therefore, the risk setting unit 127 determines that there is no possibility of colliding with the wheel stopper 324, and sets the risk to zero. Therefore, in the superimposed image, no warning display is performed on the wheel chock 324 .

By doing so, the risk setting unit 127 can set the risk more appropriately. Therefore, it becomes possible to effectively call attention to the user. Further, the display control device 100 may output a voice message, warning sound, or the like for calling attention from a speaker based on the degree of risk. In other words, the output unit 129 may generate audio data according to the degree of danger.

Also, the program for executing the above display control method can be stored using various types of non-transitory computer readable media and supplied to computers. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magneto-optical recording media (eg, magneto-optical discs), CD-ROMs (Read Only Memory), CD-Rs, CD-R/Ws, semiconductor memories (eg, mask ROMs, PROMs). (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)). The program may also be delivered to the computer on various types of transitory computer readable medium. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can deliver the program to the computer via wired channels, such as wires and optical fibers, or wireless channels.

Reference Signs List 1 vehicle 2 sensor unit 21F front camera 21R rear camera 22 ranging sensor 30 display unit 40 ECU
REFERENCE SIGNS LIST 100 display control device 110 travel information acquisition unit 111 traveling direction acquisition unit 113 speed information acquisition unit 121 image data acquisition unit 122 object detection unit 125 ranging data acquisition unit 127 risk setting unit 128 highlight display processing unit 129 output unit 301 rear wall 302 side wall 304 wheel chock 310 road 311 viaduct 312 warning sign 320 parking space 321, 322 surrounding vehicles 324 wheel chock 325 pedestrian 331, 332 warning sign

Claims (5)

an image data acquisition unit that acquires image data of an image of the exterior of the own vehicle;
an object detection unit that detects an object included in the image;
a ranging data acquisition unit that acquires ranging data relating to the distance from the own vehicle to the object;
a risk level setting unit that sets a risk level for the object according to the size of the object when the distance is equal to or less than a threshold;
a highlighting processing unit that performs highlighting processing on image data according to the degree of risk;
an output unit that outputs a highlighted image that has undergone highlighting processing;
comprising
Display controller. 2. The display control device according to claim 1, wherein the risk setting unit sets the risk to the object according to the distance to the object when the distance is greater than a threshold. The risk level setting unit
measuring at least one of the height of the object from the road surface and the horizontal position of the object;
determining whether or not the object will collide with the own vehicle based on the measurement result;
The degree of danger is set for the object according to the judgment result,
The display control device according to claim 1 or 2. a step of obtaining image data of an image of the exterior of the own vehicle;
detecting objects contained in said image;
obtaining ranging data relating to the distance from the host vehicle to the object;
setting a risk level for the object according to the size of the object when the distance is equal to or less than a threshold;
a step of performing a highlighting process on image data according to the degree of risk;
a step of outputting a highlighted image that has undergone highlighting processing;
comprising
Display control method. a step of obtaining image data of an image of the exterior of the own vehicle;
detecting objects contained in said image;
obtaining ranging data relating to the distance from the host vehicle to the object;
setting a risk level for the object according to the size of the object when the distance is equal to or less than a threshold;
a step of performing a highlighting process on the image data according to the degree of risk;
a step of outputting a highlighted image that has undergone highlighting processing;
A program that makes a computer run

Images