JP2024088276A - Display controller and image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display controller and an image display device capable of displaying natural images that allow a driver to correctly grasp information on a vehicle's surroundings.

SOLUTION: A display control device according to the present disclosure includes: a captured image acquisition unit that acquires a captured image of a surrounding area of a vehicle body; a region identification unit that identifies a first region including the vehicle body in the captured image; a monotoning processing unit that monotones an image of the first region; and a display unit that displays a display image including the monotoned image on the display unit.

SELECTED DRAWING: Figure 5

COPYRIGHT: (C)2024,JPO&INPIT

Description

This disclosure relates to a display control device and an image display device.

In order to assist vehicle operation, images captured around the vehicle have been used in the past. Specifically, instead of side mirrors mounted on the vehicle, so-called electronic exterior mirrors are known, which capture images of the rear side of the vehicle with a camera and display the captured images on a monitor installed inside the vehicle cabin.

The following Patent Document 1 describes technology for an electronic exterior mirror that uses a camera instead of a side mirror. This Patent Document 1 describes how the area including the vehicle body is colored or overwritten with a specific vehicle body shape so that the driver can easily distinguish between the actual scenery in the image captured by the camera and the scenery reflected in the vehicle body.

International Publication No. 2019/177036

As in Patent Document 1, by coloring or overwriting the shape of the vehicle body in the area including the vehicle body in the image captured by the camera, it is expected that it will be easier to distinguish between the area including the vehicle body and other areas compared to when such coloring is not performed. In addition, by performing the above-mentioned coloring, the scenery reflected in the vehicle body will no longer be displayed, which can also reduce the misidentification of the actual scenery as the scenery reflected in the vehicle body.

However, as mentioned above, if part of the image displayed on the monitor is colored or overwritten with another image, the image displayed on the monitor will be clearly different from a natural image as seen through an optical mirror. A person viewing the monitor, for example a driver of a vehicle, will recognize the vehicle body reflected in the image displayed on the monitor as their own vehicle and will then grasp its positional relationship with the rear vehicle, etc., that is displayed in the image. Therefore, if the image displayed on the monitor is unnatural, there is a risk that the driver will not be able to correctly grasp information about the surroundings of the vehicle from the image.

In view of the above-mentioned problems, the present disclosure aims to provide a display control device and an image display device capable of displaying natural images that allow the driver or other users to correctly grasp information about the vehicle's surroundings.

To achieve the above object, the display control device according to the first aspect of the present disclosure includes an image acquisition unit that acquires an image of the periphery of a vehicle body, an area identification unit that identifies a first area including the vehicle body in the image, a monotone processing unit that converts the image of the first area into monotone, and a display unit that displays a display image including the monotone image on a display device.

In the display control device as described above, the first region of the display image, which includes the vehicle body, is converted to monotone, so the amount of information provided to the driver from the first region is reduced. This allows the driver who sees the display image to pay more attention to the actual scenery rather than the scenery reflected in the vehicle body, making it easier to correctly grasp the information around the vehicle. In addition, since the portion of the display image corresponding to the first region is only converted to monotone and the outlines of the scenery reflected in the vehicle body are not erased, a more natural image can be provided to the driver than in the past.

A display control device according to a second aspect of the present disclosure is a display control device according to the first aspect of the present disclosure, wherein the monotone conversion processing unit calculates luminance information and color information of the image of the first region, calculates a first distribution median which is the median of the color information of the entire image of the first region, and replaces the color information of the entire image of the first region with the first distribution median.

In a display control device like the one described above, the actual image is converted to monotone, so a natural image can be maintained as the displayed image.

The display control device according to the third aspect of the present disclosure is the display control device according to the first aspect of the present disclosure, further including a storage unit that stores vehicle body color information of the vehicle body and a first threshold region set based on the similarity with the vehicle body color information, and the monotone conversion processing unit calculates luminance information and color information of the image of the first region, calculates a second distribution median that is the median of the color information of the image of the first region whose color information is within the first threshold region, and replaces the color information of the entire first region with the second distribution median.

In a display control device like the one described above, even if a large part of the vehicle body is covered with scenery of a color different from the vehicle body color, it is possible to convert the scenery into a monotone color close to the vehicle body color, and to provide a display image that is less likely to cause the driver to feel uncomfortable.

A display control device according to a fourth aspect of the present disclosure is a display control device according to the first aspect of the present disclosure, further including a high-brightness area identifying unit that identifies high-brightness areas having a predetermined brightness or higher in the captured image, and the monotone conversion processing unit calculates luminance information and color information of a second area obtained by excluding the high-brightness areas from the first area, calculates a third distribution median that is the median of the color information of the entire image of the second area, and replaces the color information of the entire image of the second area with the third distribution median.

In the above-mentioned display control device, even if a bright scene, such as an external light source, is reflected on the vehicle body, it is possible to prevent the vehicle body from becoming monotone with a color close to that of the external light source, and to maintain a natural image as the displayed image.

A display control device according to a fifth aspect of the present disclosure is a display control device according to any one of the first to fourth aspects of the present disclosure, further including a smoothing filter unit that smoothes the image of the first region.

In the display control device described above, by performing a smoothing process on the first region, the amount of information provided to the driver from the first region can be further reduced.

A display control device according to a sixth aspect of the present disclosure is the display control device according to the fifth aspect of the present disclosure, wherein the smoothing filter unit smoothes the image of the first region when the traveling speed of the vehicle is equal to or greater than a predetermined speed.

In the display control device described above, when the driver is paying attention to areas closer to the vehicle, such as when the vehicle is traveling at a slow speed, stopped, or in reverse, smoothing processing is not performed, so the driver is less likely to feel uncomfortable with the image in the first area.

The image display device according to the seventh aspect of the present disclosure includes a display control device according to any one of the first to sixth aspects of the present disclosure, an imaging device capable of imaging the periphery of the vehicle body, and the display device capable of displaying the display image.

In the image display device as described above, the first region of the displayed image, which includes the vehicle body, is made monotone, so the amount of information provided to the driver from the first region is reduced. This allows the driver, who sees the image displayed on the display device, to pay more attention to the actual scenery rather than the scenery reflected on the vehicle body, making it easier to correctly grasp the information around the vehicle. In addition, since the portion of the displayed image corresponding to the first region is only made monotone and the outlines of the scenery reflected on the vehicle body are not erased, a more natural image can be provided to the driver, etc., than in the past.

The display control device and image display device disclosed herein can display natural images that allow the driver to correctly grasp information about the vehicle's surroundings.

1 is a schematic diagram showing an example of a vehicle equipped with an image display device according to a first embodiment of the present disclosure. 1 is a schematic block diagram illustrating an example of a hardware configuration of an image display device according to a first embodiment of the present disclosure. 2 is a functional block diagram illustrating an example of a software configuration of a display control device according to a first embodiment of the present disclosure. FIG. 11 is a graph showing an example of a distribution of UV values of each pixel in a first region. 4A and 4B are schematic diagrams showing an example of an image before and after being converted into monotone by the monotone conversion processing unit shown in FIG. 3 . 4 is a schematic diagram showing an example of an image smoothed by a smoothing filter unit shown in FIG. 3 . 4 is a flowchart illustrating an example of a display image generation process performed by a display control device according to a first embodiment of the present disclosure. FIG. 11 is a functional block diagram illustrating an example of a software configuration of a display control device according to a second embodiment of the present disclosure. 13 is a graph showing another example of the distribution of UV values of each pixel in the first region. FIG. 10 is an explanatory diagram showing the UV color plane constituting the graph shown in FIG. FIG. 13 is a functional block diagram illustrating an example of a software configuration of a display control device according to a third embodiment of the present disclosure. 11A and 11B are schematic diagrams showing an example of an image before and after a second region is specified in a captured image.

Below, each embodiment for carrying out the present disclosure will be described with reference to the drawings. Note that below, the scope necessary for the explanation to achieve the object of the present disclosure will be shown diagrammatically, and the scope necessary for the explanation of the relevant parts of the present disclosure will be mainly explained, and the parts for which explanation is omitted will be based on publicly known technology. In addition, identical or similar reference symbols will be used for identical or corresponding components in the drawings, and duplicate explanations will be omitted. Furthermore, when a single drawing contains multiple identical or corresponding components, only some of them may be reference symbols in order to make the drawing easier to read.

As an example of the image display device according to each embodiment described below, one that functions as an electronic outer mirror used in place of a side mirror composed of an optical mirror installed on the side of a vehicle will be given. However, the display control device and image display device disclosed herein are not limited to use as an electronic outer mirror. Specifically, the image display device disclosed herein can also be used to monitor the exterior of a vehicle using multiple cameras, etc., in public transportation such as route buses. In this case, the person conducting the monitoring can be a manager other than the driver.

First Embodiment
FIG. 1 is a schematic diagram showing an example of a vehicle equipped with an image display device according to a first embodiment of the present disclosure. FIG. 2 is a schematic block diagram showing an example of a hardware configuration of the image display device according to the first embodiment of the present disclosure. As shown in FIG. 1 and FIG. 2, the image display device 10 according to the first embodiment includes a camera 20 as an example of an imaging device capable of imaging the periphery of a vehicle body 2, a monitor 30 as an example of a display device that displays a display image P (see FIG. 5(B) or FIG. 6, etc.) generated based on an imaged image P0 (see FIG. 5(A) for example) captured by the camera 20, and a display control device 40. The display image P is used as a collective term for a plurality of display images (specifically, P1, P2, and P4) described below.

The cameras 20 are installed at appropriate positions on the front sides of the vehicle 1, for example, on the left and right fenders at the front of the vehicle body 2, and can be configured to capture images of the rear of the vehicle 1. The cameras 20 can be two-dimensional cameras using imaging elements such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The orientation of the camera 20 can be changed via a user interface such as a switch (not shown).

The monitor 30 can be configured with a well-known display device such as a liquid crystal display (LCD) or an organic electro-luminescent display (OELD). This monitor 30 can be supported in the vehicle cabin, for example, at the left and right ends of the vehicle forward of the driver's seat, or in the center in the vehicle width direction.

When the image display device 10 according to this embodiment is used as an electronic outer mirror in place of a vehicle's side mirror, a camera 20 is installed on the vehicle 1 in place of an optical mirror. In this case, the camera 20 does not need to be installed in a position where it can be seen by the driver through the side window 3, as in the case of an optical mirror, so it is generally installed below the side window 3, specifically in the fender. As shown in FIG. 1, most of the height of the imaging area IA captured by the camera 20 installed in such a position corresponds to the vehicle body 2 located between the side window 3 and the tire 4, so that the vehicle body 2 is largely reflected in the captured image P0 captured by this camera 20.

The scenery around the vehicle 1, such as the road surface and vehicles behind, is reflected in the vehicle body 2. Therefore, if the captured image P0 is displayed as is on the monitor 30, the driver D may confuse the scenery reflected on the vehicle body 2 with the actual scenery. Therefore, in the image display device 10 according to this embodiment, the display control device 40 adjusts the captured image P0 so that the driver D can correctly grasp the surrounding information of the vehicle 1 from the image displayed on the monitor 30.

The display control device 40 according to this embodiment may generate a display image P to be displayed on the monitor 30 based on an image P0 captured by the camera 20. This display control device 40 may be configured, for example, by a well-known computer including a microcontroller. The computer constituting the display control device 40 may be a computer prepared separately for the image display device 10, or may be realized using an ECU (Electronic Control Unit) in the vehicle 1.

The display control device 40 may include at least a CPU (Central Processing Unit) 41 as an example of a processor, a ROM (Read Only Memory) 42 and a RAM (Random Access Memory) 43 as examples of memory, a storage 44, and an input/output interface 45. These components may be connected to each other so as to be able to communicate with each other via an internal bus.

The CPU 41 may be a central processing unit capable of executing various programs and controlling each part. Specifically, the CPU 41 may be capable of reading out various programs stored in the ROM 42 or storage 44, and executing the programs using the RAM 43 as a working area. The CPU 41 may be capable of controlling each component constituting the image display device 10 and performing various types of arithmetic processing according to the programs.

The ROM 42 may be capable of storing various programs and various data. The RAM 43 may be capable of temporarily storing programs or data as a working area.

The storage 44 can be configured with a recording medium such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. This storage 44 may store various programs including an operating system and various data necessary for operating the image display device 10 (for example, vehicle color information and various thresholds described below). In this embodiment, the ROM 42 or the storage 44 may store a program and various data for generating a display image P based on an image captured by the camera 20.

The input/output interface 45 may be an interface for transmitting and receiving various data between various components mounted or attached to the vehicle 1 that are necessary to realize at least the functions of the electronic exterior mirror. An example of the input/output interface 45 in this embodiment is one in which at least the camera 20 and the monitor 30 are electrically connected.

FIG. 3 is a functional block diagram showing an example of a software configuration of a display control device according to a first embodiment of the present disclosure. As shown in FIG. 3, the display control device 40 according to this embodiment includes an image acquisition unit 53 that acquires an image P0 of an area around the vehicle body 2 of the vehicle 1, an area identification unit 51 that identifies a first area A1 including the vehicle body 2 in the image P0, a monotone processing unit 52 that converts the image of the first area A1 in the image P0 into monotone, and a display image output unit 54 as an example of a display unit that displays a display image P including the monotone image on the monitor 30.

The area identification unit 51 can be realized mainly by the CPU 41 and the storage 44, etc., and may identify a first area A1 including the vehicle body 2 from within the captured image P0. In this embodiment, the first area A1 refers only to the area occupied by the vehicle body 2 and does not include the areas of the side glass 3 and tires 4, but the present disclosure is not limited to this. As a specific example, the first area A1 may refer to the entire area in which the vehicle 1 is reflected, including, for example, the side glass 3 and tires 4 in addition to the vehicle body 2.

The method for identifying the first area A1 is not particularly limited, but for example, during the calibration work of the image display device 10 when manufacturing the vehicle 1, the first area A1 in the captured image P0 can be manually set corresponding to the angle of the camera 20. Alternatively, a method can be adopted in which a control algorithm is used to identify the first area A1 from the captured image P0 in real time, as described in the above-mentioned Patent Document 1.

In order to more easily identify the first area A1, a chart can be arranged in the image capture area IA and the boundary of the vehicle body 2 can be identified by moving the chart up, down, left and right. The chart used in this case can be, for example, a board with a chessboard pattern in which black and white squares are arranged alternately. Then, by continuously detecting the position where the intersection of the chart becomes undetectable as the chart itself moves (i.e., the position hidden behind the vehicle body 2), the boundary between the vehicle body 2 and the actual scenery, that is, the outline of the vehicle body 2 reflected in the captured image P0, can be identified. The identification work of the first area A1 is preferably performed multiple times in accordance with the angle of view of the camera 20. As described above, if the first area A1 is identified in the calibration work at the time of manufacturing the vehicle 1, the area identification unit 51 can identify the first area A1 simply by acquiring the angle of view of the camera 20. Therefore, there is no need to separately prepare a control algorithm or the like for identifying the above-mentioned boundary in real time in the display control device 40, and the load on the CPU 41 can be reduced.

The monotone conversion processing unit 52 can be realized mainly by the CPU 41 and storage 44, etc., and serves to convert the image of the first area A1 in the captured image P0 to monotone. Here, monotone conversion refers to changing the image to a format that expresses it in shades of a single color. In the monotone conversion processing unit 52 of this embodiment, since the first area A1 is composed of an image corresponding to the vehicle body 2 portion, when the image of the first area A1 is converted to a monotone color by the monotone conversion processing, the color of the image of the first area A1 will in most cases be close to the color of the vehicle body.

To explain an example of the method of monotoning processing in the monotoning processor 52, the monotoning processor 52 first obtains the RGB values of each pixel constituting the image of the first area A1 in the captured image P0. The obtained RGB values are then converted into YUV values consisting of luminance information (more specifically, the luminance signal Y) and color information (more specifically, the color difference signals U and V). The conversion from RGB values to YUV values can be performed using the following formula (1).

FIG. 4 is a graph showing an example of the distribution of UV values of each pixel constituting the image of the first region. The horizontal and vertical axes in FIG. 4 correspond to the U value and the V value, respectively, and the vertical axis corresponds to the count number of plotted pixels. In the monotone processing unit 52, when the YUV values of each pixel of the image of the first region A1 are specified, they are plotted and tallied on the UV color plane as shown in FIG. 4, and the median values (hereinafter, this value is referred to as the "first distribution median value") U ₀ , V ₀ are calculated from the distribution state. The method of specifying the first distribution median values U ₀ , V ₀ is not particularly limited, but for example, as a result of counting all the pixels constituting the first region, the U value and V value corresponding to the position on the UV color plane with the largest count number can be specified as the first distribution median values U ₀ , V ₀ .

When the first distribution medians _U0 , _V0 are calculated, the U value and V value among the YUV values of each pixel of the image in the first region A1 are replaced with the first distribution medians _U0 , _V0 and then converted to RGB values. The conversion from the _YUV ( _YU0V0 ) values to the RGB values can be performed using the following formula (2).

The RGB values of each pixel obtained by the above-mentioned conversion process are merged into the first area A1 of the captured image P0. The image obtained by the above-mentioned merging process partially includes a monotone image in which the colors of each pixel of the image in the first area A1 are monochromatic and only the luminance information is different.

Figure 5 is a schematic diagram showing an example of an image before and after being converted to monotone by the monotone conversion processing unit shown in Figure 2, where Figure 5(A) shows an example of a captured image, and Figure 5(B) shows an example of a display image created based on the captured image of Figure 5(A). The monotone conversion processing unit 52 can change the captured image P0, which includes a landscape containing various color information reflected in the vehicle body 2 as shown in Figure 5(A), to an image in which only the image reflected in the vehicle body 2 is monotone, as shown in Figure 5(B). The image that has been partially converted to monotone by this monotone conversion processing unit 52 is then sent to the display image output unit 54 and can be displayed on the monitor 30 as the display image P1.

As can be seen from FIG. 5, the display image P1, in which the image of the first area A1 has been converted to monotone, reduces the amount of information that the driver D can obtain from the image of the first area A1 compared to the captured image P0. Therefore, when the driver D sees this display image P1, his or her attention is naturally drawn to images other than the first area A1, which contains a relatively large amount of information, and the driver D is able to obtain information about the rear side of the vehicle 1 accurately in a short period of time.

The captured image acquisition unit 53 may be connected to the camera 20 and may be capable of acquiring the captured image P0 captured by the camera 20. The display image output unit 54 may be connected to the monitor 30 and may output the display image P generated by the area identification unit 51 and the monotone conversion processing unit 52, etc., to the monitor 30. Both the captured image acquisition unit 53 and the display image output unit 54 may be realized mainly by the input/output interface 45.

Incidentally, the display control device 40 according to this embodiment may further include, in addition to the configuration described above, a smoothing filter unit 55 that smoothes the image of the first area A1, as shown in FIG. 3.

The smoothing filter unit 55 may be capable of performing a smoothing process, or in other words, a "blurring" process, on a specific image. There are no particular limitations on the specific method for performing this smoothing process, but examples include a method that uses a filter (moving average filter), Gaussian filter, median filter, or the like that can flatten (smooth) the luminance values of each pixel that constitutes an image.

It is preferable that the smoothing filter unit 55 be turned on and off according to the traveling speed of the vehicle 1. Specifically, the smoothing filter unit 55 may be directly or indirectly connected to a vehicle speed sensor 60 mounted on the vehicle 1, and the smoothing process is performed only when the traveling speed detected by the vehicle speed sensor 60 is equal to or higher than a predetermined speed (e.g., any speed between 20 and 30 km/h). The vehicle speed sensor 60 may be configured, for example, with a sensor capable of detecting the rotation speed of the output shaft. The detection result of the vehicle speed sensor 60 may be obtained directly or via an ECU in the vehicle 1.

When the vehicle 1 is traveling at a low speed, stopped, or reversing, the driver D tends to try to obtain information about a relatively close position (e.g., about several meters) behind the vehicle 1 from the image displayed on the monitor 30. Therefore, at such times, the driver D can obtain more information from the display image P if the display image P is not blurred. Also, when the vehicle 1 is stopped, the driver D can focus his or her eyes on the monitor 30 for a longer period of time than when the vehicle 1 is traveling at high speed, so it is preferable to display a more natural image that is not blurred, as this makes the driver D less likely to feel uncomfortable with the display image P.

Conversely, when the vehicle 1 is traveling at high speed, the driver D tends to try to obtain information about a relatively distant position (e.g., about 10 meters) behind the vehicle 1 from the image displayed on the monitor 30. For this reason, it is preferable for the amount of information in the first area A1 to be small, since this allows the driver D, who sees the display image P, to immediately focus his or her attention on the actual scenery. In other words, performing blurring processing when traveling at high speed can better help the driver D to obtain information about the side and rear of the vehicle 1 accurately in a short period of time.

Figure 6 is a schematic diagram showing an example of an image that has been smoothed by the smoothing filter unit shown in Figure 3. In Figure 6, an image (hereinafter, referred to as "display image P2") obtained by blurring the display image P1 shown in Figure 5 (B) is shown. As shown in Figure 6, when the image of the first area A1 is monotone-toned and smoothed and displayed on the monitor 30 as the display image P2, the amount of information that the driver D can obtain from the first area A1 is further reduced. Therefore, the driver D can accurately obtain information about the rear side of the vehicle 1 in a short time. This can also significantly reduce the driver D's confusion between the actual scenery in the display image P2 and the scenery reflected in the vehicle body 2.

Next, an example of the process for generating the display image P2 by the display control device 40 according to this embodiment will be briefly described with reference to FIG. 7.

Figure 7 is a flowchart showing an example of a display image generation process by the display control device according to the first embodiment of the present disclosure. When the image display device 10 starts operating due to starting the vehicle 1, the camera 20 starts capturing an image, and as shown in Figure 7, the display control device 40 first acquires the captured image P0 captured by the camera 20 using the captured image acquisition unit 53 (step S1).

After the captured image P0 is acquired by the captured image acquisition unit 53, the area identification unit 51 then identifies the first area A1 in the captured image P0 (step S2). At this time, the area identification unit 51 may acquire information on the angle of view of the camera 20 in order to identify the first area A1 in the captured image P0. Note that, if the first area A1 is defined in advance in accordance with the angle of view as in this embodiment, the first area A1 will not substantially change unless the angle of view of the camera 20 is changed. Therefore, it is sufficient to identify the first area A1 when the angle of view of the camera 20 is changed.

When the first area A1 in the captured image P0 is specified, the monotone processing unit 52 then performs monotone processing on the image in the first area A1. In detail, first, the RGB values of each pixel of the image in the first area A1 are obtained (step S3). Next, from the RGB values obtained here, the YUV values are calculated using the above-mentioned formula (1) (step S4). Then, among the YUV values corresponding to each pixel, the first distribution median _U0 and _V0 , which are the median values of the U value and V value constituting the color information, are calculated (step S5), and all the U value and V value of each pixel of the image in the first area A1 are replaced with the first distribution median _U0 and _V0 (step S6). By performing the above-mentioned replacement, the U value and V value of all pixels of the image in the first area A1 become the same value, and the image in the first area A1 is monotone-processed.

When the monotone conversion process described above is completed, the YUV values (strictly speaking _{, YU0V0 values} ) of each pixel are converted to RGB values (step S7) in order to merge the processing results into the captured image P0. The above-mentioned formula (2) may be used for this conversion. Then, the image in the first area A1 in the captured image P0 is updated based on the converted RGB values (step S8). The updated image becomes an image in which only the image in the first area A1 has been converted to monotone, as shown in FIG. 5B.

Next, the traveling speed of the vehicle 1 at the time when the captured image P0 was acquired is obtained from the vehicle speed sensor 60, and the smoothing filter unit 55 judges whether or not smoothing processing is required. Specifically, it is judged whether the traveling speed acquired from the vehicle speed sensor 60 is equal to or greater than a predetermined speed (e.g., 20 km/h) (step S9). If the result of this judgment is that the traveling speed is equal to or greater than the predetermined speed, smoothing processing is executed on the image of the first area A1 (step S10), and if the traveling speed is slower than the predetermined speed, the smoothing processing is skipped. Finally, the image obtained by the above-mentioned series of processes is output to the monitor 30 by the display image output unit 54 as the display image P2 (step S11).

According to the display control device 40 and the image display device 10 of this embodiment, the display image P2 in which only the image in the first area A1 including the vehicle body 2 is monotoned is displayed on the monitor 30. Although the first area A1 portion of this display image P2 is unified in a color close to the vehicle body color, the brightness information of the scenery reflected in the vehicle body 2 is maintained. Therefore, the driver D who sees this image hardly feels strange about the display image P2 and can understand at a glance that the first area A1 is the vehicle body 2. In addition, the driver D who sees the display image P2 will pay more attention to the area with more information than the first area A1, that is, the actual scenery displayed in the area other than the first area A1, rather than the monotoned first area A1. From the above, the driver D can correctly recognize the actual scenery in a short time without confusing the scenery reflected in the vehicle body 2 with the actual scenery. Furthermore, the driver D, who views the display images P1 and P2 generated by the image display device 10 according to this embodiment, can accurately grasp information about the surroundings of the vehicle 1.

In addition, in the display control device 40 and image display device 10 according to this embodiment, the image of the first area A1 is blurred according to the traveling speed, thereby enabling optimal display according to the state of the vehicle 1.

Second Embodiment
Incidentally, for example, in roads on which the vehicle 1 travels, locations where accidents are likely to occur, such as intersections and curves on expressways, may be painted in red or blue to alert the driver D. When traveling on a road painted in such a specific color, the color of the road may be reflected over a wide area on the vehicle body 2. Here, when the vehicle body color of the vehicle 1 is a color that is completely different from the color of the road described above, such as black or gray, the above-mentioned display control device 40 performs the monotone processing by the monotone processing unit 52 in a state in which a landscape of a color different from the color of the vehicle body 2 is reflected over a wide area of the vehicle body 2. In this case, the first distribution medians U ₀ and V ₀ to be calculated may be a color close to the color of the road, not the color of the vehicle body. If the first region A1 is unified in a color close to the color of the road and output to the monitor as a display image, the driver D may have difficulty recognizing the vehicle body 2 at a glance, or may feel uncomfortable with the display image P.

The following describes an image display device 10A and a display control device 40A according to a second embodiment that takes the above points into consideration. The image display device 10A and the display control device 40A according to this embodiment may include configurations similar to those of the image display device 10 and the display control device 40 according to the first embodiment described above, except for some configurations of the display control device 40A. Therefore, the following description will focus on the configurations of the image display device 10A and the display control device 40A according to this embodiment that differ from the image display device 10 and the display control device 40 described above, and components similar to those of the image display device 10 and the display control device 40 will be given the same reference numerals and their description will be omitted.

FIG. 8 is a functional block diagram showing an example of the software configuration of a display control device according to a second embodiment of the present disclosure. As shown in FIG. 8, the display control device 40A according to this embodiment may further include a storage unit 56 in addition to the above-mentioned area identification unit 51 and monotone processing unit 52, etc.

The memory unit 56 can be realized by, for example, the storage 44 described above, and can store at least the body color BC as the body color information of the vehicle body 2, and the first threshold area TA set based on the similarity with the body color BC. The body color BC and the first threshold area TA stored in the memory unit 56 can be used during monotone processing by the monotone processing unit 52.

Figure 9 is a graph showing another example of the distribution of UV values of each pixel in the first region. Also, Figure 10 is an explanatory diagram showing the UV color plane that constitutes the graph shown in Figure 9. Figures 9 and 10 show the distribution state of YUV values in the first region when a vehicle 1 with a gray body color BC runs on a road painted red. Also, the symbol BC in Figure 10 indicates the position of the gray vehicle body color on the UV color plane, and the symbol X indicates the approximate position of the color of the road painted red on the UV color plane.

When a vehicle 1 with a gray body color BC travels on a road painted red and the red color of the road is widely reflected on the vehicle body 2, the first area A1 of the captured image P0 may have more pixels with colors close to the road color than the vehicle color BC or colors close to the vehicle color, as shown in FIG. 9. Here, the position of the body color BC of the gray vehicle 1 on the UV color plane and the position X of the road color are far apart, as shown in FIG. 10.

Therefore, in the monotone conversion processing unit 52 of the display control device 40A according to this embodiment, the first threshold area TA stored in the storage unit 56 is used to adjust the color when monotone conversion is performed so that it is close to the vehicle body color BC. Specifically, when the monotone conversion processing unit 52 starts the monotone conversion process, first, the RGB values of each pixel of the image in the first area A1 are obtained. Next, the YUV values obtained here are calculated using the above-mentioned formula (1) from the RGB values. Then, from the YUV values corresponding to each pixel, pixels whose U and V values are included in the first threshold area TA are selected, a second distribution median, which is the median of the U and V values of the selected pixel, is calculated, and the U and V values of each pixel of the image in the first area A1 are replaced with the second distribution median.

In the example shown in Figures 9 and 10, the first threshold area TA is adjusted to an area that does not include the position X of the road color. Therefore, when the above-mentioned U value and V value replacement is performed, each pixel of the image in the first area A1 is monotone-colored to a color close to the vehicle body color BC with almost no influence from the road color.

As described above, in the display control device 40A and image display device 10A according to this embodiment, even if a landscape color different from the vehicle body color BC is reflected over a wide area of the vehicle body 2, it is possible to prevent the vehicle body 2 portion from being monotonized in a color significantly different from the original vehicle body color BC. As a result, in addition to the effects described for the display control device 40 and image display device 10 according to the first embodiment described above, the display control device 40A and image display device 10A according to this embodiment can avoid the discomfort felt by the driver caused by the vehicle body portion being monotonized in a color different from the vehicle body color BC.

Third Embodiment
When the vehicle 1 is traveling at night, external light sources such as headlights of vehicles behind, store lights, and traffic lights may be reflected on the vehicle body 2. Areas of the vehicle body 2 where the external light source is reflected have much higher brightness than areas where the external light source is not reflected, which may cause errors when performing monotone conversion processing.

The following describes a display control device 40B and an image display device 10B according to a third embodiment that takes the above points into consideration. The display control device 40B and the image display device 10B according to this embodiment may include the same configuration as the display control device 40 and the image display device 10 according to the first embodiment described above, except for a portion of the configuration of the display control device 40B. Therefore, the following description will focus on the configuration of the display control device 40B and the image display device 10B according to this embodiment that differs from the display control device 40 and the image display device 10 described above, and components similar to those of the display control device 40 and the image display device 10 will be given the same reference numerals and their description will be omitted.

FIG. 11 is a functional block diagram showing an example of the software configuration of a display control device according to a third embodiment of the present disclosure. As shown in FIG. 11, the display control device 40B according to this embodiment may further include a high-brightness area identification unit 57 in addition to the above-mentioned area identification unit 51 and monotone processing unit 52, etc.

The high-brightness area identification unit 57 may be capable of identifying areas of high brightness contained in the captured image (hereinafter referred to as "high-brightness areas HB"). Here, a high-brightness area HB refers to an area having a brightness equal to or higher than a predetermined value, for example, an area composed of pixels whose RGB values total equal to or higher than a predetermined value.

Figure 12 is a schematic diagram showing an example of an image before and after identifying a second region in a captured image. With reference to Figure 12, a part of the process of generating a display image P4 by the display control device 40B according to this embodiment will be described. Note that the following mainly describes the differences from the process of generating a display image described in the first embodiment, and a description of the process similar to that described in the first embodiment will be omitted.

In the display control device 40B according to this embodiment, when the captured image acquisition unit 53 acquires an image P3 (see FIG. 12(A)) including the headlights of the vehicle behind, the area identification unit 51 identifies a first area A1 in the captured image P3, and the high-brightness area identification unit 57 identifies a high-brightness area HB in the captured image P3.

Once the first area A1 and the high-brightness area HB in the captured image P3 have been identified, a second area A2 is identified by excluding the high-brightness area HB from the first area A1, as shown in FIG. 12(B). Then, the image in this second area A2 is subjected to monotone processing by the monotone processing unit 52. The monotone processing can be realized by setting the target area to be the second area A2 instead of the first area A1, and replacing the U value and V value of each pixel of the image in the second area A2 with a third distribution median, which is the median of the color information of the second area A2.

As described above, in the image display device 10B according to this embodiment, the area in which the external light source is reflected is excluded from the area in which the monotone conversion process is performed. As a result, in addition to the effects described for the image display device 10 according to the first embodiment described above, the image display device 10B according to this embodiment can prevent the entire vehicle body 2 from being monotone-converted with a color close to that of the external light source. The generated display image P4 is then a natural image that is unlikely to cause discomfort when viewed by the driver D.

The configurations described as the second and third embodiments can also be used in combination. In other words, the display control device can include both the memory unit 56 and the high-brightness area identification unit 57 described above. In this case, for example, after excluding the high-brightness area HB from the area to be converted to monotone in the captured image, the distribution median, which is the median of the color information of the area to be converted to monotone within the first threshold area TA, can be calculated to perform the monotone conversion process. Therefore, it is possible to obtain the same effects as those described in each of the above-mentioned embodiments.

In each of the above embodiments, the processing executed by the CPU 41 after reading the program stored in the ROM 42 or storage 44 may be executed by various processors other than the CPU 41. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) that is a processor having a circuit configuration designed exclusively to execute a specific process. In addition, the display image generation process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types, for example, by multiple FPGAs, or a combination of a CPU and an FPGA. In addition, the hardware configuration of these various processors may be an electric circuit that combines circuit elements such as semiconductor elements.

This disclosure is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of this disclosure. All of these modifications are included in the technical ideas of this disclosure. In this disclosure, each component may exist alone or in two or more instances, as long as no contradiction arises.

1 Vehicle 2 Vehicle body 10, 10A, 10B Image display device 20 Camera (an example of an imaging device)
30 Monitor (an example of a display device)
40, 40A, 40B Display control device 51 Area identification unit 52 Monotone conversion processing unit 53 Captured image acquisition unit 54 Display image output unit (an example of a display unit)
55 Smoothing filter unit 56 Storage unit 57 High brightness area identification unit A1 First area A2 Second area D Driver P0, P3 Captured image P, P1, P2, P4 Display image TA First threshold area BC Vehicle body color HB High brightness area

Claims (7)

an image acquisition unit that acquires an image of the periphery of a vehicle body;
an area specifying unit that specifies a first area including the vehicle body in the captured image;
a monotone conversion processing unit that converts the image in the first region into monotone;
A display unit that displays a display image including a monotone image on a display device.
Display control device. the monotone conversion processing unit calculates luminance information and color information of the image of the first region, calculates a first distribution median which is a median of the color information of the entire image of the first region, and replaces the color information of the entire image of the first region with the first distribution median.
The display control device according to claim 1 . a storage unit configured to store vehicle color information of the vehicle body and a first threshold region that is set based on a similarity with the vehicle color information;
the monotone conversion processing unit calculates luminance information and color information of the image of the first region, calculates a second distribution median which is a median of the color information of images of the first region whose color information is within the first threshold region, and replaces the color information of the entire first region with the second distribution median.
The display control device according to claim 2 . a high-brightness area specifying unit that specifies a high-brightness area having a brightness equal to or higher than a predetermined value in the captured image;
the monotone conversion processing unit calculates luminance information and color information of a second region obtained by excluding the high-brightness region from the first region, calculates a third distribution median which is a median of the color information of the entire image of the second region, and replaces the color information of the entire image of the second region with the third distribution median.
The display control device according to claim 1 . Further comprising a smoothing filter unit that smoothes the image of the first region.
The display control device according to claim 1 . The smoothing filter unit smoothes the image of the first region when the traveling speed of the vehicle is equal to or higher than a predetermined speed.
The display control device according to claim 5 . A display control device according to any one of claims 1 to 6,
An imaging device capable of imaging the periphery of the vehicle body;
The display device is capable of displaying the display image.
Image display device.