JP6754993B2 - In-vehicle image display device, in-vehicle image display method, and program - Google Patents

Description

The present disclosure relates to an in-vehicle image display device, an in-vehicle image display method, and a program.

In recent years, various systems for supporting driving using an in-vehicle camera have been developed due to advances in camera technology and cost reduction. As one of the systems utilizing the in-vehicle camera, there is an electronic mirror system mounted on a vehicle such as an automobile. The electronic mirror system includes an in-vehicle camera and a display device, and the in-vehicle camera captures the situation of the area outside the vehicle, which was conventionally projected by the optical mirror, and the display device connected to the in-vehicle camera displays the image as an image.

Among the electronic mirror systems, a driving support system that can assist the driver's driving by clarifying the contour of the own vehicle such as the contour of the bumper in a dark situation such as at night is proposed. (See Patent Document 1).

Japanese Unexamined Patent Publication No. 2006-173835

In the case of an electronic mirror system used in place of a side mirror mounted on a vehicle, it may be difficult for an occupant looking at the display device to recognize the boundary between the vehicle and the road, depending on the surrounding conditions. In such a situation, even if the contour image representing the boundary between the area of the own vehicle and other areas is superimposed on the image captured by the in-vehicle camera, the occupant looking at the display device may see the boundary depending on the display color of the contour image. Situations that are difficult to recognize are assumed.

An object of the present disclosure is to provide an in-vehicle image display device that allows an occupant viewing the display device to more clearly and easily recognize the boundary between the area of the own vehicle and other areas.

The in-vehicle image display device according to one aspect of the present disclosure refers to the own vehicle region and the non-own vehicle region from the information indicating the shape of the own vehicle with respect to the captured image acquired from the vehicle exterior imaging unit including a part of the own vehicle in the imaging region. contour image having a boundary specifying unit for specifying a boundary, a color extraction unit for extracting a feature of the color before Symbol vehicle region or the non-subject vehicle area, the display color corresponding to the color feature of including the boundary of the A contour image generation unit that generates a contour image, an image superimposition unit that superimposes the contour image on the captured image to generate a superimposed image, a first region of the non-own vehicle region of the captured image corresponding to a road, and A road shape detecting unit for detecting a second region other than the first region in the non-own vehicle region is provided, and the boundary specifying portion includes the own vehicle region and the first region. The first boundary and the second boundary between the own vehicle region and the second region are specified, and the color extraction unit determines the characteristics of the first color for the first region and the second. The second color feature of the region is extracted individually, and the contour image generation unit determines the display color of the first boundary according to the first color feature, and determines the display color of the first boundary according to the first color feature. The display color of the boundary is determined according to the characteristics of the second color .

In the in-vehicle image display method according to one aspect of the present disclosure, with respect to the captured image acquired from the vehicle exterior imaging unit including a part of the own vehicle in the imaging region, the own vehicle region and the non-own vehicle region are obtained from the information indicating the shape of the own vehicle. the step of generating identifying a boundary extracting pre Symbol vehicle region or the color of the non-vehicle region feature, the outline image having the display color corresponding to the color feature of including the boundary A step of superimposing the contour image on the captured image to generate a superimposed image, and the first region of the captured image corresponding to a road and the first region of the non-own vehicle region. A step of detecting a second region which is a region other than the region of the above, a first boundary between the own vehicle region and the first region, and a second boundary between the own vehicle region and the second region. The step of individually extracting the first color feature for the first region and the second color feature for the second region, and the display of the first boundary. It includes a step of determining a color according to the characteristics of the first color and determining a display color of the second boundary according to the characteristics of the second color .

The program according to one aspect of the present disclosure relates to an captured image obtained from an external imaging unit that includes a part of the own vehicle in the imaging region by a computer of an in-vehicle image display device that displays an image captured from the external imaging unit on the display unit. , the self-boundary specifying unit configured to specify the boundary of the shape from the information indicating the own vehicle region and the non-subject vehicle region of the vehicle, the color extraction section for extracting a feature of the color before Symbol vehicle region or the non-subject vehicle area, wherein contour image generating section that generates a contour image with a display color corresponding to the color characteristics of the inclusive and, before Symbol image superimposition section that generates a superimposed image by superimposing the contour image on the captured image, and, before Symbol imaging It functions as a road shape detecting unit that detects a first region corresponding to a road in the non-own vehicle region of the image and a second region other than the first region in the non-own vehicle region . In the program, the boundary specifying unit identifies a first boundary between the own vehicle region and the first region and a second boundary between the own vehicle region and the second region, and the color. The extraction unit individually extracts the first color feature for the first region and the second color feature for the second region, and the contour image generation unit is the first The display color of the boundary is determined according to the characteristics of the first color, and the display color of the second boundary is determined according to the characteristics of the second color .

According to the present disclosure, it is possible to provide an in-vehicle image display device that allows an occupant viewing the display device to more clearly and easily visually recognize the boundary between the area of the own vehicle and other areas.

It is a block diagram of the vehicle-mounted image display device which concerns on 1st Embodiment. This is an example of an operation flowchart of the in-vehicle video display device according to the first embodiment. It is a figure which shows an example of the captured image which includes a part of own vehicle. This is an example of an average value histogram of the RGB component values of the color of the own vehicle area. This is an example of the RGB component value histogram of the display color of the contour image. It is a figure which shows an example of the contour image. It is a figure which shows an example of the superimposition image. It is a figure which shows an example of the superimposition image which concerns on 2nd Embodiment. It is a block diagram of the vehicle-mounted image display device which concerns on 3rd Embodiment. It is a figure which shows an example of the superimposition image which concerns on 3rd Embodiment. It is a block diagram of the vehicle-mounted image display device which concerns on 4th Embodiment. It is a block diagram of the vehicle-mounted image display device which concerns on 5th Embodiment. It is an operation flowchart of the own vehicle shape extraction part of the vehicle-mounted image display device which concerns on 5th Embodiment. It is a figure which shows an example of the hardware configuration of a computer.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

(First Embodiment)
FIG. 1 is a configuration diagram of an in-vehicle image display device 100 according to the first embodiment. The in-vehicle image display device 100 includes an external image pickup unit 110, a control unit 120, a storage unit 122, a boundary identification unit 123, and a display unit 130.

The vehicle exterior imaging unit 110 captures an image including a part of the own vehicle and outputs the captured image. In one example, the vehicle exterior imaging unit 110 is fixed to the own vehicle. For example, the vehicle exterior imaging unit 110 is a side camera fixed on the right side surface or the left side surface of the own vehicle.

The control unit 120 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. For example, the CPU reads a program according to the processing content from the ROM, develops it in the RAM, and centrally controls the operation of each block of the in-vehicle video display device 100 in cooperation with the expanded program. The control unit 120 functions as a boundary identification unit 123, a color extraction unit 124, a contour image generation unit 125, and an image superimposition unit 126.

The storage unit 122 stores information indicating the shape of a portion of the own vehicle that is included in the image pickup region of the vehicle exterior image pickup unit 110 as, for example, two-dimensional image information. In one example, the storage unit 122 is a non-volatile memory.

The boundary specifying unit 123 identifies the boundary between the own vehicle region and the non-own vehicle region in the captured image of the external imaging unit 110 from the information indicating the shape of the own vehicle. Here, the own vehicle area means an area in which the own vehicle is reflected in the captured image, and the non-own vehicle area means an area other than the own vehicle area in the captured image. In one example, the boundary specifying unit 123 reads information indicating the shape of the own vehicle from the storage unit 122. The color extraction unit 124 extracts the color characteristics of the own vehicle region or the non-own vehicle region in the captured image of the external imaging unit 110.

The contour image generation unit 125 includes the boundary between the own vehicle region and the non-own vehicle region specified by the boundary identification unit 123, and generates a contour image based on the color characteristics extracted by the color extraction unit 124. Here, the contour image generation unit 125 determines the display color of the contour image according to the characteristics of the colors extracted by the color extraction unit 124. In one example, the inverted color of the color extracted by the color extraction unit 124 is determined as the display color of the contour image. Further, in one example, when the color extracted by the color extraction unit 124 is an achromatic color, a conspicuous color such as red is determined as the display color of the contour image. By changing the display color of the contour image to an inverted color or a conspicuous color, the occupant who sees the superimposed image can clearly and easily recognize the boundary between the area of the own vehicle and other areas. The positional relationship between the vehicle and other vehicles can be grasped more easily.

The image superimposing unit 126 superimposes the contour image generated by the contour image generation unit 125 on the captured image output by the vehicle exterior imaging unit 110 to generate a superposed image.

The display unit 130 displays the superimposed image on the occupant. For example, the display unit 130 is an electronic mirror mounted on the own vehicle instead of the optical mirror.

Hereinafter, the operation of the vehicle-mounted image display device 100 according to the first embodiment will be described with reference to the flowchart. FIG. 2 is an example of an operation flowchart of the vehicle-mounted image display device 100 according to the first embodiment. This process is realized by, for example, when the engine of the own vehicle is started, the CPU of the in-vehicle image display device 100 reads and executes the program stored in the ROM.

First, in step S101, the control unit 120 inputs an captured image including a part of the own vehicle.

FIG. 3 is a diagram showing an example of a captured image including a part of the own vehicle. The captured image shown in FIG. 3 is a captured image obtained by capturing the rear of the own vehicle, and includes the own vehicle region 300 and the non-own vehicle region 200. For example, the own vehicle area 300 includes a pillar portion 310, a window portion 320, a window frame portion 330, and a vehicle body portion 340.

Next, in step S102, the control unit 120 acquires the color of the own vehicle area of the captured image, and in step S103, the average values of the RGB component values of the own vehicle area of the captured image are _Rave1 , _Gave1 , and _Bave1 . Extracted as a color feature of the own vehicle area.

FIG. 4A is an example of an average value histogram of the RGB component values of the color of the own vehicle area. The color extraction unit 124 sums the pixel values of the pixels belonging to the own vehicle area in the captured image for each of the R component, the G component, and the B component, for example, and the pixels of the pixels belonging to the own vehicle area in the captured image. By dividing by the number, it is possible to obtain an average value histogram of the RGB component values of the colors of the own vehicle area. In the example shown in FIG. 4A, the average values of the RGB component values of the colors in the own vehicle region are _Rave1 , _Gave1 , and _Bave1 , respectively.

Next, in step S104, the control unit 120 calculates the ratios α and β of the RGB component values. α is the ratio of the second largest value to the maximum value among the average RGB component values of the color of the own vehicle region, R _ave1 , G _{ave 1} , and B _{ave 1} , and β is the second of R _{ave 1} , G _{ave 1} , and B _{ave 1.} The ratio of the minimum value to the large value. The ratios α and β can be calculated by using the following (Equation 1), with the average values R _ave1 , _Gave1 , and _Bave1 being X1, X2, and X3 in descending order.

α = X2 / X1, β = X3 / X2 ... (Equation 1)

As is clear from (Equation 1), the ratios α and β are numbers of 0 or more and 1 or less, respectively.

Next, in step S105, the control unit 120 determines whether or not the extracted color is achromatic. In one example, if the ratios α and β are larger than the predetermined ratio, the contour image generation unit 125 determines that the extracted color is achromatic, and if either of the ratios α and β is smaller than the predetermined ratio, the control is performed. The unit 120 determines that the color of the own vehicle area is not achromatic.

If it is determined that the color is achromatic (step S105: YES), in step S106, the control unit 120 sets the display color of the contour image to a predetermined color (step S106), and proceeds to step S108.

In one example, in step S106, the control unit 120 determines a conspicuous predetermined color such as red as the display color of the contour image. Here, the predetermined ratio is an arbitrary value larger than 0 and smaller than 1, for example, 0.9. For example, when the color of the own vehicle is white, the display color of the contour image is likely to be red.

On the other hand, if it is determined that the color is not achromatic (step S105: NO), in step S107, the control unit 120 determines the inverted color as the display color of the contour image, and proceeds to step S108. In one example, in step S107, the contour image generation unit 125 sets the RGB component values Y1, Y2, Y3 of the display color of the contour image corresponding to the average RGB component values X1, X2, X3 to Y1 = α × β × Calculate C, Y2 = β × C, Y3 = C. Here, C is a predetermined arbitrary value, and for example, when the RGB component value is equal to the maximum value that can be taken and the RGB component value is each represented by 8 bits, C = 255. Alternatively, for example, C = 250 or C = 200 may be used.

In the example shown in FIG. 4A, _Gave1 = α × R _ave1 and _Bave1 = β × _Gave1 , and α <0.9 and β <0.9 hold. In this case, the inverted color of the color of the own vehicle area is determined as the display color of the contour image.

FIG. 4B is an example of RGB component values of the display color of the contour image. While the relationship of _{R ave1>} G _{ave1> B} ave1 is established between the average value _{R ave1,} G _{ave1, B} ave1 color RGB component values of the vehicle region shown in Figure 4A, Figure 4B The relationship of R _out <G _out <B _out is established between the RGB component values R _out , G _out , and B _out of the outline image shown, and the relationship of the magnitude of the RGB component values is reversed. This is because the inverted color of the color of the own vehicle area is determined as the display color of the contour image.

Next, in step S108, the control unit 120 generates a contour image.

FIG. 5 is a diagram showing an example of a contour image. The contour image includes at least a part of the boundary included in the imaging region, and the image is not particularly limited as long as the occupant can clearly recognize the boundary between the own vehicle area and the non-own vehicle area. For example, as shown in FIG. 5, the contour image includes a boundary 312 between the pillar portion and the non-own vehicle region and a boundary 342 between the vehicle body portion and the non-own vehicle region. Further, for example, as shown in FIG. 5, the contour image may further include an image that highlights the window frame portion 332 as well as the boundary. Further, the contour image may further include an image in which the own vehicle area is hatched or a translucent image filled with a translucent color.

Next, in step S109, the control unit 120 superimposes the contour image on the captured image.

FIG. 6 is a diagram showing an example of a superimposed image. In the superimposed image shown in FIG. 6, a contour image including the boundary 350 between the vehicle body portion and the pillar portion and the non-own vehicle region and the window frame portion 334 is superimposed on the captured image obtained by capturing the rear of the vehicle. .. As described above, the display color of the contour image is determined according to the color characteristics of the own vehicle region or the non-own vehicle region extracted by the color extraction unit 124.

As described above, the in-vehicle image display device 100 according to the first embodiment is specified from the information indicating the shape of the own vehicle with respect to the captured image acquired from the external imaging unit 110 including a part of the own vehicle in the imaging region. It has a boundary specifying unit 123 for specifying the boundary between the own vehicle area and the non-own vehicle area, a color extracting unit 124 for extracting the color characteristics of the own vehicle area, and a display color including the boundary and corresponding to the color characteristics. A configuration including a contour image generation unit 125 for generating a contour image and an image superimposition unit 126 for superimposing a contour image on an captured image to generate a superposed image is adopted.

According to the in-vehicle image display device 100 according to the first embodiment, the occupant looks at the superimposed image instead of looking at the image of the rear of the vehicle as it is, thereby viewing the own vehicle area and the non-own vehicle. The boundary of the area can be clearly and easily recognized, and the positional relationship between the own vehicle and another vehicle can be more easily grasped.

(Second Embodiment)
FIG. 7 is a diagram showing an example of a superimposed image according to the second embodiment. In the first embodiment, the color extraction unit 124 extracts the color of the own vehicle area 400, whereas in the second embodiment, the color extraction unit 124 extracts the color of the non-own vehicle area 500. The second embodiment is different from the first embodiment in that the first embodiment is extracted.

As described above, the in-vehicle image display device 100 according to the second embodiment is specified from the information indicating the shape of the own vehicle with respect to the captured image acquired from the external imaging unit 110 including a part of the own vehicle in the imaging region. A boundary specifying unit 123 that specifies the boundary between the own vehicle area and the non-own vehicle area, a color extraction unit 124 that extracts the color characteristics of the non-own vehicle area, and a display color that includes the boundary and corresponds to the color characteristics. A configuration is adopted in which a contour image generation unit 125 that generates a contour image having the contour image and an image superimposition unit 126 that superimposes the contour image on the captured image to generate a superimposed image are provided. In this case, the display color of the boundary 502 between the own vehicle area 400 and the non-own vehicle area 500 is, for example, the inverted color of the color of the non-own vehicle area 500.

According to the in-vehicle image display device 100 according to the second embodiment, even when the own vehicle enters the tunnel, another vehicle approaches, or the like, the occupant is in the own vehicle area. And the boundary of the non-own vehicle area can be clearly and easily recognized.

(Third Embodiment)
FIG. 8 is a configuration diagram of the vehicle-mounted image display device 600 according to the third embodiment. The in-vehicle image display device 600 includes an external image pickup unit 110, a storage unit 122, a control unit 620, and a display unit 130. Of these, the vehicle exterior imaging unit 110, the storage unit 122, and the display unit 130 are the same as those provided in the vehicle-mounted image display device 100 according to the first embodiment, and thus description thereof will be omitted here.

The control unit 620 functions as a road shape detection unit 622, a boundary identification unit 623, a color extraction unit 624, a contour image generation unit 625, and an image superimposition unit 626.

The road shape detection unit 622 detects a first region corresponding to the road in the non-own vehicle region of the captured image and a second region excluding the first region in the non-own vehicle region. The region corresponding to the road can be detected by a known image processing means such as detecting a step or a white line from the captured image and detecting the region corresponding to the road based on the detected step or the white line.

The boundary specifying unit 623 identifies the first boundary between the own vehicle region in the captured image and the first region detected by the road shape detecting unit 622, and further, the own vehicle region and the road shape in the captured image. The second boundary with the second region detected by the detection unit 622 is specified.

The color extraction unit 624 has a first color feature for the first region detected by the road shape detection unit 622 and a second color feature for the second region detected by the road shape detection unit 622. Extract individually. The individually extracted first color features and the second color features are generally different.

The contour image generation unit 625 determines the display color of the first boundary specified by the boundary identification unit 623 according to the characteristics of the first color extracted by the color extraction unit 624, and the boundary identification unit 623 specifies the first color. The display color of the boundary of 2 is determined according to the characteristics of the second color extracted by the color extraction unit 624. The contour image generation unit 625 generates a contour image based on the determined display color.

The image superimposing unit 626 superimposes the contour image generated by the contour image generation unit 625 on the captured image output by the vehicle exterior imaging unit 110 to generate a superposed image.

FIG. 9 is a diagram showing an example of a superimposed image according to the third embodiment. The road shape detection unit 622 is a second region of the non-own vehicle region 500 in the captured image excluding the first region 510 corresponding to the road and the non-own vehicle region 500 excluding the first region 510. Region 520 and is detected.

The boundary specifying unit 623 identifies a first boundary 512 between the own vehicle area 400 and the first area 510 and a second boundary 522 between the own vehicle area 500 and the second area 520. The color extraction unit 624 individually extracts the first color feature for the first region 510 and the second color feature for the second region 520. Next, the contour image generation unit 625 generates a contour image including the first boundary 512 and the second boundary 522 and having a display color corresponding to the first color feature and the second color feature.

Here, the display color of the first boundary 512 is determined according to the characteristics of the first color individually extracted for the first region 510, and the display color of the second boundary 522 is the second region. It is determined according to the characteristics of the second color extracted individually for 520. As a result, as shown in FIG. 9, the boundaries between the first region 510 and the second region 520 and the own vehicle region 400 are displayed in different display colors.

As described above, the in-vehicle image display device 600 according to the third embodiment has the first region 510 corresponding to the road and the first region 500 in the non-own vehicle region 500 of the non-own vehicle region 500 of the captured image. A road shape detecting unit 622 for detecting a second area 520 which is a region other than the above is further provided, and the boundary specifying unit 623 includes a first boundary 512 between the own vehicle area 400 and the first area 510 and the own vehicle area 400. The second boundary 522 between the and the second region 520 is specified, and the color extraction unit 624 determines the first color feature for the first region 510 and the second color feature for the second region 520. And are individually extracted, and the contour image generation unit 625 determines the display color of the first boundary 512 according to the characteristics of the first color, and determines the display color of the second boundary 622 as the second color. Determined according to the characteristics of.

According to the vehicle-mounted image display device 600 according to the third embodiment, even when the first region 510 and the second region 520 have a relationship close to the inverted color, the occupant can use the own vehicle region. And the boundary of the non-own vehicle area can be clearly and easily recognized.

(Fourth Embodiment)
FIG. 10 is a configuration diagram of the vehicle-mounted image display device 700 according to the fourth embodiment. The in-vehicle image display device 700 includes an external image pickup unit 110, a storage unit 122, an in-vehicle image pickup unit 140, a control unit 720, and a display unit 130. Of these, the vehicle exterior imaging unit 110, the storage unit 122, and the display unit 130 are the same as those provided in the vehicle-mounted image display device 100 according to the first embodiment, and thus the description thereof will be omitted here.

The vehicle interior imaging unit 140 is provided inside the vehicle and images the driver. The in-vehicle imaging unit 140 is, for example, a surveillance camera that monitors the state of the driver.

The control unit 720 includes a line-of-sight detection unit 722, a boundary identification unit 723, a color extraction unit 724, a contour image generation unit 725, and an image superimposition unit 726.

The line-of-sight detection unit 722 analyzes the driver's image captured by the in-vehicle image pickup unit 140 and detects the driver's line-of-sight direction. As a means for analyzing the image of a person and detecting the line-of-sight direction of the person, a known means may be used, and description thereof will be omitted here.

The boundary specifying unit 723 identifies the boundary between the own vehicle region and the non-own vehicle region in the captured image from the information indicating the shape of the own vehicle. The color extraction unit 724 extracts the color characteristics of the own vehicle region or the non-own vehicle region in the captured image.

The contour image generation unit 725 generates a contour image based on the shape of the own vehicle and the color characteristics extracted by the color extraction unit 724. Here, the display color of the contour image is determined according to the characteristics of the colors extracted by the color extraction unit 724.

Further, the contour image generation unit 725 changes the display color of the contour image based on the line-of-sight direction detected by the line-of-sight detection unit 722. For example, when it is detected that the driver's line of sight is in the direction of the display unit 130, the contour image generation unit 725 changes the display color of the contour image to a brighter color, and if not, the contour image. Change the display color of to a darker color. The image superimposing unit 726 superimposes the contour image generated by the contour image generation unit 725 on the captured image output by the vehicle exterior imaging unit 110 to generate a superposed image.

As described above, the in-vehicle image display device 700 according to the fourth embodiment analyzes the image of the driver output by the in-vehicle image pickup unit 140 that images the inside of the vehicle and detects the driver's line-of-sight direction 722. The contour image generation unit 725 changes the display color based on the line-of-sight direction.

According to the in-vehicle image display device 700 according to the fourth embodiment, the occupant looking at the display unit 130 can clearly and easily identify the own vehicle area and the non-own vehicle area while reducing the power consumption of the display unit 130. You will be able to recognize it. Further, when the line-of-sight direction of the driver is not directed to the display unit 130, the display color of the contour image is darkened, which has the effect of reducing the annoyance for the driver. The side mirrors are in the driver's field of view even if the driver is not gazing at the side mirrors while driving. Therefore, if the contour image is always bright, the driver's attention will be directed to the side mirror even in a scene where the driver's field of view does not need to be watched. Therefore, when the driver's line of sight does not face the direction of the display unit 130, the display color of the contour image is darkened so that the driver does not need to look at the side mirror. You can prevent it from happening.

(Fifth Embodiment)
FIG. 11 is a configuration diagram of the vehicle-mounted image display device 800 according to the fifth embodiment. The in-vehicle image display device 800 includes an external image pickup unit 110, a storage unit 122, a control unit 820, and a display unit 130. Of these, the vehicle exterior imaging unit 110, the storage unit 122, and the display unit 130 are the same as those provided in the vehicle-mounted image display device 100 according to the first embodiment, and thus description thereof will be omitted here.

The control unit 820 functions as the own vehicle shape extraction unit 822, the boundary identification unit 823, the color extraction unit 824, the contour image generation unit 825, and the image superimposition unit 826.

The own vehicle shape extraction unit 822 detects an edge portion from the captured image output by the vehicle exterior imaging unit 110, and extracts the shape of the portion of the own vehicle included in the imaging region based on the detected edge portion. .. The portion included in the imaging region is, for example, a pillar portion, a window portion, a window frame portion, and a vehicle body portion. Since the edge portion can be detected by using a known method such as image processing using the Prewitt operator or the Sobel operator, the description thereof is omitted here. In one example, the storage unit 122 stores information indicating the shape extracted by the own vehicle shape extraction unit 822.

The boundary specifying unit 823 identifies the boundary between the own vehicle region and the non-own vehicle region in the captured image from the shape extracted by the own vehicle shape extraction unit 822. In one example, the boundary specifying unit 823 reads the information indicating the shape from the storage unit 122. The color extraction unit 824 extracts the color characteristics of the own vehicle region or the non-own vehicle region in the captured image.

The contour image generation unit 825 generates a contour image based on the shape extracted by the own vehicle shape extraction unit 822 and the color characteristics extracted by the color extraction unit 824. Here, the display color of the contour image is determined according to the characteristics of the colors extracted by the color extraction unit 824.

The image superimposing unit 826 superimposes the contour image generated by the contour image generation unit 825 on the captured image output by the vehicle exterior imaging unit 110 to generate a superposed image.

As described above, the in-vehicle image display device 800 according to the fifth embodiment detects the edge portion from the captured image output by the vehicle exterior imaging unit 110 and extracts the shape of the own vehicle based on the edge portion. The extraction unit 822 is further provided, and the boundary identification unit 823 specifies the boundary based on the shape extracted by the own vehicle shape extraction unit 822.

According to the in-vehicle image display device 800 according to the fifth embodiment, even when the in-vehicle image display device 800 is retrofitted to the own vehicle, the in-vehicle image display device 800 has the own vehicle area and the non-own vehicle area. The boundary with and can be displayed on the display unit 130. Therefore, the in-vehicle image display device 800 can be sold not only as a genuine product of the own vehicle in a state of being mounted on the own vehicle, but also by itself.

FIG. 12 is an operation flowchart of the own vehicle shape extraction unit 822 of the vehicle-mounted image display device 800 according to the fifth embodiment.

First, in step S201, the control unit 820 detects the edge portion from the captured image output by the vehicle exterior imaging unit 110. Next, in step S202, the control unit 820 determines whether or not the shape of the own vehicle can be extracted. When the extraction is possible (step S202: YES), in step S203, the control unit 820 stores the extracted contour shape of the own vehicle in the storage unit 122, and ends the flow. If extraction is not possible (step S202: NO), the process returns to step S201.

The cycle for the own vehicle shape extraction unit 822 to extract the shape of the own vehicle is, for example, every time the engine is started, every predetermined period (1 month, 3 months), etc., so that the external image pickup unit 110 due to aging changes. It is possible to deal with the misalignment of.

FIG. 13 is a diagram showing an example of the hardware configuration of the computer 2100. The functions of each part in each of the above-described embodiments and modifications are realized by a program executed by the computer 2100.

As shown in FIG. 13, the computer 2100 includes an input device 2101 such as an input button and a touch pad, an output device 2102 such as a display and a speaker, a CPU (Central Processing Unit) 2103, a ROM (Read Only Memory) 2104, and a RAM (Random). Access Memory) 2105 is provided. The computer 2100 is a reading device that reads information from a hard disk device, a storage device 2106 such as an SSD (Solid State Drive), a recording medium such as a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. 2107, a transmission / reception device 2108 that communicates via a network is provided. The above-mentioned parts are connected by the bus 2109.

Then, the reading device 2107 reads the program from the recording medium on which the program for realizing the functions of the above parts is recorded, and stores the program in the storage device 2106. Alternatively, the transmission / reception device 2108 communicates with the server device connected to the network, and stores the program downloaded from the server device for realizing the functions of the above parts in the storage device 2106.

Then, the CPU 2103 copies the program stored in the storage device 2106 to the RAM 2105, and sequentially reads and executes the instructions included in the program from the RAM 2105, thereby realizing the functions of the above-mentioned parts. Further, when the program is executed, the information obtained by the various processes described in each embodiment is stored in the RAM 2105 or the storage device 2106, and is appropriately used.

(Other embodiments)
In one example, the contour image generation unit 125 adjusts the brightness of the display color of the generated contour image according to the captured image output by the vehicle exterior imaging unit 110. For example, when the average image level (APL) of the captured image output by the vehicle exterior imaging unit 110 increases, the contour image generation unit 125 increases the brightness of the display color of the generated contour image. By doing so, for example, even when receiving a high beam from the rear vehicle, the occupant looking at the display unit 130 can clearly and easily recognize the boundary between the own vehicle area and the non-own vehicle area. ..

In one example, the contour image generation unit 125 applies a gradation in the contour width direction of the boundary to the portion corresponding to the boundary included in the contour image. In this case, the brightness of the gradation may increase from the own vehicle region to the non-own vehicle region in the superimposed image. In this way, the occupant looking at the display unit 130 can clearly define the boundary between the own vehicle area and the non-own vehicle area as compared with the case where the line showing the shape of the own vehicle is displayed in a single color on the display unit 130. It will be easy to recognize.

In the fifth embodiment, the storage unit 122 stores the information indicating the shape extracted by the own vehicle shape extraction unit 822, and the boundary specifying unit 823 reads out the information indicating the shape stored in the storage unit 122. Instead of this, the boundary specifying unit 823 may directly acquire the shape extracted by the own vehicle shape extracting unit 822.

The in-vehicle image display device according to the present disclosure is suitable to be mounted on a vehicle instead of a side mirror and used as a side mirror.

100 In-vehicle image display device 110 External imaging unit 120 Control unit 122 Storage unit 123 Boundary specifying unit 124 Color extraction unit 125 Contour image generation unit 126 Image superimposition unit 130 Display unit 140 In-vehicle imaging unit 200 Non-own vehicle area 300 Own vehicle area 310 Pillar Part 312 Boundary 320 Window part 330,332,334 Window frame part 340 Body part 342 Boundary 350 Boundary 400 Own vehicle area 500 Non-own vehicle area 502 Boundary 510 First area 512 First boundary 520 Second area 522 Second area Boundary

Claims (14)

With respect to the captured image acquired from the external imaging unit including a part of the own vehicle in the imaging region, the boundary specifying portion for specifying the boundary between the own vehicle region and the non-own vehicle region from the information indicating the shape of the own vehicle, and the boundary specifying portion.
A color extraction unit that extracts color characteristics of the own vehicle region or the non-own vehicle region,
A contour image generation unit that generates a contour image including the boundary and having a display color corresponding to the characteristics of the color,
An image superimposing unit that superimposes the contour image on the captured image to generate a superposed image,
A road shape detection unit that detects a first region corresponding to a road in the non-own vehicle region of the captured image and a second region other than the first region in the non-own vehicle region.
With
The boundary specifying unit identifies a first boundary between the own vehicle region and the first region and a second boundary between the own vehicle region and the second region.
The color extraction unit individually extracts the first color feature for the first region and the second color feature for the second region.
The contour image generation unit determines the display color of the first boundary according to the characteristics of the first color, and determines the display color of the second boundary according to the characteristics of the second color. In-vehicle video display device to decide. Wherein the color extraction unit, the average value of the RGB component values of the vehicle region or the non-subject vehicle area, extracted as color feature of the vehicle region or the non-subject vehicle region,
When the contour image generation unit determines that the color of the own vehicle region or the non-own vehicle region is not achromatic, the contour image generation unit displays the contour image by displaying the inverted color of the color of the own vehicle region or the non-own vehicle region. The vehicle-mounted image display device according to claim 1 , which is determined as a color. Wherein the color extraction unit, the average value of the RGB component values of the vehicle region or the non-subject vehicle area, extracted as color feature of the vehicle region or the non-subject vehicle region,
The vehicle-mounted image display according to claim 1, wherein the contour image generation unit determines the display color of the contour image to be red when the color of the own vehicle region or the non-own vehicle region is determined to be achromatic. apparatus. The vehicle-mounted image display device according to claim 2 or 3 , wherein the contour image generation unit applies a gradation in the contour width direction to a portion corresponding to a boundary included in the contour image. The vehicle-mounted image display device according to claim 4 , wherein the contour image generation unit increases the brightness of the contour image from the own vehicle region to the non-own vehicle region in the gradation. It is further equipped with a line-of-sight detection unit that analyzes the driver's image output by the in-vehicle image pickup unit that captures the inside of the vehicle and detects the driver's line-of-sight direction.
The vehicle-mounted image display device according to any one of claims 2 to 5 , wherein the contour image generation unit changes the display color based on the line-of-sight direction. A self-vehicle shape extraction unit is further provided, which detects an edge portion from an image captured by the external imaging unit and extracts the shape of the own vehicle based on the edge portion.
The vehicle-mounted image display device according to any one of claims 2 to 6, wherein the boundary specifying unit specifies the boundary based on the shape extracted by the own vehicle shape extracting unit. The vehicle-mounted image display device according to any one of claims 2 to 7, wherein the contour image generation unit adjusts the brightness of the display color of the contour image according to the average image level of the captured image. With respect to the captured image acquired from the external imaging unit including a part of the own vehicle in the imaging region, the step of identifying the boundary between the own vehicle region and the non-own vehicle region from the information indicating the shape of the own vehicle, and
The step of extracting the color characteristics of the own vehicle region or the non-own vehicle region, and
A step of generating a contour image including the boundary and having a display color corresponding to the characteristics of the color, and
A step of superimposing the contour image on the captured image to generate a superposed image,
A step of detecting a first region corresponding to a road in the non-own vehicle region of the captured image and a second region other than the first region in the non-own vehicle region.
A step of identifying a first boundary between the own vehicle region and the first region and a second boundary between the own vehicle region and the second region.
A step of individually extracting the first color feature for the first region and the second color feature for the second region.
A step of determining the display color of the first boundary according to the characteristics of the first color, and determining the display color of the second boundary according to the characteristics of the second color. In-vehicle image display method to be provided. When it is determined that the color of the own vehicle region or the non-own vehicle region is not achromatic in the step of generating the contour image, the inverted color of the color of the own vehicle region or the non-own vehicle region is the contour. The vehicle-mounted image display method according to claim 9, which is determined as an image display color. In the step of generating a pre-Symbol outline image, the color of the vehicle region or the non-subject vehicle region if it is determined that the achromatic, display color of the contour image is determined to red, to claim 9 The in- vehicle image display method described . A computer of an in-vehicle image display device that displays an image captured from an external imaging unit on the display unit.
A boundary specifying unit that identifies the boundary between the own vehicle region and the non-own vehicle region from the information indicating the shape of the own vehicle with respect to the captured image acquired from the external imaging unit that includes a part of the own vehicle in the imaging region.
A color extraction unit that extracts color characteristics of the own vehicle region or the non-own vehicle region,
A contour image generation unit that generates a contour image including the boundary and having a display color corresponding to the characteristics of the color.
An image superimposing unit that superimposes the contour image on the captured image to generate a superposed image, and
It functions as a road shape detecting unit that detects a first region corresponding to a road in the non-own vehicle region of the captured image and a second region other than the first region in the non-own vehicle region. It ’s a program that lets you
The boundary specifying unit identifies a first boundary between the own vehicle region and the first region and a second boundary between the own vehicle region and the second region.
The color extraction unit individually extracts the first color feature for the first region and the second color feature for the second region.
The contour image generation unit determines the display color of the first boundary according to the characteristics of the first color, and determines the display color of the second boundary according to the characteristics of the second color. The program to decide. Before SL contour image generating section, wherein when the color of the vehicle region or the non-subject vehicle region is determined not to be achromatic, the inverted color of the color of the vehicle region or the non-subject vehicle region, of the contour image The program according to claim 12 , which is determined as a display color. Before SL contour image generating section, wherein when the color of the vehicle region or the non-subject vehicle region is determined to be achromatic, determines the display color of the contour image to red, the program according to claim 12 ..

Images