WO2016185678A1

WO2016185678A1 - Blind-spot display device and blind-spot display method

Info

Publication number: WO2016185678A1
Application number: PCT/JP2016/002215
Authority: WO
Inventors: 大貴五藤; 勝之福田; 明江頭
Original assignee: 株式会社デンソー
Priority date: 2015-05-15
Filing date: 2016-04-27
Publication date: 2016-11-24
Also published as: JP2016215725A

Abstract

A blind-spot display device equipped with an image acquisition unit (21), an image generation unit (22), and an image display unit (23). The image acquisition unit (21) acquires a peripheral image that captures a vehicle periphery region which has a blind spot region outside the vehicle caused by blockage of the field of view of a driver by a vehicle frame including a pillar. The image generation unit (22) generates a boundary-visible image which makes a region boundary between the blind spot region and the adjacent region thereto in the vehicle periphery region visible, with the peripheral image acquired by the image acquisition unit as the original image therefor. An image display unit (23) displays the boundary-visible region generated by the image generation unit on a display device provided inside the vehicle compartment.

Description

Blind spot display device and blind spot display method

Cross-reference of related applications

This application is based on Japanese Patent Application No. 2015-1000016 filed on May 15, 2015, the contents of which are incorporated herein by reference.

The present disclosure relates to a blind spot display device and a blind spot display method for displaying an image of a blind spot area by a pillar.

Conventionally, a technology has been proposed in which a display is provided inside each of the pair of left and right front pillars in a vehicle, and an image of a blind spot area outside the vehicle that is generated when the driver's field of view is blocked by the front pillars is displayed (patent) Reference 1).

International Publication No. 2009/157446

In the above-described prior art, for example, when an image of the blind spot area is presented to the driver in an easy-to-understand manner so that the pillar is transmitted, the display must be arranged along the pillar. There is a problem that the vehicle design including the shape of the frame becomes complicated.

The present disclosure has been made in view of the above points, and provides a blind spot display device and a blind spot display method capable of presenting a blind spot image or the like to a driver in an easy-to-understand manner without complicating vehicle design. It is aimed.

The blind spot display device according to an aspect of the present disclosure includes an image acquisition unit, an image generation unit, and an image display unit. The image acquisition unit acquires a peripheral image obtained by imaging a vehicle peripheral region having a blind spot region outside the vehicle, which is generated when a driver's field of view is blocked by a frame including a vehicle pillar.

The image generation unit generates a boundary visualized image in which the region boundary between the blind spot region and the adjacent region in the vehicle peripheral region is visualized using the peripheral image acquired by the image acquisition unit as an original image. The image display unit displays the boundary visualized image generated by the image generation unit on a display device provided in the vehicle interior.

According to such a configuration, by visualizing the region boundary in the display image, it is possible to intuitively let the driver know the image portion corresponding to the blind spot region, and further, the blind spot region and the vehicle peripheral region (or It is possible for the driver to instantly understand the correspondence relationship with the adjacent region) regardless of the shape or arrangement of the display device. Therefore, according to the present disclosure, an image of a blind spot area or the like can be presented to the driver in an easy-to-understand manner without complicating the vehicle design.

In addition, the blind spot display method according to another aspect of the present disclosure acquires and acquires a peripheral image obtained by capturing a vehicle peripheral area having a blind spot area outside the vehicle that is generated when a driver's view is blocked by a frame including a vehicle pillar. A display device that generates a boundary visualization image that visualizes a region boundary between a blind spot region and an adjacent region in a vehicle peripheral region using the peripheral image as an original image, and the generated boundary visualization image is provided in a vehicle interior Including the display.

According to this blind spot display method, the same effects as those already described in the blind spot display device according to the aspect of the present disclosure can be obtained.

The above and other objects, features, and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. The drawing
FIG. 1 is a block diagram illustrating an overall configuration of a blind spot display device according to an embodiment of the present disclosure. FIG. 2 is a top view of a vehicle equipped with a blind spot display device. FIG. 3 is a view showing the interior of a vehicle equipped with a blind spot display device. FIG. 4 is a block diagram showing a functional configuration of the ECU, FIG. 5 is a flowchart of image generation processing according to the first embodiment of the present disclosure. FIG. 6A is a diagram showing a boundary visualized image including a contour line superimposed image; FIG. 6B is a diagram showing a boundary visualization image including a transparent superimposed image; FIG. 7 is a flowchart of image generation processing according to the second embodiment of the present disclosure. FIG. 8 is a diagram illustrating a boundary visualized image including a mask superimposed image.

Hereinafter, embodiments to which the present disclosure is applied will be described with reference to the drawings.

(First embodiment)
A blind spot display device 1 shown in FIG. 1 includes an ECU (Electronic Control Unit) 2, a right camera 3, a left camera 4, a brightness detection unit 5, a right display 6, a left display 7, and a driver camera 8. A body shape DB (Data Base) 9.

The right camera 3 and the left camera 4 have a function of imaging the vehicle peripheral area including the blind spot area of the left and right

front pillars

31 and 32 shown in FIG. The blind spot region is a region outside the vehicle that is generated when the driver's field of view is blocked by the vehicle frame including the left and right

front pillars

31 and 32. Each of the right camera 3 and the left camera 4 is configured by a CMOS camera or the like, and captures an image of a region (that is, a vehicle peripheral region) within the range of the imaging angle of view F (hereinafter referred to as “peripheral image”). Is output to the ECU 2. Hereinafter, of the vehicle peripheral area, the area other than the blind spot area is referred to as an adjacent area, and the boundary between the blind spot area and the adjacent area is referred to as an area boundary.

Referring back to FIG. 1, the ECU 2 is an electronic control unit that controls the entire apparatus. The ECU 2 is configured mainly by the CPU 10 and includes a memory 11 such as a ROM, a RAM, and a flash memory, an input signal circuit, an output signal circuit, a power supply circuit, and the like. In the ECU 2, the CPU 10 acquires peripheral images from the right camera 3 and the left camera 4 based on the program stored in the memory 11, and the blind spot region in the vehicle peripheral region is used as an original image for each acquired peripheral image. Various processes such as generating boundary visualized images in which the boundary between the region and the adjacent region is visualized are performed.

The right display 6 and the left display 7 each have a function of displaying the boundary visualized image generated by the ECU 2. For example, as shown in FIG. 3, the right display 6 is configured by a liquid crystal display or the like provided in the vicinity of the right front pillar 31 in the vehicle interior, and boundary visualization using the peripheral image of the right camera 3 as an original image Display an image. Similarly, the left display 7 is configured by a liquid crystal display or the like provided in the vicinity of the left front pillar 32 in the vehicle interior, and displays a boundary visualized image using a peripheral image of the left camera 4 as an original image.

The brightness detection unit 5 is configured by an illuminance sensor or the like that detects brightness outside the vehicle, and outputs the detection result to the ECU 2.

The driver camera 8 is arranged in the passenger compartment so as to capture a face area including the driver's eyes. The driver camera 8 detects a three-dimensional position of each of the driver's eyes in the captured image using, for example, the eye or corneal reflection as a reference point and the iris or pupil as a moving point using a known gaze detection technique. Then, the driver's line of sight is detected based on the position of the moving point with respect to the reference point. Basically, for example, if the iris of the left eye is far from the eye, the driver is looking at the left side, and if the iris is close to the eye of the left eye, the driver is looking at the right side. By applying such a basic principle, the driver's line-of-sight direction can be obtained in a three-dimensional space from the three-dimensional relative position of the moving point with respect to the reference point. Accordingly, the driver camera 8 functions as an eye point detection unit. The driver camera 8 outputs information indicating the driver's eye position and line-of-sight direction thus detected (hereinafter referred to as “driver camera information”) to the ECU 2.

The body shape DB 9 stores body shape data indicating the three-dimensional position of each part of the frame such as a front pillar constituting the vehicle body. The body shape DB 9 may be built in the memory 11.

Next, the functional configuration of the ECU 2 will be described with reference to the block diagram of FIG.

The ECU 2 functionally includes an image acquisition unit 21, an image generation unit 22, and an image display unit 23. Note that processing for realizing the functions as the image acquisition unit 21, the image generation unit 22, and the image display unit 23 is executed by the CPU 10 based on a program stored in the memory 11.

The image acquisition unit 21 acquires peripheral images from each of the right camera 3 and the left camera 4 in time series, and supplies the acquired peripheral images to the image generation unit 22.

The image generation unit 22 executes a process (hereinafter referred to as “image generation process”) for generating a boundary visualized image using the peripheral image supplied from the image acquisition unit 21 as an original image for each of the right camera 3 and the left camera 4. Then, each of the generated boundary visualized images is supplied to the image display unit 23 in time series.

The image display unit 23 displays the boundary visualized images supplied from the image generation unit 22 in time series for each of the right camera 3 and the left camera 4 as videos on the right display 6 and the left display 7, respectively. Specifically, the boundary visualized video corresponding to the right camera 3 is displayed on the right display 6, and the boundary visualized video corresponding to the left camera 4 is displayed on the left display 7.

Next, image generation processing executed by the image generation unit 22 will be described with reference to the flowchart of FIG. Note that this processing is repeatedly started at predetermined cycles while a switch (not shown) for inputting operation details of start and stop regarding the blind spot display function is on, for example. Moreover, in this process, in order to avoid complexity, the processing target is described as a peripheral image supplied from the image acquisition unit 21 without particularly distinguishing the right camera 3 and the left camera 4, and the right display 6 and the left display 7. To do.

When this processing is started, the image generation unit 22 first acquires driver camera information from the driver camera 8 in step (hereinafter simply referred to as “S”) 110.

Subsequently, in S120, the body shape data is read from the body shape DB 9.

Next, in S130, based on the positional relationship between the position of the driver's eyes and the position of the front pillar, a blind spot area when the driver views the vehicle peripheral area is set on the peripheral image. The driver's eye position is read from the driver camera information, and the front pillar position is extracted from the body shape data. Specifically, using the camera parameters of the

cameras

3 and 4, the blind spot area corresponding to the vector from the driver's eye position to the front pillar position is converted from world coordinates into a camera image. At this time, the vector can also be corrected based on the driver's line-of-sight direction.

Subsequently, in S140, the contour line superimposed image shown in FIG. 6A is generated by superimposing the contour line image corresponding to the region boundary on the peripheral image. As the contour image, an image that transmits the region boundary of the peripheral image is preferable, and an image in which image attributes such as hue, brightness, saturation, and luminance are set to default values in advance is used.

Next, in S150, a transparent superimposed image shown in FIG. 6B is generated by superimposing a transparent image corresponding to the blind spot area on the peripheral image. Specifically, a transparent superimposed image is generated by performing a known filter process on the blind spot area portion of the peripheral image. The transparent image preferably has a vehicle body color or a color imitating translucent acrylic so as to remind the driver of the frame portion including the front pillar, in order to transmit the blind spot area portion of the surrounding image. The default value of which the image attribute is set in advance is used.

Subsequently, in S160, an attribute change process for changing the image attributes of the contour superimposed image and the transparent superimposed image is performed, and the peripheral image including the contour superimposed image and the transparent superimposed image subjected to the attribute change process (that is, boundary visualization) Image) is supplied to the image display unit 23, and this process ends.

For example, in the attribute change process, the brightness and / or brightness relating to each of the contour superimposed image and the transparent superimposed image is changed according to the brightness of the vehicle peripheral region. Specifically, based on the detection result of the brightness detection unit 5, when the illuminance of the vehicle surrounding area is low, the brightness and / or luminance relating to each of the outline superimposed image and the transmission superimposed image is increased, and the vehicle surrounding area is When the illuminance is high, the brightness and / or luminance relating to each of the contour superimposed image and the transparent superimposed image is lowered.

Also, for example, in the attribute change process, the image attributes relating to each of the outline superimposed image and the transparent superimposed image are changed so that the contrast with respect to the surrounding image is higher than a predetermined value. Specifically, when the hue, brightness, saturation, and / or luminance related to the peripheral image supplied from the image acquisition unit 21 is low, the hue, brightness, saturation, and / Or increase the brightness. Further, when the hue, lightness, saturation, and / or luminance related to the peripheral image supplied from the image acquisition unit 21 is high, the hue, lightness, saturation, and / or luminance related to each of the contour superimposed image and the transparent superimposed image. Lower.

According to the first embodiment described in detail above, the following effects can be obtained.

(1a) By visualizing the region boundary in the display image, it is possible to intuitively inform the driver of the image portion corresponding to the blind spot region, and further, the blind spot region and the vehicle peripheral region (or adjacent region) The correspondence can be instantly understood by the driver regardless of the shape and arrangement of the

displays

6 and 7. Therefore, the blind spot area can be presented to the driver in an easy-to-understand manner with an image without complicating the vehicle design.

(2a) Based on the positional relationship between the position of the driver's eyes and the position of the front pillar, in order to set a blind spot area on the peripheral image when the driver views the vehicle peripheral area, for example, a region boundary is displayed in the display image. The contour line to be represented can be moved in accordance with the position of the driver's eyes, and the blind spot area can be presented to the driver more intuitively and in an intuitive manner.

(3a) By superimposing a contour image corresponding to the region boundary on the peripheral image, the driver can clearly know the region boundary in the display image.

(4a) By superimposing a transmission image corresponding to the blind spot area on the peripheral image, the blind spot area can be presented to the driver more intuitively and easily in the display image.

(5a) By changing the image attributes of the contour superimposed image and the transparent superimposed image, for example, by resetting the image attribute so that the contrast with respect to the brightness of the vehicle peripheral region and the peripheral image becomes high, the display image Visibility can be improved.

(Second Embodiment)
Since the basic configuration of the second embodiment is the same as that of the first embodiment, the description of the common configuration will be omitted, and the description will focus on the differences.

In the first embodiment described above, when the boundary visualization image is generated, the transmission image corresponding to the blind spot area is superimposed on the peripheral image. In contrast, the second embodiment is different from the first embodiment in that a mask image corresponding to an adjacent region is superimposed on a peripheral image when generating a boundary visualized image.

Next, image generation processing executed by the image generation unit 22 of the second embodiment instead of the image generation processing (FIG. 5) of the first embodiment will be described with reference to the flowchart of FIG. Note that the processing in S210 to S240 in FIG. 7 is the same as the processing in S110 to S140 in FIG.

When this process is started, the image generation unit 22 first acquires driver camera information from the driver camera 8 in S210.

Subsequently, in S220, the body shape data is read from the body shape DB9.

Next, in S230, based on the positional relationship between the position of the driver's eyes and the position of the front pillar, a blind spot area when the driver views the vehicle peripheral area is set on the peripheral image.

Subsequently, in S240, a contour line superimposed image shown in FIG. 6A is generated by superimposing a contour line image corresponding to the region boundary on the peripheral image.

Next, in S250, the mask superimposed image shown in FIG. 8 is generated by superimposing the mask image corresponding to the adjacent region on the peripheral image. Specifically, a mask superimposed image is generated by performing a known filter process on the adjacent region portion of the peripheral image. As this mask image, the adjacent region portion of the peripheral image is inconspicuous so as to remind the driver that it is a portion other than the blind spot region in the vehicle peripheral region, or that the driver can directly recognize the region. A default value for which the image attribute is set in advance is used.

Subsequently, in S260, an attribute changing process for changing the image attributes of the contour superimposed image and the mask superimposed image is performed, and a peripheral image including the contour superimposed image and the mask superimposed image subjected to the attribute changing process (that is, boundary visualization) Image) is supplied to the image display unit 23, and this process ends.

For example, in the attribute changing process, the brightness and / or luminance relating to each of the contour superimposed image and the mask superimposed image is changed according to the brightness of the vehicle peripheral region. Specifically, based on the detection result of the brightness detection unit 5, when the illuminance in the vehicle peripheral region is low, the brightness and / or luminance relating to each of the contour superimposed image and the mask superimposed image is increased, and the vehicle peripheral region When the illuminance is high, the brightness and / or luminance relating to each of the contour superimposed image and the mask superimposed image is lowered.

Also, for example, in the attribute change process, the image attributes relating to each of the contour superimposed image and the mask superimposed image are changed so that the contrast with respect to the surrounding image is high. Specifically, when the hue, lightness, saturation, and / or luminance related to the peripheral image supplied from the image acquisition unit 21 is low, the hue, lightness, saturation, and / Or increase the brightness. Further, when the hue, lightness, saturation, and / or luminance related to the peripheral image supplied from the image acquisition unit 21 is high, the hue, lightness, saturation, and / or luminance related to each of the contour superimposed image and the mask superimposed image. Lower.

According to the second embodiment described in detail above, in addition to the effects (1a) to (3a) of the first embodiment described above, the following effects can be obtained.

(1b) By superimposing the mask image corresponding to the adjacent area on the peripheral image, the blind spot area in the display image can be presented to the driver more intuitively and easily.

(2b) By changing the image attributes of the contour superimposed image and the mask superimposed image, for example, by resetting the image attribute so that the contrast with respect to the brightness of the vehicle peripheral region and the peripheral image becomes high, the display image Visibility can be improved.

(Other embodiments)
As mentioned above, although embodiment of this indication was described, this indication can take various forms, without being limited to the above-mentioned embodiment.

In the above embodiment, the image attributes of the contour superimposed image and the transparent superimposed image, or the contour superimposed image and the mask superimposed image are changed, but the present invention is not limited to this. For example, at least one image attribute of the contour superimposed image, the transparent superimposed image, and the mask superimposed image may be changed.

In the above embodiment, the boundary visualized image is generated by visualizing the region boundary between the blind spot region of the left and right

front pillars

31 and 32 and the adjacent region. However, the present invention is not limited to this. For example, the boundary visualized image may be generated in the same manner for the blind area of other pillars such as a center pillar and a rear pillar.

The functions of one component in the above embodiment may be distributed as a plurality of components, or the functions of a plurality of components may be integrated into one component. Further, at least a part of the configuration of the above embodiment may be replaced with a known configuration having the same function. Moreover, you may abbreviate | omit a part of structure of the said embodiment. In addition, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified only by the wording described in the claims are embodiments of the present disclosure.

In addition to the blind spot display device 1 described above, a system including the blind spot display device 1 as a constituent element, one or more programs for causing a computer to function as the blind spot display device 1, and one or more programs recording at least a part of the program. The present disclosure can also be realized in various forms such as a plurality of media and a blind spot display method.

Although the present disclosure has been described with reference to the embodiments, it is understood that the present disclosure is not limited to the embodiments and structures. The present disclosure includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more or less, are within the scope and spirit of the present disclosure.

Claims

An image acquisition unit (21) for acquiring a peripheral image obtained by imaging a vehicle peripheral region having a blind spot region outside the vehicle, which is generated when a driver's field of view is blocked by a frame including a pillar of the vehicle;
An image generation unit (22) that generates a boundary visualization image that visualizes a region boundary between the blind spot region and an adjacent region in the vehicle peripheral region, using the peripheral image acquired by the image acquisition unit as an original image;
An image display unit (23) for displaying the boundary visualization image generated by the image generation unit on a display device provided in the vehicle interior;
A blind spot display device comprising:
The blind spot display device according to claim 1,
An eye point detector (8) for detecting the position of the driver's eyes;
Further comprising
The image generation unit (22) is configured to display the vehicle surrounding area when the driver looks at the vehicle surrounding area based on a positional relationship between the eye position detected by the eye point detection unit (8) and the pillar position. Setting a blind spot area on the peripheral image,
Blind spot display device.
The blind spot display device according to claim 1 or 2,
The image generation unit (22) superimposes a contour image corresponding to the region boundary on the peripheral image when generating the boundary visualized image.
Blind spot display device.
A blind spot display device according to any one of claims 1 to 3,
The image generation unit (22) superimposes a transmission image corresponding to the blind spot region on the peripheral image when generating the boundary visualization image.
Blind spot display device.
A blind spot display device according to any one of claims 1 to 4,
The image generation unit (22) superimposes a mask image corresponding to the adjacent region on the peripheral image when generating the boundary visualization image.
Blind spot display device.
A blind spot display device according to any one of claims 1 to 5,
When generating the boundary visualization image, the image generation unit (22) generates an outline image corresponding to the region boundary, a transmission image corresponding to the blind spot region, or the adjacent region on the peripheral image. Changing at least one image attribute of hue, brightness, saturation, and luminance for a superimposed image in which a corresponding mask image is superimposed;
Blind spot display device.
The blind spot display device according to claim 6,
The image generation unit (22) changes at least one of the brightness and the luminance among the image attributes related to the superimposed image according to the brightness of the vehicle surrounding area.
Blind spot display device.
The blind spot display device according to claim 6 or 7,
The image generation unit (22) changes the image attribute relating to the superimposed image so that a contrast is higher than a predetermined value with respect to a peripheral image acquired by the image acquisition unit (21).
Blind spot display device.
A peripheral image obtained by imaging a vehicle peripheral area having a blind spot area outside the vehicle that is generated when a driver's field of view is blocked by a frame including a vehicle pillar is obtained (21);
Using the peripheral image as an original image, generating a boundary visualized image in which a region boundary between the blind spot region and an adjacent region in the vehicle peripheral region is visualized (22);
A blind spot display method including displaying the boundary visualized image on a display device provided in a room of the vehicle (23).