JP2019001226A - Electronic mirror device - Google Patents

Abstract

To provide an electronic mirror device which can perform brightness control suitable for snowy road travel.SOLUTION: An electronic mirror device 100 comprises a color sensor 120 on a front face of a half mirror 162. The electronic mirror device 100 compares an RGB value of light inputted to the color sensor 120 and an RGB value of a snow surface estimation region of imaging data imaged by an imaging part 110 with each other, and determines that a vehicle travels on a snow surface when both are coincident with each other. Since reflectance from the snow surface enters in the vehicle when it is determined that the vehicle travels on the snow surface, control for increasing a brightness of a back light 166 is performed.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an electronic mirror device mounted on a vehicle, and more particularly to brightness control of the electronic mirror device.

  Electronic mirrors have been proposed as an alternative to conventional room mirrors. The electronic mirror captures the rear of the vehicle with an imaging camera, displays the captured image on a display such as a liquid crystal, and displays various warnings for approaching vehicles behind the vehicle. By using such an electronic mirror, the blind spot of the optical room mirror and the poor visibility at night can be improved.

  Various techniques have been proposed for controlling the brightness of electronic mirror displays. For example, Patent Document 1 determines the brightness outside the vehicle using an in-vehicle camera, and controls the brightness of the electronic mirror based on the determination result. Patent document 2 adjusts the brightness | luminance of an electronic mirror based on the weather information and time information which were acquired from the outside.

JP 2010-173344 A JP 2007-43333 A

  FIG. 7A shows the operation of the electronic mirror when traveling on a normal road surface. An electronic mirror 2 is installed on the ceiling 1, and the electronic mirror 2 is configured by laminating a backlight 3, a liquid crystal display 4 and a half mirror 5, for example. When the surroundings are dark, such as at night, the half mirror 5 functions as a transmission window, and the driver U can view the captured image behind the vehicle displayed on the liquid crystal display 4 via the half mirror 5. Further, when the surroundings are bright such as in the daytime, the half mirror 5 also functions as a reflection mirror, and the driver U can see the scenery behind the vehicle reflected by the half mirror 5 and is displayed on the liquid crystal display 4 at the same time. An image such as a warning can be seen superimposed on the reflected image. When traveling on a normal road surface, the brightness of the ceiling 1 behind the driver U is not reflected in the half mirror 5 because the ceiling 1 is dark even during the daytime.

  FIG. 7B shows the operation of the electronic mirror when traveling on a snowy road. When traveling on a snowy road, if sunlight is reflected on the snow surface, the reflected light may enter the vehicle and the ceiling 1 behind the driver U may become bright. When the ceiling 1 becomes bright, the brightness is reflected on the half mirror 5, and the driver U has difficulty in viewing the image of the half mirror 5. In particular, when there is residual snow while it is fine, there are many irregular reflections of sunlight, and a part of it is strongly incident on the ceiling 1. In such a case, by increasing the luminance of the backlight 3, it is possible to relatively reduce the brightness on the ceiling 1 side and reduce the reflection of light on the ceiling 1. However, since the conventional luminance control of the electronic mirror is linked to an illumination signal or an external light sensor built in the electronic mirror 2, the luminance control corresponding to the brightness of the ceiling 1 cannot be performed.

  The present invention solves such a conventional problem, and an object thereof is to provide an electronic mirror device capable of performing brightness control suitable for running on a snowy road.

  An electronic mirror device according to the present invention includes an imaging unit that captures an image of the periphery of a vehicle, a display unit that can display imaging data captured by the imaging unit, and detection that detects color information of light incident on the display unit. Means for comparing the color information detected by the detection means with the color information of the specific area of the imaging data, and a brightness control means for controlling the brightness of the display unit based on the comparison result of the comparison means Have.

  Preferably, the luminance control unit increases the luminance of the display unit when the color information of the detection unit matches or approximates the color information of the specific area. Preferably, the color information is an R, G, B value. Preferably, the specific area is an area estimated to be white. Preferably, the specific area is an area estimated to be a snow surface. Preferably, the mirror device further includes estimation means for estimating the specific area based on the imaging data. Preferably, the display unit includes a half mirror, a liquid crystal display, and a backlight, and the luminance control unit controls the luminance of the backlight. Preferably, the electronic mirror device further includes an image control unit that controls an image displayed on the display unit, and the image control unit displays an image of a vehicle rear or an augmented reality image on the display unit.

  According to the present invention, since the luminance of the display unit is controlled based on the color information of the light incident on the display unit in the vehicle and the color information of the specific area of the imaging data, the display unit suitable for running on a snowy road Brightness control can be performed.

It is a figure which shows the structure and arrangement | positioning in a vehicle of the electronic mirror apparatus based on the Example of this invention. It is a figure which shows the functional structure of the brightness | luminance control program which concerns on the Example of this invention. It is an illustration of the output waveform of the color sensor which concerns on the Example of this invention. It is an illustration of the imaging data which concerns on the Example of this invention. It is the figure which represented typically the RGB value calculated by the snow surface area | region RGB calculation part which concerns on the Example of this invention. It is a flowchart which shows operation | movement of the electronic mirror apparatus based on the Example of this invention. FIG. 7A is a diagram for explaining the operation of the conventional electronic mirror during normal road surface travel, and FIG. 7B is a diagram for explaining the operation of the conventional electronic mirror during snowy road travel.

  Next, an embodiment of the present invention will be described. In a preferred embodiment, the electronic mirror device of the present invention functions as a room mirror or a rearview mirror installed in a vehicle such as an automobile. In a preferred embodiment, the electronic mirror device of the present invention includes a liquid crystal display and a half mirror disposed on the front surface of the liquid crystal display, and displays a captured image of the rear of the vehicle or an image for augmented reality on the liquid crystal display. By doing so, a variety of information is provided to the driver.

  FIG. 1 is a diagram illustrating a configuration of an electronic mirror device according to an embodiment of the present invention. The electronic mirror device 100 of this embodiment includes an imaging unit 110, a color sensor 120, a mode switching information acquisition unit 130, an LCD driving unit 140, a backlight driving unit 150, a display unit 160, and a control unit 170. . Although not shown here, the electronic mirror device 100 may include a memory that stores image data and the like for displaying augmented reality.

  The imaging unit 110 captures the rear of the vehicle and provides the captured video data to the control unit 170. The control unit 170 can detect the imaged video data on the display unit 160 or perform image processing on the video data, thereby detecting an object such as a rear vehicle imaged there.

  The display unit 160 includes a half mirror 162, a liquid crystal display 164, a backlight 166, and a support substrate (not shown) in order from the surface side visually recognized by the driver. The laminated structure of the half mirror 162, the liquid crystal display 164, and the backlight 166 constitutes a mirror part, and the mirror part is fixed on the support substrate. An external light sensor that detects external light may be provided around the support substrate.

  The half mirror 162 transmits part of the incident light and reflects the rest. The reflectance and the transmittance are not necessarily equal. The half mirror 162 functions as a mirror (reflecting mirror) from the bright side, and functions as a translucent window from the dark side. Therefore, for example, if the driver's seat side is bright during daytime driving, the half mirror 162 can function as a mirror that optically reflects the scenery behind the vehicle. At this time, an image is displayed on the liquid crystal display 164. For example, the image can be superimposed on the reflected image of the half mirror 162. In one example, when a rear vehicle approaching the host vehicle is detected, an augmented reality (AR) graphic such as a warning frame surrounding the rear vehicle is displayed on the liquid crystal display 164. The detection of the rear vehicle is, for example, detected from the difference between the imaging data obtained by imaging the rear. Further, if the driver's seat side is dark when traveling at night, a part of the image generated on the liquid crystal display 164 passes through the half mirror 162, and the driver U can see this image. At this time, the image data of the vehicle rear imaged by the imaging unit 110 is displayed on the liquid crystal display 164. At this time, the half mirror 162 can play a role of an anti-glare function for the headlight light of the vehicle behind. .

  The liquid crystal display 164 is a display medium that controls the amount of transmitted light and displays an image, and is disposed on the back side of the half mirror 162. As is well known, the liquid crystal display 164 arranges a plurality of liquid crystal elements in a two-dimensional manner, and drives each liquid crystal element from the X direction and the Y direction, thereby transmitting light through the entire screen or a selected region. Can be varied. For example, a selected area of the liquid crystal display 164 can be used as a reflection area, an AR image can be displayed on the liquid crystal display 164, or an image captured by the imaging unit 110 can be displayed. Preferably, the liquid crystal display 164 is normally black and does not transmit light and reflects light from the half mirror 162 when the power is turned off.

  The backlight 166 is disposed on the back side of the liquid crystal display 164. For example, when displaying an image on the liquid crystal display 164, the backlight 164 is operated. The backlight 164 includes, for example, a light emitting diode (LED) that emits white light, and the light from the backlight 164 irradiates the liquid crystal display 164 from the back side, and increases the brightness or contrast of the image displayed on the liquid crystal display 164. Increase.

  The color sensor 120 is attached to the display unit 160, and more preferably is installed on the front surface of the half mirror 162. The color sensor 120 detects components of the three primary colors of red (R), blue (B), and green (G) of incident light. The control unit 170 can control the luminance of the display unit 160 based on the detection result of the color sensor 120.

  For example, when the electronic mirror device 100 has a plurality of modes, the mode switching information acquisition unit 130 acquires information for determining which mode to operate. In one example, the mode switching information acquisition unit 130 acquires detection information of an external light sensor built in the electronic mirror device 100, and acquires an illumination signal indicating whether the light of the vehicle is turned on or off. The controller 170 can select or switch modes based on detection information from an external light sensor or an illumination signal.

  The control unit 170 controls the image display of the liquid crystal display 164 via the LCD drive unit 140 based on information from the color sensor 120 and the mode switching information acquisition unit 130, and the backlight via the backlight drive unit 150. The luminance control of 166 is performed. In a preferred embodiment, the control unit 170 is configured by a microcontroller including a ROM, a RAM, and the like, and controls the entire operation of the electronic mirror device 100 by executing a program stored in the ROM or the RAM.

  In the present embodiment, the control unit 170 executes a brightness control program that controls the brightness of the liquid crystal display 164. The brightness control program of the present embodiment determines whether or not the vehicle is running on a snowy road based on the RGB information detected by the color sensor 120 and the RGB information of a specific area of the imaging data captured by the imaging camera. Based on the determination result, the luminance of the backlight 166 is controlled. FIG. 2 shows a functional configuration of the brightness control program 200 of the present embodiment. The brightness control program 200 includes an RGB information acquisition unit 210, an imaging data acquisition unit 220, a snow surface area estimation unit 230, a snow surface area RGB calculation unit 240, a snow surface travel determination unit 250, and a brightness control unit 260.

  The RGB information acquisition unit 210 acquires RGB information of incident light in the vehicle detected by the color sensor 120. The graph shown in FIG. 3 is an example of the output waveforms of R, G, and B of the color sensor 120. The horizontal axis represents wavelength and the vertical axis represents sensitivity. If sunlight reflected by the snow surface enters the vehicle and the incident light is detected by the color sensor 120, as shown in FIG. 3, a waveform Z1 having approximately equal peak sensitivities of R, G, and B. That is, white light is detected. The RGB information acquisition unit 210 converts the peak values of the R, G, and B waveforms shown in FIG. 3 into digital information. For example, when the liquid crystal display 164 displays one pixel with 24-bit data that defines the R, G, and B gradations with 8 bits, the RGB information acquisition unit 210 converts the R, G, and B information into 8-bit digital data. Convert to

  The imaging data acquisition unit 220 acquires imaging data behind the vehicle imaged by the imaging unit 110. The snow surface area estimation unit 230 performs image analysis on the captured image data and estimates a snow surface area therefrom. In one preferable example, a road reflected in the imaging data may be identified, and an area other than the road (for example, a roadside belt or a sidewalk) may be estimated as a snow surface area. Alternatively, in another preferred example, one or more predetermined regions in the imaging data are estimated as the snow surface region. The snow surface area estimation unit 230 cuts out the imaging data of the estimated area.

  FIG. 4 is an example of image data when the vehicle rear is imaged. The imaging data shows the road R1 and areas (hatching areas) 300 and 310 existing on both sides of the road R1. The snow surface area estimation unit 230 identifies the road R1 from the imaging data, estimates the other areas 300 and 310 as snow surface areas, and cuts out the imaging data of the areas 300 and 310.

  The snow surface area RGB calculation unit 240 calculates the R, G, and B values of the snow surface area estimated by the snow surface area estimation unit 230. For example, if the liquid crystal display 164 displays one pixel using 8-bit grayscale R, G, B data, one pixel in the estimated snow region is composed of 8-bit R, G, B values. . In one example, the snow surface area RGB calculation unit 240 calculates R, G, and B values that constitute one pixel of the estimated snow surface area. Alternatively, the snow surface area RGB calculation unit 240 extracts a plurality of pixels in the estimated snow surface area, and calculates an average of R, G, and B values constituting each of the plurality of pixels. Alternatively, the snow surface area calculation unit 240 calculates an average of R, G, and B values constituting all pixels of the estimated snow surface area. In addition, when there are a plurality of estimated snow surface areas, the snow surface area RGB calculation unit 240 calculates the R, G, and B values of each estimated snow surface area.

  FIG. 5 is a diagram for explaining the RGB values calculated by the snow surface region RGB calculation unit 240. If the regions 300 and 310 are snow surface regions, the light reflected by the regions 300 and 310 has an R, G, and B waveform Z2 as shown in FIG. The same as the waveform Z1 of R, G, B of the color sensor 120 shown in FIG.

  The snow surface running determination unit 250 compares the RGB value generated by the RGB information acquisition unit 210 with the RGB value calculated by the snow surface region RGB calculation unit 240, and the two RGB values match or approximate. In this case, it is determined that the vehicle is traveling on the snow surface. For example, if the peak wavelengths of R, G, and B values and their sensitivities are within a certain range, it is considered that there is a correlation between the reflected light captured by the color sensor 120 and the imaging unit 110, and the vehicle is running on the snow surface. Is determined. The luminance control unit 260 controls the BL driving unit 150 to further increase the luminance of the backlight 166 when the snow surface traveling determination unit 250 determines that the vehicle is traveling on the snow surface.

  When the RGB value of the color sensor 120 and the RGB value of the snow surface area estimated from the imaging data are the same or approximate, it is considered that the reflected light from the snow surface is incident on the interior of the vehicle. In such a case, the light on the ceiling 180 is reflected on the half mirror 162 by the light incident on the inside of the vehicle, and the visibility of the display unit 160 may be reduced. Therefore, in the present embodiment, the brightness control unit 260 increases the brightness of the backlight 166 to relatively darken the ceiling side and suppress the reflection of ceiling light on the half mirror 162.

  Next, the operation of the electronic mirror device of this embodiment will be described with reference to the flowchart of FIG. The electronic mirror device 100 is operated in a predetermined mode (S100). For example, when the illumination signal is off (when the vehicle light is off), the control unit 170 can operate the electronic mirror device 100 in the daytime mode. In the daytime mode, since the driver side is bright, the half mirror 162 functions as a reflection mirror, and the driver can see the AR image such as a warning displayed on the liquid crystal display 164 superimposed on the reflected image of the half mirror 162. . When the illumination signal is on (when the vehicle light is on), the control unit 170 can operate the electronic mirror device 100 in the night mode. In the night mode, since the driver side is dark, the half mirror 162 functions as a transmission window, and the driver can see the image behind the vehicle captured by the imaging unit 110 displayed on the liquid crystal display 164. In addition to the illumination signal, the control unit 170 switches between the day mode and the night mode using information included in the mode switching information acquisition unit 130, for example, a detection result of an external light sensor built in the display unit 160. It is also possible to perform control. Furthermore, the control unit 170 can operate in a mode other than the day mode or the night mode.

  When displaying an image or video on the liquid crystal display 164, the controller 170 controls the luminance of the backlight 166. For example, when the controller 170 displays on the liquid crystal display 164 in the daytime mode, the brightness of the backlight 166 is somewhat increased because the driver's seat side is bright, and when displayed on the liquid crystal display 164 in the night mode, the driver side is dark. Reduce the backlight brightness somewhat. In addition to the operation mode, the control unit 170 may control the luminance of the backlight 166 based on detection information of an external light sensor built in the display unit 160. For example, when a decrease in the amount of light is detected by the external light sensor in the daytime mode, the control unit 170 reduces the luminance of the backlight 166 somewhat, and in the night mode, the control unit 170 makes a constant by the external light sensor. When the above light quantity is detected, it is possible to perform control such that the luminance of the backlight 166 is somewhat increased.

  When the electronic mirror device 100 is operated in a predetermined mode, the RGB information acquisition unit 210 sequentially acquires RGB information of the color sensor 120 and calculates digitized RGB values (S102). Further, the snow surface area estimation unit 230 estimates the snow surface area from the imaging data behind the vehicle imaged by the imaging unit 110 (S104), and the RGB value of the snow surface area estimated by the snow surface area RGB calculation unit 240 Is calculated (S106). The snow running determination unit 250 compares both RGB values (S108), and determines that the vehicle is traveling on the snow surface when both RGB values match or approximate (S110). If it is determined that the vehicle is traveling on the snow surface, the luminance control unit 260 performs control to further increase the current luminance of the backlight 166 (S112). By increasing the luminance of the backlight 166, the ceiling side becomes relatively dark, and the light on the ceiling 180 is prevented from being reflected on the half mirror 162, thereby making it difficult to view the image on the liquid crystal display 164.

  As described above, in this embodiment, the color sensor 120 built in the electronic mirror device 100 is compared with the color information of the specific area of the imaging data, and if both match or approximate, the vehicle is traveling on the snow surface. Control is performed to increase the luminance of the backlight 166 by determining. Thereby, when the sunlight reflected on the snow surface reaches the ceiling 180 of the vehicle, the light of the ceiling 180 is suppressed from being reflected on the half mirror 162, and the visibility of the electronic mirror device can be improved.

  In the above embodiment, the RGB values of the imaging data behind the vehicle are used. However, the present invention is not limited to this, and the vehicle travels on the snow surface using the RGB values of the imaging data of the front or side of the vehicle. It may be determined whether or not it is. Moreover, in the said Example, although the snow surface area estimation part 230 estimated the snow surface area by recognizing the road etc. in imaging data, a snow surface area may be a white area. The white region reflects sunlight light, and since the reflected light is likely to enter the vehicle, the white region can be a region equivalent to the snow surface region. The estimation of the white area may be performed in the same manner as the estimation of the snow surface area, or an area where the R, G, and B values are equal or within a certain range is estimated as the white area. May be.

  Furthermore, in the said Example, although the snow surface area estimation part 230 estimated the area | regions other than a road as a snow surface area, this is an example and you may estimate a road as a snow surface area. In this case, first, the RGB value of the road cut out from the imaging data is compared with the RGB value of the color sensor 120, and if they match, there is snow on the road. It is determined that the vehicle is traveling on the snow surface. If they do not match, it means that there is no snow on the road. In that case, the area other than the road is estimated as the snow surface area as in the above-described embodiment, and whether or not the vehicle is running on the snow surface. Determine whether.

  Further, in the above-described embodiment, the configuration in which the display unit 160 includes the liquid crystal display is shown. However, the display unit 160 is not limited to the liquid crystal display, and may be a display medium such as an organic EL display, for example. When an organic EL display is used, no backlight is required, and the control unit 170 controls the luminance of the organic EL display based on the determination result of the snow surface traveling determination unit 250.

  Furthermore, in the said Example, although the display part 160 showed the structure provided with the half mirror 162, the display part 160 does not necessarily need to provide a half mirror. Even if the display unit 160 does not include a half mirror, it is possible to improve the visibility of the image on the display unit 160 by increasing the luminance of the display unit 160 when the ceiling 180 becomes bright. For example, the present invention can also be applied to such an electronic mirror device.

  The preferred embodiment of the present invention has been described in detail above. However, the present invention is not limited to the specific embodiment, and various modifications and variations are possible within the scope of the gist of the invention described in the claims. It can be changed.

DESCRIPTION OF SYMBOLS 100: Electronic mirror apparatus 110: Imaging part 120: Color sensor 130: Mode switching information acquisition part 140: LCD drive part 150: Backlight drive part 160: Display part 162: Half mirror 164: Liquid crystal display 166: Backlight 170: Control unit 180: ceiling

Claims (8)

Imaging means for imaging the surroundings of the vehicle;
A display unit capable of displaying imaging data imaged by the imaging means;
Detecting means for detecting color information of light incident on the display unit;
Comparing means for comparing the color information detected by the detecting means with the color information of a specific area of the imaging data;
Brightness control means for controlling the brightness of the display unit based on the comparison result of the comparison means;
An electronic mirror device. 2. The electronic mirror device according to claim 1, wherein the luminance control unit increases the luminance of the display unit when the color information of the detection unit and the color information of the specific area match or approximate. The electronic mirror device according to claim 1, wherein the color information is an R, G, B value. The electronic mirror device according to claim 1, wherein the specific region is a region estimated to be white. The electronic mirror device according to claim 1, wherein the specific region is a region estimated to be a snow surface. The electronic mirror device according to claim 1, further comprising an estimating unit that estimates the specific region based on the imaging data. The display unit includes a half mirror, a liquid crystal display and a backlight,
The electronic mirror device according to claim 1, wherein the luminance control unit controls the luminance of the backlight. The electronic mirror device further includes image control means for controlling an image displayed on the display unit,
The electronic mirror device according to claim 1, wherein the image control unit displays an image of a vehicle rear or an augmented reality image on the display unit.

Images