JP2012162126A - Luminance control system, luminance control program and luminance control method of in-vehicle display unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide comfortable driving environment by appropriate control of luminance.SOLUTION: A luminance control system 10 includes a computer 12 and a CPU 12a of the computer 12 detects the line-of-sight direction of a driver from a shot image of a first camera 14. The luminance distribution in a driver's visible area is calculated from a shot image of a second camera 16. In addition, driver's gazing luminance is detected from the line-of-sight direction. When a driver is viewing an in-vehicle display unit 20, the luminance of the in-vehicle display unit 20 is controlled in view of the luminance in the line-of-sight direction moved to the outside of the vehicle after the viewing operation. When the line-of-sight direction is moved from the outside of the vehicle to the in-vehicle display unit 20, the luminance of the in-vehicle display unit 20 is taken to be equivalent to the gazing luminance of a position that has been viewed until that time. Accordingly, adaptation of eyes is quick. When the line of sight is directed to the outside of the vehicle, the luminance of the in-vehicle display unit 20 is reduced. Thus, there is little adverse effect of reflection or glare.

Description

  The present invention relates to a brightness control device, a brightness control program, and a brightness control method for an in-vehicle display device, and more particularly to, for example, a brightness control device, a brightness control program, and a brightness control method for an in-vehicle display device installed in an automobile.

  An example of this type of in-vehicle display device brightness control device is disclosed in Patent Document 1. In the driving operation assisting device disclosed in Patent Document 1, a driver's face image is taken, and the face direction and the line-of-sight direction are detected based on the face image data obtained by the photographing. When the movement of the face is small, it is determined whether the line-of-sight direction is within a predetermined meter viewing angle range. When the angle of the line of sight is within the range of the predetermined meter viewing angle, that is, when the line of sight faces downward, it is determined that the driver is checking the meter display, and the brightness of the meter display is determined. The information is increased to such an extent that the information is easily visible. For example, the display brightness of the meter display is increased or the display contrast is increased. Further, when the driver's face direction changes greatly to the left and right, it is determined that it is a front obstacle confirmation operation or the like, and the irradiation direction of the auxiliary light is adjusted to the face direction.

  Another example of the brightness control device for this type of in-vehicle display device is disclosed in Patent Document 2. In the illumination control system disclosed in Patent Document 2, when the outside of the vehicle is darker than a predetermined brightness, the front seat occupant does not feel annoying the illumination of the in-vehicle device from the normal first brightness. The in-vehicle device is darkened to the second brightness, and the device to which the line of sight of the front-seat boarding vehicle is directed is changed from the second brightness to the first brightness so that the device can be easily seen.

Patent No. 3228086 [B60Q 1/18, B60K 35/00, B60Q 1/08] JP 2006-215591 [B60Q 3/02, B60K 35/00, B60Q 3/04]

  However, with the techniques disclosed in Patent Document 1 and Patent Document 2, when the line of sight is directed to an in-vehicle device such as a main display, basically the brightness is simply increased. Therefore, when changing from a dark looking outside to a bright in-vehicle device display, if the difference in light and dark is large, it takes time for the eyes to adapt and the time not looking forward. There is a risk of becoming longer. Further, when the line of sight is returned to the outside of the vehicle after viewing the display of the bright in-vehicle device, if the difference in brightness is large, it takes time until the eyes adapt and the front may be difficult to see. In Patent Document 1 and Patent Document 2, it is merely determined whether or not the outside of the vehicle is dark, and the brightness of the direction or place where the driver is looking is not determined. May not be adjusted.

  Therefore, a main object of the present invention is to provide a novel brightness control device, brightness control program, and brightness control method for an in-vehicle device.

  Another object of the present invention is to provide a luminance control device, a luminance control program, and a luminance control method for an in-vehicle device that can adjust the brightness appropriately.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate correspondence relationships with embodiments described later to help understanding of the present invention, and do not limit the present invention in any way.

  A first invention is a brightness control device for an in-vehicle display device provided in an automobile, a brightness distribution detecting means for detecting a brightness distribution in a range visible to a driver, and a gaze detection for detecting a driver's gaze direction. Means, gaze determination means for determining whether the driver is gazing at the on-vehicle display device based on the gaze direction detected by the gaze detection means, the gaze direction detected by the gaze detection means, and the luminance distribution detection means Gaze luminance detection means for detecting the driver's gaze luminance from the detected luminance distribution, and the gaze luminance before the change when the driver is gazing from the state in which the driver is not gazing A luminance control device for an in-vehicle display device, comprising: luminance control means for controlling the luminance of the in-vehicle display device based on the gaze luminance detected by the detection means.

  In the first invention, the luminance control device (10) of the in-vehicle display device (20) provided in the automobile includes the luminance distribution detection means (12a, S3), the line-of-sight detection means (12a, S9), and the gaze determination means (12a). , S15), gaze luminance detecting means (12a, S11), and luminance control means (12a, S23, S27, S33, S41). The luminance distribution detecting means detects a luminance distribution in a range that can be visually recognized by the driver. The line-of-sight detection means detects the driver's line-of-sight direction. The gaze determination unit determines whether the driver is gazing at the in-vehicle display device based on the gaze direction detected by the gaze detection unit. The gaze luminance detection means detects the gaze luminance of the driver from the gaze direction detected by the gaze detection means and the luminance distribution detected by the luminance distribution detection means. That is, the direction or position (location) where the driver is looking from the line-of-sight direction is detected, and the luminance at the corresponding position in the luminance distribution is detected as the gaze luminance. When the driver changes from a state in which the driver is not gazing to the in-vehicle display device to a state in which the driver is gazing, the luminance control unit is configured to determine the luminance of the in-vehicle display device based on the gaze luminance detected by the gaze luminance detection unit before the change. To control. For example, the brightness of the in-vehicle display device is controlled so that it matches or substantially matches the brightness before the change.

  According to the first invention, since the brightness of the in-vehicle display device is controlled based on the gaze brightness before the change, for example, the difference in brightness before and after the change can be reduced, and thus the brightness is appropriately adjusted. Can be adjusted. For this reason, a comfortable driving environment can be provided.

  The second invention is dependent on the first invention and further comprises probability distribution storage means for storing the probability distribution of the gaze position for the driver's visible range, and the brightness control means is provided on the vehicle-mounted display device by the driver. When the state of gazing is continued, the luminance of the in-vehicle display device is controlled based on the luminance distribution detected by the luminance distribution detecting unit and the probability distribution stored in the probability distribution storing unit.

  In the second invention, the luminance control apparatus further includes a probability distribution storage means (12b). The probability distribution storage means stores the probability distribution of the gaze position for the range that is visible to the driver. For example, the probability distribution indicates where and how long (time length) the driver sees in the range that the driver can visually recognize. The brightness control means stores the brightness distribution detected by the brightness distribution detection means and the probability distribution storage means when the driver is gazing at the in-vehicle display device (“YES” in S15 and S17). The brightness of the in-vehicle display device is controlled based on the stored probability distribution. Therefore, for example, the luminance of the in-vehicle display device is controlled so as to approach the gaze luminance of the gaze position after the predicted movement.

  According to the second invention, when viewing other places such as the outside of the vehicle from the state of viewing the in-vehicle display device, the brightness is controlled to the in-vehicle display device so as to match the gaze luminance after movement. The driver's eyes can be adapted relatively quickly.

  The third invention is dependent on the first or second invention, and the luminance control means changes the gaze luminance before the change when the driver changes from the state where the driver is not gazing to the state where the driver is gazing. The brightness of the in-vehicle display device is set to a value that correlates with the gaze brightness detected by the detection means.

  In the third invention, when the driver changes from a state in which the driver is not gazing to the vehicle-mounted display device to a state in which the driver is gazing (“NO” in step S7), the luminance control unit detects the luminance before the change. The brightness of the in-vehicle display device is set to a value that correlates with the detected gaze brightness. For example, the value correlated with the gaze luminance is a value that matches or substantially matches the gaze luminance or a predetermined ratio value of the gaze luminance. Thereby, the brightness of the in-vehicle display device is controlled so as to be close to the brightness of the place or position that was viewed before changing the line of sight.

  According to the third aspect of the invention, the brightness of the in-vehicle display device is controlled so as to approach the brightness of the place or position that was viewed before changing the line of sight, so the driver's eyes can be adapted quickly. The time for watching the vehicle-mounted display device, that is, the time for not watching the front, can be reduced as much as possible.

  A fourth invention is dependent on any one of the first to third inventions, and the luminance control means reduces the luminance of the in-vehicle display device when the driver is not gazing at the in-vehicle display device.

  In the fourth invention, the brightness control means is in a state where the driver is not gazing at the vehicle-mounted display device (“NO” in step S15), that is, when the driver is looking forward (outside the vehicle). Reduces the brightness of the in-vehicle display device.

  According to the fourth aspect of the invention, when the driver is looking forward, the brightness of the in-vehicle display device is reduced, so that the illumination is prevented from being reflected on the windshield or causing glare. can do. Therefore, a comfortable driving environment can be provided.

  The fifth invention is dependent on the first to fourth inventions, and the luminance distribution and the gaze luminance are average values for a predetermined time.

  According to the fifth aspect, since the average value for a predetermined time is used, the luminance of the in-vehicle display device can be controlled without being affected by a temporary change.

  A sixth aspect of the present invention is a luminance control device for an in-vehicle display device provided in an automobile, a luminance distribution detecting means for detecting a luminance distribution in a range visible to the driver, and a gaze detection for detecting a driver's gaze direction Means, gaze determination means for determining whether or not the driver is gazing at the vehicle-mounted display device based on the line-of-sight direction detected by the line-of-sight detection means, and the state where the driver is gazing at the vehicle-mounted device An on-vehicle display comprising luminance control means for controlling the luminance of the in-vehicle display device based on the luminance distribution detected by the luminance distribution detection means and the probability distribution of the gaze position for the driver's visible range. It is the brightness | luminance control apparatus of an apparatus.

  A seventh invention is a luminance control device for an in-vehicle display device provided in an automobile, and includes luminance distribution detection means for detecting a luminance distribution in a range visible to the driver, and a gaze detection for detecting a driver's gaze direction. Means, gaze luminance detection means for detecting the driver's gaze luminance from the gaze direction detected by the gaze detection means and the luminance distribution detected by the luminance distribution detection means, and the gaze luminance detected by the gaze luminance detection means And a luminance control device for the in-vehicle display device, comprising luminance control means for controlling the luminance of the in-vehicle display device based on the adaptation of the driver's eyes.

  An eighth invention is a luminance control program for a luminance control device of an in-vehicle display device provided in an automobile, wherein a luminance distribution detection step of detecting a luminance distribution in a range visible to a driver by a processor of the luminance control device Detecting by the gaze detection step for detecting the driver's gaze direction, the gaze determination step for judging whether the driver is gazing at the vehicle-mounted display device based on the gaze direction detected by the gaze detection step, and the gaze detection step Gaze luminance detection step for detecting the driver's gaze luminance from the sight line direction and the luminance distribution detected by the luminance distribution detection step, and the driver is gazing from the state in which the driver is not gazing at the on-vehicle display device When the state changes, the luminance of the in-vehicle display device is adjusted based on the gaze luminance detected by the gaze luminance detection step before the change. To execute Gosuru luminance control step, a brightness control program.

  A ninth aspect of the present invention is a luminance control program for a luminance control device of an in-vehicle display device provided in an automobile, wherein a luminance distribution detection step of detecting a luminance distribution in a range visible to a driver by a processor of the luminance control device A gaze detection step for detecting the driver's gaze direction, a gaze judgment step for judging whether the driver is gazing at the on-vehicle display device based on the gaze direction detected by the gaze detection step, and the driver is on-vehicle When the state of gazing at the device continues, the luminance of the in-vehicle display device is determined based on the luminance distribution detected by the luminance distribution detection step and the probability distribution of the gaze position for the driver's visible range. This is a brightness control program for executing a brightness control step for controlling

  A tenth aspect of the invention is a luminance control program for a luminance control device of an in-vehicle display device provided in an automobile, wherein a luminance distribution detection step of detecting a luminance distribution in a range visible to a driver by a processor of the luminance control device A gaze detection step for detecting the driver's gaze direction, a gaze luminance detection step for detecting the gaze luminance of the driver from the gaze direction detected by the gaze detection step and the luminance distribution detected by the luminance distribution detection step, And a luminance control program for executing a luminance control step for controlling the luminance of the in-vehicle display device based on the gaze luminance detected by the gaze luminance detection step and the adaptation of the driver's eyes.

  An eleventh aspect of the invention is a luminance control method for a luminance control device of an in-vehicle display device provided in an automobile, wherein (a) a luminance distribution in a range visible to a driver is detected, and (b) a driver's line of sight (C) Based on the line-of-sight direction detected in step (b), it is determined whether the driver is gazing at the vehicle-mounted display device, and (d) the line-of-sight detected in step (b) From the direction and the luminance distribution detected in step (a), the driver's gaze luminance is detected, and (e) the driver changes from the state of not gazing at the vehicle-mounted display device to the state of gazing. In this case, the brightness control method controls the brightness of the in-vehicle display device based on the gaze brightness detected in step (d) before the change.

  A twelfth aspect of the invention is a luminance control method for a luminance control device of an in-vehicle display device provided in an automobile, wherein (a) a luminance distribution in a range visible to the driver is detected, and (b) the driver's line of sight (C) based on the line-of-sight direction detected in step (b), it is determined whether the driver is gazing at the in-vehicle display device, and (d) the driver is gazing at the in-vehicle device. Controlling the luminance of the in-vehicle display device based on the luminance distribution detected in step (a) and the probability distribution of the gaze position for the driver's visible range, This is a luminance control method.

  A thirteenth aspect of the invention is a luminance control method for a luminance control device of an in-vehicle display device provided in an automobile, wherein (a) a luminance distribution in a range visible to a driver is detected, and (b) a driver's line of sight Detecting the direction, (c) detecting the driver's gaze luminance from the line-of-sight direction detected in step (b) and the luminance distribution detected in step (a), and (d) step (c) Is a luminance control method for controlling the luminance of the in-vehicle display device based on the gaze luminance detected by the driver and the adaptation of the driver's eyes.

  In the sixth to thirteenth inventions, as in the first invention, the brightness can be appropriately controlled, and a comfortable driving environment can be provided.

  According to this invention, since the brightness of the in-vehicle display device is controlled based on the gaze brightness before the driver's line of sight changes, the brightness can be controlled so as to reduce the difference in brightness before and after the change. . That is, it is possible to provide a comfortable driving environment by appropriately controlling the brightness.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

FIG. 1 is a block diagram showing an electrical configuration of a luminance control device of an in-vehicle display device according to an embodiment of the present invention. FIG. 2 is an illustrative view showing a state in an automobile to which the luminance control device shown in FIG. 1 is applied. FIG. 3 is a table showing the contents of the experimental video and the experimental conditions. FIG. 4 is an illustrative view showing an example of an auxiliary screen for experiments. FIG. 5 is a graph showing experimental results. FIG. 6 is a graph showing another experimental result. FIG. 7 is a graph showing other experimental results. FIG. 8 is an illustrative view for explaining an environmental luminance distribution based on a photographed image of the second camera shown in FIG. FIG. 9 is an illustrative view for explaining the probability distribution (weight) of the gaze position of the driver of the automobile. FIG. 10 is an illustrative view showing an example of a memory map of the RAM shown in FIG. FIG. 11 is a table showing brightness values corresponding to panel brightness levels and a table showing panel target brightness levels corresponding to distributed brightness time averages. FIG. 12 is a table showing the panel target luminance level corresponding to the gaze luminance time average according to the gaze mode. FIG. 13 is a flowchart showing a part of the luminance control processing of the CPU shown in FIG. FIG. 14 is another flowchart showing another part of the luminance control processing of the CPU shown in FIG. FIG. 15 is another part of the luminance control processing of the CPU shown in FIG. 1, and is a flowchart subsequent to FIG. FIG. 16 is an explanatory diagram for explaining an operation example of the luminance control apparatus of this embodiment.

  Referring to FIG. 1, the brightness control device 10 of the in-vehicle display device of this embodiment includes a computer 12, and a first camera 14, a second camera 16, and a database (DB) 18 are connected to the computer 12. .

  In this embodiment, the DB 18 is provided for easy understanding, but instead of the DB 18, an HDD or ROM (not shown) built in the computer 12 or an SD card attached to the computer 12. A memory card (not shown) may be used.

  The computer 12 is a general-purpose computer such as a PC, and includes a CPU 12a and a RAM 12b. The first camera 14 is an infrared camera and is used to detect the driver's line-of-sight direction. The second camera 16 is a monochrome camera and is used to detect a luminance distribution in a range that can be visually recognized by the driver. Therefore, as the second camera 16, a camera having a relatively low resolution (for example, 300,000 pixels) as compared with the first camera 14 can be used. The DB 18 stores a gaze position probability distribution (see FIG. 8) and various table data (see FIGS. 9 and 10), which will be described later.

  2 (A) and 2 (B) are illustrative views for explaining the installation status of the first camera 14 and the second camera 16 shown in FIG. As shown in FIG. 2A, a certain automobile 100 is provided with an in-vehicle display device 20. For example, the in-vehicle display device 20 includes a display device 20a of an instrument panel (instrument panel) 102, a display screen 104a of a car navigation system (car navigation system), a display device 20b of an operation panel 104b, and a display device 20c of an air conditioning operation panel 106. Including. However, when the car audio is provided separately from the car navigation system, the display screen and the operation panel display device are also included.

  As shown in FIG. 2A, the first camera 14 is installed above the left end of the instrument panel 102. However, if it does not obstruct driving, it may be installed on the upper center of the instrument panel 102 so that the driver's face can be photographed from the front. As shown in FIG. 2B, the second camera 16 is installed on the passenger seat side, for example, above the driver seat. Although illustration is omitted, the second camera 16 is an upper part of the passenger seat and may be installed on the driver's seat side.

  In such a brightness control device 10 of the in-vehicle display device 20, conventionally, the brightness outside the vehicle is detected, the driver's line of sight is detected, and when the driver is driving while looking outside the vehicle, the on-vehicle display is displayed. The brightness of the device 20 is reduced to prevent the display contents such as the instrument panel 102 from appearing on the windshield. When the driver looks at the in-vehicle display device 20, the brightness of the in-vehicle display device 20 is increased to make the display contents easier to see.

  However, the brightness outside the vehicle simply determines whether it is bright or dark. The brightness when the driver is gazing (gazing brightness), and when looking at a bright place from a dark place and vice versa. No consideration is given to the adaptation speed of the eye. However, the gaze luminance means the luminance in the direction or position (place) where the driver is looking (gazing). Specifically, when the driver is looking at a device other than the vehicle-mounted display device 10 (including outside the vehicle), it is the luminance in the line-of-sight direction, and when the driver is looking at the vehicle-mounted display device 20, It is the brightness for lighting.

  For example, as a disturbing factor of visibility when driving with vehicle interior lighting including the in-vehicle display device 20, the vehicle interior lighting is reflected on the windshield, and there is a brighter area in the peripheral visual field than the central visual field. Visibility deteriorates due to the difference in brightness between different areas to be watched during driving, such as “glare”, which hinders the visibility of the central visual field, and the equipment screen (in this embodiment, the display devices 20a, 20b, and 20c). “Adaptation” is known.

  Here, “adaptation” refers to a process (adaptation process) in which the properties of the visual system are adapted in response to a change in luminance received by the retina of the eye, and a familiar condition (adaptation state). However, in this embodiment, simply “adaptation” mainly means “adaptation process”.

  Of these, “reflection” and “glare” are basically obstructive factors caused by illumination that is simultaneously illuminated when the device screen is not viewed, whereas “adaptation” obstructions are observed when looking outside the vehicle. This is because the effect of the difference in brightness between the area watched at different timings and the brightness outside the vehicle remains. Since “reflection” and “glare” basically occur when you are not looking at the equipment screen, for example, use gaze information to turn off the lights when the driver is gazing outside the vehicle. It is highly possible. A system that aims to improve visibility by actually lowering the illumination level at the time of non-gazing in conjunction with the line of sight is disclosed in Patent Document 2 shown in the background art. Therefore, one of the purposes of the experiment described later is to focus on the driver's line-of-sight direction and reduce the illumination, that is, the luminance level of the in-vehicle display device 20 when the driver is not gazing. The purpose is to confirm that the visibility is improved.

  On the other hand, interference due to “adaptation” occurs when the user looks away from a dark vehicle and looks at the bright device screen, or when the user looks back from the bright device screen to the outside of the vehicle. Therefore, it is considered that the degree of interference is reduced by adjusting the brightness of the device screen according to the outside of the vehicle. As an example of an on-vehicle display control that takes into account the adaptation characteristics, a method is being considered that realizes an on-vehicle display that is easy to view even in high-luminance viewing environments such as during snowy weather by correcting brightness and saturation. Yes (Sakaguchi, Higuchi, Nakano, Yamamoto: “Display of in-vehicle display considering eye adaptation characteristics”, Toyota Central R & D Review, 33, 2, pp. 37-45 (1998)). In this method, the visibility is improved by correcting the display brightness so that the change in subjective brightness against a bright outside of the vehicle is suppressed based on the adaptation characteristics of human vision.

  However, when the outside of the vehicle has a relatively low brightness, such as during night driving, the visibility of the device itself is likely to be impaired if the brightness of the display display is reduced to a level that matches the outside of the vehicle. Therefore, in the experiment described below, the brightness of the display display is adjusted stepwise in conjunction with the line of sight, ensuring the driver's visibility and generating the “adaptation” that occurs when the driver returns the line of sight outside the vehicle. The method to suppress the interference by "" was also taken into consideration. In addition, we examined the relationship with visibility for multiple gaze-linked patterns by subject experiments.

  Briefly, in the experiment, for the night driving when the driver's visual recognition is difficult, by combining the line-of-sight measurement technology and the interior lighting control (luminance control of the in-vehicle display device 20), In order to realize a comfortable driving environment for both drivers with high visibility, we investigated the relationship between lighting environment and visual recognition ability. In the experiment, a video in front of the vehicle (main screen) capturing a night driving scene and a graphic display (auxiliary screen) imitating the device panel are presented to the subject by a liquid crystal monitor (not shown) in a dark room environment, We evaluated the results of detection tasks on the main and auxiliary screens due to differences in lighting patterns, and differences in response time.

  In the experiment of this embodiment, three liquid crystal monitors are arranged horizontally on a desk. Among these, the LCD monitor (main screen) in the center shows the environment outside the vehicle during driving (the state that the driver can see through the windshield), and the LCD monitors (auxiliary screens) on both sides show the car navigation / instrument. In-vehicle devices (vehicle-mounted display device 20) such as speedometers, tachometers, fuel meters, water temperature meters, distance meters, car audio, heaters and air conditioner operating devices, etc. are assumed. The subject sits on a chair so as to face the screen of the central liquid crystal monitor with keyboard input enabled.

In addition, PC (OS: Windows (registered trademark)) is used for stimulus control, and Adobe (registered trademark) Flash CS4 is used for stimulus generation. For the stimulus presentation, as described above, three liquid crystal monitors are used. However, the display resolution of each liquid crystal monitor is 1920 × 1200 dpi. As for the brightness of the liquid crystal monitor used as the main screen, the brightness is set to the minimum, the contrast is adjusted according to the actual driving situation, and the maximum brightness is set to 28.38 cd / m ² . On the other hand, regarding the brightness of the liquid crystal monitor used as the auxiliary screen, the brightness and contrast are set to the maximum, and the maximum luminance is set to 386.50 cd / m ² . Further, a keyboard numeric keypad is used to obtain the subject's response. However, the “0” button is used to obtain the response of the stimulus detection task (main task) using the main screen, and “4”, The “6” button is used.

  A detection task for promptly and accurately responding to the target presented on the main screen was performed while changing the display brightness of the auxiliary screen according to the experimental conditions described later. Further, during the experiment, each subject was asked to keep an eye on one of the auxiliary screens specified for each subject in the time interval indicated by the voice.

  Although illustration is omitted, in the experiment, in order to simulate an actual night driving scene, an image taken in front of the vehicle was taken on a national road No. 163 connecting Kyoto, Nara, and Osaka at night from 22:00 to 23:00 on weekdays. The humanoid target was superimposed in gray on the image as the target of the detection task. The subject was instructed to press the “0” button after confirming the target. The display contents of the auxiliary screens at both ends were adjusted according to the experimental conditions. In order to ensure gaze on the auxiliary screen, the letters “E” or “F” were displayed (presented) on the liquid crystal monitor, and a discrimination task was imposed on the subject.

  FIG. 3A shows the contents of the driving scene image used in the experiment in a table. Each scene is a video of about 8 minutes that combines a scene with low traffic volume and a scene with high traffic volume for about 4 minutes each. As will be described later, an experiment was performed on illumination patterns of four different auxiliary screens for each subject. For each lighting pattern, one of the images of the driving scene shown in FIG. 3A was selected without duplication so that one subject did not repeatedly view the same driving scene.

  In addition, in order to offset the difference in the results due to the driving scene, the combination of the illumination pattern of the auxiliary screen and the video of the driving scene was changed for each subject.

  Next, the target displayed superimposed on the video of the driving scene for the detection problem on the main screen will be described. The target to be presented is a gray (achromatic) humanoid figure as described above, and the luminance to be displayed is set in the range of 16 to 48 according to the brightness of the scene. However, the system brightness is set to 0 (minimum) -255 (maximum). The number of target presentations was 36 under each experimental condition (see FIG. 3 (B)), and the targets were presented 12 times in three areas, left, right, or center, except for the top and bottom of the main screen. Moreover, it adjusted so that the frequency | count of showing a target immediately after gazes at an auxiliary | assistant screen might become the same (3 times) in each area | region.

  In each experimental condition, the instruction sound for requesting the subject to pay attention to the auxiliary screen was presented a total of nine times. However, the timing of gaze and the situation of the scene are different in each scene.

  Next, the content presented on the auxiliary screen will be described. On the auxiliary screen, the illumination pattern is switched according to the experimental conditions. As a visual stimulus, an image in which one alphabet letter (E or F) was displayed in a square display frame was presented. When the test sound is presented while performing the detection task on the main screen, the subject moves the line of sight to the auxiliary screen, determines the alphabet (E, F) displayed in the presented image, While responding with the keyboard, it was required to watch the screen until the beep sound disappeared. As an instruction sound, a Windows standard chime sound was used and presented for 10 seconds per time. Further, the image on which the alphabet is displayed is displayed 200 ms after the start of the presentation of the instruction sound, and is erased at the same time as the instruction sound is turned off. The presentation position of the visual stimulus, that is, the display position of the image on which the alphabet is displayed, is a combination of the position of the height 1/3, 2/3 and the position of the width 1/3, 2/3 in the auxiliary screen. One place was randomly determined. However, the brightness of the alphabet-displayed image differs for each experimental condition. As part of the experimental conditions, examples of the auxiliary screen display are shown in FIGS. 4A to 4C as the line-of-sight interlocking conditions for changing the luminance of the auxiliary screen in time synchronization with the gaze instruction for the subject. Indicated. In brief, FIG. 4A shows an auxiliary screen in the case of high luminance. FIG. 4B shows an auxiliary screen in the case of low luminance. FIG. 4C shows an auxiliary screen in the case of medium luminance (brightness intermediate between high luminance and low luminance).

  As shown in FIG. 3B, the experimental conditions were set to four cases (Bright-Off, Dark-Off, Dark-On, Gaze). As shown in FIG. 3B, in each experimental condition, different contents are set for the brightness of the auxiliary screen, the line-of-sight accompanying, and the line-of-sight accompanying luminance change. Under the condition of Bright-Off, the brightness of the auxiliary screen is “high luminance”, the line of sight is “none”, and the luminance change at the time of line of sight is “none”. Further, under the Dark-Off condition, the brightness of the auxiliary screen is “low luminance”, the line of sight accompanied is “none”, and the luminance change during the line of sight is also “none”. Further, under the Dark-On condition, the brightness of the auxiliary screen is “low luminance”, the line-of-sight accompanying is “present (high intensity)”, and the line-of-sight accompanying luminance change is “none”. Furthermore, under the Gaze condition, the brightness of the auxiliary screen is “low luminance”, the line-of-sight accompanying is “present (medium luminance)”, and the line-of-sight accompanying luminance change is also “present”.

  However, with regard to the line of sight, the brightness (brightness) of the auxiliary screen during trial is fixed at “None”, and the brightness is changed so that the entire auxiliary screen on the side that is watched at the time of gaze is bright at “Yes”. However, the luminance of the auxiliary screen (non-gaze screen) that is not gazing is not changed. Also, under the Gaze conditions, the luminance of the auxiliary screen was reduced 2000 ms after the start of gaze on the auxiliary screen, and the aim was to alleviate interference due to adaptation when the line of sight was returned to the dark field (main screen).

  In each experimental condition, the target is not displayed on the auxiliary screen while the subject is gazing at the main screen.

  As described above, the video of the actual night driving scene is presented on the main screen, the image with the alphabet “E” or “F” is presented on the auxiliary screen, and each subject responds with the keyboard. To do. The experimental tasks are as follows.

(1) Detection of stimulus presented on main screen A test subject observes a moving image as a driver and, after detecting a target, presses the “0” button with the left thumb as quickly and accurately as possible.

(2) Discrimination of stimuli presented on the auxiliary screen When the instruction sound sounds, an image with alphabets displayed on the auxiliary screen is presented. While the instruction sound is sounding, the subject pays attention to the image on which the alphabet is displayed, discriminates whether the alphabet is “E” or “F”, and responds with the keyboard. Further, no stimulus is presented on the main screen while the instruction sound is sounding.

  However, the test subjects were 8 healthy adults (5 men and 3 women), and the left and right auxiliary screens were switched for each test subject and balanced by 4 subjects.

  The results of the detection task (main task) on the main screen are shown in FIG. In FIG. 5A, the correct answer rate (%) for each experimental condition is shown as a graph as the result of the detection task. As can be seen from FIG. 5A, the Bright-Off condition in which the auxiliary screen is always lit with high luminance has a lower result than the other conditions (ratio p <0.05). This is considered to be because the influence of “glare” or “reflection” due to the fact that the auxiliary screen is always lit even when gazing at the main screen is large.

  Moreover, the reaction time in the main subject is shown in FIG. However, the reaction time is the time from when the target is presented until the button is pressed, and an average value is obtained. As can be seen from FIG. 5B, the bright-off condition has the longest reaction time compared to the other conditions. However, there is no significant difference between the conditions.

  Furthermore, the subjective evaluation about the difficulty level in the main task is shown in FIG. As shown in FIG. 6A, the Bright-Off condition has the highest difficulty level, and a significant difference is observed between the Bright-Off condition and the Gaze condition. It can be seen from this result that visual recognition is hindered by “glare” or “reflection”.

  On the other hand, there was no significant difference in the performance of the detection task on the main screen between the Dark-Off condition and the Dark-On condition, which differed in brightness when the subject gazes at the auxiliary screen. This means that “glare” or “reflection”, which is the influence of the illumination of the auxiliary screen at the time of gazing at the main screen, is dominant among the disturbing factors of the main problem.

  However, as shown in FIG. 6 (B) and FIG. 7 (A), a significant difference was recognized in the subjective evaluation about the comfort of the auxiliary screen and the influence of the auxiliary screen on the main problem. In other words, compared to the Dark-On condition where the brightness is high when gazing at the auxiliary screen, the Dark-Off condition, which maintains a low luminance even when gazing, is more comfortable and has an effect on the main problem of the auxiliary screen. It was also evaluated to be few. Under the Dark-On condition, it is necessary to shift the line of sight to the dark field (main screen) immediately after gazing at a bright screen (auxiliary screen), and it may be difficult to visually recognize the main screen due to adaptation.

  In FIG. 6B and FIG. 7A, a significant difference was recognized for each combination between the (Bright-Off, Dark-On) condition and the (Dark-Off, Gaze) condition. .

  FIG. 7B is a graph showing the reaction time for each condition in the alphabet discrimination task on the auxiliary screen. Here, the Dark-On and Gaze conditions for performing illumination line-of-sight interlocking tended to have longer reaction times than the Bright-Off and Dark-Off conditions for non-line-of-sight interlocking. There is a possibility that a shift in the timing of the line-of-sight movement of each subject to the auxiliary screen and the switching of the illumination condition in the experiment of this example affects the visibility.

  From the above, in the whole experiment, it was confirmed that the result of the detection task under the condition of Bright-Off was the lowest, and the brightness of the auxiliary screen not directly presenting the stimulus of the detection task affected the detection task of the main screen. . This may be due to both “glare”, which makes visual recognition of the central visual field difficult due to the presence of a brighter region in the peripheral visual field, and “reflection”, in which peripheral illumination light is reflected on the main screen. Even during actual night driving, it is difficult to grasp the dark road surface condition due to high-illuminance lighting such as vehicle interior lighting including the on-vehicle display device 20 and headlights of oncoming vehicles, or reflection of those lighting on the windshield Similarly, it is routinely experienced that it is difficult to observe (view) the outside of the vehicle. The fact that the main screen is difficult to see due to the high-luminance illumination (auxiliary screen) in the peripheral visual field has also been clarified in the subjective evaluation shown in FIG. For this reason, it is reasonable to reduce the illuminance of the in-vehicle display device 20 when it is not needed, and by detecting the driver's gaze direction, the interior lighting ( It was suggested that the control for reducing the brightness (luminance) of the in-vehicle display device 20) is effective.

  On the other hand, these in-vehicle display devices 20 are information providing devices and need to be visually recognized by the driver comfortably. As a result of the experiment, the Dark-On condition, which increases the brightness of the auxiliary screen only at the time of gaze, has improved the results of the main task compared to the Bright-Off condition, which always maintains high brightness, but the comfort of the auxiliary screen In the subjective evaluation regarding the influence of the auxiliary screen on the main problem, the evaluation was lower than the Dark-Off condition. Under the Dark-On condition, when the subject is gazing at the auxiliary screen, the brightness level is the same as the Bright-Off condition, so the difference in brightness level from the main screen seems to have led to a decrease in evaluation. It is done. In other words, although the effects of reflection (reflection) and glare can be suppressed by brightness control linked to the line of sight, it can be said that as long as the user is gazing at a high-brightness monitor, many subjects have difficulty in visually recognizing the dark field immediately after. . If the luminance at the time of gaze is low as in the Dark-Off condition, the adaptation problem is less likely to occur, but conversely, the visibility of the auxiliary screen may be reduced.

  Considering this point, under the Gaze condition examined in the above-described experiment, the luminance after the gaze is maintained while maintaining a relatively high luminance at the initial gaze where proper line-of-sight movement to the presentation information in the auxiliary screen is important. Intended to lower and promote adaptation to dark field. When this is applied to the driving scene of the experiment, the luminance of the display devices 20a, 20b, and 20c to be watched is reduced while suppressing the amount of light of the entire in-vehicle display device 20 in order to suppress glare and reflection on the windshield. Increase the visibility by setting it higher. In addition, it is possible to make it difficult to cause interference when the line of sight is returned to the dark field (outside the road surface or the like) by adding temporal control to gradually reduce the luminance after gaze. As a result of the above-described experiment, the average result was the highest under the Gaze condition and the effect was confirmed even though the average luminance at the time of gaze was higher than the Dark-Off condition. This can be said to be a result suggesting the possibility that the illumination pattern considering temporal control according to the gaze state can suppress the effects of glare and adaptation.

  In summary, adjusting the amount of light on the auxiliary screen in conjunction with the line of sight is effective for suppressing interference (reflection) due to glare and reflection during gaze outside the vehicle. In addition, in order to avoid interference due to adaptation, if the brightness of the auxiliary screen is lowered when the brightness is lowered, the visibility of the auxiliary screen will be reduced, but by adjusting the brightness in conjunction with gaze confirmation based on the line-of-sight information, There is a high possibility that the balance between adaptability and adaptation suppression can be achieved.

  Therefore, in the brightness adjusting apparatus 10 of this embodiment, the brightness of the in-vehicle display device 20 is controlled in consideration of the effects of glare and reflection and taking into account the driver's eye adaptation. Here, as will be briefly described, it is assumed that the automobile 100 is traveling in a certain place at night (including a dark field place such as in a tunnel even in the daytime).

  When the driver is looking out of the vehicle, the gaze luminance is relatively low, and thus the luminance of the in-vehicle display device 20 is reduced in order to avoid the effects of glare and reflection. However, when the driver returns the line of sight from the in-vehicle display device 20 to the outside of the dark vehicle, the brightness of the in-vehicle display device 20 is lowered thereafter. When the driver views the in-vehicle display device 20 from the outside of the vehicle, the luminance of the in-vehicle display device 20 is set to the same level as the luminance (gaze luminance) in the direction or position that has been viewed. Furthermore, when the driver is continuously watching (watching) the vehicle-mounted display device 20, it matches the brightness of the direction or position predicted to move the line of sight after that (predicted watching brightness). Or the brightness | luminance of the said vehicle-mounted display apparatus 20 is set so that it may correspond substantially (approaching). This will be specifically described below.

  First, it is necessary to know which direction the driver is looking in order to avoid the effects of glare and reflection and to consider the adaptation of the driver's eyes. That is, it is necessary to detect (track) the driver's gaze direction (gaze position). In this embodiment, a driver's face image is photographed by the first camera 14, and the driver's line of sight is detected from the photographed face image. As the line-of-sight detection method, a technique (Japanese Patent Application Laid-Open No. 2008-102902) previously filed by the present applicant and published as an application can be used. Briefly, the center of the eyeball is estimated from the driver's face image, the iris center is extracted, and the direction of the three-dimensional straight line connecting the estimated eyeball center and the extracted iris center is detected as the driver's gaze direction. To do. However, the line-of-sight detection is executed every predetermined time (for example, 1 frame = 1/15 seconds).

  Although detailed description is omitted, the position of the driver's head is measured in advance, and the gaze direction is the reference direction when the direction in which the driver is looking at the front is the reference direction. It is represented by an angle formed in the vertical direction (vertical direction) and the horizontal direction (horizontal direction) with respect to the vector indicating.

  However, the gaze direction detection method need not be limited to the above-described technique, and any other known technique can be used.

  Further, in this embodiment, in order to detect the luminance (gaze luminance) in the driver's gaze direction (gaze position) and to predict the gaze luminance at the location or position (gaze position) where the gaze after movement is directed, A luminance distribution (hereinafter referred to as “environmental luminance distribution”) is detected from the captured image of the camera 16. In this embodiment, the second camera 16 is set so as to be capable of photographing a range that can be visually recognized by the driver during driving (the entire gaze range). Although the detailed description is omitted, for example, the entire gaze range includes a dashboard including the outside and the instrument panel 102, the car navigation display screen 104a and the operation panel 104b, and the air conditioning operation panel 106 that the driver views through the windshield. It is.

Although detailed description is omitted, as described above, the position of the driver's head is measured in advance, and the line of sight is specified so that the gaze position in the entire gaze range can be specified in association with the line of sight. A table (not shown) in which the direction and the gaze position are associated with each other is stored in the computer 12 or DB 18 in advance. However, the gaze position specified from the line-of-sight direction includes a plurality of small regions R _ij to be described later, and the gaze luminance is obtained by calculating an average value of the luminances T _ij (to be described later) of the plurality of small regions R _ij. It is done.

In this embodiment, in order to detect the environmental luminance distribution, the entire gaze range is divided into matrix-like small regions R, and the luminance of each small region R is obtained. For example, as shown in FIG. 8, the entire gaze range is divided into 100 × 100 small areas R. However, the row number for identifying each small region R is represented by a variable i, and the column number is represented by a variable j. Further, as can be seen from FIG. 8, the variable i and the variable j of the upper left small region R _ij are 1, and the variable i and the variable j of the lower right small region R _ij are 100. As described above, in this embodiment, the second camera 16 can obtain a captured image of 300,000 pixels (640 × 480 pixels), so the luminance of one small region R _{ij is} the luminance of 7 × 5 pixels. It is calculated by the average value of. However, the luminance of the pixel on the boundary between the adjacent small regions R _ij is used to calculate the luminance of both small regions R _ij .

Further, the probability distribution (weight) of the gaze position is set for the entire gaze range. This is a probability of where (gaze position) and how long (time length) the driver views during driving, and weighting is performed for each small region R _{ij of the} entire gaze range. Although detailed explanation is omitted, the probability distribution of the gaze position is obtained by detecting how much and how much within a unit time (for example, 60 seconds) when the driver is actually driving. It is done. As shown in FIG. 9, the weight w _ij is assigned to each of the small regions R _ij shown in FIG. The sum total of the weights w _ij is 1, and the weight w _ij is larger as the small region R _ij has a higher probability that the driver sees while driving.

Therefore, as described above, when the driver is driving while looking outside the vehicle (front), relatively low gaze luminance is detected from the detected line-of-sight direction. Therefore, the brightness of the in-vehicle display device 20 is reduced. That is, the influence of “reflection” and “glare” is avoided. However, when the driver turns his gaze from the in-vehicle display device 20 to the outside of the vehicle, the luminance of the in-vehicle display device 20 is controlled in accordance with the luminance of the in-vehicle display device 20 that has been viewed so far. Is reduced. In this embodiment, when the driver has changed the line of sight, the last few seconds including gaze luminance v _g just before changing the line of sight (in this example, 5 seconds) Time Average (gaze luminance gaze luminance v _g of The luminance of the in-vehicle display device 20 is controlled based on the time average) v _g /. However, “/” means an average and is displayed next to the character for convenience of description, but in reality, “−” is displayed on the character (see FIG. 11). The same applies hereinafter.

Similarly, when the driver turns his / her line of sight toward the in-vehicle display device 20 from outside the vehicle, the luminance of the in-vehicle display device 20 is controlled in accordance with the luminance outside the vehicle that has been viewed so far. At this time, as described above, the luminance of the in-vehicle display device 20 is controlled based on the gaze luminance time average v _g /. Therefore, as described above, even when the driver looks at the in-vehicle display device 20 from outside the dark vehicle (gazing), the driver's eyes adapt relatively quickly because there is little difference in brightness. For this reason, the time for viewing the contents of the in-vehicle display device 20 can be made relatively short, and the time for not looking forward can be reduced as much as possible.

  Further, when the driver is continuously watching the display device 20, by using the probability distribution of the gaze position, the brightness of the direction or position (including outside the vehicle) predicted to move the line of sight afterwards is used. The driver's eyes are adapted relatively quickly when the line of sight is changed by controlling the brightness of the in-vehicle display device 20 to approach.

Specifically, luminance (distribution average luminance) v _d averaged over the entire gaze range is calculated from the environmental luminance distribution obtained from the captured image by the second camera 16 and the probability distribution of the gaze position. Specifically, the distribution average luminance v _d is calculated according to Equation 1. However, the luminance of the small region R _ij is T _ij . As described above, the variables i and j are integers from 1 to 100, respectively.

[Equation 1]

The brightness of the in-vehicle display device 20 is controlled so as to approach the distribution average brightness v _d calculated in this way. However, in this embodiment, the average value of distribution average luminance v _d (hereinafter, “distributed luminance time average”) for the past several seconds (5 seconds in this embodiment) including the frame immediately before (one immediately before) the current frame. V _d / is used.

  FIG. 10 is an illustrative view showing an example of a memory map 300 of the RAM 12b shown in FIG. As illustrated in FIG. 10, the RAM 12 b includes a program storage area 302 and a data storage area 304. The program storage area 302 stores a luminance control program for the in-vehicle display device 20, and the luminance control program includes an overall processing program 302a, a gaze detection program 302b, a gaze luminance detection program 302c, an environmental luminance distribution detection program 302d, and a distribution luminance calculation program. 302e, a brightness level control program 302f, and the like.

The overall processing program 302a is a program for executing the main routine of the brightness control process of this embodiment. The line-of-sight detection program 302b is a program for detecting the line-of-sight direction from the captured image of the first camera 14 as described above. Gaze brightness detecting program 302c specifies the gaze position of the gaze entire range from the detected viewing direction in accordance with the sight line detection program 302 detects a gaze brightness v _g of the identified gaze position is its time average gaze This is a program for calculating the luminance time average v _g /.

The environmental luminance distribution detection program 302d is a program for calculating the luminance T _ij for each small region R _ij in the entire gaze range from the captured image of the second camera 16 and obtaining the environmental luminance distribution. Distribution luminance calculation program 302e includes environmental brightness distribution detected by the environmental luminance distribution detecting program 302, and a probability distribution of the gaze position indicated by probability distribution data 304e described later, to calculate the distribution average luminance v _d according to Equation 1 This is a program for calculating the distribution luminance time average v _d / that is the time average. The brightness level control program 302f is a program for controlling the panel brightness level L of the in-vehicle display device 20.

  Although illustration is omitted, the program storage area 302 also stores other programs necessary for the brightness control process.

  The data storage area 304 is provided in the face image data buffer 304a, the environmental luminance data buffer 304b, the gaze luminance data buffer 304c, and the distribution average luminance data buffer 304d. The data storage area 304 stores probability distribution data 304e, line-of-sight data 304f, gaze luminance time average data 304g, distribution luminance time average data 304h, table data 304i, and luminance level data 304j. Further, the data storage area 304 is provided with a gaze flag 304k.

  The face image data buffer 304a is a buffer for storing image data corresponding to the captured image of the first camera 14 every frame. In this embodiment, the image data buffer 304a can store the current frame and several seconds (for example, five seconds) of image data. When the storage area is full, the image data of the current frame is stored. , The oldest image data is deleted.

  The environmental luminance data buffer 304b is a buffer for storing the environmental luminance distribution data (environment luminance distribution data) detected according to the environmental luminance detection program 302d for each frame. In this embodiment, the environmental luminance data buffer 304b can store environmental luminance distribution data for several seconds (for example, five seconds) including the current frame. When the storage area is full, the environmental luminance distribution data of the current frame is filled. Is stored, the oldest environmental luminance distribution data is deleted.

Gaze luminance data buffer 304c is a buffer for each frame stores data (gaze luminance data) for the fixation intensity v _g detected according gaze brightness detecting program 302c. In this embodiment, the gaze luminance data buffer 304c can store gaze luminance data for the current frame and several seconds (for example, 5 seconds), and after the gaze luminance data for several seconds is stored, the gaze luminance data for the current frame is gazeed. When the luminance data is stored, the oldest gaze luminance data is deleted.

Distribution average luminance data buffer 304d is a buffer for each frame stores data (distribution mean luminance data) of the distribution average luminance v _d calculated in accordance with the distribution luminance calculation program 302 e. In this embodiment, the distribution average luminance data buffer 304d can store distribution average luminance data for several seconds (for example, 5 seconds) including the current frame, and when the storage area is full, the distribution average luminance data of the current frame is filled. Is stored, the oldest distribution average luminance data is deleted.

The probability distribution data 304e is data regarding the probability distribution of the gaze position as shown in FIG. As described above, the probability distribution of the gaze position, that is, the weight w _ij is assigned to each small region R _{ij based} on the result measured in advance for the driver, and the sum of the weights w _ij is 1. The line-of-sight data 304f is data regarding the line-of-sight direction detected according to the line-of-sight detection program 302b.

Gaze luminance time-averaged data 304g is gazing luminance v time average for several seconds past including one previous frame of the current frame _g (gaze luminance time-averaged v illustrating gaze luminance data stored in the gaze luminance data buffer 304c is It is the data about _g /). Distribution luminance time-averaged data 304h, the time average (distribution luminance of the past few seconds containing one previous frame of the current frame of the distribution average luminance v _d shown the distribution average luminance data stored in the distribution average luminance data buffer 304d Data on time average v _d /).

  The table data 304i is data for each of the tables 18a, 18b, 18c, and 18d shown in FIGS. 11 and 12, and is read from the DB 18 and stored in the RAM 12b. Here, each table 18a-18d will be described.

As shown in FIG. 11A, the table 18a indicates the luminance (cd / m ² ) corresponding to the panel luminance level L. In this embodiment, the panel luminance level L is expressed in five steps (1-5), and the luminance value corresponding to each level L is described. In this embodiment, a maximum luminance (400cd / ^{m 2)} and no greater than the minimum luminance (25cd / ^{m 2)} generally range in which vehicle display device 20 used, to vary the brightness in five steps, The table 18a is set. As shown in FIG. 11A, when the panel luminance level L is 1, the luminance is 25 (cd / m ² ). Further, when the panel luminance level L is 2, the luminance is 75 (cd / m ² ). Further, when the panel luminance level L is 3, the luminance is 150 (cd / m ² ). Furthermore, when the panel luminance level L is 4, the luminance is 300 (cd / m ² ). When the panel luminance level L is 5, the luminance is 400 (cd / m ² ). Therefore, when the panel brightness level L is determined, the brightness of the in-vehicle display device 20 (display devices 20a, 20b, 20c) is determined.

As shown in FIG. 11B, the table 18b describes the panel target luminance level L _d corresponding to the range of the distribution luminance time average v _d / (cd / m ² ). In this example, the table 18b is referred to when the driver is gazing at the vehicle display device 20, the panel target luminance level L _d at the panel luminance level L is determined. For example, when the distribution luminance time average v _d / (cd / m ² ) is 100 or more and less than 200, the panel target luminance level Ld is determined to be 3. At this time, for example, when the current panel brightness level L is 1 or 2, the panel brightness level L is increased by 1. Further, for example, when the current panel brightness level L is 4 or 5, the panel brightness level L is decreased by 1. Therefore, the luminance of the in-vehicle display device 20 (here, the display devices 20a, 20b, and 20c to which attention is paid) is controlled so as to match or substantially match the predicted gaze luminance. Therefore, even when the driver returns his / her line of sight to the outside (front) of the vehicle thereafter, the gaze luminance does not change significantly, and there is almost no interference due to adaptation.

As shown in FIG. 12 (A), the table 18c, corresponding to the range of the gaze luminance time-averaged ^{_{v g / (cd / m 2}} ), the panel target luminance level _{L T} is described. In this embodiment, the table 18c is referred to when device gaze mode is TRUE, when gaze from a state in which the driver is not gazing at the vehicle display device 20, the panel target luminance level L _T in the panel luminance level L Is determined. Here, the device gazing mode indicates whether or not the driver is gazing at the in-vehicle display device 20. Therefore, when the device gazing mode is TRUE, it indicates that the driver is gazing at the in-vehicle display device 20. On the other hand, when the device gazing mode is FALSE, it indicates that the driver is not gazing at the in-vehicle display device 20. However, the device gaze mode is set for each of the display devices 20a, 20b, and 20c. For example, if the average gaze luminance time ^{_{v g / (cd / m 2}} ) is less than 100 or 200, the panel target luminance level _{L T} is determined to 3, which is set in the panel luminance level L. Therefore, as shown in FIG. 11A, the brightness of the in-vehicle display device 20 (here, the display devices 20a, 20b, and 20c to which attention is paid) is set to 150 (cd / m ² ). That is, the difference between light and dark before and after the line of sight changes. The same applies to other cases.

In this embodiment, when the driver gazes from the state where he / she is not gazing at the vehicle-mounted display device 20, the table 18c is used to match or almost match the gaze luminance time average v _g / (cd / m ² ). Thus, the brightness of the in-vehicle display device 20 is set. However, depending on the performance of the in-vehicle display device 20, the luminance may not be allowed to follow so as to match or substantially match the gaze luminance time average v _g / (cd / m ² ). Accordingly, a predetermined percentage of the average gaze luminance time ^{_{v g / (cd / m 2}} ) ( e.g., 5-7 percent) may be set the brightness of the vehicle display device 20 in. Even in such a case, the luminance of the in-vehicle display device 20 can be controlled based on the gaze luminance, and the driver's eyes can be adapted quickly.

  Thus, as shown in FIGS. 11 (A), 11 (B) and 12 (A), the tables 18a-18c are set and used, so that the adaptation of the driver's eyes is considered and the current The luminance of the in-vehicle display device 20 can be set according to the gaze luminance and the predicted gaze luminance. That is, the luminance of the in-vehicle display device 20 can be controlled based on the gaze luminance and the adaptation of the driver's eyes.

Also, as shown in FIG. 12B, the table 18d describes the panel target luminance level L _F corresponding to the range of the gaze luminance time average v _g / (cd / m ² ), similarly to the table 18c. Is done. The table 18c, the device gaze mode is referred to If FALSE, if the driver does not watch the vehicle display device 20, the panel target luminance level L _F on the panel luminance level L is determined. For example, when the gaze luminance time average v _g / (cd / m ² ) is 100 or more and less than 200, the panel target luminance level L _F is determined to be 2, and this is set to the panel luminance level L. Therefore, as shown in FIG. 11A, the brightness of the in-vehicle display device 20 (here, all the display devices 20a, 20b, and 20c) is set to 75 (cd / m ² ). That is, when the table 18d is set as shown in FIG. 12B corresponding to the table 18a shown in FIG. 11A and the driver is not gazing at the in-vehicle display device 20, this table 18d is used. And since the brightness | luminance of the vehicle-mounted display apparatus 20 falls according to gaze brightness | luminance, reflection and glare can be avoided (suppressed).

Figure 12 (A) and as can be seen from FIG. 12 (B), the numerical and numerical ranges of gaze luminance time-averaged ^{_{v g / (cd / m 2}} ) shown in table 18d, the gaze luminance time-averaged _{v g} shown in table 18c / (Cd / m ² ) and the numerical value range are set larger. For this reason, when the device gazing mode is FALSE, the brightness of the in-vehicle display device 20 is controlled to be lowered.

The tables 18a-18d are merely examples, and should not be limited to this. Depending on the product to be adopted, the numerical value and numerical value range are variably set as appropriate. In some cases, it is necessary to prepare a set of individual tables 18a-18d for each of the display devices 20a, 20b, 20c. However, the correspondence relationship between the table 18a and the table 18b, the correspondence relationship between the table 18a and the table 18c, the correspondence relationship between the table 18a and the table 18d, and the magnitude relationship between the numerical values and the numerical ranges between the table 18c and the table 18d are maintained. Need to be done. Further, the panel luminance level L and the panel target luminance levels L _d , L _T , and L _F may be divided into more levels as long as they are two or more levels.

  Returning to FIG. 10, the luminance level data 304j is data indicating the panel luminance level L set or updated in accordance with the luminance level control program 302f. The gaze flag 304k is a flag for determining whether or not the in-vehicle display device 20 is being watched. In this embodiment, since the three display devices 20a, 20b, and 20c are provided, the gaze flag 304k includes a 3-bit register.

  For example, the most significant bit of the register is assigned to the display device 20a, the least significant bit is assigned to the display device 20c, and the middle bit is assigned to the display device 20b. When the display devices 20a, 20b, and 20c are being watched (device watching mode = TRUE), the data value “1” is set in the corresponding bit. On the other hand, when the display devices 20a, 20b, and 20c are not watched (device watch mode = FALSE), the data value “0” is set in the corresponding bit. Therefore, when the display device 20a is watched, the value of the register indicates “100”. When the display device 20b is watched, the register value indicates “010”. Further, when the display device 20c is watched, the value of the register indicates “001”. Furthermore, when the driver is looking out of the vehicle (front) and the display devices 20a to 20c are not watched, the value of the register indicates “000”.

  Although illustration is omitted, the data storage area 304 stores other data necessary for the brightness control process, and is provided with other flags and timers (counters) necessary for the brightness control process.

FIG. 13 to FIG. 15 are flowcharts showing the overall luminance control processing of the CPU 12a shown in FIG. As shown in FIG. 13, when starting the luminance control process, the CPU 12a sets the apparatus gaze mode to FALSE and sets an initial value (L = _LINIT ) to the panel luminance level L in step S1. As described above, the device gazing mode is set for each of the display devices 20a, 20b, and 20c, and in step S1, the device gazing mode is set to FALSE for all the display devices 20a to 20c. That is, in step S1, the CPU 12a resets the gaze flag 304k, and the value of the register is set to “000”. The initial value L _INIT of the panel brightness level L is a value set by the designer, programmer or driver of the brightness control apparatus 10, and is set to any value 1-5 (for example, 5). .

In the subsequent step S3, the environmental luminance distribution is measured. That is, the CPU 12a calculates the environmental luminance distribution by calculating the luminance T _ij for each small region R _ij using the luminance for each pixel obtained from the captured image data from the second camera 16. Then, the CPU 12a stores the environmental luminance distribution data corresponding to the obtained environmental luminance distribution in the environmental luminance data buffer 304b. Although illustration is omitted, the photographing process of the second camera 16 is started when the entire process is started. When the entire process is started, the shooting process of the first camera 14 is also started, and the shot image data is stored for each frame in a buffer area (not shown). However, the captured image data of the first camera 14 can be stored for the current frame and several seconds (for example, for 5 seconds). Therefore, when the storage area is full, the oldest photographed image data is deleted when the photographed image data of the current frame is stored.

In the next step S5, the distribution average luminance v _d is calculated. Here, the distribution average luminance v _d is calculated according to the equation 1 from the environmental luminance distribution (luminance T _ij ) measured in step S3 and the weight w _ij indicated by the probability distribution data 304e, and the corresponding distribution average luminance data is the distribution average. Stored in the luminance data buffer 304d. In step S7, the distribution luminance time average v _d / is updated. In other words, it calculates the average value of the distribution average brightness v _d of the past few seconds, including a distribution average luminance v _d of the current frame calculated in step S5, the calculated distribution luminance time-averaged v _{d /} a distributed brightness time corresponding The average data 304h is stored (overwritten) in the data storage area 304.

Next, in step S9, the line of sight is detected. That is, as described above, the CPU 12a calculates the driver's line-of-sight direction from the captured image of the first camera 14. Subsequently, in step S11, it calculates the gaze luminance _{v g.} Here, attention luminance v _g gaze position determined by the viewing direction of the driver, calculates the distribution average luminance data buffer 304d distribution average luminance indicated by the distribution average luminance data of the current frame stored (brightness T _ij) Is done. The gaze luminance data corresponding to the gaze luminance v _g calculated is stored in the gaze luminance data buffer 304c.

Furthermore, in step S13, the gaze luminance time average v _g / is updated. Here, the average value of the gaze luminance v _g the past few seconds including gaze luminance v _g of the immediately preceding frame of the current frame stored in the gaze luminance data buffer 304c, the calculated gaze luminance time-averaged v _The gaze luminance time average data 304g corresponding to _g / is stored (overwritten) in the data storage area 304. In step S15, it is determined whether the line of sight faces the target device. Here, the CPU 12a determines whether or not the driver is looking at the display devices (20a, 20b, and 20c) that the driver pays attention to according to the line-of-sight direction detected in step S9.

  In step S15 and subsequent steps, for the sake of simplicity, one of the target display devices (20a, 20b, and 20c) is set as a target device, and only processing for the one target device is illustrated. Therefore, actually, the processing after step S15 is executed for each of the display devices 20a-20c.

  If “YES” in the step S15, that is, if the line of sight faces the target device, the process proceeds to a step S17 shown in FIG. On the other hand, if “NO” in the step S15, that is, if the line of sight does not face the target device, the process proceeds to a step S35 shown in FIG.

In step S <b> 17 shown in FIG. 14, it is determined whether or not the device watching mode for the target device is TRUE. Here, the CPU 12a refers to the gaze flag 304k, and determines whether the data value of the bit assigned to the target device (display devices 20a, 20b, 20c) is “1”. The same applies to step S35 described later. If “YES” in the step S17, that is, if the device gazing mode is TRUE, it is determined that the state of watching the target device is continued, and in step S19, the panel is calculated from the distribution luminance time average v _d /. setting a target luminance level _{L d.} That is, in step S19, the CPU 12a refers to the table 18b included in the table data 304i, and the panel target luminance level L corresponding to the distribution luminance time average v _d / indicated by the distribution luminance time average data 304h calculated in step S7. Determine _d .

Subsequently, in step S21, the panel luminance level L is larger than the panel target luminance level L _d, and panel luminance level L is determined whether greater than 1 (minimum value). If in step S21 "YES", i.e. the panel luminance level L is larger than the panel target luminance level L _d, and if the panel luminance level L is greater than 1, in step S23, the panel brightness level L 1 is subtracted (L = L-1), the process returns to step S3. That is, in step S23, the panel luminance level L is matched to the panel target luminance level L _d, or to approach. The same applies to step S27 described later.

On the other hand, if "NO" in the step S21, i.e. the panel luminance level L is less than or equal to the panel target luminance level L _d, or the panel luminance level L is 1, or, in the case of both, step in S25, the panel luminance level L is smaller than the panel target luminance level L _d, and panel luminance level L is 5 determines whether less than (maximum value).

If "NO" in the step S25, that is, the panel brightness level L is the panel target luminance level L _d or more, or a panel luminance level L is 5 (maximum value), or, in the case of both, as is Return to step S3. That is, in such a case, the panel brightness level L is the maximum value or the minimum value, or can not be increased or decreased, since the panel brightness level L and the panel target luminance level L _d are the same, to adjust the panel luminance level L There is no need. On the other hand, if "YES" in step S25, that is, if the panel luminance level L is smaller than the panel target luminance level L _d, and panel luminance level L is smaller than 5, in step S27, the panel luminance level L Is increased by 1 (L = L + 1), and the process returns to step S3.

  If “NO” in the step S17, that is, if the gazing mode for the target device is FALSE, it is determined that the target device has changed from a state of not gazing to a state of gazing, and in a step S29. The apparatus gaze mode for the target apparatus is set to TRUE. That is, in step S29, in the gaze flag 304k, the CPU 12a sets the data value “1” to the bits assigned to the display devices 20a, 20b, and 20c that the driver gazes, and sets the data value “0” to the other bits. Set.

In step S31, it determines a panel target luminance level _{L T} from the gaze luminance time-averaged _{v g} /. Here, since device gaze mode is TRUE, CPU 12a refers to the table 18c included in the table data 304i, determines the panel target luminance level _{L T.} Then, in step S33, sets the panel target luminance level _{L T} to the panel luminance level L (L _{= L} T), the flow returns to step S3.

  As described above, when the line of sight does not face the target device, “NO” is determined in the step S15, and it is determined whether or not the device gazing mode is TRUE in a step S35 shown in FIG. If “NO” in the step S35, that is, if the device gazing mode is FALSE, it is determined that the state in which the target device is not being watched is continued, and the process proceeds to a step S39 as it is. On the other hand, if “YES” in the step S35, that is, if the device gazing mode is TRUE, it is determined that the target device is changed from a state of gazing to the state of not gazing the target device, and the step S37. The apparatus gaze mode is set to FALSE, and the process proceeds to step S39.

In step S39, the panel target luminance level L _F is determined from the gaze luminance time average v _g /. Here, since device gaze mode is FALSE, CPU 12a refers to the table 18d included in the table data 304i, determines the panel target luminance level _{L F.} Then, at step S41, sets the panel target luminance level L _F on the panel luminance level L, the flow returns to the step S3 shown in FIG. 13.

Although illustration is omitted, when the panel luminance level L is set in steps S23, S27, S33 and S41, the CPU 12a determines the luminance corresponding to the panel luminance level L based on the table 18a shown in FIG. The controller or driver of the target device is driven so as to emit light at (cd / m ² ).

FIG. 16 shows an example in which the brightness control apparatus 10 of this embodiment is operated. As shown in FIG. 16, in the section A and the section C, the gaze target is other than the in-vehicle display device 20 (display devices 20a, 20b, 20c). In the section B, the gaze target is one of the display devices 20a, 20b, and 20c. It is assumed that the distribution luminance time average v _d / and the gaze luminance time average v _g / change with time as shown in FIG. As described above, since the gaze target changes, the apparatus gaze mode is set to FALSE in the section A, set (changed) to TRUE after entering the section B, and then set to FALSE again after entering the section C. (Changed). In such a case, the panel luminance level changes from the initial value L _INIT (= 5) according to time t.

Further, as described above, the tables (18c, 18d) to be referred to are different depending on whether the device gazing mode is TRUE or FALSE. Therefore, when the apparatus gaze mode is FALSE, the panel target luminance level L _F is determined according to the gaze luminance time average v _g / as shown on the left side of FIG. On the other hand, if the device gaze mode is TRUE, as shown on the right side of FIG. 16, panel target luminance level L _T in accordance with the gaze luminance time-averaged v _{g /} are determined.

As shown in FIG. 16, in the section A and section C, as described above, fixation luminance time-averaged v _{g /} target luminance level L _F according to is determined, the luminance of the panel luminance level L (display device 20a-20c ) Is set (changed). Therefore, the panel luminance level L is increased when the gaze luminance time average v _g / is high in preparation for a sudden panel gaze while suppressing the occurrence of reflection and glare. Further, in the section B, and becomes an apparatus gaze mode, this time, the panel target luminance level L _T in accordance with the gaze luminance time-averaged v _{g /} are determined, the panel luminance level L is set. Therefore, the luminance is set to be easy to see first. After that, after moving the line of sight to a device other than the device (including outside the vehicle), the panel target luminance level L _D is determined according to the distribution luminance time average v _d / so that no interference due to adaptation remains, and the panel luminance level L Is set (changed) step by step.

However, in the example shown in FIG. 16, in the section B, the panel target luminance level L _D determined according to the distribution luminance time average v _d / does not change (L _D = 2). Therefore, in Section B, the panel luminance level L, when the apparatus fixation mode becomes to TRUE, the after rising once in response to the panel target luminance level L _T, decreases to approach the panel target luminance level L _D It is then set to the panel target luminance level L _D.

  According to this embodiment, reflection and glare due to light emission of the in-vehicle display device are avoided, and brightness of the in-vehicle display device is controlled in consideration of adaptation of the driver's eyes, so that the brightness can be adjusted appropriately. Therefore, a comfortable driving environment can be provided.

  In this embodiment, the luminance is adjusted in the entire instrument panel, but the luminance may be adjusted in units of measuring devices.

  In this embodiment, the environmental luminance distribution is detected using a camera capable of capturing a monochrome image, but the present invention is not limited to this. For example, an optical sensor array capable of detecting luminance may be provided. However, an LED or a photodiode can be used as the optical sensor.

  Further, in this embodiment, the driver is determined in advance, the position of the driver's head is also measured in advance, and a table describing the gaze position corresponding to the line-of-sight direction at the head position is stored. However, it is not necessary to be limited to this. For example, by providing two cameras or one stereo camera, it is possible to calculate the position of the driver's head based on captured images of the driver's head or upper body taken from two different directions. . The gaze position may be calculated from the position of the head and the line-of-sight direction.

DESCRIPTION OF SYMBOLS 10 ... Brightness control apparatus 12 ... Computer 12a ... CPU
12b RAM
14, 16 ... Camera 18 ... Database

Claims (13)

A brightness control device for an in-vehicle display device provided in an automobile,
A luminance distribution detecting means for detecting a luminance distribution within a range visible to the driver;
Line-of-sight detection means for detecting the driver's line-of-sight direction;
Gaze determination means for determining whether the driver is gazing at the in-vehicle display device based on the gaze direction detected by the gaze detection means;
Gaze luminance detection means for detecting gaze luminance of the driver from the gaze direction detected by the gaze detection means and the luminance distribution detected by the luminance distribution detection means, and the driver uses the in-vehicle display device In-vehicle equipped with brightness control means for controlling the brightness of the in-vehicle display device based on the gaze brightness detected by the gaze brightness detection means before the change when changing from a non-gaze state to a gaze state Brightness control device for display device. Probability distribution storage means for storing a probability distribution of the gaze position for the visible range of the driver,
The luminance control means includes a luminance distribution detected by the luminance distribution detection means and a probability stored in the probability distribution storage means when the driver is gazing at the in-vehicle display device. The brightness control device for an in-vehicle display device according to claim 1, wherein the brightness of the in-vehicle display device is controlled based on the distribution.   When the driver changes from a state in which the driver is not gazing to the vehicle-mounted display device to a state in which the driver is gazing, the luminance control unit is set to a value that correlates with the gaze luminance detected by the gaze luminance detection unit before the change. The brightness control device for an in-vehicle display device according to claim 1, wherein the brightness of the in-vehicle display device is set.   The in-vehicle display device according to any one of claims 1 to 3, wherein the luminance control means reduces the luminance of the in-vehicle display device when the driver is not gazing at the in-vehicle display device.   The brightness control device for an in-vehicle display device according to any one of claims 1 to 4, wherein the brightness distribution and the gaze brightness are average values for a predetermined past time including a current time. A brightness control device for an in-vehicle display device provided in an automobile,
A luminance distribution detecting means for detecting a luminance distribution within a range visible to the driver;
Line-of-sight detection means for detecting the driver's line-of-sight direction;
Based on the line-of-sight direction detected by the line-of-sight detection means, gaze determination means for determining whether or not the driver is gazing at the vehicle-mounted display device, and a state where the driver is gazing at the vehicle-mounted device The luminance for controlling the luminance of the in-vehicle display device based on the luminance distribution detected by the luminance distribution detecting means and the probability distribution of the gaze position for the visible range of the driver when continuing A brightness control device for an in-vehicle display device, comprising control means. A brightness control device for an in-vehicle display device provided in an automobile,
A luminance distribution detecting means for detecting a luminance distribution within a range visible to the driver;
Line-of-sight detection means for detecting the driver's line-of-sight direction;
The gaze luminance detecting means for detecting the driver's gaze luminance from the gaze direction detected by the gaze detection means and the luminance distribution detected by the luminance distribution detection means, and detected by the gaze luminance detection means A luminance control device for an in-vehicle display device, comprising: luminance control means for controlling the luminance of the in-vehicle display device based on gaze luminance and adaptation of the driver's eyes. A brightness control program for a brightness control device of an in-vehicle display device provided in an automobile,
In the processor of the brightness control device,
A luminance distribution detection step for detecting a luminance distribution within a range that is visible to the driver;
A line-of-sight detection step for detecting the driver's line-of-sight direction;
A gaze determination step of determining whether the driver is gazing at the in-vehicle display device based on the gaze direction detected by the gaze detection step;
A gaze luminance detection step for detecting the gaze luminance of the driver from the gaze direction detected by the gaze detection step and the luminance distribution detected by the luminance distribution detection step, and the driver gazes at the in-vehicle display device A luminance control step for executing a luminance control step for controlling the luminance of the in-vehicle display device based on the gaze luminance detected by the gaze luminance detection step before the change when the state changes from a non-gated state to a gaze state Control program. A brightness control program for a brightness control device of an in-vehicle display device provided in an automobile,
In the processor of the brightness control device,
A luminance distribution detection step for detecting a luminance distribution within a range that is visible to the driver;
A line-of-sight detection step for detecting the driver's line-of-sight direction;
Based on the line-of-sight direction detected by the line-of-sight detection step, a gaze determination step of determining whether the driver is gazing at the vehicle-mounted display device, and a state where the driver is gazing at the vehicle-mounted device The luminance for controlling the luminance of the in-vehicle display device based on the luminance distribution detected by the luminance distribution detecting step and the probability distribution of the gaze position for the visible range of the driver when continuing A brightness control program for executing control steps. A brightness control program for a brightness control device of an in-vehicle display device provided in an automobile,
In the processor of the brightness control device,
A luminance distribution detection step for detecting a luminance distribution within a range that is visible to the driver;
A line-of-sight detection step for detecting the driver's line-of-sight direction;
The gaze luminance detection step for detecting the driver's gaze luminance from the gaze direction detected by the gaze detection step and the luminance distribution detected by the luminance distribution detection step, and the gaze luminance detection step detected A luminance control program for executing a luminance control step for controlling luminance of the in-vehicle display device based on gaze luminance and adaptation of the driver's eyes. A brightness control method for a brightness control device of an in-vehicle display device provided in an automobile,
(A) Detecting the luminance distribution in a range that is visible to the driver,
(B) detecting the driver's line-of-sight direction;
(C) Based on the line-of-sight direction detected in step (b), it is determined whether the driver is gazing at the in-vehicle display device,
(D) detecting the driver's gaze luminance from the line-of-sight direction detected in step (b) and the luminance distribution detected in step (a); and (e) the driver displays the vehicle-mounted display. A luminance control method for controlling the luminance of the in-vehicle display device based on the gaze luminance detected by the step (d) before the change when the device changes from a state where the device is not gazing to a state where the device is gazing. A brightness control method for a brightness control device of an in-vehicle display device provided in an automobile,
(A) Detecting the luminance distribution in a range that is visible to the driver,
(B) detecting the driver's line-of-sight direction;
(C) based on the line-of-sight direction detected in the step (b), it is determined whether the driver is gazing at the in-vehicle display device; and (d) the driver is gazing at the in-vehicle device. The in-vehicle display device brightness based on the luminance distribution detected in step (a) and the probability distribution of the gaze position for the visible range of the driver. Control method, brightness control method. A brightness control method for a brightness control device of an in-vehicle display device provided in an automobile,
(A) Detecting the luminance distribution in a range that is visible to the driver,
(B) detecting the driver's line-of-sight direction;
(C) detecting the driver's gaze luminance from the line-of-sight direction detected in step (b) and the luminance distribution detected in step (a); and (d) according to step (c). A brightness control method for controlling brightness of the in-vehicle display device based on detected gaze brightness and adaptation of the driver's eyes.