JP2008001182A - Vehicular visual information presenting device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent visibility of visual information from being degraded by a change of luminance of a background corresponding to a position for presenting the visual information. <P>SOLUTION: A background luminance estimation means 5 estimates luminance of a plurality of points in a front scene of a driver. An image control means 7 controls the luminance of a display image displayed in the front visual field of the driver based on the luminance of the plurality of points estimated by the background luminance estimation means 5. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a visual information presentation device for a vehicle and a visual information presentation method for a vehicle that assist a driver in driving the vehicle by presenting visual information within the visual field of the driver.

Conventionally, as a first vehicle visual information presentation device, the brightness of a display image background and a region other than the background is detected using an optical sensor, and the brightness of the display image is adjusted based on the detected brightness. Thus, there is known a display device that presents a driver with a display image having an optimum luminance regardless of day or night (see Patent Document 1). Further, as the second visual information presentation device for a vehicle, the illuminance around the vehicle is detected at every predetermined timing that changes according to the vehicle speed, and the luminance of the display by the combiner is adjusted based on the average value of the detected illuminance. Thus, a display device is known that reduces flickering in display brightness accompanying a sudden change in brightness around the vehicle (see Patent Document 2). Further, as the third vehicle visual information presenting device, the relationship between the illuminance in the passenger compartment and the background luminance of the display image is obtained in advance in the fine weather and in the cloudy weather, and the range in which the data at both times is the upper limit and the lower limit There is known a display device that adjusts the luminance of a display image to adjust the luminance of the display image to an optimum state according to background luminance or weather conditions (see Patent Document 3).
JP 2004-314860 A Japanese Patent No. 2914066 JP 7-132758 A

  However, since the first visual information presentation device for a vehicle is configured to detect the background brightness at the two points of the background and the non-background area, the background brightness is accurately detected depending on the background contrast. There are things you can't do. For this reason, according to the 1st visual information presentation apparatus for vehicles, since the brightness | luminance of a display image is not optimized with respect to the brightness | luminance of actual background, the visibility of a display image may deteriorate. The second visual information presentation device for a vehicle is configured to optimize the luminance of the display image based on the illuminance around the vehicle, and accurately estimates the luminance of the background at the position where the display image is displayed. Inability to do so may degrade the visibility of the display image. Furthermore, since the third vehicular visual information presenting apparatus is configured to indirectly estimate the background luminance from the room illuminance using the relationship between the background luminance and the room illuminance, a dark region or a bright light is included in the background. If there is a region or the like and a distribution state exists in luminance, the background luminance cannot be accurately estimated, and the visibility of the display image may be deteriorated.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to prevent a deterioration in the visibility of visual information due to a change in luminance of the background corresponding to a position where the visual information is presented. It is providing the visual information presentation apparatus for vehicles, and the visual information presentation method for vehicles.

  The visual information presentation device for a vehicle and the visual information presentation method for a vehicle according to the present invention estimate the luminance of a plurality of points in a landscape, and control the luminance of the visual information based on the estimated luminances of the plurality of points.

  According to the visual information presentation device for a vehicle and the visual information presentation method for a vehicle according to the present invention, the background luminance corresponding to the position where the visual information is presented can be accurately estimated for each pixel of the imaging device. It is possible to prevent the visibility of visual information from deteriorating due to a change in the brightness of the background corresponding to the position where the image is presented.

  Hereinafter, with reference to the drawings, the configuration and operation of a vehicular visual information presentation device according to an embodiment of the present invention will be described.

[Configuration of visual information presentation device for vehicles]
As shown in FIG. 1, a visual information presentation device for a vehicle according to an embodiment of the present invention is provided in a vehicle 1, and includes a host vehicle running state estimation unit 2, a vehicle exterior environment estimation unit 3, and an eyeball position estimation unit 4. The background luminance estimation means 5, the data storage means 6, the image control means 7, and the visual information display means 8 are provided as main components. The functions of the vehicle running state estimation means 2, the vehicle exterior environment estimation means 3, the eyeball position estimation means 4, the background luminance estimation means 5, and the image control means 7 are realized by an in-vehicle computer executing a control program. .

  The own vehicle traveling state estimation means 2 includes a traveling direction of the vehicle 1, a horizontal position in front of the vehicle 1, a speed and a steering angle of the vehicle 1, a level of the vehicle 1 (pitch, roll), a headlight signal (high beam, low beam), Information representing the running state of the vehicle 1 such as the presence / absence of a wiper signal and a blinker signal and level information of the level riser is detected, and the running state of the vehicle 1 is estimated based on the detection result. The own vehicle traveling state estimation unit 2 records the estimated traveling state data in the data storage unit 6.

  The outside environment estimation means 3 estimates the outside environment of the vehicle 1 such as time information, direction information, weather conditions outside the vehicle 1, position information of the vehicle 1, and map information around the vehicle 1, and data of the estimated outside environment Is stored in the data storage means 6.

  The eyeball position estimating means 4 estimates the eyeball position of the driver who drives the vehicle 1. Specifically, the eyeball position estimation means 4 sets a driver having a standard body shape as a driver driving the vehicle 1 and outputs the set driver's eye position as an arbitrary fixed value. Thus, the eyeball position of the driver driving the vehicle 1 is estimated.

  The eyeball position estimating means 4 detects the position of the driver's seat and the angle of the mirror, and the driver's seat detected from the table data indicating the correspondence between the position of the driver's seat and the angle of the mirror and the eyeball position of the driver. The eyeball position of the driver who drives the vehicle 1 may be estimated by searching the driver's eyeball position corresponding to the position of the vehicle and the angle of the mirror (at least one of the side mirror and the rearview mirror). The eyeball position estimation means 4 may estimate the driver's left and right eyeball positions by capturing and analyzing the driver's face image using an imaging device and a distance sensor.

  The background luminance estimation means 5 is constituted by an imaging device such as a C-MOS camera installed in the front part of the vehicle 1 or in the passenger compartment, and estimates the luminance of the area in the driver's front view in units of pixels. In addition, you may estimate the brightness | luminance of all the area | regions in a driver | operator's front visual field with one imaging device by providing a wide angle lens to an imaging device. Further, by setting a plurality of imaging devices, the range in which the luminance can be estimated may be widened.

  The data storage means 6 is composed of a storage device such as a RAM (Random Access Memory), and the processing results of the vehicle running state detection means 2, the vehicle exterior environment estimation means 3, the eyeball position estimation means 4, and the background luminance estimation means 5 are displayed. Remember.

  The image control means 7 controls the display position, contrast, color, shape, and display on / off of the visual information presented to the driver of the vehicle 1 with reference to the information stored in the data storage means 5.

  The visual information display means 8 is constituted by a display device such as a head-up display, a projector, a transparent liquid crystal, and an LED, and in the windshield area of the vehicle 1 according to the control of the image control means 7, more specifically, the driver's display. Visual information is superimposed on the front scenery position that does not interfere with vehicle travel.

  In the vehicular visual information presentation device having such a configuration, the image control means 7 executes the following display control processing, whereby the visibility of the visual information is determined by the luminance change of the background corresponding to the position where the visual information is presented. To prevent deterioration. The operation of the image control means 7 when executing the display control process will be described below with reference to the flowchart shown in FIG.

[Display control processing]
The flowchart shown in FIG. 2 starts when the power supply of the visual information presentation device for vehicles is turned on, and the display control process proceeds to step S1.

  In the process of step S 1, the image control unit 7 acquires the size of the visual information (hereinafter referred to as a display image) presented to the driver and the coordinate value of the center point, and stores the acquired data in the data storage unit 6. To do. The coordinate value of the center point of the display image has a standard body shape and indicates the coordinate value of the center point of the display image viewed from the viewpoint of the driver sitting in the driver's seat. Thereby, the process of step S1 is completed and a display control process progresses to the process of step S2.

  In the process of step S <b> 2, the eyeball position estimation unit 4 estimates the eyeball position of the driver sitting in the driver's seat, and stores the estimated eyeball position data in the data storage unit 6. Thereby, the process of step S2 is completed, and the display control process proceeds to the process of step S3.

  In the process of step S3, the background luminance estimation unit 5 captures a background image in the driver's forward visual field, and drives the background image captured based on the driver's eyeball position stored in the data storage unit 6. By converting the viewpoint to an image viewed from the viewpoint of the person (hereinafter referred to as a converted image), the captured background image is made to correspond to the display area of the display image. Thereby, the process of step S3 is completed, and the display control process proceeds to the process of step S4.

  In the process of step S4, the background luminance estimation unit 5 specifies a position in the converted image on which the display image is superimposed and displayed based on the driver's eyeball position stored in the data storage unit 6. Thereby, the process of step S4 is completed, and the display control process proceeds to the process of step S5.

  In the process of step S5, the background luminance estimation means 5 extracts the luminance values of a plurality of pixels near the position specified by the process of step S4, and the pixel luminance value and the aperture value of the imaging device when the background image is imaged. The extracted luminance values of the plurality of pixels are converted into background luminance values with reference to a table indicating the correspondence relationship between and the background luminance. Note that the number of pixels from which the luminance value is extracted varies depending on the size of the display image and the processing capability of the apparatus. For example, the upper left, upper, upper right, left, center, and right of the superimposed position of the display image It is preferable to set a 3 × 3 matrix in the section, lower left, lower, and lower right, and calculate the average value of the luminance values of the pixels constituting the 3 × 3 matrix. Thereby, the process of step S5 is completed, and the display control process proceeds to the process of step S6.

  In the process of step S7, the image control means 7 keeps the brightness above a certain value, for example, with respect to an average value of 80% of the background brightness value, or the background brightness and the brightness contrast of the display image become constant for each pixel. For example, the luminance value of the display image is set according to the background luminance value converted by the process of step S6. Note that the image control means 7 may calculate the maximum value, the minimum value, the average value, and the standard deviation value of the background luminance value, and set the luminance value of the display image according to the calculated value. Thereby, the process of step S7 is completed, and the display control process proceeds to the process of step S8.

  In the process of step S8, the visual information display means 8 superimposes and displays the display image on the front landscape with the brightness value set by the process of step S7. Thereby, the process of step S8 is completed and a series of display control processes are complete | finished.

  According to such display control processing, for example, as shown in FIG. 3, when the display image A indicating the inter-vehicle distance with the preceding vehicle 11 is superimposed on the front landscape, the preceding vehicle 11 turns on the brake lamp. When the background luminance at the position where the display image A is superimposed is increased, the visibility of the display image A is reduced by increasing the luminance of the display image A in accordance with the lighting of the brake lamp as shown in FIG. Can be prevented. In addition, since the luminance of the background on which the display image is superimposed is estimated in consideration of the driver's eyeball position, even if the driver's viewpoint changes due to changes in the posture of different drivers or drivers The brightness of the display image can be optimally controlled.

  When controlling the luminance value of the display image according to the background luminance value, the image control means 7 holds the luminance value of the display image at the current value for a predetermined time so that the luminance value of the display image is constant for a predetermined time. You may make it change to a setting value later. Further, the image control means 7 may perform control so that the luminance value of the display image becomes a set value continuously or stepwise over a predetermined time. According to such processing, it is possible to prevent the driver from feeling troublesome with respect to the display image or reducing the visibility of the display image due to frequent changes in the luminance value of the display image. The predetermined time is preferably about several seconds such as the time until the driver's eyes adapt to light and dark at the entrance of the tunnel.

  Further, the image control means 7 determines whether or not the display image includes information whose positional relationship with the background is important. If the positional relationship with the background is important, the image control means 7 determines the position of the display image. May follow or move in front of the scenery. According to such a process, the display position can be optimally controlled according to the content of the display image to improve the visibility of the display image, and the viewing efficiency can be increased. Examples of information in which the positional relationship with the background is important include location information indicating the destination as shown in FIG. 5, destination orientation, terrain information, route guidance information, location information of pedestrians and approaching vehicles, etc. can do. In addition, examples of information whose positional relationship with the background is not important include travel speed information and warning information of the vehicle 1, and in this case, the position of the display image is fixed.

  Further, the image control means 7 estimates a specific object existing ahead from the distribution state of the background luminance value, and prevents the display image from being superimposed on the object, for example, so that the display image for the estimated object is displayed. The position may be controlled. According to such processing, the visibility of the display image can be improved and the driver can be prevented from feeling bothersome about the display image without hindering the visibility of the specific object. In addition, as a specific target object, the label | marker 12 as shown in FIG. 6, a signal, etc. can be illustrated. As described above, when the display image includes information related to the target object, the image control unit 7 may superimpose at least a part of the display image on the target object.

  In addition, the image control means 7 changes background luminance values such as a road area and an area where the background luminance value is relatively small, such as an empty area above the horizon, and a left and right area of the vehicle 1 where external information is mixed. The converted image may be divided into relatively large areas, and the display image may be superimposed and displayed on the optimum divided area. In this case, in order to reduce the processing load, it is desirable to perform the division process only on the range of the background image that can be visually recognized by the driver. Further, the image control means 7 estimates road terrain information such as the positions of the left and right white lines and the center line as the road boundary, the position of the horizon, and the like based on the estimated road terrain information from the background luminance value distribution state. The image may be divided and the display image may be superimposed and displayed on the optimum divided area. Specifically, the image control means 7 is defined in advance as shown in FIG. 8 around a vanishing point B (see FIG. 7) defined by the intersection of the straight line L1 and the horizontal line L2 indicating the traveling direction of the vehicle. The converted image is divided into regions having continuous boundary values and having a boundary line indicating a change in luminance value equal to or greater than a predetermined value. In addition, areas (A) to (F) shown in FIG. 8 are road, headlight irradiation range (road), hood tip visible range of vehicle 1, horizon L2 information area, front landscape left area, and front landscape, respectively. The right area is shown. According to such processing, the display image can always be displayed at a position that is easy for the driver to visually recognize, so that the visibility of the display image is improved and the driver feels bothered by the display image. Can be prevented.

  Further, the image control means 7 gives priority to the divided areas in the order of the areas where the display image is easily visible based on the maximum value, the minimum value, the average value, and the standard deviation value of the background luminance values. You may make it display a display image in a division area according to order. Specifically, when a display image is displayed in an area with high luminance, the display image is difficult to see because the luminance contrast between the background and the display image is low, and the display image is displayed in an area with large luminance variations. In this case, it becomes difficult to see the entire display. For this reason, the image control means 7 gives priority to the divided regions in the order of the luminance not being large and the standard deviation of the luminance distribution being small.

  In addition, when the luminance of the display image does not have a sufficient difference from the luminance of the background after the display image is displayed in the divided area, the image control unit 7 again divides the front landscape into a plurality of areas, The display image may be controlled to move to other divided areas according to the priority order. Specifically, when an arrow display for route guidance is displayed as a display image superimposed on the front landscape, and when the display image is displayed in a road area where the luminance change is small, it is preceded on the road. When a vehicle, an obstacle, or the like is present, the change in luminance of the road area increases, so the image control means 7 may display a display image in an empty area above the horizontal line. Further, as shown in FIG. 9, when the brake lamp of the preceding vehicle 11 is turned on, the image control means 9 may control the movement of the display image to the left background area in consideration of the visibility during night driving. Similarly, when the display image overlaps the headlamp irradiation range of the oncoming vehicle 13 as shown in FIG. 10, the image control means 9 moves the display area of the display image from the road area to the upper horizon area. Also good. Similarly, when a light source having a higher luminance than that of the surroundings such as a streetlight is estimated during night driving, the image control means 9 desirably moves the display area of the display image.

  Further, the image control means 7 may control the position of the display image according to the traveling state of the vehicle 1 estimated by the own vehicle traveling state estimating means 2. In this case, specifically, the image control means 7 estimates the vanishing point based on the traveling direction and the front horizontal position of the vehicle 1 estimated by the own vehicle traveling state estimation means 2. Then, the image control means 7 divides the front landscape into a plurality of regions based on the estimated vanishing point, optimizes the luminance contrast with the background, and controls the display image to the region with the highest priority. At this time, the image control means 7 may control the position of the display image so that the display image is always displayed in the traveling direction area of the vehicle 1. When the display area of the display image is changed, the image control means 7 changes the display area to the target area after holding the display area in the current display area for a predetermined time such as several seconds, or the display area over the predetermined time. May be moved continuously to the target area. Further, the image control means 7 changes the priority order of the display image display areas according to the running state of the vehicle 1 estimated by the own vehicle running state estimating means 2 such as the presence or absence of the headlight lighting operation and the wiper operation. May be. Specifically, when the vehicle running state estimating unit 2 estimates that the vehicle 1 is traveling on a curved road or an up / downhill, the image control unit 7 frequently determines that the luminance of the background region is high. The display image may be displayed in an area other than the changing area.

  In addition, the image control means 7 is used for the vehicle 1 estimated by the outside environment estimation means 3 when the state of the front field of view changes greatly due to bad weather such as rain or fog, or when it differs greatly from the normal time such as driving at night. The position of the display image may be controlled according to the environmental state. At this time, the image control means 7 may control the position of the display image so that the display image is always displayed in the traveling direction area of the vehicle 1.

  In general, since the background luminance is extremely low outside the headlight irradiation range, the image control means 7 does not perform background area control, but the head estimated by the vehicle running state estimation means 2 or the outside environment estimation means 3. Estimate the headlight irradiation range from the light on / off, high beam, low beam, leveler level information, vehicle attitude information, etc., and display the display image so that the display image is not displayed across the boundary of the headlight irradiation range You may make it display on one of a bright area | region and a dark area | region.

  Finally, FIG. 11 shows an example in which the display position of the display image and the control of the brightness contrast based on the background brightness value are summarized for each traveling scene and time zone of the vehicle 1.

  As mentioned above, although the embodiment to which the invention made by the present inventors was applied has been described, the present invention is not limited by the description and the drawings that form part of the disclosure of the present invention according to this embodiment. That is, it should be added that other embodiments, examples, operation techniques, and the like made by those skilled in the art based on this embodiment are all included in the scope of the present invention.

It is a block diagram which shows the structure of the visual information presentation apparatus for vehicles used as embodiment of this invention. It is a flowchart figure which shows the flow of the display control process used as embodiment of this invention. It is a figure which shows the state which superimposed and displayed the display image showing the inter-vehicle distance with a preceding vehicle on the front scenery. It is a figure which shows the state which increased the brightness | luminance of the display image according to lighting of the brake lamp. It is a figure which shows the state which displayed the positional information which shows the destination as a display image. It is a figure for demonstrating the process which controls the position of a display image according to the positional relationship of a target object and a display image. It is a figure for demonstrating the definition of a vanishing point. It is a figure for demonstrating the process which divides | segments a converted image into a some area | region. It is a figure for demonstrating the process which carries out the movement control of a display image when the brake lamp of a preceding vehicle lights. It is a figure for demonstrating the process which controls a display image when a display image overlaps with the headlamp irradiation range of an oncoming vehicle. It is the figure which put together the control of the display position of the display image based on a background luminance value, and brightness | luminance contrast according to the driving | running | working scene and time slot | zone of a vehicle.

Explanation of symbols

1: vehicle 2: own vehicle running state estimation means 3: vehicle exterior environment estimation means 4: eyeball position estimation means 5: background luminance estimation means 6: data storage means 7: image control means 8: visual information display means

Claims (15)

A visual information presentation device for a vehicle that supports a driver's vehicle driving operation by displaying visual information superimposed on a landscape in the driver's front vision,
Background luminance estimating means for estimating the luminance of a plurality of points in the landscape;
A visual information presentation device for a vehicle, comprising: image control means for controlling the brightness of the visual information based on the brightness of a plurality of points estimated by the background brightness estimation means.   The visual information presentation device for a vehicle according to claim 1, wherein the background luminance estimation means corresponds to a part of the overlapping area of the visual information or a part of a vicinity area of the visual information including the overlapping area. A visual information presentation device for a vehicle, wherein a luminance distribution state of a landscape is estimated, and the image control unit controls the luminance of visual information based on the luminance distribution state estimated by a background luminance estimation unit.   3. The visual information presentation device for a vehicle according to claim 2, wherein the image control means changes the luminance of the visual information continuously or stepwise after holding the luminance of the visual information constant for a predetermined time. A visual information presentation device for a vehicle, which is characterized.   4. The visual information presenting device for a vehicle according to claim 1, wherein the image control means drives a landscape area in which a luminance distribution state is estimated by the background luminance estimation means. A visual information presentation device for a vehicle, wherein the visual information is displayed in association with a region in a display region of visual information viewed from a general viewpoint position of the person.   5. The visual information presentation device for a vehicle according to claim 4, further comprising eyeball position estimating means for estimating the driver's eyeball position, wherein the image control means calculates the eyeball position estimated by the eyeball position estimating means. The visual information visually recognized by the driver is estimated to be superimposed on the landscape, the background luminance estimation means estimates a luminance distribution of a part of the area estimated by the image control means, and the image The vehicle visual information presentation device, wherein the control means controls the luminance of the visual information based on the luminance distribution estimated by the background luminance estimating means.   The visual information presentation device for a vehicle according to any one of claims 1 to 5, wherein the image control unit fixes a display position of visual information according to a content of the visual information or the A visual information presentation device for a vehicle characterized by following or moving in a landscape.   The visual information presentation device for a vehicle according to any one of claims 1 to 6, wherein the background luminance estimation means estimates the presence of a specific object from the estimated luminance, and specifies When there is a target object, the image control means displays visual information at a position superimposed on at least a part of the specific object estimated by the background luminance estimation means or a position not superimposed on the specific object. A visual information presentation device for a vehicle characterized by the above.   8. The visual information presenting device for a vehicle according to claim 1, wherein the background luminance estimation means has at least a small change in background luminance of the landscape based on the estimated luminance. Dividing into two or more areas, the image control means displays visual information superimposed on the area divided by the background luminance estimation means.   9. The visual information presentation device for a vehicle according to claim 1, wherein the background luminance estimation unit estimates road terrain information in the landscape based on the estimated luminance. The vehicle is characterized in that the landscape is divided into at least two or more regions based on the estimated road terrain information, and the image control means displays visual information superimposed on the areas divided by the background luminance estimating means. Visual information presentation device.   10. The vehicular visual information presentation device according to claim 8 or 9, wherein the image control means displays visual information for each area according to a luminance distribution state for each area divided by the background luminance estimation means. A visual information presenting apparatus for a vehicle, wherein a priority order to be set is set, and an area for displaying visual information is variably controlled according to the set priority order.   The visual information presentation device for a vehicle according to any one of claims 8 to 10, further comprising own vehicle running state estimating means for estimating a running state of the own vehicle, wherein the image control means includes: A visual information presentation device for a vehicle, characterized in that the visual information is displayed in an optimum region in accordance with the running state estimated by the own vehicle running state estimation means.   The visual information presentation device for a vehicle according to any one of claims 8 to 11, further comprising a vehicle environment estimation unit that estimates an environmental state outside the host vehicle, wherein the image control unit includes the image control unit, A visual information presentation device for a vehicle, characterized in that the visual information is displayed in an optimum region in accordance with an environmental state estimated by an outside environment estimation means.   13. The visual information presentation device for a vehicle according to claim 8, wherein the image control means holds the visual information in the current display area for a predetermined time or displays the current visual information. A visual information presentation device for a vehicle, wherein a visual information display region is moved by continuously moving from the region to a region to be displayed next.   14. The visual information presenting device for a vehicle according to claim 8, wherein the image control unit is configured to visually recognize the visual information corresponding to a region for displaying the visual information. A vehicular visual information presentation device that controls an area so that information is displayed in an optimal area. A visual information presentation method for a vehicle that supports a driver's vehicle driving operation by superimposing visual information on a landscape in a driver's front vision,
Estimating the brightness of a plurality of points in the landscape;
Controlling the brightness of the visual information based on the estimated brightness of the plurality of points.

Images