JP2008162550A - External environment display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an external environment display device capable of making a driver grasp an external environment to be recognized in an early time even in the case wherein eyes of a driver are not directed to an appropriate direction. <P>SOLUTION: This external environment display device 1 comprises an eyes detecting means for detecting eyes a driver, an image pick-up means for picking up images of external environment of the vehicle, a determining means for determining whether the eyes detected by the eyes detecting means exist within a predetermined range or not, and a display position control means for displaying images of the external environment projected to the eyes of the driver on the supposition that the eyes of the driver exist within the predetermined range in a direction corresponding to the eyes of the driver by using the images picked up by the image pick-up means when the determining means determines that the eyes of the driver do not exists within the predetermined range. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an external environment display device that displays an image of an external environment at an appropriate position to assist a driver's vision.

Conventionally, a gaze direction detection unit that detects the gaze direction of the driver is further provided, and is detected by the gaze direction detection unit when the gaze direction of the driver detected by the gaze direction detection unit is the gaze direction when looking aside. There is known a vehicle display device that displays guidance information such as an arrow for guiding a vehicle occupant's line of sight outside the display area of the display means on a display means having a display area in the vicinity of the sight line direction (for example, Patent Documents). 1).
JP-A-2005-59660

  However, in the invention described in Patent Document 1, the driver does not consider the time lag until the driver notices an arrow as guidance information and changes the line of sight according to the direction of the arrow. That is, in the invention described in Patent Document 1, the external environment to be recognized by the driver after the guidance information is displayed until the driver notices the arrow and changes the line of sight according to the direction of the arrow. There is a problem that it cannot be grasped.

  Therefore, an object of the present invention is to provide an external environment display device that allows a driver to quickly recognize an external environment to be recognized even when the driver's line-of-sight direction is an inappropriate direction.

In order to achieve the above object, an external environment display device according to a first aspect of the present invention includes gaze direction detection means for detecting the gaze direction of a driver,
Imaging means for imaging the external environment of the vehicle;
Determining means for determining whether or not the line-of-sight direction detected by the line-of-sight direction detecting means is within a predetermined range;
When it is determined by the determination means that the driver's line-of-sight direction is not within a predetermined range, driving is performed when it is assumed that the driver's line-of-sight direction is within the predetermined range using an image captured by the imaging unit. Display position control means for displaying an image of the external environment seen by the driver's eyes in a direction corresponding to the driver's line-of-sight direction.

A second invention is the external environment display device according to the first invention,
The display position control means displays the image of the external environment ahead of the driver's line-of-sight direction.

A second invention is the external environment display device according to the first invention,
The display position control means displays the image of the external environment in the vicinity of the driver's line of sight in the direction of the predetermined range.

  ADVANTAGE OF THE INVENTION According to this invention, even when a driver | operator's gaze direction is an unsuitable direction, the external environment display apparatus which can make a driver | operator grasp | ascertain the external environment which should be recognized early is obtained.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing a main configuration of an external environment display device 1 according to a first embodiment of the present invention. FIG. 2 is a block diagram showing the main configuration of the display control apparatus 10A.

  The external environment display device 1 is configured around a display control device 10A. The display control apparatus 10A has a hardware configuration mainly composed of an appropriate processor or microcomputer, a CPU that performs various processes described below, and a program and data used to perform various processes described below. Includes a ROM in which is stored, a readable / writable RAM for storing operation results, a timer, a counter, an input interface, an output interface, and the like.

  The display control apparatus 10A includes cameras 30, 31, and 32 that image the external environment. Each camera 30, 31 and 32 is arranged in an appropriate line-of-sight direction (optical axis direction) at an appropriate position such as a rearview mirror, and is provided by an image sensor such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). , Image the external environment. The camera 30 images the external environment on the left side of the vehicle. The camera 31 images the external environment in front of the vehicle. The camera 32 images the external environment on the right side of the vehicle. The cameras 30, 31, and 32 may acquire an external environment image in real time while the vehicle is traveling and supply the images to the display control apparatus 10 </ b> A in a stream format with a predetermined frame period. The cameras 30, 31, and 32 may be replaced with one wide-angle camera having a fisheye lens, or may be replaced with one camera whose optical axis direction is variable. Moreover, you may have a 4th camera which images the external environment behind a vehicle.

  A driver monitor ECU 42 is connected to the display control apparatus 10A. A driver monitor camera 40 is connected to the driver monitor ECU 42. The driver monitor camera 40 includes, for example, a color or infrared sensitive CCD sensor array. The driver monitor camera 40 is provided at an appropriate location of the vehicle so that the front surface of the driver (for example, the driver's face from the front) can be captured. For example, the driver monitor camera 40 is installed on a dashboard of a vehicle instrument panel, a steering column, a room mirror, or the like. The driver monitor camera 40 obtains an image of the driver's face (hereinafter referred to as “face image”) in real time while the vehicle is running, and is typically in a stream format of 30 fps (frame per second). The ECU 42 may be supplied.

  The driver monitor ECU 42 recognizes the direction of the driver's face by image processing on the face image acquired by the driver monitor camera 40, and information on the direction of the driver's line of sight based on the recognized face direction (hereinafter, referred to as the driver's face direction). Also referred to as “line-of-sight information”). The driver monitor ECU 42 generates line-of-sight information in real time while the vehicle is traveling, and supplies the generated line-of-sight information to the display control apparatus 10A.

  For example, the driver monitor ECU 42 first extracts representative points (feature points) of the face from the face image acquired by the driver monitor camera 40. This feature extraction method may be any appropriate method, and for example, a technique based on Active Appearance Model (AAM) may be used. An appropriate edge detection algorithm (eg, Sobel's edge detection algorithm) is then applied to extract the boundary between the face and facial features. Next, using the edges and feature points extracted as described above, the facial parts are separated as shown in FIG. 3, and the shape of the facial parts is extracted. FIG. 3 shows only a part of the face as a representative, and shows an image in which the eyebrows and eyes are extracted as parts. The driver monitor ECU 42 matches the position or orientation of the extracted facial parts with each of the pre-stored postures (for example, the posture when the display device 21 is watched, the posture when looking right and left, right and left, etc.). The current posture (face orientation) is detected by comparing the degree of matching with the position or orientation of the part. The orientation of the face may be represented by, for example, a rotation angle about three axes with the front direction of the face in a normal posture as one axis. The driver monitor ECU 42 generates line-of-sight information by determining the driver's line-of-sight direction under the assumption that the driver's face direction corresponds to the driver's line-of-sight direction (line-of-sight direction). Note that the gaze method detection method by the image recognition process in the driver monitor ECU 42 may be various, and the present invention is not limited to the above-described one. For example, the driver monitor ECU 42 extracts the outline of the eyeball as a face part, and based on the position of the eyeball within the eye outline and the driver's face orientation recognized as described above, the driver's line-of-sight direction May be detected.

  A display device group 20 is connected to the display control device 10A. The display device group 20 includes four display devices 21 to 26 as an example. The display device 21 is a display of a navigation device arranged on the instrument panel or the like. The display devices 22 and 23 are provided on the left and right side mirrors, the display device 24 is provided on the rearview mirror, and the display devices 25 and 26 are provided on the left and right windows (side glasses), respectively. The display devices 22 to 26 may project a video (image) onto a corresponding mirror or glass in the form of a head-up display (HUD), or may be provided on the corresponding mirror or glass. In addition, it may be constituted by a display body of a type having optical transparency when no image is displayed, or may be constituted by a transmissive liquid crystal display panel provided on a corresponding mirror or glass. The display device group 20 may include a fifth display device such as a display device provided in a meter.

  As shown in FIG. 2, the display control device 10 </ b> A includes, as function blocks that realize its main functions, a warning direction determination unit 12 </ b> A, a gaze direction evaluation unit 14 </ b> A, a display position determination unit 16 </ b> A, a display image generation unit 18 </ b> A, and The display control unit 19A is provided. The function of each part of the display control apparatus 10A will be described with reference to FIG.

  FIG. 4 is a flowchart showing a flow of main processing realized in the display control apparatus 10A according to the present embodiment. The process shown in FIG. 4 may be repeatedly executed while the vehicle is running until the ignition switch is turned off after the ignition switch is turned on.

  In step 100, the alerting direction determination unit 12A determines whether or not a situation that should alert the driver has been detected based on the external environment images obtained from the cameras 30, 31, and 32. The situation that should alert the driver is, for example, a situation where there is a pedestrian trying to cross the front of the vehicle, a situation where the other vehicle in front of the vehicle suddenly decelerates, and the distance between the vehicles suddenly decreases. This type of situation may be detected based on the monitoring result of a radar or the like instead of or in addition to each external environment image obtained from the cameras 30, 31, and 32. If a situation that should alert the driver is detected, the process proceeds to step 102; otherwise, the processing routine of the current cycle ends.

  In step 102, the alerting direction determination unit 12A determines a direction to alert the driver based on each external environment image obtained from the cameras 30, 31, and 32. That is, the alerting direction determination unit 12A detects and determines the direction in which an object that should alert the driver exists. For example, in a situation where there is a pedestrian who tries to cross the front of the vehicle, the attention direction determination unit 12A detects and determines the direction in which the pedestrian exists as the attention direction, and other vehicles in front of the vehicle suddenly decelerate, etc. In a situation where the inter-vehicle distance rapidly decreases, the direction in which the other vehicle exists is detected and determined as the alerting direction.

  In step 104, the line-of-sight direction evaluation unit 14 </ b> A is a desirable range as the driver's line-of-sight direction (hereinafter referred to as “appropriate range”) based on the attention direction determined by the attention direction determination unit 12 </ b> A in step 102. ) Is set. The appropriate range of the line-of-sight direction may be set as a predetermined range centered on the alerting direction, and when the driver's line-of-sight direction is within that range, the external environment (external object) that the driver should be aware of It is set to a range that can be recognized.

  In step 106, the line-of-sight direction evaluation unit 14A obtains line-of-sight information from the driver monitor ECU 42 and grasps the current line-of-sight direction of the driver.

  In step 108, the line-of-sight direction evaluation unit 14A determines whether or not the current driver's line-of-sight direction obtained in step 106 is within the appropriate range set in step 104 above. If the driver's line-of-sight direction is within the appropriate range, it is not necessary to alert the driver in particular, so the processing routine for the current cycle ends as it is. On the other hand, if the driver's line-of-sight direction is not within the appropriate range, the process proceeds to step 110.

  In step 110, the display position determination unit 16 </ b> A is based on the current driver's line-of-sight direction obtained in step 106, and is one display device ahead of the driver's line-of-sight direction from the display device group 20. Are selected as output target display devices. Here, the display device ahead of the driver's line-of-sight direction is typically a display device in a direction that matches the driver's line-of-sight direction. In this case, the driver's line-of-sight direction may consider only the direction in the horizontal plane (see FIG. 6A), or may consider the direction in the vertical plane as an auxiliary (see FIG. B) Reference). Alternatively, the previous display device in the driver's line-of-sight direction may include a display device in a direction that can be seen even when the driver maintains the current driver's line-of-sight direction. In any case, if there is no display device in the direction that matches the driver's line-of-sight direction, one of the display devices 20 that is present in the direction closest to the driver's line-of-sight direction. The display device may be selected as a display device to be output. In this step 110, for example, when the left side mirror exists ahead of the driver's line of sight, the display position determination unit 16A uses the display device 22 provided in the left side mirror as the output target display device. Choose as.

  In step 120, the display image generation unit 18A selects the external environment image in the alerting direction determined in step 102 from the external environment images obtained from the cameras 30, 31, and 32, and displays the display image. Generate. That is, a display image that will appear to the driver's eyes when the driver sees the alerting direction is generated. For example, when the alerting direction is the front direction, the display image generation unit 18A generates a display image based on the external environment image captured by the camera 31. At this time, the external environment image from the camera 31 may be used as a display image as it is, or a display image may be generated by appropriately correcting the external environment image from the camera 31. In the latter case, for example, a display image may be generated by correcting the external environment image from the camera 31 like a landscape that can be seen by the driver's eyes, or a partial image including an object to be alerted (for example, The image around the pedestrian, the image near the traffic light, the image near the brake lamp of the other vehicle ahead) may be cut out to generate the display image. In addition, a display image may be generated by superimposing an emphasis effect or color on the external environment image from the camera 31. Further, a display image may be generated by superimposing a message such as “pedestrian attention” on the external environment image from the camera 31. Furthermore, a display image may be generated by superimposing an arrow graphic display or a message indicating a direction in which an object to be alerted exists on the external environment image from the camera 31. In any case, the display image is generated as an image that allows the driver to immediately grasp the object to be alerted, and is generated based on each external environment image obtained from the cameras 30, 31, and / or 32.

  In step 122, the display control unit 19 </ b> A selects the display image generated by the display image generation unit 18 </ b> A in step 120 above from the display device selected in the above step 110 (selection from among the display devices 21 to 26). Any one of them). Thereby, the driver reacts to the display image being output to the display device ahead of the line-of-sight direction and looks at the display image before changing the line-of-sight direction to the alerting direction. It is possible to grasp the situation that exists in the alert direction.

  In subsequent step 130, the line-of-sight direction evaluation unit 14A monitors the driver's line-of-sight direction after outputting the display image in step 122, and determines whether or not the driver's line-of-sight direction has changed within the appropriate range. To do. If the driver's line-of-sight direction has changed within the appropriate range, the process proceeds to step 140; otherwise, the process returns to step 110. In this case, the processing of steps 110 to 122 is repeated, and the display image generated based on the latest external environment image is output to the display device selected according to the latest driver's line-of-sight direction. .

  In step 140, the display control unit 19A stops outputting the display image on the display device executed in step 122 described above. Therefore, the display image generated based on the external environment image is continuously output on the predetermined display device until the driver's line-of-sight direction changes within the appropriate range. The display control unit 19A also displays the display image on the display device executed in step 122 even when the situation detected in step 100 is no longer detected due to an environmental change associated with traveling of the vehicle. Stop output.

  According to the present embodiment described above, even when the driver's line-of-sight direction is not directed to the environment to which attention should be paid, the image obtained by capturing the environment to be noted on the display device ahead of the driver's line-of-sight direction. Therefore, the driver should be careful by looking at the output display image before changing the direction of the line of sight to the direction in which the environment to be noted exists (that is, the alerting direction). It is possible to grasp the environment. That is, the driver can grasp the environment to be noted without time lag.

  In this embodiment, various modifications and additional configurations are possible.

  For example, in the above-described embodiment, the camera 30 that monitors the side of the vehicle in consideration of the fact that an environment to be watched also exists on the side of the vehicle in preparation for an encounter collision at an intersection, for example. 32 is provided, but only the camera 31 for monitoring the front of the vehicle may be set simply. In this case, a desirable range (= appropriate range) in the driver's line-of-sight direction may be a fixed range set around the front direction. That is, the line-of-sight direction evaluation unit 14A detects whether or not the driver's face is facing the front. And when it is not facing the front (that is, when looking aside), the display control unit 19A displays the display device in the direction of the driver's line of sight similarly selected by the display position determination unit 16A. The display image similarly generated by the display image generation unit 18A may be output.

  In addition, in the above-described embodiment, when the driver's line-of-sight direction is not within the appropriate range, various warnings are additionally issued (alarms are output earlier than usual), or collision prediction is performed. Safety may be ensured from other viewpoints by setting a threshold value for execution of control or brake assist control to be lower, or by increasing the brake assist amount than usual. In addition, when the driver's line-of-sight direction is not within the appropriate range, the necessary information can be easily provided by additionally increasing the volume of the voice guidance of the navigation device or changing the volume of the audio. May be.

  In the above-described embodiment, when the driver's line-of-sight direction is not within the appropriate range, a deceleration control for automatically decelerating the vehicle may be additionally executed. In this case, for example, for the purpose of avoiding a rear-end collision from the rear, if the following vehicle is detected and it can be determined that it is necessary according to the relative distance, relative speed difference or relative speed difference between the following vehicle and the host vehicle, stop The lamp may be turned on. In this case, the deceleration may be canceled when the driver's line-of-sight direction returns to the appropriate range. If the driver's line-of-sight direction does not return within the appropriate range, a dangerous state (sleeping or sudden illness) may occur. Etc.), and finally, the vehicle is decelerated to a stop (and automatic steering control is executed to a safe road shoulder if necessary), and the hazard lamp may be blinked.

  The second embodiment is mainly different from the first embodiment described above in the method for determining the display position of the display image. Other configurations may be the same as those of the first embodiment described above, and the configuration peculiar to the second embodiment will be mainly described below.

  FIG. 5 is a flowchart showing the flow of main processing realized in the display control apparatus 10B according to the present embodiment. Regarding the description of FIG. 5, the same processes as those in FIG. 4 according to the first embodiment are denoted by the same reference numerals and the description thereof is omitted. The process shown in FIG. 5 may be repeatedly executed while the vehicle is running after the ignition switch is turned on until the ignition switch is turned off.

  In step 112, the display position determination unit 16B determines, based on the current driver's line-of-sight direction obtained in step 106, an appropriate range side in the vicinity of the current driver's line-of-sight direction from the display device group 20. Is selected as the output target display device. For example, as shown in FIG. 6A, it is assumed that the appropriate range of the driver's line-of-sight direction is the diagonally forward right direction, but the driver's line-of-sight direction is the direction toward the passenger side dashboard. . In this case, a display device 22 provided in the left side mirror, a display device 21 as a display of a navigation device arranged in an instrument panel, and a display device 24 provided in the room mirror exist in the vicinity of the line-of-sight direction. Although the display device 22 is closer to the driver's line-of-sight direction than the other display devices 21 and 24, the display position determining unit 16B exists on the appropriate range side of the display devices 21, 22, and 24. The display devices 21 and 24 to be selected are selected as display devices to be output. Note that, as in the example shown in FIG. 6A, when there is substantially no difference in the directions of the display devices 21 and 24 when viewed in the horizontal plane, the display position determination unit 16B displays the display devices 21 and 24. Both may be selected. Alternatively, one of the display devices 21 and 24 may be selected in consideration of the driver's line-of-sight direction with respect to the horizontal plane. For example, as in the example illustrated in FIG. 6B, the display device 21 closer to the driver's line-of-sight direction as viewed in the vertical plane may be selected as the output target display device.

  In step 122, the display control unit 19B outputs the display image generated by the display image generation unit 18B in step 120 to the display device selected in step 110. Thereby, the driver reacts to the display image being output to the display device ahead of the line-of-sight direction and looks at the display image before changing the line-of-sight direction to the alerting direction. It is possible to grasp the situation that exists in the alert direction.

  According to the present embodiment described above, even when the driver's line-of-sight direction is not directed to the environment to be watched, an image of the environment to be watched is output to the display device in the vicinity of the driver's line-of-sight direction. Therefore, the driver changes the line-of-sight direction by looking at the output display image before changing the line-of-sight direction to the direction in which the environment to be noted exists (that is, the direction to call attention). I can grasp it.

  In particular, according to the present embodiment, as described above, an image that captures the environment to which attention should be paid is output to the display device in the direction of the aptitude range in the vicinity of the driver's line-of-sight direction. Compared to a comparative configuration in which an image that captures an environment to be watched is output to a display device (display device 22 in the example of FIG. 6A) in the direction opposite to the aptitude range in the vicinity, In a short time after viewing the output display image, the direction of the line of sight can be changed to a direction in which the environment to be watched can be actually visually observed (that is, within an appropriate range). In other words, the driver changes the line-of-sight direction slightly when viewing the output display image, but is careful to change the line-of-sight direction continuously in the same direction as the change direction. The line-of-sight direction can be changed in a direction in which the power environment can be actually seen (that is, within the appropriate range). In other words, the driver can change the line-of-sight direction from the line-of-sight direction in which the displayed image is viewed to the line-of-sight direction in which the environment to be watched can be actually visually observed in a natural flow. Thereby, a driver | operator's gaze direction can be guide | induced effectively to the gaze direction which can actually visually observe the environment which should be careful.

  In the present embodiment, various modifications can be considered in addition to the additional configuration described above in connection with the first embodiment.

  For example, when selecting a display device to be output, driver characteristics may be taken into consideration. This is because, as described above, when selecting a display device in a direction shifted from the driver's line-of-sight direction to the appropriate range side, if the shift amount is large, some drivers may not notice. The characteristics of the driver depend on the response at the time of past display image output (for example, presence or absence of movement to an appropriate range of the line-of-sight direction, time from display image output to movement, change of line-of-sight direction at the time of display image output, etc.) Data that is determined based on the characteristics of the driver and may be accumulated for each driver in a driver database, for example. In such a modification, for example, in the example illustrated in FIG. 6, when the display device 21 is selected and the reaction is delayed due to the characteristics of the driver, the display position determination unit 16B causes the display device 24 to be the output target. It is good also as selecting as a display apparatus.

  The third embodiment is mainly different from the first embodiment described above in that the display position of the display image is gradually changed in a direction in which the driver's line-of-sight direction is guided to an appropriate range. Other configurations may be the same as those of the first embodiment described above, and the configuration unique to the third embodiment will be mainly described below.

  In the third embodiment, the display device group 20 </ b> C is a display provided on a windshield glass (front glass) instead of the above-described display devices 21 to 26, or in addition to all or any of them. Device 27 is included. As schematically shown in FIG. 7, the display device 27 has a plurality of display areas 27 a to 27 h on the windshield glass along the vehicle width direction. In the display areas 27a to 27h, an image is not normally displayed and remains transparent. The display of images in the display areas 27a to 27h may be realized by, for example, a plurality of head-up display devices provided corresponding to the display areas 27a to 27h, or the display areas 27a to 27h may be respectively displayed. You may implement | achieve by comprising with a transmissive liquid crystal.

  FIG. 8 is a flowchart showing the flow of main processing realized in the display control apparatus 10C according to the present embodiment. In addition, regarding the description of FIG. 8, the same processes as those of FIG. 4 according to the first embodiment described above are denoted by the same reference numerals, and the description thereof is omitted. The process shown in FIG. 8 may be repeatedly executed while the vehicle is running after the ignition switch is turned on until the ignition switch is turned off.

  In step 116, the display position determination unit 16C is based on the current driver's line-of-sight direction obtained in step 106, and is near the current driver's line-of-sight direction among the display areas 27a to 27h of the display device 27. The display area in the direction of the appropriate range is selected as the display area to be output. For example, in the example illustrated in FIG. 9, the alerting direction corresponds to the direction of the region 27 g of the display device 27, and accordingly, the appropriate range corresponds to the directions of the regions 27 f to 27 h of the display device 27, while The direction is the left side mirror direction that deviates greatly from the appropriate range. In this case, the display position determination unit 16C selects the display area 27a in the direction of the appropriate range near the current driver's line-of-sight direction as the display area to be output.

  In step 123, the display control unit 19C outputs the display image generated by the display image generation unit 18C in step 120 to the display area of the display device 27 selected in step 110. Thus, the driver reacts to the display image being output to the display area of the display device 27 in the vicinity of the line-of-sight direction, and changes the line-of-sight direction to the alerting direction by viewing the display image. Before, it is possible to grasp the situation that exists in the alert direction.

  In step 130, the line-of-sight direction evaluation unit 14C monitors the driver's line-of-sight direction after outputting the display image in step 123, and determines whether or not the driver's line-of-sight direction has changed within the appropriate range. . If the driver's line-of-sight direction changes within the appropriate range, the process proceeds to step 140; otherwise, the process proceeds to step 132.

  In step 132, the line-of-sight direction evaluation unit 14C monitors the driver's line-of-sight direction after outputting the display image in step 123, and the driver's line-of-sight direction is displayed on the display device 27 on which the display image is output. It is determined whether or not it has changed in the direction of the area. If the driver's line-of-sight direction has changed to the direction of the display area of the display device 27 from which the display image has been output, the process proceeds to step 134; otherwise, the process returns to step 116. In the case of returning to step 116, in the subsequent steps 120 and 123, the display image generated based on the latest external environment image is output to the region of the display device selected according to the latest driver's line-of-sight direction. Will be.

  In step 134, the display position determination unit 16 </ b> C selects a display area in a direction closer to the appropriate range than the display area of the current display device 27 as a display area to be output. For example, in the example shown in FIG. 9, when the display area of the current display device 27 is the display area 27a, the display area 27b in the direction of the appropriate range near the display area 27a is selected as the display area to be output. . In this way, the display area of the display device 27 to be output is gradually changed toward the appropriate range until the driver's line-of-sight direction changes within the appropriate range (until affirmative determination in step 130). That is, in the example shown in FIG. 9, the display area of the display device 27 to be output is gradually changed in the order of the display areas 27a to 27b, 27c, 27d, and 27e.

  According to the present embodiment described above, even when the driver's line-of-sight direction is not directed to the environment where attention should be paid, the environment where attention should be paid is captured in the display area of the display device 27 in the vicinity of the driver's line-of-sight direction. The driver can view the displayed image before changing the direction of the line of sight to the direction in which the environment to be noted exists (that is, the direction of alerting). The environment that should be understood.

  In addition, according to the present embodiment, as described above, an image that captures the environment to which attention should be paid is output to the display area of the display device 27 in the direction of the aptitude range in the vicinity of the driver's line of sight. Compared with a comparative configuration in which an image of the environment to be watched is output to the display area of the display device 27 in the direction opposite to the aptitude range in the vicinity of the visual line direction of the driver, the driver has output the output. In a short time after viewing the display image, the direction of the line of sight can be changed to a direction in which the environment to be watched can be actually visually observed (that is, within the appropriate range).

  In particular, according to the present embodiment, as described above, the display area of the display device 27 where the display image is output is gradually changed in the direction toward the aptitude range, so that the driver's line-of-sight direction is within the aptitude range. Can be effectively induced.

  In the present embodiment, various modifications can be considered in addition to the additional configuration described above in connection with the first embodiment.

  For example, in the present embodiment, the display area of the display device 27 that is the first display target is the display area of the display device 27 that is in the direction of the aptitude range near the driver's line-of-sight direction according to the determination method of the second embodiment described above. However, according to the determination method of the first embodiment, the display area (or other display device) of the display device 27 ahead of the driver's line of sight may be set. For example, in the example illustrated in FIG. 9, the first display target display device may be set to the display device 22 (display device provided in the left side mirror) that is ahead of the driver's line of sight. In this case, the display device of the output target and the display area of the display device 27 are gradually changed from the display device 22 in the order of the display areas 27a, 27b, 27c, 27d, and 27e of the display device 27.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

  For example, in the above-described embodiment, the display image is displayed on a display device arranged at a predetermined position, but may be output by forming an image at an arbitrary spatial position using, for example, holography technology. . In this case, when the display image is formed ahead of the driver's line of sight, the driver's visibility is taken into consideration (that is, the focal position can be changed to a minimum), and the driver is to some extent. The display image may be combined at a position far from the eye position.

It is a figure which shows the main structures of the external environment display apparatus 1 by Example 1. FIG. It is a block diagram which shows the main structures of display control apparatus 10A. It is a figure which shows the image from which the eyebrows and eye part of the face were extracted as parts. 6 is a flowchart illustrating a flow of main processing realized in the display control apparatus 10A according to the first embodiment. It is a flowchart which shows the flow of the main processes implement | achieved by the display control apparatus 10B by Example 2. FIG. FIG. 6 (A) is a diagram schematically showing the relationship between the driver's line-of-sight direction and the appropriate range in the horizontal plane, and FIG. 6 (B) shows the driver's line-of-sight direction and the appropriate range in the vertical plane. FIG. It is a figure which shows roughly the some display area 27a-27h provided on a windshield glass. It is a flowchart which shows the flow of the main processes implement | achieved by 10 C of display control apparatuses by Example 3. FIG. It is a figure which shows roughly the relationship between the driver | operator's eyes | visual_axis direction in a horizontal surface, an appropriate range, and display area 27a-27h.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 External environment display apparatus 10A Display control apparatus 12A Attention direction determination part 14A Gaze direction evaluation part 16A Display position determination part 18A Display image generation part 19A Display control part 20 Display apparatus group 22 thru | or 27 Display apparatus 30, 31, 32 Camera 40 Driver monitor camera 42 Driver monitor ECU

Claims (3)

Gaze direction detection means for detecting the driver's gaze direction;
Imaging means for imaging the external environment of the vehicle;
Determining means for determining whether or not the line-of-sight direction detected by the line-of-sight direction detecting means is within a predetermined range;
When it is determined by the determination means that the driver's line-of-sight direction is not within a predetermined range, driving is performed when it is assumed that the driver's line-of-sight direction is within the predetermined range using an image captured by the imaging unit. An external environment display device comprising: display position control means for displaying an image of the external environment that is visible to the driver's eyes in a direction corresponding to the driver's line-of-sight direction.   The external environment display device according to claim 1, wherein the display position control unit displays an image of the external environment ahead of a driver's line of sight.   The external environment display device according to claim 1, wherein the display position control unit displays an image of the external environment in a direction in the predetermined range side in the vicinity of the driver's line of sight.

Images