JP2015077876A - Head-up display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly control display range of an AR image that is displayed on a head-up display of a vehicle.SOLUTION: A head-up display device detects a line of site direction of a driver and a speed of an own vehicle, and according to the detected line of site direction and car speed, decides a display target range in front of the vehicle for displaying an AR (augmented reality) image. Among targets (100-107) present in front of the vehicle, the information related to the targets (100-102) contained in the decided display target range is specified. On a display where an image is projected by superposing on a front driving view of a driver of a vehicle, the AR images (200-202) based on the specified information are displayed at a position corresponding to a real scene of the corresponding targets (100-102).

Description

  The present invention relates to a head-up display device for a vehicle that superimposes and displays an image on a real scene in the driving field of view of the driver of the vehicle.

  Conventionally, as a form of providing information such as facilities to the driver of the vehicle, information on the facility existing in the driver's driving field of view is displayed on the head-up display provided near the front window of the vehicle and the actual scenery of the facility A technique for displaying images so as to overlap each other is known (for example, see Patent Document 1).

JP 2005-208158 A

  By adding an augmented reality image (hereinafter also referred to as an AR image) to a real landscape by a technology applying this type of augmented reality (AR), it is possible to provide information effective for the driver. On the other hand, if the AR image is displayed excessively, driving may be hindered or the driver may feel bothered. Therefore, a device for preventing the AR image from being excessively displayed is required.

  With regard to this point, for example, the location where the driver can easily recognize the AR image in the driving field of view differs depending on the direction of the driver's line of sight and the speed of the scenery flowing through the driving field of view due to the speed of the vehicle. . For example, it is easy to recognize an object in the direction in which the driver is looking, or when the vehicle speed is high, it is easy to recognize an object farther away. AR information added to an object in a place that is easily recognized by the driver can be a useful information source for the driver. However, AR information added to an object that is out of a place where it is easy for the driver to recognize may cause troubles such as inconvenience and a delay in judgment. Under such circumstances, it is considered effective to suppress excessive display by appropriately limiting the range in which the AR image is displayed according to the situation of the driver and the vehicle.

  The present invention has been made to solve the above problems, and an object thereof is to provide a technique for appropriately controlling the display range of an AR image displayed on a head-up display of a vehicle.

  In order to achieve the above object, the present invention is directed to a head-up display device that projects an image onto a windshield or combiner disposed in front of a vehicle driver and superimposes the image on the driver's driving field of view. In this case, a line-of-sight detection unit, a vehicle speed detection unit, a determination unit, a specifying unit, a display unit, and a control unit are provided.

  The line-of-sight detection means detects the direction of the driver's line of sight. The vehicle speed detection means detects the speed of the vehicle. The determining unit determines a display target range in front of the vehicle according to the line-of-sight direction detected by the line-of-sight detecting unit and the speed detected by the vehicle speed detecting unit. The specifying unit specifies image information representing information on the specific target included in the display target range determined by the determining unit among the specific target existing in front of the vehicle. The display means displays the image information specified by the specifying means at a position associated with the actual scenery of the specific object visually recognized in the driving field of view ahead of the driver. The control means controls the display means.

  According to the present invention, a display target range for displaying an AR image representing information on a specific object is determined according to the detected driver's line-of-sight direction and vehicle speed, and the display target range is included in the determined display target range. An AR image can be added only to a specific object. In this way, the display target range can be controlled so that only the AR image that is easy for the driver to recognize is displayed according to the driver's line-of-sight direction and vehicle speed, so the AR image is displayed unconditionally. Excessive display can be suppressed as compared with the case of doing so.

The block diagram which shows schematic structure of a head-up display system. The flowchart which shows the procedure of AR image display processing. Explanatory drawing which shows the example of a display of AR image. Explanatory drawing which shows the example of a display of AR image. Explanatory drawing which shows the reference example when not restrict | limiting the display object range of AR image.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In addition, this invention is not limited to the following embodiment, It is possible to implement in various aspects.
[Description of head-up display system configuration]
The head-up display system according to the embodiment includes a windshield (front window) or a combiner (half mirror) 32 that is mounted on a vehicle and displays information related to an object such as a facility that exists in front of the vehicle. It is projected on the screen and displayed superimposed on the actual scenery in the driving field of view visually recognized by the driver. As shown in FIG. 1, the head-up display system of the present embodiment includes a HUD drawing ECU 10, a state detection unit 12 connected to the HUD drawing ECU 10, an AR information database 22, a navigation device 24, an external information acquisition unit 26, an operation An input unit 28 and a HUD 30 display unit are provided.

  The state detection unit 12 is a group of various sensors and control devices for inputting various information related to the state of the vehicle or the driver to the HUD drawing ECU 10. The state detection unit 12 includes a front detection unit 14, a collision prevention control unit 16, a vehicle speed detection unit 18, a driver state detection unit 20, and a viewpoint time determination unit 21. The front detection unit 14 is a sensor that detects obstacles such as other vehicles and pedestrians existing on the traveling road ahead of the host vehicle. For example, a camera, a millimeter wave radar, a laser radar, a radio for inter-vehicle communication, etc. Embodied. These cameras, radars, radios, etc. may be used alone or in combination. When the collision prevention control unit 16 determines that there is a possibility of colliding with an obstacle (vehicle, pedestrian) on the traveling road based on the detection result by the front detection unit 14, vehicle control such as warning and brake assist is performed. It is a vehicle control device that performs automatically. The vehicle speed detector 18 is a sensor that detects the speed of the host vehicle. The driver state detection unit 20 is a device that detects a driver's state such as a face position, a face direction, and a line-of-sight direction from a driver's face image captured by an in-vehicle camera. The viewpoint time determination unit 21 is a device that measures the time during which the driver is gazing at a specific place based on the detection result of the driver's viewpoint direction.

  The AR information database 22 is a storage device that stores information for displaying an AR image to be presented to the driver. The AR information database 22 stores a large amount of data representing the name, attribute, position, information content to be provided, and the like of a specific object (for example, a facility) that is an AR image display target. Examples of the specific object that is the display target of the AR image include a store, a customer collection facility, a dining facility, a resting facility, a sight, an event venue, a registered facility (for example, a destination for route guidance), and the like.

  The navigation device 24 is a device that constitutes a known navigation system that performs route guidance based on the current location of the host vehicle and map data. The navigation device 24 identifies the current location and traveling direction of the host vehicle on the road based on a positioning result obtained by a GPS (Global Positioning System) receiver or an autonomous navigation sensor and map matching using map data. The map data used by the navigation device 24 for route guidance includes road data indicating the connection status, position, shape, etc. of intersections and roads, facility data indicating the position, shape, name, attributes, etc. of facilities existing on the map, etc. It is included. The navigation device 24 provides the HUD drawing ECU 10 with the present location and traveling direction on the road of the host vehicle, road data, and facility data regarding the facility that is the display target of the AR image. Since other functions and configurations of the navigation device 24 are in accordance with known techniques, a detailed description thereof is omitted here.

  The external information acquisition unit 26 is for acquiring information related to a facility that is an AR image display target from an external device. The external information acquisition unit 26 performs, for example, a communication device for communication connection to the Internet, a communication device for communication with a mobile communication terminal such as a high-function mobile phone (smart phone) owned by the driver, or inter-vehicle communication. It is embodied in a communication device or the like for performing. The external information acquisition unit 26 accesses a web server on the Internet, a mobile communication terminal, or other vehicles existing around the host vehicle, acquires information related to various facilities that are the display target of the AR image, and performs HUD drawing. Provided to the ECU 10.

  The operation input unit 28 is an input device for inputting an operation instruction from the driver. The operation input unit 28 is, for example, a pointing device (so-called remote touch controller) capable of operating various on-vehicle devices at the driver's hand, a voice recognition device that inputs a command recognized from a voice input from the driver, It is embodied by an operation switch attached to the steering wheel or console. In the head-up display system of the present embodiment, the operation input unit 28 is used to switch on / off the display of the AR image. In addition, by using a pointing device or a voice recognition device that constitutes the operation input unit 28, an arbitrary AR image can be designated from the AR images displayed on the HUD display unit 30.

  The HUD display unit 30 is a display device that projects an image on a windshield or combiner 32 provided in front of the driver so that the image is superimposed on an actual scene in the driver's driving field of view. For example, the HUD display unit 30 projects an image by projecting light from an LED light source or a halogen lamp light source onto a transmissive liquid crystal display or a reflective display, and projects the image rendered on the display onto a windshield or combiner 32. The configuration is illustrated. Or the structure which reflects the light from the laser light source which forms an image with the mirror comprised by MEMS, and projects an image on the windshield or the combiner 32 may be sufficient. An image projected by the windshield or combiner 32 by these methods is recognized as a virtual image displayed far away from the driver.

  The HUD drawing ECU 10 is an electronic control unit (ECU) including a CPU, a ROM, a RAM, a GDC, an input / output port with an external device, and the like. The HUD drawing ECU 10 performs overall control of the entire system according to a stored program. The outline of the process in which the HUD drawing ECU 10 displays the AR image on the HUD display unit 30 is as follows.

  The HUD drawing ECU 10 acquires the current location, traveling direction, and map data of the host vehicle from the navigation device 24. In addition, information about the facility that is the display target of the AR image is acquired from the AR information database 22, the navigation device 24, and the external information acquisition unit 26. Note that the information regarding the facility that is the display target of the AR image includes heading information to be displayed at the first stage and detailed information to be displayed at the second and third stages. The HUD drawing ECU 10 specifies a visually recognizable facility that is a display target of the AR image existing in front of the vehicle based on the acquired various information. Then, the HUD drawing ECU 10 detects the facility in the vicinity of the position where the identified facility is actually visually recognized by the driver based on the distance and relative orientation between the host vehicle and the facility, the face position and the face direction of the driver, and the like. The display position of the AR image is calculated so that the AR image corresponding to is projected. The HUD drawing ECU 10 draws an AR image based on the display position obtained as a result of the calculation, and causes the HUD display unit 30 to display the drawn AR image.

  In this way, the AR image related to the facility existing in the driver's driving field of view can be displayed in association with the position of the actual scenery of the facility visually recognized by the driver. The HUD drawing ECU 10 determines a part of the display target range for displaying the AR image in accordance with the driver's line-of-sight direction and the vehicle speed from the entire actual scenery in front of the host vehicle, and determines the display target thus determined. Only AR images of facilities included in the range are displayed. Detailed procedures of this series of processing will be described later.

[Description of AR image display processing]
The procedure of the AR image display process executed by the HUD drawing ECU 10 will be described with reference to the flowchart of FIG. This process is a process that is executed on condition that the display of the AR image is designated ON by the operation input unit 28.

  In S <b> 100, the HUD drawing ECU 10 determines whether or not there is a possibility that the host vehicle collides with an obstacle based on the determination status of the collision risk by the collision prevention control unit 16. When it is determined that there is no possibility of a collision (S100: YES), the HUD drawing ECU 10 proceeds to S102. In S <b> 102, the HUD drawing ECU 10 acquires the detection result of the driver's current line-of-sight direction by the driver state detection unit 20. In S <b> 104, the HUD drawing ECU 10 acquires the detection result of the current speed of the vehicle by the vehicle speed detection unit 18.

  In S106, the HUD drawing ECU 10 determines a display target range, which is an AR image display target in front of the vehicle, based on the driver's line-of-sight direction acquired in S102 and the vehicle speed acquired in S104. . Specifically, the HUD drawing ECU 10 determines the horizontal range of the display target range according to the direction of the line of sight in the left-right direction toward the front in the traveling direction of the vehicle. For example, when the driver's line of sight is directed to the right in the direction of travel, the region on the right side of the front in the direction of travel is set as the lateral range of the display target range. On the other hand, when the driver's line of sight is directed to the left in the front direction of travel, the region on the left side of the front in the direction of travel is defined as the lateral range of the display target range.

  Further, the HUD drawing ECU 10 determines a distance range facing the front in the traveling direction of the display target range according to the level of the vehicle speed. The distance range here is a range represented by the distance from the position of the host vehicle to the near end and the edge of the display target range. For example, when the vehicle speed is relatively high, a range farther from the host vehicle is determined as the display target range than when the vehicle speed is relatively low. When the vehicle speed is high, the scenery in which the driving field of view flows quickly, the period during which the AR image can be displayed is shortened. Therefore, when the vehicle speed is high, the AR image is displayed at an early stage, so that a time margin for the driver to grasp the displayed AR image can be provided.

  Conversely, when the vehicle speed is relatively low, a range closer to the host vehicle is determined as the display target range than when the vehicle speed is relatively high. When the vehicle speed is low, the scenery of the driving field of view flows slowly, so that the driver has time to grasp the AR image. Therefore, by displaying the AR image closer, it is possible to reduce the troublesomeness caused by continuing to display the AR image from a long distance. Note that the correspondence relationship between the distance range of the display target range and the vehicle speed may be registered in advance in a memory or the like as a design value. Alternatively, it is conceivable that the driver can select a setting value that defines the correspondence between the distance range of the display target range and the vehicle speed.

  In next S108, the HUD drawing ECU 10 specifies information related to the AR image corresponding to the object located within the display target range determined in S106 among the objects such as the facilities existing in front of the host vehicle. Specifically, facilities that are visible in the driver's driving field of view from the information about the facilities acquired from the AR information database 22, the navigation device 24, and the external information acquisition unit 26, and are located within the display target range Identify information about.

  At this time, the structure which sorts | selects the facility used as the display object of AR image according to the attribute of a facility may be sufficient. For example, it is conceivable that the driver can specify the type of facility (store, customer collection facility, dining facility, rest facility, famous place, event hall, registered facility, etc.) that is the display target of the AR image. In this case, the HUD drawing ECU 10 sets a facility corresponding to the designated facility type among the facilities located within the display target range as a display target of the AR image. Alternatively, it may be configured to determine the type of facility that is the display target of the AR image according to the time zone. For example, when the current time corresponds to lunch time or dinner time, the HUD drawing ECU 10 sets a dining facility as a display target of an AR image from facilities located within the display target range. Alternatively, when information that is limited in application period, such as big sale, time service, or special event, is transmitted to the driver's car, it is possible to use the limited information as an interrupt image to be displayed as an AR image.

  Alternatively, it may be configured to select a facility having a relationship with a driver's behavior history as a facility to be displayed as an AR image. Specifically, the HUD drawing ECU 10 acquires history information such as a browsing history and search history of a web page using a high-performance mobile phone possessed by the driver, an e-mail transmission / reception history, and the like, and is located within the display target range. Among the facilities to be used, the facility having the relationship with the acquired history information is set as the display target of the AR image. For example, when a browsing history and a search history of a web page related to a ramen store are stored in a high-function mobile phone, the ramen store is set as an AR image display target.

  In next S110, the HUD drawing ECU 10 determines whether or not the driver has watched the facility that is the display target of the AR image when the HUD display unit 30 displays the first-stage AR image. . Here, for example, using the detection result by the viewpoint time determination unit 21, a predetermined time (for example, 0.3 seconds) remains in a state where the driver's line-of-sight direction is in the direction in which the facility that is the display target of the AR image exists. ) It is determined that the driver has watched the facility on the condition that the operation has been continued. When the driver is not paying attention to the facility that is the display target of the AR image (S110: NO), the HUD drawing ECU 10 proceeds to S112.

  In S112, the HUD drawing ECU 10 determines whether or not an operation instruction for designating any of the displayed AR images is input when the HUD display unit 30 displays the first-stage AR image. . When the operation instruction for specifying the AR image is not input (S112: NO), the HUD drawing ECU 10 proceeds to S114. In S114, the HUD drawing ECU 10 draws an AR image relating to all facilities within the display target range specified in S108. Here, the HUD drawing ECU 10 calculates the display position of the AR image so that the AR image corresponding to the facility is displayed in the vicinity of the position where the facility to be displayed is visually recognized in the driving field of view of the driver. Then, the HUD drawing ECU 10 draws the AR image based on the display position obtained by the calculation. In the AR image drawn in S112, headline information regarding the facility is described as information provided in the first stage.

  In next S120, the HUD drawing ECU 10 outputs the image information of the first-stage AR image drawn in S114 to the HUD display unit 30, and causes the HUD display unit 30 to display the AR image. A display example of the AR image at this time will be described with reference to FIG. FIG. 3 schematically shows the state of the AR image displayed in a superimposed manner on the actual scenery of the facility in the driver's driving field of view. In the example of FIG. 3, it is assumed that a plurality of facilities 100 to 107 exist on both sides of the road on which the host vehicle travels in the driving field of view ahead of the vehicle.

  In the example shown in FIG. 3, since the driver's line-of-sight direction is directed to the right side in the traveling direction of the host vehicle, the display target range of the AR image is limited to the right side in the traveling direction of the host vehicle. . Further, based on the vehicle speed of the host vehicle, the distance range ahead of the display target range of the AR image is limited to a range of Am to Bm from the host vehicle toward the front. The facilities included in this display target range are facilities 100, 101, and 102. As a result, among the plurality of facilities 100 to 107, the AR images 200, 201, and 202 corresponding to the facilities 100, 101, and 102 that fall within the display target range are displayed in the vicinity of the actual scenery of each facility. . In the AR image displayed in the first stage, characters and symbols indicating headline information regarding facilities are described.

  As a comparison target for the display example of FIG. 3, a reference example assuming a state in which an AR image is displayed without limiting the display target range in the same driving field of view as the example of FIG. 3 is illustrated in FIG. In the reference example shown in FIG. 5, AR images are added to all the facilities 100 to 107 existing in the driving field of view regardless of the driver's line-of-sight direction and vehicle speed. In the present embodiment, the display target range is limited according to the driver's line-of-sight direction and the vehicle speed, and therefore, the display of the aspect illustrated in FIG. 5 is not performed.

  Returning to the flowchart of FIG. On the other hand, if it is determined in S110 that the driver has watched the facility that is the display target of the AR image when the first-stage AR image is displayed (S110: YES), the HUD drawing ECU 10 proceeds to S116. . In S116, the HUD drawing ECU 10 draws the second-stage AR image related to the facility that is watched. Alternatively, in S112, when it is determined that an operation instruction for designating any AR image is input when the first-stage AR image is displayed (S112: YES), the HUD drawing ECU 10 proceeds to S118. In S <b> 118, the HUD drawing ECU 10 draws the third-stage AR image related to the facility corresponding to the AR image designated via the operation input unit 28. In S116 and S118, the HUD drawing ECU 10 draws the second-stage or third-stage AR image so that the AR image is displayed in the vicinity of the position where the corresponding facility is visually recognized in the driver's driving field of view. In the AR image drawn in S116 and S118, more detailed information than the information provided in the first stage is described.

  In the next S120, the HUD drawing ECU 10 outputs the image information of the second or third stage AR image drawn in S116 or S118 to the HUD display unit 30, and causes the HUD display unit 30 to display the AR image. A display example at this time will be described with reference to FIG. In the case of FIG. 4, in the display state illustrated in FIG. 3, the driver gazes at the facility 101 on which the AR image 201 is superimposed and corresponds to the facility 101 among the AR images 200 to 202. Assume a situation after an operation instruction for specifying an AR image 201 to be input is input. When the facility 101 is watched or the AR image 201 corresponding to the facility 101 is designated, an AR image 203 for presenting detailed information corresponding to the facility 101 is displayed as shown in FIG. It is newly displayed as an AR image of the second stage or the third stage. At this time, the first-stage AR images 200 and 202 other than the AR image 201 are not displayed. The first stage AR image 201 may be left displayed or may be hidden by replacing the newly displayed AR image 203. Alternatively, a blinking switching display may be used. In the newly displayed AR image 203, characters and symbols indicating detailed information (for example, sale information) regarding the facility 101 are described.

  Returning to the flowchart of FIG. On the other hand, if it is determined in S100 that there is a possibility of a collision (S100: NO), the HUD drawing ECU 10 proceeds to S122. In S122, the HUD drawing ECU 10 stops (hides) display of all AR images. Then, the HUD drawing ECU 10 returns to S100.

[effect]
The head-up display system according to the embodiment has the following effects. In accordance with the detected driver's line-of-sight direction and vehicle speed, a display target range for displaying an AR image related to a facility or the like in the driving field of view is determined, and only the actual scenery of the facility included in the determined display target range An AR image can be added. For example, by displaying the AR image only in the direction in which the driver's line of sight is directed, the AR image can be easily visually recognized, and an extra AR image in an area where the line of sight is not easily viewed is difficult. You can prevent it from being displayed. In addition, the AR image is displayed from a distance when the vehicle speed is high, and the AR image is displayed near when the vehicle speed is low. Can be determined. In this way, the display target range can be controlled so that the AR image is displayed in a manner that is easy for the driver to recognize, depending on the driver's line-of-sight direction and vehicle speed (see FIG. 3). Compared with the case of displaying an image (see FIG. 5), the recognizability is improved and excessive display can be suppressed.

  Further, when a facility corresponding to the displayed AR image is watched or an operation instruction for designating the displayed AR image is received, detailed AR information regarding the facility corresponding to the designated AR image (FIG. 4). Reference) can be displayed. Thereby, the driver's intention can be reflected in the change in the display of the AR information.

[Modification]
In the above-described embodiment, the example in which the display target range of the AR image is determined according to the driver's line-of-sight direction and the vehicle speed has been described. Not only this but the structure which changes the size of AR image displayed according to speed may be further provided. For example, the HUD drawing ECU 10 may draw a larger AR image for display on the HUD display unit 30 as the detected vehicle speed is higher. In this way, it is possible to improve the recognizability of the AR image in a situation where the scenery of the driving field of view flows fast. Further, the visibility of the AR image can be improved by changing the size of the AR image or changing the arrangement so that the driver can easily see the AR image according to the distance between the facility that is the display target of the AR image and the driver.

  DESCRIPTION OF SYMBOLS 10 ... HUD drawing ECU, 12 ... State detection part, 14 ... Front detection part, 16 ... Collision prevention control part, 18 ... Vehicle speed detection part, 20 ... Driver state detection part, 22 ... AR information database, 24 ... Navigation apparatus, 26 ... External information acquisition unit, 28 ... Operation input unit, 30 ... HUD display unit, 32 ... Windshield or combiner.

Claims (4)

A head-up display device that projects an image on a windshield or combiner arranged in front of a driver of a vehicle, and displays the image superimposed on the driving field of view of the driver,
Gaze detection means (20, S102) for detecting the gaze direction of the driver;
Vehicle speed detection means (18, S104) for detecting the speed of the vehicle;
A determining means (10, S106) for determining a display target range in front of the vehicle according to the line-of-sight direction detected by the line-of-sight detecting means and the speed detected by the vehicle speed detecting means;
A specifying means (10, S108) for specifying image information representing information related to the specific object included in the display target range determined by the determining means among the specific objects existing in front of the vehicle;
Display means (30) for displaying the image information specified by the specifying means at a position associated with the actual scenery of the specific object visually recognized in the driving field of view ahead of the driver;
Control means (10, S112) for controlling the display means;
A head-up display device comprising: The head-up display device according to claim 1,
Receiving means (28) for receiving an instruction for designating specific image information displayed by the display means from the driver;
When the control means (10, S114) receives an operation instruction for designating specific image information by the accepting means, the display means displays second image information related to the specific object corresponding to the designated image information. To display
A head-up display device. The head-up display device according to claim 1 or 2,
The determining means determines the range in the lateral direction with respect to the driving field of view based on the line-of-sight direction as the display target range, and determines the distance range directed forward from the vehicle based on the vehicle speed;
A head-up display device. The head-up display device according to any one of claims 1 to 3,
The control means causes the display means to display the image information having different sizes according to the vehicle speed detected by the vehicle speed detection means;
A head-up display device.