JPWO2020170380A1 - Display control device and display control method - Google Patents

Abstract

An object of the present invention is to avoid over-display of objects for route guidance. The display control device according to the present invention includes an information acquisition unit (11) that acquires vehicle position information, a vehicle's planned travel route, line-of-sight information that is information on the line-of-sight direction of the vehicle driver, and map data. , The gaze point recognition unit (12) that recognizes the gaze point of the driver in front of the vehicle based on the position information, the line of sight information, and the map data, and the junction or intersection of the gaze point and the driving road in front of the vehicle. An object creation unit (13) that creates an object for route guidance based on the difference from the planned travel route ahead of the singular point, and a display control unit (14) that displays the object on the HUD (35) mounted on the vehicle. ) And.

Description

The present invention relates to a display control of a HUD (Head-Up Display).

As a method of guiding the planned travel route to the driver of the vehicle, Patent Document 1 discloses a technique of displaying an arrow object indicating the planned travel route superimposed on the planned travel route on the HUD.

Japanese Unexamined Patent Publication No. 2015-31700

However, such a display method has a problem that it is difficult for the driver to see the foreground because the object is always superimposed on the foreground during the route guidance guidance.

The present invention has been made to solve the above problems, and an object of the present invention is to avoid over-display of objects for route guidance.

The first display control device of the present invention includes an information acquisition unit that acquires vehicle position information, a vehicle's planned travel route, line-of-sight information that is information on the line-of-sight direction of the driver of the vehicle, and map data. A gaze point recognition unit that recognizes the gaze point of the driver in front of the vehicle based on position information, line-of-sight information, and map data, and a singular point that is a junction or intersection of the gaze point and the driving road in front of the vehicle. It is provided with an object creation unit that creates an object for route guidance based on the difference from the planned travel route, and a display control unit that displays the object on the HUD mounted on the vehicle.

The second display control device of the present invention includes an information acquisition unit that acquires vehicle position information, line-of-sight information that is information on the line-of-sight direction of the driver of the vehicle, and map data, and position information, line-of-sight information, and a map. An object creation unit that creates an object for route guidance based on the difference between the gaze point recognition unit that recognizes the gaze point of the driver in front of the vehicle based on the data and the gaze point and the specific planned travel route of the vehicle. And a display control unit that displays the object on the HUD mounted on the vehicle, and the specific planned travel route of the vehicle is visually recognized discontinuously with the road on which the vehicle is currently traveling among the planned travel routes of the vehicle. It is a road to be driven.

The present invention causes the HUD to display an object for route guidance based on the difference between the gaze point and the planned travel route or the specific travel schedule route ahead of the singular point. Therefore, the object is not displayed constantly, and overdisplay can be avoided. Objectives, features, aspects, and advantages of the present invention will be made more apparent with the following detailed description and accompanying drawings.

It is a block diagram which shows the structure of the display control device of Embodiment 1. FIG. It is a flowchart which shows the operation of the display control device of Embodiment 1. FIG. It is a block diagram which shows the structure of the display control device of Embodiment 2. It is a flowchart which shows the operation of the display control device of Embodiment 2. It is a figure which shows the situation which a driver is paying attention to the road other than the planned travel route. It is a figure which shows the display example of the HUD in the situation shown in FIG. It is a figure which shows the situation which a driver is paying attention to the planned travel route beyond a singular point. It is a figure which shows the display example of the HUD in the situation shown in FIG. 7. It is a figure which shows the display example of a correct answer object. It is a figure which shows the display example of a correct answer object. It is a figure which shows the display example of a correct answer object. It is a figure which shows the display example of a correct answer object. It is a figure which shows the display example of an incorrect answer object. It is a figure which shows the display example of the correct answer notification object. It is a figure which shows the display example of an incorrect answer object and a correct answer notification object. It is a flowchart which shows the operation when the display control device of Embodiment 2 displays an incorrect answer object. It is a block diagram which shows the structure of the display control device of Embodiment 3. It is a flowchart which shows the operation of the display control device of Embodiment 3. It is a figure which shows the display example of the object which urges a driver to make a gesture. It is a block diagram which shows the structure of the display control device of Embodiment 4. It is a flowchart which shows the operation of the display control device of Embodiment 4. It is a figure which shows the display example of the object which prompts a driver to speak a keyword. It is a figure which shows the display example of the HUD in the situation where a road can be seen at a distant mountain foot. It is a figure which shows the display example of the HUD when the driver gazes at the road at the foot of a mountain in the situation shown in FIG. It is a flowchart which shows the operation of the display control device of Embodiment 5. It is a figure which shows the hardware configuration of a display control device. It is a figure which shows the hardware configuration of a display control device. It is a figure which shows the configuration example of the display control device by a vehicle and a server.

<A. Embodiment 1>
<A-1. Configuration>
FIG. 1 is a block diagram showing the configuration of the display control device 101 of the first embodiment. The display control device 101 controls the display of the HUD 25. The HUD 25 is mounted on the vehicle. Hereinafter, the term "vehicle" in the present specification refers to a vehicle equipped with HUD25.

The display control device 101 includes an information acquisition unit 11, a gazing point recognition unit 12, an object creation unit 13, and a display control unit 14. The information acquisition unit 11 acquires map data, vehicle position information, line-of-sight information including the line-of-sight direction of the driver of the vehicle, and the planned travel route of the vehicle.

The gaze point recognition unit 12 recognizes the gaze point of the driver in front of the vehicle based on the vehicle position information, line-of-sight information, and map data acquired by the information acquisition unit 11. The gaze point is the point where the driver is gazing. For example, when the line-of-sight direction continues to stay at the same point for a certain period of time or longer, the gazing point recognition unit 12 recognizes that point as the gazing point.

The object creation unit 13 creates an object for route guidance based on the difference between the gazing point and the planned travel route ahead of the singular point in front of the vehicle. Here, the singular point is a point where the course of the vehicle may change, such as an intersection or a branch point. Since multiple routes on which the vehicle can travel are assumed at the singularity, the driver needs to know which route is the planned travel route.

The display control unit 14 displays the object created by the object creation unit 13 on the HUD 25.

<A-2. Operation>
FIG. 2 is a flowchart showing the operation of the display control device 101. Hereinafter, the operation of the display control device 101 will be described with reference to FIG. First, the information acquisition unit 11 acquires various types of information (step S101). The information acquired here includes the position information of the vehicle, the planned travel route of the vehicle, the line-of-sight information of the driver, and the map data.

Next, the gazing point recognition unit 12 recognizes the gazing point based on the vehicle position information, the line-of-sight information, and the map data (step S102).

Next, the object creation unit 13 creates an object for route guidance based on the difference between the gazing point and the planned travel route ahead of the singular point (step S103).

Then, the display control unit 14 controls the display of the HUD 25 (step S104), and the object is displayed on the HUD 25.

<A-3. Effect>
As described above, the display control device 101 of the first embodiment includes an information acquisition unit 11, a gazing point recognition unit 12, an object creation unit 13, and a display control unit 14. The information acquisition unit 11 acquires the position information of the vehicle, the planned travel route of the vehicle, the line-of-sight information which is the information of the line-of-sight direction of the driver of the vehicle, and the map data. The gaze point recognition unit 12 recognizes the gaze point of the driver in front of the vehicle based on the position information, the line of sight information, and the map data. The object creation unit 13 creates an object for route guidance based on the difference between the gazing point and the planned travel route ahead of the singular point which is a branch point or an intersection of the travel road in front of the vehicle. The display control unit 14 displays the object on the HUD 25 mounted on the vehicle. Therefore, the driver can grasp from the display status of the object of the HUD 25 whether or not the point he / she gazes at is the planned travel route. Further, since the object is displayed according to the driver's gaze point, the object is not always displayed, and the obstruction of the driver's forward view is suppressed.

<B. Embodiment 2>
<B-1. Basic configuration>
FIG. 3 is a block diagram showing the configuration of the display control device 102 according to the second embodiment. The display control device 102 is connected to the map storage device 21, the locator 22, the line-of-sight detection device 23, the route setting device 24, and the HUD 25, and is configured to be available.

Map data is stored in the map storage device 21, and the map data is output to the information acquisition unit 11 and the locator 22. The locator 22 measures the position of the vehicle by a GNSS or a vehicle sensor (not shown). The locator 22 may acquire map data from the map storage device 21 and correct the position of the vehicle measured by map mapping. The locator outputs the position information of the vehicle to the route setting device 24 and the information acquisition unit 11. The line-of-sight detection device 23 detects the line-of-sight of the occupant from the image of the eyes of the driver of the vehicle, and outputs line-of-sight information including information on the line-of-sight direction to the information acquisition unit 11. The route setting device 24 acquires the position information of the vehicle from the locator 22, sets the planned travel route from the current position of the vehicle to the destination, and outputs the planned travel route to the information acquisition unit 11. That is, the information acquisition unit 11 acquires map data from the map storage device 21, vehicle position information from the locator 22, line-of-sight information from the line-of-sight detection device 23, and a planned travel route from the route setting device 24.

In FIG. 3, the map storage device 21, the locator 22, the line-of-sight detection device 23, and the route setting device 24 are outside the display control device 102, but at least one of these is a configuration of the display control device 102. good.

<B-2. Basic operation>
The operation of the display control device 102 will be described with reference to the flowchart of FIG. First, the information acquisition unit 11 acquires various types of information (step S201). This step is the same as step S101 of FIG. Next, the gazing point recognition unit 12 determines whether or not there is a singular point in front of the vehicle based on the position information of the vehicle and the map data (step S202). In this step, the gazing point recognition unit 12 determines whether or not a singular point exists within a certain distance from the vehicle, for example, within 300 m in front of the vehicle. If there is no such singularity, the display control device 102 returns to the process of step S201.

If there is a singular point in front of the vehicle (Yes in step S202), the gazing point recognition unit 12 recognizes the gazing point based on the vehicle's position information, line-of-sight information, and map data (step S203). This step is the same as step S102 in FIG.

Next, the gazing point recognition unit 12 determines whether or not the gazing point is the planned travel route ahead of the singular point (step S204). If the gazing point is not the planned travel route ahead of the singular point (No in step S204), the display control device 102 returns to the process of step S201, and the object is not displayed on the HUD 25.

FIG. 5 shows a situation in which the driver is gazing at a road other than the planned travel route. In FIG. 5, there are intersections P1 and P2, which are singular points, in front of the road 31 on which the vehicle 34 travels. Here, it is assumed that the planned travel route of the vehicle is a route that travels on the road 32 by turning left at the intersection P2, as shown by the arrow 36. The gazing point 35 is not on the road 32, but on the road 31 ahead of the intersection P2. In this case, as shown in FIG. 6, the object for route guidance is not displayed on the HUD 25. Although the eye mark representing the gazing point 35 is shown in FIG. 6, this merely represents the gazing point and is not actually displayed on the HUD 25.

Returning to the flowchart of FIG. 4, in step S204, if the gazing point is the planned travel route ahead of the singular point, the object creation unit 13 creates a correct answer object (step S205).

FIG. 7 shows a situation in which the driver is gazing at the planned travel route beyond the singularity. In FIG. 7, the same as in FIG. 5 except for the position of the gazing point 35. The gazing point 35 is on the road 32, which is the planned travel route ahead of the intersection P2. In this case, as shown in FIG. 8, the HUD 25 displays the arrow icon 37 at a position overlapping the point near the intersection P2 on the road 32. The arrow icon 37 is a symbol that can be visually understood to turn left at the intersection P2, whereby the driver can grasp that the road 32 is the planned travel route. That is, the arrow icon 37 is a correct answer object indicating that the driver's gaze point is on the planned travel route. Although the eye mark representing the gazing point 35 is shown in FIG. 8, this merely represents the gazing point and is not actually displayed on the HUD 25.

Although not shown in FIG. 3, the display control device 102 may include a peripheral detection device that captures an image of the front of the vehicle and recognizes the feature in front by image processing. By collating the position of the feature in front recognized by the main change detection device with the map data, the display control unit 14 determines the display position of the object in the HUD 25 with higher accuracy and superimposes it on the target position more accurately. Can be displayed.

<B-3. Correct object>
In the above, the arrow icon 37 has been described as an example of the correct answer object, but various other modes are assumed for the correct answer object. The correct object may be any object that can notify the driver that the gazing point is on the planned travel route. For example, the correct answer object may be the object 38 of the arrow line shown in FIG. The object 38 is displayed along the roads 31, 32, that is, the planned travel route, before and after the intersection P2. Further, the correct answer object may be the linear object 39 shown in FIG. The object 39 is displayed along the road 32, which is the planned travel route, and is not displayed in front of the intersection P2. Further, the correct answer object may be a plurality of rod-shaped objects 40 shown in FIG. The objects 40 are erected on the roads 31, 32, that is, the planned travel route, before and after the intersection P2, and are arranged along the planned travel route.

Further, the display control unit 14 may notify the driver of the planned travel route by making the brightness of the region superimposed on the planned travel route in the display region of the HUD 25 brighter than the brightness of the other regions. FIG. 12 shows such a display example. In FIG. 12, of the display areas of the HUD 25, the area 41 superimposed on the planned travel route is displayed with higher brightness than the other areas 42. In other words, in FIG. 12, the object covering the entire display area of the HUD 25 is the correct answer object, and among the correct answer objects, the area 41 superimposed on the planned travel route on the HUD 25 is displayed with higher brightness than the other areas 42. Will be done.

<B-4. Transformation example at the time of incorrect answer>
In FIG. 4-6, it has been explained that the display control device 102 does not display the object related to the route guidance on the HUD 25 when the driver gazes at a place other than the planned travel route ahead of the singular point. However, in such a case, that is, at the time of incorrect answer, the display control device 102 provides an incorrect answer object indicating that the driver is gazing at a place other than the planned travel route, or a correct answer notification object representing the planned travel route. It may be displayed on the HUD25. A display example of the incorrect answer object and the correct answer notification object will be described below.

FIG. 13 shows a display example of an incorrect answer object. In FIG. 13, when the driver's gaze point 35 is on the road 33 even though the road 32 is the planned travel route, the cross mark icon 43 is displayed on the road 33 in the HUD 25. The icon 43 is an incorrect answer object indicating that the road 33 that the driver is gazing at is not the planned travel route. Here, an example is shown in which the incorrect answer object is superimposed and displayed on the road having the gaze point 35, but the incorrect answer object is not necessarily displayed superimposed on the road having the gaze point 35. For example, the icon 43 may be displayed in a place that does not interfere with the driver's front view, such as the upper right corner of the display area of the HUD 25.

FIG. 14 shows a display example of the correct answer notification object. In FIG. 14, when the driver's gaze point 35 is on the road 33 even though the road 32 is the planned travel route, the road 31 which is the planned travel route before and after the intersection P2 which is a singular point on the HUD 25. , 32 is displayed along the arrow object 38. The object 38 is a correct answer notification object indicating that the road 32 is a planned travel route.

The incorrect answer object and the correct answer notification object may be displayed at the same time. FIG. 15 shows such a display example. In FIG. 15, the cross-shaped icon 43 is displayed on the road 33 which is not the planned travel route, and the linear object 39 is superimposed and displayed on the road 32 which is the planned travel route ahead of the intersection P2 which is a singular point. Has been done. The icon 43 is an incorrect answer object, and the object 39 is a correct answer notification object.

When displaying the incorrect answer object or the correct answer notification object, the display control device 102 does not display the correct answer object.

FIG. 16 is a flowchart showing an operation when the display control device 102 displays an incorrect answer object. The flowchart of FIG. 16 is obtained by substituting step S204 and step S205 of the flowchart of FIG. 4 with steps S204A and S205A. In step S204A, the gazing point recognition unit 12 determines whether or not the gazing point is the planned travel route ahead of the singular point. If the gazing point is the planned travel route beyond the singular point (Yes in step S204A), that is, if the gazing point of the driver is correct, the operation of the display control device 102 returns to step S201, and the object is displayed on the HUD25. Do not show. On the other hand, if the gazing point is not the planned travel route beyond the singular point (No in step S204A), that is, if the gazing point of the driver is an incorrect answer, the object creation unit 13 creates an incorrect answer object (step S205A). ), The display control unit 14 displays the incorrect answer object on the HUD 25 (step S206).

In this way, the display control device 102 sets the object at either the correct answer when the gazing point is the planned travel route ahead of the singular point and the incorrect answer when the gazing point is not the planned travel route ahead of the singular point. Display on HUD25. As a result, the object related to the route guidance is prevented from being always displayed on the HUD 25, and the obstruction of the driver's forward view is suppressed.

<B-5. Other variants>
In the above, the correct answer object, the incorrect answer object, and the correct answer notification object have been described as the objects related to the route guidance. The display control unit 14 may continue to display these objects once displayed on the HUD 25 as long as the driver's gaze point does not change, or may erase the display under certain conditions. For example, the display control unit 14 may limit the display period to a certain time such as 10 seconds, and after a certain time elapses, the display of the object may be erased even if the driver's gaze continues. Further, in the display control devices 103 and 104 of the third and fourth embodiments described later, the display of the object may be deleted when the utterance of a specific gesture or a specific keyword is detected. This makes it possible to avoid over-displaying objects.

Further, the display control device 102 may limit the singular points to which the object is displayed based on the gazing point. For example, the display control device 102 may limit the target to a singular point in a road structure that is easily mistaken. A road structure that is easily mistaken is, for example, a road structure in which intersections or branch points are continuously present at short intervals, or a branch point that branches into three or more roads. The above-mentioned "short interval" is about 10 m on a general road and about 30 m on an expressway, although it depends on the speed of the vehicle or the type of road.

The arrow icon 37 in FIG. 8, the icon 43 in FIG. 13, and the icon 43 in FIG. 15 are displayed on the road on which the gazing point 35 exists, and whether or not the road the driver is gazing at is the planned travel route. Has the role of notifying the driver. The display control unit 14 may display these icons on the gazing point 35. In this case, since the icons are displayed at the exact place where the driver is watching, there is an effect that the driver can easily see these icons.

When the gazing point 35 is on the road in front of the singularity, that is, on the road on which the vehicle is currently traveling, neither the correct answer object nor the incorrect answer object is displayed on the HUD 25. Normally, the driver visually recognizes the road on which the vehicle is currently traveling until the driver approaches the left / right turn point. In such a case, the object is not displayed on the HUD 25, thereby preventing over-display to the driver.

In the above, the difference between the gazing point and the planned travel route beyond the singular point was considered. However, depending on the road structure or terrain, it may be difficult for the driver to directly see the road itself. For example, if the road ahead of the singularity or the road currently on the road has a slope, or if the road is behind a feature, building, or vehicle in front. In these cases, the gaze point recognition unit 12 may determine from the direction of the gaze point that the road that is hidden and invisible is actually being visually recognized, and may determine the difference between the gaze point and the planned travel route.

Further, in the above, it has been described that the correct answer object or the like is displayed when the vehicle is within a certain distance from the singular point. However, the display of the correct answer object or the like may be performed after the route guidance for the singular point is performed by the navigation device or the like.

<B-6. Effect>
In the display control device 102 of the second embodiment, the object creation unit 13 is an object indicating that the gaze point of the driver is the planned travel route when the gazing point matches the planned travel route ahead of the singular point. Create a correct object. Therefore, the driver can grasp from the correct object that his / her gaze point is the planned travel route. Further, since the correct answer object is not displayed unless the gazing point matches the planned travel route, the correct answer object is not over-displayed and the forward view is not obstructed.

Further, when the correct answer object is superimposed and displayed on the planned travel route ahead of the singular point, the driver can intuitively grasp the planned travel route ahead of the singular point from the display position of the correct answer object.

Further, when the correct answer object is displayed superimposed on the gazing point, the correct answer object can be easily visually recognized without moving the line of sight from the gazing point.

Further, when the correct answer object is an icon representing the course direction of the planned travel route at the singular point, the driver can easily grasp the course direction at the singular point from the correct answer object.

Further, when the correct answer object is a linear object displayed along the planned travel route ahead of the singular point, the driver can intuitively grasp the planned travel route ahead of the singular point.

In addition, when the correct answer object is a plurality of rod-shaped objects erected on the road along the planned travel route before and after the singular point, the driver intuitively grasps the planned travel route ahead of the singular point. can do.

Further, the correct answer object is an object that covers the entire display area of the HUD, and the display brightness of the portion that overlaps the planned travel route ahead of the singular point of the object is the portion that overlaps the road that is not the planned travel route ahead of the singular point. It may be higher than the display brightness of. In this case, the driver can intuitively grasp the planned travel route ahead of the singular point.

Further, the object creation unit 13 creates an incorrect answer object which is an object indicating that the driver's gaze point is not the planned travel route when the gazing point coincides with a road other than the planned travel route ahead of the singular point. You may. In this case, the driver can grasp from the incorrect answer object that the current gazing point is not the planned travel route.

Further, when the incorrect answer object is displayed overlapping on the road of the gazing point, the driver can easily visually recognize the incorrect answer object.

Further, when the gazing point coincides with a road other than the planned travel route ahead of the singular point, the object creation unit 13 may create a correct answer notification object which is an object representing the planned travel route ahead of the singular point. This correct answer notification object is displayed superimposed on the planned travel route ahead of the singular point. Therefore, the driver can grasp the planned travel route by the correct answer notification object.

Then, when the correct answer notification object is a linear object displayed along the planned travel route ahead of the singular point, the driver can intuitively grasp the planned travel route.

Further, when the gazing point matches a road other than the planned travel route ahead of the singular point, the object creation unit 13 is an object representing the planned travel route ahead of the singular point in addition to the incorrect answer object, which is a correct answer notification object. May be created. This correct answer notification object is displayed superimposed on the planned travel route ahead of the singular point. Therefore, the driver can grasp that the route is not the planned travel route from his / her own gaze point of view and intuitively grasp the planned travel route.

<C. Embodiment 3>
<C-1. Configuration>
FIG. 17 is a block diagram showing the configuration of the display control device 103 according to the third embodiment. The display control device 103 is connected to the gesture recognition device 26 and is configured to be usable, and other configurations are the same as those of the display control device 102 of the second embodiment.

The gesture recognition device 26 recognizes the gesture by the driver from the image taken by the driver by the camera mounted on the vehicle, and when it detects that a specific preset gesture has been performed, the information acquisition unit notifies that fact. Output to 11.

<C-2. Operation>
FIG. 18 is a flowchart showing the operation of the display control device 103. In the flow of FIG. 18, steps S301 to S304 are the same as steps S201 to S204 of FIG. In step S304, the gazing point recognition unit 12 determines whether or not the gazing point is the planned travel route ahead of the singular point. If the gazing point is not the planned travel route beyond the singular point (No in step S304), the operation of the display control device 103 returns to step S301, and the object is not displayed on the HUD 25.

If the gazing point is the planned travel route ahead of the singular point (Yes in step S304), the object creation unit 13 determines whether or not the gesture recognition device 26 has detected a specific gesture (step S305). This determination is made within a certain period of time, for example, within 10 seconds from the recognition of the gazing point in step S303, for example. If no specific gesture is detected within this time, the operation of the display control device 103 returns to step S301.

When the gesture recognition device 26 detects a specific gesture in step S305, the object creation unit 13 creates a correct answer object (step S306). Then, the display control unit 14 displays the correct answer object on the HUD 25 (step S307). Step S306 and step S307 are similar to steps S205 and S206 of FIG.

In this way, according to the display control device 103, the correct answer object is displayed on the HUD 25 only after the driver gazes at the planned travel route ahead of the singular point and then makes a specific gesture. Therefore, the correct object is not displayed on the HUD 25 contrary to the driver's intention, and the over-display of the object on the HUD 25 can be prevented.

In the flowchart of FIG. 18, the operation when the display control device 103 displays the correct answer object has been described, but the same applies to the display of the incorrect answer object or the correct answer notification object.

Further, according to the flowchart of FIG. 18, the display control device 103 detects the gesture in step S305 after recognizing the gazing point in step S303. However, gaze and gesture may be performed continuously, for example, within 10 seconds, and the order may be reversed. That is, the display control device 103 may display the correct answer object when the driver gazes at the planned travel route ahead of the singular point after performing a specific gesture.

Further, after recognizing that the gazing point is on the planned travel route ahead of the singular point (Yes in step S304 of FIG. 18), the display control device 103 displays an object prompting the driver to make a gesture on the HUD 25. Is also good. FIG. 19 shows a display example of an object that prompts the driver to make a gesture. In FIG. 19, the road 32 is the planned travel route, and the gazing point 35 is on the road 32. Then, the display control device 103 displays the object 44 that prompts the driver to make a gesture in the upper left corner of the display area of the HUD 25.

<C-3. Effect>
In the display control device 103 of the third embodiment, the information acquisition unit 11 acquires the gesture information from the gesture recognition device 26 that recognizes the gesture of the driver of the vehicle, and the display control unit 14 obtains the gesture information based on the gesture information. Display the object on HUD25. Therefore, the object is not displayed on the HUD 25 against the intention of the driver, and the over-display of the object on the HUD 25 can be prevented.

<D. Embodiment 4>
<D-1. Configuration>
FIG. 20 is a block diagram showing the configuration of the display control device 104 according to the fourth embodiment. The display control device 104 is connected to the voice recognition device 27 and is configured to be usable, and other configurations are the same as those of the display control device 102 of the second embodiment.

When the voice recognition device 27 detects that the driver has spoken a preset keyword from the utterance voice of the driver acquired by the microphone mounted on the vehicle, the voice recognition device 27 outputs to that effect to the information acquisition unit 11.

<D-2. Operation>
FIG. 21 is a flowchart showing the operation of the display control device 104. In the flow of FIG. 21, steps S401 to S404 are the same as steps S201 to S204 of FIG. In step S404, the gazing point recognition unit 12 determines whether or not the gazing point is the planned travel route ahead of the singular point. If the gazing point is not the planned travel route beyond the singular point (No in step S404), the operation of the display control device 104 returns to step S401, and the object is not displayed on the HUD 25.

If the gazing point is the planned travel route ahead of the singular point (Yes in step S404), the object creation unit 13 determines whether or not the voice recognition device 27 has detected a specific keyword (step S405). This determination is made, for example, within a certain period of time, for example, within 10 seconds from the recognition of the gazing point in step S403. If a specific keyword is not detected within this time, the operation of the display control device 104 returns to step S401.

When the voice recognition device 27 detects a specific keyword in step S405, the object creation unit 13 creates a correct answer object (step S406). Then, the display control unit 14 displays the correct answer object on the HUD 25 (step S407). Step S406 and step S407 are similar to steps S305 and S306 of FIG.

In this way, according to the display control device 104, the correct answer object is displayed on the HUD 25 only after the driver gazes at the planned travel route ahead of the singular point and then utters a specific keyword. Therefore, the correct object is not displayed on the HUD 25 contrary to the driver's intention, and the over-display of the object on the HUD 25 can be prevented.

In the flowchart of FIG. 21, the operation when the display control device 104 displays the correct answer object has been described, but the same applies to the display of the incorrect answer object or the correct answer notification object.

Further, according to the flowchart of FIG. 21, the display control device 104 detects the keyword in step S405 after recognizing the gazing point in step S403. However, the gaze and the utterance of the keyword may be performed continuously within a certain period of time, for example, within 10 seconds, and the order may be reversed. That is, the display control device 104 may display the correct answer object when the driver gazes at the planned travel route ahead of the singular point after uttering a specific keyword.

Further, the display control device 104 displays an object on the HUD 25 that prompts the driver to speak a keyword after recognizing that the gazing point is on the planned travel route ahead of the singular point (Yes in step S404 of FIG. 21). You may. FIG. 22 shows a display example of an object that prompts the driver to speak a keyword. In FIG. 22, the road 32 is the planned travel route, and the gazing point 35 is on the road 32. Then, the display control device 104 displays the object 45 that prompts the driver to speak the keyword in the upper left corner of the display area of the HUD 25.

<D-3. Effect>
The information acquisition unit 11 acquires the utterance voice of the driver of the vehicle from the microphone mounted on the vehicle, and the display control unit 14 causes the HUD to display the object based on the utterance voice. Therefore, the object is not displayed on the HUD 25 against the intention of the driver, and the over-display of the object on the HUD 25 can be prevented.

<E. Embodiment 5>
<E-1. Configuration>
The configuration of the display control device 105 of the fifth embodiment is as shown in FIG. 3, and is the same as that of the display control device 102 of the second embodiment.

In the second embodiment, when there is a singular point in front of the vehicle, an object related to route guidance is displayed on the HUD 25 based on the difference between the gazing point and the planned travel route ahead of the singular point.

On the other hand, in the fifth embodiment, when there is a specific travel schedule route in front of the vehicle, an object related to route guidance is displayed on the HUD 25 based on the difference between the specific travel schedule route and the gazing point. A specific travel schedule route will be described with reference to FIG. 23.

FIG. 23 shows a display example of the HUD 25 in a situation where the road 46 can be seen at the foot of a distant mountain. In FIG. 23, the vehicle is traveling on the road 31, and there is a road 46 at the foot of the mountain in the distance. Road 46 is a planned travel route for vehicles. The road 31 and the road 46 on which the vehicle is currently traveling are connected at a distance, but in the driver's field of view, there is a feature such as a building 47 between the road 31 and the road 46, so that the road is a road. 46 is visually recognized discontinuously with the road 31. Such a road is referred to as a specific planned travel route in the present embodiment.

FIG. 24 shows a display example of the HUD 25 when the driver gazes at the road 46 in the situation shown in FIG. When the gazing point 35 coincides with the road 46, which is the specific planned travel route, the linear object 48 is displayed at a position overlapping the road 46 in the HUD 25, whereby the road 46 is highlighted. The object 48 is a correct answer object indicating that the gazing point 35 is a planned travel route. However, the correct object is not limited to the object 48 shown in FIG. 24 as long as it indicates that the road 46 is the planned travel route.

<E-2. Operation>
FIG. 25 is a flowchart showing the operation of the display control device 105. Hereinafter, the operation of the display control device 105 will be described with reference to FIG. 25. Steps S501 to S503 of FIG. 25 are the same as steps S201 to S203 of FIG. After recognizing the gazing point (step S503), the gazing point recognition unit 12 determines whether or not the gazing point is a specific planned travel route (step S504). Since the specific travel schedule route is a travel schedule route far from the vehicle, the gaze point recognition unit 12 allows a slight deviation between the gaze point and the specific travel schedule route when making a determination in step S504. Is also good. For example, in the situation of FIG. 24, when the gazing point 35 is not on the road 46 but is on the mountain where the road 46 is laid, the gazing point recognition unit 12 sets the gazing point on the specific planned travel route in step S504. You may judge that there is.

When the gazing point is not the specific travel schedule route (No in step S504), the display control device 105 returns to the process of step S501, and the object is not displayed on the HUD 25. On the other hand, when the gazing point is a specific planned travel route (Yes in step S504), the object creation unit 13 creates a correct answer object (step S505). Then, the display control unit 14 displays the correct answer object on the HUD 25 (step S506), and the display control device 105 returns to the process of step S501.

<E-3. Effect>
The display control device 105 of the fifth embodiment includes an information acquisition unit 11 that acquires vehicle position information, line-of-sight information that is information on the line-of-sight direction of the driver of the vehicle, and map data, position information, line-of-sight information, and An object that creates an object for route guidance based on the difference between the gaze point recognition unit 12 that recognizes the gaze point of the driver in front of the vehicle based on the map data and the gaze point and the specific planned travel route of the vehicle. It includes a creation unit 13 and a display control unit 14 for displaying an object on a HUD mounted on a vehicle. Therefore, according to the display control device 105, it is possible to easily grasp from the object whether or not the road in the distant view is the planned travel route. Further, since the object is displayed according to the gazing point, it is not displayed constantly, and the obstruction of the driver's field of vision is suppressed.

<F. Hardware configuration>
The information acquisition unit 11, the gazing point recognition unit 12, the object creation unit 13, and the display control unit 14 in the display control device 101-105 described above are realized by the processing circuit 81 shown in FIG. That is, the processing circuit 81 includes an information acquisition unit 11, a gazing point recognition unit 12, an object creation unit 13, and a display control unit 14 (hereinafter, referred to as “information acquisition unit 11 and the like”). Dedicated hardware may be applied to the processing circuit 81, or a processor that executes a program stored in the memory may be applied. The processor is, for example, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like.

When the processing circuit 81 is dedicated hardware, the processing circuit 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable). Gate Array), or a combination of these. Each function of each part such as the information acquisition unit 11 may be realized by a plurality of processing circuits 81, or the functions of each part may be collectively realized by one processing circuit.

When the processing circuit 81 is a processor, the functions of the information acquisition unit 11 and the like are realized by a combination of software and the like (software, firmware or software and firmware). Software and the like are described as programs and stored in memory. As shown in FIG. 27, the processor 82 applied to the processing circuit 81 realizes the functions of each part by reading and executing the program stored in the memory 83. That is, when the display control device 101-104 is executed by the processing circuit 81, the position information of the vehicle, the planned travel route of the vehicle, the line-of-sight information which is the information of the line-of-sight direction of the driver of the vehicle, and the line-of-sight information. A step to acquire map data, a step to recognize the driver's gaze point in front of the vehicle based on position information, line-of-sight information, and map data, and a junction or intersection of the gaze point and the driving road in front of the vehicle. As a result, the step of creating an object for route guidance based on the difference from the planned travel route ahead of the singular point and the step of displaying the object on the HUD mounted on the vehicle are executed. A memory 83 for storing a program is provided. Further, when the display control device 105 is executed by the processing circuit 81, the step of acquiring the position information of the vehicle, the line-of-sight information which is the information of the line-of-sight direction of the driver of the vehicle, and the map data, and the position information. , The step of recognizing the gaze point of the driver in front of the vehicle based on the line-of-sight information and the map data, and the step of creating an object for route guidance based on the difference between the gaze point and the specific planned travel route of the vehicle. And a step of displaying the object on the HUD mounted on the vehicle, and a memory 83 for storing a program to be executed as a result. In other words, it can be said that this program causes the computer to execute the procedure or method of the information acquisition unit 11 or the like. Here, the memory 83 includes, for example, non-volatile such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). Or with volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disk) and its drive device, etc., or any storage medium used in the future. There may be.

The configuration in which each function of the information acquisition unit 11 and the like is realized by either hardware or software has been described above. However, the present invention is not limited to this, and a configuration may be configured in which a part of the information acquisition unit 11 or the like is realized by dedicated hardware and another part is realized by software or the like. For example, the information acquisition unit 11 realizes its function by a processing circuit as dedicated hardware, and other than that, the processing circuit 81 as a processor 82 reads and executes a program stored in the memory 83 to execute the function. It is possible to realize.

As described above, the processing circuit can realize each of the above-mentioned functions by hardware, software, or a combination thereof. Further, although the display control device 101-105 has been described above as an in-vehicle device, it is installed in an in-vehicle device, a PND (Portable Navigation Device), a communication terminal (for example, a mobile terminal such as a mobile phone, a smartphone, and a tablet), and the like. It can also be applied to a system constructed as a system by appropriately combining the functions of various applications and servers. In this case, each function or each component of the display control device 101-105 described above may be distributed and arranged in each device for constructing the system, or may be centrally arranged in any device. May be good. FIG. 28 shows a configuration example of the display control devices 102 and 105 by the vehicle 34 and the server 70. In the example of FIG. 28, the display control unit 14 and the HUD 25 are arranged on the vehicle 34, and the map storage device 21, the locator 22, the line-of-sight detection device 23, the route setting device 24, the information acquisition unit 11, and the gazing point recognition unit 12 are arranged on the server 70. And the object creation unit 13 is arranged.

In the present invention, each embodiment and each modification can be freely combined, and each embodiment and each modification can be appropriately modified or omitted within the scope of the invention.

Although the present invention has been described in detail, the above description is exemplary in all embodiments and the present invention is not limited thereto. A myriad of variants not illustrated can be envisioned without departing from the scope of the invention.

11 Information acquisition unit, 12 Gaze point recognition unit, 13 Object creation unit, 14 Display control unit, 21 Map storage device, 22 Locator, 23 Line-of-sight detection device, 24 Route setting device, 25 HUD 26 Gesture recognition device, 70 server, 81 Processing circuit, 82 processors, 83 memories.

Claims (18)

An information acquisition unit that acquires vehicle position information, a planned travel route of the vehicle, line-of-sight information that is information on the line-of-sight direction of the driver of the vehicle, and map data.
A gaze point recognition unit that recognizes the gaze point of the driver in front of the vehicle based on the position information, the line of sight information, and the map data.
An object creation unit that creates an object for route guidance based on the difference between the gazing point and the planned travel route ahead of a singular point that is a branch point or an intersection of the travel road in front of the vehicle.
A display control unit for displaying the object on the HUD mounted on the vehicle is provided.
Display control device. When the gaze point coincides with the planned travel route ahead of the singular point, the object creation unit creates a correct answer object which is the object indicating that the gaze point of the driver is the planned travel route. ,
The display control device according to claim 1. The correct object is displayed superimposed on the planned travel path ahead of the singular point.
The display control device according to claim 2. The correct answer object is displayed superimposed on the gazing point.
The display control device according to claim 3. The correct answer object is an icon representing the course direction of the planned travel route at the singularity.
The display control device according to claim 3. The correct answer object is a linear object displayed along the planned travel path beyond the singular point.
The display control device according to claim 3. The correct answer object is a plurality of rod-shaped objects erected on the road along the planned travel route before and after the singular point.
The display control device according to claim 2. The correct answer object is an object that covers the entire display area of the HUD.
The display brightness of the portion of the object that overlaps the planned travel route ahead of the singular point is higher than the display brightness of the portion that overlaps the road that is not the planned travel route ahead of the singular point.
The display control device according to claim 2. When the gaze point coincides with a road other than the planned travel route ahead of the singular point, the object creation unit is an incorrect answer indicating that the gaze point of the driver is not the planned travel route. Create an object,
The display control device according to claim 1. The incorrect answer object is displayed so as to overlap on the road of the gazing point.
The display control device according to claim 9. The incorrect answer object is an icon,
The display control device according to claim 10. When the gazing point coincides with a road other than the planned travel route ahead of the singular point, the object creation unit creates a correct answer notification object which is the object representing the planned travel route ahead of the singular point. ,
The correct answer notification object is displayed superimposed on the planned travel route ahead of the singular point.
The display control device according to claim 1. The correct answer notification object is a linear object displayed along the planned travel route beyond the singular point.
The display control device according to claim 12. When the gazing point coincides with a road other than the planned travel route ahead of the singular point, the object creation unit indicates the object representing the planned travel route ahead of the singular point in addition to the incorrect answer object. Create a correct answer notification object that is
The correct answer notification object is displayed superimposed on the planned travel route ahead of the singular point.
The display control device according to claim 9. The information acquisition unit acquires the information of the gesture from the gesture recognition device that recognizes the gesture of the driver of the vehicle.
The display control unit causes the object to be displayed on the HUD based on the information of the gesture.
The display control device according to claim 1. The information acquisition unit acquires the voice of the driver of the vehicle from the microphone mounted on the vehicle, and obtains the voice of the driver of the vehicle.
The display control unit causes the object to be displayed on the HUD based on the spoken voice.
The display control device according to claim 1. An information acquisition unit that acquires vehicle position information, line-of-sight information that is information on the line-of-sight direction of the driver of the vehicle, and map data.
A gaze point recognition unit that recognizes the gaze point of the driver in front of the vehicle based on the position information, the line of sight information, and the map data.
An object creation unit that creates an object for route guidance based on the difference between the gaze point and the specific planned travel route of the vehicle.
A display control unit for displaying the object on the HUD mounted on the vehicle is provided.
The specific travel route of the vehicle is a road that is visually recognized discontinuously from the road on which the vehicle is currently traveling among the planned travel routes of the vehicle.
Display control device. The position information of the vehicle, the planned travel route of the vehicle, the line-of-sight information which is the information of the line-of-sight direction of the driver of the vehicle, and the map data are acquired.
Based on the position information, the line-of-sight information, and the map data, the gaze point of the driver in front of the vehicle is recognized.
An object for route guidance is created based on the difference between the gaze point and the planned travel route ahead of the singular point which is a branch point or an intersection of the travel road in front of the vehicle.
Displaying the object on the HUD mounted on the vehicle,
Display control method.

Images