JP2014174879A - Information processor and information program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To select a guide object intended by a driver from among a plurality of guide objects.SOLUTION: A navigation device 1 is configured to photograph a scene in the visual field of a driver by a front camera 31, and to perform the image recognition of a guide image from a photographed visual field image, and to observe the visual line of the driver by a visual line camera 32. Then, the navigation device 1 is configured to specify a guide object focused by the visual line by collating the visual line with the guide object, and to display guide information corresponding to the guide object such that it is superimposed on the visual field of the driver by an HUD(Head-Up Display) 45. Then, the navigation device 1 is configured to, when a plurality of guide objects are overlapped, detect the change of the positions of the eyes of the driver. For example, when the guide object overlapped with the left front side is intended by the driver, the driver is expected to peek into the guide object by moving the head to the left side. Therefore, the navigation device 1 is configured to, when the driver moves the eyes to the left side, select the guide object overlapped on the left side.

Description

  The present invention relates to an information providing apparatus and an information providing program, and relates to, for example, an apparatus that provides information to a driver of a vehicle.

Conventional vehicle navigation devices use a small touch panel screen installed near the driver's seat, in-car speakers, microphones, etc. to accept input from the driver (driver), directions, road information, commercial facility information Or provide guidance information.
In recent years, attempts have been made to provide information using a head-up display, as in “Vehicle Information Display Device” of Patent Document 1.
This technique detects a driver's line of sight and projects guidance information for a guidance target object such as a building in the line of sight on a windshield or the like.
With this technology, the driver can obtain desired guidance information while looking at the foreground without looking at the touch panel screen.

  By the way, in the technique described in Patent Document 1, the driver's line of sight functions as a remote control means for designating a guidance object. For example, the guidance objects at the line of sight overlap or approach each other. When a plurality of guidance objects are guidance candidates, there is a problem that it is impossible to determine which is the guidance object intended by the driver.

JP 2006-242859 A

  An object of the present invention is to select a driver's intention from a plurality of guidance objects.

(1) In the first aspect of the present invention, the image acquisition means for acquiring an image corresponding to the field of view of the driver of the vehicle, the recognition means for recognizing the guidance object shown in the acquired image, and the driving Gaze detection means for detecting the gaze of the person, and specifying means for identifying the guidance object in the direction of the detected gaze by associating the recognized guidance object with the detected gaze in the acquired image. When there are a plurality of the specified guidance objects and a part of the plurality of guidance objects is overlapped with the acquired image, the overlapping of other guidance objects with respect to any one of the guidance objects Based on the overlapping direction setting means for setting the direction, the movement detecting means for detecting the moving direction of the position of the eyes of the driver, the set overlapping direction, and the moving direction of the detected eye position, Identified Providing information providing apparatus characterized by comprising a selection means for selecting any one of the guidance object number.
(2) In the invention described in claim 2, the movement detecting means further detects the amount of movement of the eye, and uses the movement of the vehicle and at least one of the distance to the guidance object. Threshold selection means for dynamically setting a threshold value for the amount, and the selection means, when the detected movement amount of the eye exceeds a predetermined threshold value, the set overlapping direction and the detected movement direction The information providing apparatus according to claim 1, wherein the guidance object is selected based on the information.
(3) In the invention according to claim 3, the overlapping direction setting means overlaps other guidance countermeasure objects based on the guidance object in the foreground among the plurality of overlapping guidance objects. The information providing apparatus according to claim 1 or 2, wherein a direction is set.
(4) In the invention according to claim 4, an image acquisition function for acquiring an image corresponding to the field of view of the driver of the vehicle, a recognition function for recognizing a guidance object in the acquired image, and the driving A line-of-sight detection function for detecting a person's line of sight, and a specifying function for associating the recognized guidance object with the detected line of sight in the acquired image to identify a guidance object in the direction of the detected line of sight When there are a plurality of the specified guidance objects and a part of the plurality of guidance objects is overlapped with the acquired image, the overlapping of other guidance objects with respect to any one of the guidance objects Based on the overlapping direction setting function for setting the direction, the movement detecting function for detecting the moving direction of the position of the eyes of the driver, the set overlapping direction, and the moving direction of the detected eye position, Identified Providing information providing program characterized by implementing the selection function of selecting any one of the guidance object number, with the computer.

  According to the present invention, an appropriate item can be selected from a plurality of candidates for guidance objects.

It is a figure for demonstrating the outline | summary of a navigation apparatus. It is a system configuration figure of a navigation device. It is a figure for demonstrating the functional structure of a navigation apparatus. It is a figure for demonstrating the process at the time of overlap. It is a figure for demonstrating the 1st method of the process at the time of approach. It is a figure for demonstrating the 2nd method of the process at the time of approach. It is a figure for demonstrating the 3rd method of the process at the time of approach. It is a flowchart for demonstrating guidance information provision processing. It is a flowchart for demonstrating imaging | photography and image recognition processing. It is a flowchart for demonstrating the process at the time of overlap. It is a flowchart for demonstrating the procedure of the process at the time of approach (1st method). It is a flowchart for demonstrating the procedure of the process at the time of approach (2nd method). It is a flowchart for demonstrating the procedure of the process at the time of approach (3rd method).

(1) Outline of Embodiment The navigation device 1 (FIG. 1) captures a scene in the driver's field of view with the front camera 31, and recognizes a guide image object from the captured visual field image. Furthermore, the navigation device 1 observes the driver's line of sight with the line-of-sight camera 32.
The navigation device 1 then collates the line of sight with the guidance object, identifies the guidance object having the focus of the line of sight, and displays guidance information for this as a head-up display (hereinafter simply referred to as HUD) 45. Is superimposed on the driver's field of view.

For example, if there are multiple guidance objects that overlap or approach (close) in the line of sight, and it is difficult to determine the driver's intention from the line of sight, perform overlap processing and approach processing, It is estimated which guidance object the driver intends, selects a thing that is likely to be intended by the driver, and outputs guidance information.
The overlapping process and the approaching process both use natural physical reactions based on the driver's intention.

  The overlapping process detects a change in the position of the driver's eyes when a plurality of guidance objects overlap. For example, when the driver intends a guidance object that is overlapped in the left front, it is expected that the driver moves the head to the left side and tries to look into the guidance object. Therefore, when the eye moves to the left side, the navigation device 1 selects a guidance object that overlaps the left side.

In the approaching process, when a plurality of guidance objects are approaching, the driver's line of sight is guided to the intended guidance object using an animation (moving image) displayed on the HUD 45. That is, when a plurality of guidance objects are approaching, by moving or reducing the area corresponding to each guidance object, the driver's line of sight is guided, and whether the line of sight moves to the guided guidance object. The guidance object intended by the driver is selected depending on whether or not it is present.
For example, when two guidance objects are approaching in front, if an image is displayed as an animation that separates them, the driver's line of sight is expected to go toward the intended guidance object. Therefore, the navigation apparatus 1 selects the guidance object ahead of the driver's line of sight. There are two other forms of approach processing.

  Further, when there are a plurality of guidance objects (including duplication and approach), the guidance object intended by the driver is selected by measuring the gaze time for the guidance object. At this time, the guidance object intended by the driver can be more accurately selected by guiding the driver's line of sight by moving or reducing the area corresponding to each guidance object.

(2) Details of Embodiment FIG. 1 is a diagram for explaining an outline of the navigation device 1 of the present embodiment.
The front camera 31 is installed toward the front of the vehicle, and continuously captures a field image in front of the vehicle as viewed from the driver (user of the navigation device 1).
On the other hand, the line-of-sight camera 32 is installed toward the driver's head, and monitors the driver's line of sight.

The information processing apparatus 20 recognizes the field-of-view image captured by the front camera 31 and identifies guidance objects such as buildings, vehicles, intersections, and pedestrians.
Furthermore, the information processing device 20 specifies the guidance object intended by the driver by associating the driver's line of sight with the guidance object.
Then, the information processing apparatus 20 outputs guidance information regarding the identified guidance object to the HUD 45.
In response to this, the HUD 45 superimposes and displays the guidance information on the scene viewed by the driver.

In this way, the navigation device 1 uses the two cameras to, for example, pre-read from the driver's line of sight what happens when turning the intersection ahead, and notifies the driver of the landmark with the HUD 45 or voice. When he asks "What is that?", He understands (guesses) what the driver is seeing, such as notifying him of "Sky Tower." From the line of sight, it is possible to provide a service that pre-reads the driver's intention.
It is also possible to alert the driver when he / she is not looking at an object that the driver should be careful about, such as a pedestrian walking in a direction in which the driver's line of sight is not facing.

FIG. 2 is a system configuration diagram of the navigation device 1 used in the present embodiment.
The navigation device 1 is mounted on a vehicle and includes a current position detection device 10, an information processing device 20, a photographing device 30, an input / output device 40, and a storage device 50.

The current position detection device 10 includes an orientation sensor 12, a distance sensor 13, a GPS receiver 14, a geomagnetic sensor 15, a vibration sensor (not shown), and the like.
The azimuth sensor 12 is a means for detecting an angle that has changed relative to a reference angle (absolute azimuth), and uses, for example, a gyro sensor that detects a change in angle using an angular velocity.
An optical rotation sensor attached to the rotating part of the handle, a rotary resistance volume, or an angle sensor attached to the wheel part may be used.
The geomagnetic sensor 15 is a means for detecting an absolute azimuth, and detects which direction the vehicle is directed from the detection in the N direction based on the magnet.
The navigation apparatus 1 can determine the forward direction (that is, the traveling direction) of the host vehicle based on the direction detected by the direction sensor 12 or the geomagnetic sensor 15.

The distance sensor 13 is a means that can measure the moving distance of the vehicle, and for example, a sensor that detects and counts the rotation of a wheel or a sensor that detects acceleration and integrates it twice.
Although not shown, the vibration sensor detects the vibration of the vehicle.
A GPS (Global Positioning System) receiver 14 receives signals from a plurality of GPS satellites orbiting the earth. This signal includes time information, and the information processing device 20 processes the time information, whereby the navigation device 1 can obtain the position information of the host vehicle.

The information processing device 20 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 24, a clock 28, a sensor input interface 23, a communication interface 25, and the like.
The information processing device 20 performs various calculations for navigation based on information input from the current position detection device 10, the imaging device 30, the input / output device 40, and information stored in the storage device 50, and a display 42. , HUD45, speaker 44 and the like are controlled, and calculation results are output to them.

  The CPU 21 is a central processing unit that performs transmission / reception of information via the transmission path 26 (bus line) and executes programs stored in the ROM 22 and the storage device 50, so that the overall navigation device 1 is generalized. Performs proper calculation and control. In the present embodiment, a guidance object is specified from the driver's line of sight, and guidance information related to this is output from the HUD 45, the speaker 44, or the like.

The ROM 22 is a read-only storage device, and stores a basic program for operating the information processing apparatus 20. Further, the navigation program 51 stored in the storage device 50 can be stored here.
The ROM 22 may be divided into a first ROM and a second ROM, and a navigation program related to voice guidance may be stored in the second ROM, and other programs may be stored in the first ROM.

The RAM 24 provides a working memory when the CPU 21 performs various calculations and controls. The CPU 21 uses the memory space of the RAM 24 to perform image recognition processing or when there are a plurality of guidance target candidates to be described later. Or select this.
The RAM 24 stores information input by the user (driver and other passengers) such as destination information and passing point information input by the input device 41 and based on user input information. The result calculated by the CPU 21, the result of the route search, or the map information read from the storage device 50 is stored.

The sensor input interface 23 is an interface connected to various sensors such as the azimuth sensor 12, the distance sensor 13, the geomagnetic sensor 15, and the vibration sensor.
The communication interface 25 is an interface connected to various peripheral devices via the transmission path 27.
Specifically, the GPS receiver 14, the input device 41, the storage device 50, the front camera 31, the line-of-sight camera 32, and the like are connected via the transmission path 27.

The clock 28 is a transmitter configured using, for example, a crystal resonator, and provides operation timing of each part of the navigation device 1. The CPU 21 can also measure a gaze time, which will be described later, or obtain the current time using the clock 28.
In addition, for the information processing apparatus 20, image information dedicated to image processing for processing vector information processed by the CPU 21 into image information by the display 42 or the HUD 45, and image information processed by the image processor. An audio processor dedicated to audio processing for processing the audio information read from the image memory to be stored and the storage device 50 and outputting it to the speaker 44 may be provided.

The imaging device 30 includes a front camera 31 and a line-of-sight camera 32 each using a CCD (Charge-Coupled Device) camera or the like.
The front camera 31 continuously captures a scene viewed by the driver in the traveling direction at a predetermined sampling rate and outputs the captured scene to the information processing apparatus 20. The visual field image by the front camera 31 corresponds to the visual field that the driver obtains during driving.
The line-of-sight camera 32 continuously captures the head including the driver's eyes at a predetermined sampling rate and outputs the image to the information processing apparatus 20. From this image, the driver's line-of-sight information (information specifying the point of view of the driver, such as the line-of-sight vector, focus coordinate value, and viewpoint position) can be obtained, and the focus by the line of sight on the field-of-view image can be calculated. .
In the present embodiment, the front camera 31 and the line-of-sight camera 32 capture visible light, but a special camera according to the purpose, such as an infrared camera, may be used.

The input / output device 40 includes an input device 41, a display 42, a HUD (Head-Up Display) 45, a speaker 44, a microphone 46, and the like.
The input device 41 includes, for example, a touch panel, a touch switch, a joystick, a key switch, and the like, and receives input of data such as a destination, a passage point, and a search condition from a user, for example.
The display 42 is a small display installed in the vicinity of the driver's seat. The display 42 is composed of, for example, a liquid crystal display, and a guidance screen such as a map around the current location and a travel route to the destination, and data displayed by the user. Displays an image such as an input screen for inputting.

The HUD 45, for example, projects the image on the windshield using a projector, or displays the image on a transparent screen installed on the windshield, thereby superimposing the guidance information on the driver. provide.
The speaker 44 provides voice guidance information to the driver, for example, “Turn right 30 meters ahead”.
The microphone 46 receives input from the user by voice. For example, when the user utters “XX station” in the destination setting, XX station is input as the destination.

The storage device 50 includes a large-capacity storage medium and a drive device that drives the storage medium.
As the storage medium, for example, a DVD-ROM or CD-ROM that is an optical storage medium, a hard disk that is a magnetic storage medium, or the like can be used. The storage medium includes various information storage media such as a magneto-optical disk and various semiconductor memories. You can also
Information that needs to be rewritten may be constituted by a rewritable hard disk or flash memory, and other fixed information may be a ROM such as a CD-ROM or DVD-ROM.

The storage device 50 stores a navigation program 51, a map DB (database) 52, a guidance information DB 54, a traffic information DB 55, an image recognition DB 56, and the like.
The navigation program 51 is a program for causing the CPU 21 to exhibit functions for performing various calculations and controls necessary for navigation.

The map DB 52 is a database composed of 2D map data, 3D map data, and the like.
In the two-dimensional map data, two-dimensional information necessary for route search and route guidance is registered.
In the 3D map data, in addition to normal map information, 3D information of features such as the shape and height of structures such as buildings and signals, and the height of topography is registered.
Based on this three-dimensional information, a virtual simulated image corresponding to the visual field image is calculated.
Further, in the 3D map data, attributes (buildings, intersections, etc.) for collating with the image recognition result are stored corresponding to the feature objects.

The guide information DB 54 is a database in which guide information for structures such as buildings and intersections included in 2D map data and 3D map data is registered.
The guide information is composed of, for example, the name of the structure, detailed information of the structure (name of facility occupying the building, number of floors, genre, etc.), and is displayed on the HUD 45.

The traffic information DB 55 is a database that stores traffic information necessary for route search and route guidance, for example, that a certain section of a road is disconnected.
The image recognition DB 56 is a database in which a pattern for extracting and recognizing a guidance target object by pattern recognition from a visual field image captured by the front camera 31 is registered.

More specifically, the image recognition DB 56 stores an image pattern that is a feature of a building, an image pattern that is a feature of an intersection, an image pattern that is a feature of a vehicle, and the like. The apparatus 20 matches this with the visual field image, and extracts and recognizes guidance objects such as buildings and intersections in the visual field image.
In addition, extraction is to extract a portion having a matching pattern from the field image, and recognition is, for example, a building, a bridge, an intersection, or a vehicle. It is to understand that.

FIG. 3 is a diagram for explaining a functional configuration of the navigation device 1.
Each of these functions is realized by executing the navigation program 51 by the CPU 21 of the information processing apparatus 20.
The image recognition function unit 60 captures a field-of-view image corresponding to the driver's field of view (field of view in front of the vehicle), and recognizes the image as a front camera 31, a photographing unit 61, an image processing recognition unit 62, The image recognition DB 56 is used.
Among these, the imaging unit 61 and the image processing recognition unit 62 are configured by executing the navigation program 51.

The photographing unit 61 controls the front camera 31 to photograph the front of the vehicle and generates a visual field image.
Thus, the imaging unit 61 functions as an image acquisition unit that acquires an image corresponding to the field of view of the driver of the vehicle.
The image processing recognizing unit 62 performs image processing for extracting the guidance object from the visual field image by collating the visual field image generated by the photographing unit 61 with the pattern registered in the image recognition DB 56 (pattern recognition).
Then, the image processing recognition unit 62 recognizes what the extracted guidance object is (for example, a vehicle, a bridge, a building, a traffic light, an intersection, a pedestrian, etc.), and outlines the recognized guidance object. After that, enclose it with a rectangle.
As described above, the image processing recognition unit 62 functions as a recognition unit that recognizes the guidance object in the acquired image.

This rectangle is a determination area for determining whether or not the driver is looking at the guidance object. In the navigation device 1, when the driver's line of sight is within the rectangle, the driver is surrounded by the rectangle. It is determined that the target object is being viewed.
Note that the rectangular determination area is merely an example, and may be any area that surrounds (includes) the guidance object, such as a circle, an ellipse, or a polygon.

  When the image processing recognizing unit 62 generates a rectangle surrounding the guidance target object in this way, the rectangular coordinate value (the coordinate value specifying the area surrounded by the rectangle, the coordinate value of the four vertices of the rectangle) and the recognition result (Building, vehicle, intersection, etc.) is output to the line-of-sight information service unit 65 as guidance object information.

  The line-of-sight recognition function unit 63 includes a line-of-sight camera 32, a line-of-sight management unit 64, a line-of-sight information service unit 65, and the like. Among these, the line-of-sight management unit 64 and the line-of-sight information service unit 65 are configured by executing the navigation program 51.

The line-of-sight management unit 64 recognizes the eye part from the image of the driver's head photographed by the line-of-sight camera 32, and calculates eyeball center coordinates from the curvature of the eyeball shape. Further, the line-of-sight management unit 64 specifies a black eye (pupil) region from the eye, and calculates the black eye (pupil) center coordinates.
The line-of-sight management unit 64 defines the direction from the eyeball center coordinates thus obtained to the black-eye center coordinates as the line-of-sight direction, and outputs the eyeball center coordinates and the line-of-sight direction to the line-of-sight information service unit 65 as line-of-sight information.
In this way, the line-of-sight management unit 64 functions as a line-of-sight detection unit that detects the driver's line of sight.

This processing can be performed for one eye, but may be performed for both eyes and the results may be averaged into one.
Even when the driver has a blind eye, the pupil can be black, so this method can be used.
In addition, you may use other devices, such as using other methods, such as a cornea reflection method and a scleral reflection method using infrared rays, or mounting | wearing a driver | operator's eyes.

The line-of-sight information service unit 65 outputs information from the HUD 45 and the speaker 44 based on the rectangle, the recognition result, and the driver's line-of-sight information as follows.
First, the line-of-sight information service unit 65 receives guidance object information from the image recognition function unit 60 and receives line-of-sight information from the line-of-sight management unit 64.

Then, the line-of-sight information service unit 65 calculates the coordinate value of the focus of the line of sight (coordinate value of the line-of-sight destination in the visual field) from the line-of-sight information, and identifies the rectangle with the focus.
Since the guidance object is specified by the focused rectangle, the line-of-sight information service unit 65 associates the recognized guidance object with the detected line of sight in the acquired image, and is in the direction of the detected line of sight. It functions as a specifying means for specifying.

The line-of-sight information service unit 65 determines the driver's intention by the method described later when it is difficult to identify the guidance object intended by the driver due to overlapping or approaching a plurality of rectangles. Is selected, and one is selected from a plurality of guidance objects.
Thus, the line-of-sight information service unit 65 functions as a selection unit that selects any one of the plurality of guidance objects when there are a plurality of identified guidance objects.

Further, the line-of-sight information service unit 65 obtains the current position and the traveling direction from the navigation function unit 68, and uses the three-dimensional map data stored in the map DB 52 to generate a simulated image of the driver's line-of-sight direction using the ray tracing method. It is generated using a technique such as
The simulated image is a virtual image obtained by using a three-dimensional map data of a scene image corresponding to the visual field image.

The line-of-sight information service unit 65 collates (matches) the guidance object information (rectangular coordinate values and recognition results) with the coordinate values and attributes of the feature objects in the simulated image.
This collation is performed, for example, by collating a rectangular coordinate value with a coordinate value of a feature object, and further collating a recognition result (such as a building) with an attribute of the feature object (such as a building).
This matching may be performed for all rectangles, or may be performed for some rectangles such as landmarks.
Thereafter, the line-of-sight information service unit 65 acquires the guidance information of the guidance object intended by the driver from the guidance information DB 54 and outputs the guidance information to the HUD display control unit 72 and the voice control unit 73.
Thus, the line-of-sight information service unit 65 functions as an information providing unit that provides the driver with information on the selected guidance object.

The voice recognition function unit 66 includes a microphone 46, a voice recognition unit 67, and the like.
The line-of-sight information service unit 65 can also receive an input by voice from the voice recognition function unit 66.
For example, when the driver turns his gaze toward the guidance object and utters “What is this?”, The microphone 46 inputs this to the speech recognition unit 67, and the speech recognition unit 67 interprets the utterance content.
In response to this, the line-of-sight information service unit 65 outputs “Sky tower” or the like to the HUD display control unit 72 or the voice control unit 73.

The navigation function unit 68 includes a guidance processing unit 69, the GPS receiver 14, each sensor 71, a map DB 52, a traffic information DB 55, a guidance information DB 54, and the like.
Among these, the guidance processing unit 69 is configured by executing the navigation program 51, and each sensor 71 is configured by the azimuth sensor 12, the distance sensor 13, the geomagnetic sensor 15 and the like of FIG.

  The guidance processing unit 69 performs conventional navigation using the display 42 and the speaker 44 such as route search and route guidance, provides 3D map data and guidance information according to the request of the line-of-sight information service unit 65, and GPS The line-of-sight recognition function unit 63 assists in providing information using the HUD 45, such as notifying the current position and the traveling direction by the receiving device 14 or each sensor 71.

  The HUD display control unit 72 and the voice control unit 73 are configured by executing the navigation program 51, and display the guidance information output from the line-of-sight information service unit 65 on the HUD 45 and the speaker 44, respectively.

FIG. 4 is a diagram for explaining the overlapping process when rectangles overlap.
When a plurality of rectangles overlap in the driver's line-of-sight direction, the line-of-sight information service unit 65 uses the following method to estimate what the driver intended and selects a rectangle to be guided To do.

As shown in FIG. 4A, in the field-of-view image 81, since the object B exists on the left front side of the object A, the object A and the object B overlap, and which object the driver 2 is gazing at. Judgment is difficult from the line-of-sight direction.
Thus, when the object B is behind the object A and the driver 2 is gazing at the object B, the driver 2 can expect to move or tilt his head to the left.
Therefore, the navigation device 1 observes the movement of the eye of the driver 2 (for example, movement in a two-dimensional plane perpendicular to the traveling direction), whereby the driver 2 gazes at either the object A or B. Judgment is made.
As described above, the navigation device 1 includes movement detection means for detecting the movement direction of the position of the driver's eyes.

This will be described in more detail with reference to FIG.
It is assumed that the set rectangles A to D overlap in this order from the driver side. Then, it is assumed that the focus 82 of the driver's line of sight is on the rectangle A.
In this case, the navigation device 1 sets the rectangle A with the driver's focus (the one closest to the viewpoint position) as the reference rectangle.
Then, the navigation device 1 extracts another rectangle that directly overlaps the reference rectangle. In the example of the figure, the rectangles B and C overlap the reference rectangle A, and the rectangle D does not overlap. Therefore, the navigation device 1 extracts the rectangles B and C.

Next, the navigation device 1 assigns an overlapping direction to the extracted rectangle from the positional relationship of the extracted rectangle with respect to the reference rectangle.
Here, for simplification, as an example, it is assumed that the overlapping direction is an up-down direction and a left-right direction, and the navigation apparatus 1 assigns the overlapping direction with the larger area of the portion not overlapping the reference rectangle.

In the example shown in the figure, since the area of the rectangle B that does not overlap the rectangle A is larger in the left direction than in the upward direction, the navigation device 1 assigns the left direction to the rectangle B. Similarly, an upward direction is assigned to the rectangle C.
As described above, the navigation device 1 sets the overlapping direction of the plurality of guidance objects when there are a plurality of identified guidance objects and a part of the plurality of guidance objects overlaps the acquired images. Overlapping direction setting means is provided.

After setting the reference rectangle and assigning the positional relationship between the extracted rectangles, the navigation apparatus 1 sets a determination area corresponding to the overlapping direction with the focus of the line of sight as the center, and accompanies movement and rotation of the driver's head. A rectangle in the direction in which the eye movement is equal to or greater than the determination area is selected.
Thus, the navigation apparatus 1 selects a guidance target object based on the set rectangular overlapping direction and the detected eye movement direction.

Note that the size of the determination area is set to be larger as the guidance object is farther away and as the vibration of the vehicle is larger.
This is because the farther away the focal point is, the more difficult the focus is, and the greater the vibration, the more likely the position of the eye to vary.
As described above, the navigation device 1 includes threshold setting means for dynamically setting a threshold for the amount of movement using at least one of the vibration of the vehicle and the distance to the guidance object.
Then, when the detected eye movement amount exceeds a predetermined threshold (determination area), the navigation device 1 selects a guidance object based on the rectangular overlapping direction and the eye movement direction.

  Further, the distance to the guidance object may be calculated from the current position and the position of the guidance object, or since there is three-dimensional data of the size of the guidance object, the size of the guidance object in the view image And the size of the three-dimensional data may be compared.

Here, as an example, it is assumed that the navigation device 1 sets an error circle 83 centered on the focal point 82 as a determination region.
In this case, the navigation device 1 selects the rectangle B as an object intended by the driver when the driver's eyes move to the left beyond the error circle 83, and the rectangle when the driver moves upward. Select C.
Since the head is easier to move in the left-right direction than in the up-down direction, the error circle 83 may be an ellipse that is long in the left-right direction.

As described above, when a plurality of rectangles are overlapped, the navigation device 1 can select a rectangle that is estimated to be intended by the driver as a guidance target.
Further, other means for conveying the driver's will, such as a switch or a gesture (for example, blinking), may be used in combination with the movement of the eye position.

Next, a case where the rectangles are approaching will be described. When a plurality of rectangles are close to each other, there are cases where the driver intends a neighboring rectangle even when a certain rectangle has a focus.
Therefore, when a plurality of rectangles are approaching, the navigation device 1 estimates the driver's intention and selects a rectangle to be guided.
Here, three methods of the approaching process of selecting from the approaching rectangle will be described with reference to FIGS.

Each figure of FIG. 5 is a figure for demonstrating the 1st method of the process at the time of approach. In this method, a rectangle is enlarged and separated, and a change in the driver's line of sight is observed.
As shown in FIG. 5A, consider a case where a rectangle A and a rectangle B (both rectangular regions surrounded by wavy lines) are close to each other.
In this case, the navigation apparatus 1 enlarges the areas of the rectangles A and B so that the driver can easily recognize the reaction area, sets the reaction area (the hatched portion is the enlarged portion), and displays it on the HUD 45.
Thus, when there are a plurality of specified guidance objects, the navigation apparatus 1 displays a region (reaction area) surrounding the plurality of guidance objects for each guidance object in the driver's field of view. It has.

Next, as shown in FIG. 5B, the navigation device 1 connects the midpoints of the rectangles A and B, and continuously animates both reaction regions in the direction away from each other along the straight line. Move to.
Thus, by clearly separating the approaching (approaching) rectangles in a form that can be recognized by the driver, it is possible to clarify each gaze on each rectangle. Further, when the driver intends any one of the objects, it is expected that the viewpoint moves to the intended rectangle along with the animation.
Therefore, the navigation device 1 selects the rectangle on the side on which the driver's line of sight has moved.

Thus, the navigation device 1 includes area moving means for moving the displayed area away from the displayed area, and line-of-sight change acquisition means for acquiring changes in the driver's line of sight when moving these areas. A guidance object is selected based on the change.
The area moving means continuously moves the area by animation from the position before the movement start to the position after the movement.

Note that the reason why the rectangle is gradually moved by animation is that it is easier to guide the driver's line of sight than when the rectangle is moved away.
In addition, in the rectangle selection determination, the larger rectangle may be selected by comparing the sum of the gaze times in each rectangle.

  Here, the dividing distance y that separates the rectangles A and B is, for example, the distance between opposite sides of the rectangle, the distance to the object is x, and the error angle is θ, as shown in FIG. , Y = x × tan θ. Here, the error angle is half of the angle formed by opposing sides of the rectangle as viewed from the driver.

Each figure of FIG. 6 is a figure for demonstrating the 2nd method of the process at the time of approach. In this method, the driver guides the line of sight by reducing the target to which the line of sight is directed.
Assume that there are rectangles A, B, and C that are close to the current focal position, as shown in FIG.
The navigation device 1 superimposes the rectangles A, B, and C, and displays rectangular reaction areas 91, 92, and 93 surrounding the rectangles for observing the driver's reaction on the HUD 45.
The reaction areas 91, 92, and 93 are displayed in a conspicuous state so that the driver can easily see them. It should be noted that the reaction area may be displayed only for the rectangle with the focus of the driver's line of sight.

After displaying the reaction area in this manner, the navigation device 1 determines whether the driver's intention is a currently focused rectangle or not while the driver is viewing the focused reaction area. The animation is gradually and continuously reduced to guide the driver's line of sight to the center of the guidance object.
At that time, it is more effective to make it inconspicuous by changing the color of other reaction areas or rectangles or making them thinner.
That is, the navigation apparatus 1 temporarily selects the reaction area in order to confirm whether the focused reaction area is intended by the driver, and reduces the observation area to observe a change in line of sight.

  As described above, when there are a plurality of specified guidance objects, the navigation device 1 provisionally selects one of the plurality of guidance objects, and an area surrounding the provisionally selected guidance object. A display means for displaying the (reaction area) in the driver's field of view, an area changing means for changing the area of the displayed area (in this example, reduction), and a change in the driver's line of sight when the area changes A line-of-sight change acquisition means is provided.

  Since the reaction area may be reduced as much as necessary, the reaction area is not made smaller than the outline of the guidance target object. The rectangle and the reaction area are managed separately, and the rectangle is not reduced. This is because the rectangle is an area for specifying the guidance object due to the focus of the line of sight, and therefore, when the rectangle is made small, the range for detecting the line of sight is narrowed. The display of the reaction area is reduced in order to guide the line of sight to the last.

When the line of sight is guided as described above, it can be expected that the driver keeps his gaze when the driver intends the guidance target object and shifts the line of sight when the driver does not intend.
Therefore, the navigation device 1 measures the gaze time (the dwell time of the focus of the line of sight) for the rectangle superimposed on the reduced reaction area. The method for measuring the gaze time is the same as the third method described below.
When the gaze time exceeds a predetermined determination time (threshold value), the driver determines that the guidance object is intended, and selects the guidance object as a guidance object.
As described above, the navigation device 1 selects a region when the driver's line of sight stays in the region where the area is changed for a predetermined time or longer (that is, when the driver gazes for a predetermined time or longer).

For example, as shown in FIG. 6B, if the focus of the line of sight is in the rectangle B, the navigation device 1 gradually reduces the reaction area 92 to the reaction area 94 as indicated by the arrow. , Change the other display state to make it inconspicuous.
And the navigation apparatus 1 measures the gaze time with respect to the reaction area | region 94 of a driver | operator, and when this exceeds predetermined determination time, the rectangle B is selected.

Each drawing of FIG. 7 is a diagram for explaining a third method of the approaching process. In this method, the gaze time of the focus of the line of sight with respect to each rectangle is measured.
As shown in FIG. 7A, it is assumed that there are rectangles A, B, and C that are close to each other and the guidance object intended by the driver is unclear.
In this case, the navigation device 1 counts and adds the time that the driver's line of sight stays (stays) in each rectangle over a predetermined determination time, and thus how long the candidate guidance object has been viewed. When the gaze time reaches a predetermined determination time (threshold), the rectangular guide object is selected.

In FIG. 7A, the transition of the focus, which is the direction of the driver's line of sight, is indicated by a point and an arrow line connecting it. For example, as shown in FIG. 7A, it is assumed that the object moves from the rectangle A to the rectangle C from the focal point 101 to the focal point 103.
In this case, the navigation device 1 accumulates the time during which the focal point stays in each rectangle, for example, in milliseconds.

FIG. 7B is a diagram showing an example of the gaze time in FIG.
The left end of the figure is the determination start time, and the right end is the determination time. The gaze times 107, 108, and 109 of the rectangles A, B, and C are shown from the top of the figure, and the gaze time 108 of the rectangle B is the longest.
In this way, the navigation device 1 measures the gaze time in each rectangle, and when it reaches the determination time, selects the rectangular guidance object that has reached the determination time as the guidance object.
As described above, when there are a plurality of specified guidance objects, the navigation device 1 measures the time (gaze time) during which the driver's line of sight stays on the plurality of guidance objects for each guidance object. A means is provided and a guidance object is selected based on the measured time.

Next, the operation of the navigation device 1 configured as described above will be described.
The following information processing operation and control operation are realized by the CPU 21 (FIG. 2) executing the navigation program 51.
FIG. 8 is a flowchart for explaining the guidance information providing process using the line of sight.
First, the navigation apparatus 1 performs a photographing / image recognition process on the visual field image, extracts a guidance object, and understands the extracted guidance object (step 5).
And the navigation apparatus 1 sets the rectangle of the magnitude | size surrounding this with respect to the guidance target object extracted and understood (step 10).

Next, the navigation device 1 determines whether or not the rectangle set in the driver's line-of-sight direction is in a position overlapping another rectangle (step 15).
When the plurality of rectangles are at the overlapping position (step 15; Y), the navigation device 1 performs the overlapping process (step 20) and selects a rectangle to be guided.
Then, the navigation device 1 provides the driver with guidance information for the selected rectangular guidance object on the HUD 45 or outputs it from the speaker 44 (step 25).

On the other hand, when the rectangles are not at the overlapping position (step 15; N), the navigation device 1 determines whether or not the plurality of rectangles are close to each other in the driver's line-of-sight direction (step 35).
The determination of whether or not they are approaching (approaching) determines that they are approaching when the interval between the sides of the rectangle is equal to or less than a predetermined threshold.
When a plurality of rectangles are approaching (step 35; Y), the navigation device 1 performs approach processing (any of methods 1 to 3) (step 40), and guides the selected rectangular guidance object. Information is provided (step 25).
On the other hand, when the plurality of rectangles are not close to each other in the line-of-sight direction (step 35; N), the navigation device 1 provides guidance information for the rectangular guide object in the line-of-sight direction (step 25).

  Thus, after providing the guidance information (step 25), the navigation device 1 determines whether or not to continue the guidance (step 30). If the guidance is to be continued (step 30; Y), step 5 When the guidance is not continued (for example, when the driver stops the guidance by operating the navigation device 1) (step 30; N), the guidance information providing process is terminated.

FIG. 9 is a flowchart for explaining the photographing / image recognition processing in step 5.
The following processing is performed by the image recognition function unit 60 (FIG. 3) of the navigation device 1.
The navigation apparatus 1 acquires a visual field image using the front camera 31 (step 50).
Then, the navigation device 1 cuts out the visual field image in the line-of-sight tracking range (the range in which the line of sight changes), and extracts this (step 55).
This is because the image captured by the front camera 31 and the driver's field of view are misaligned, and the captured image is adjusted so that the range for image recognition processing matches the driver's field of view. Thereafter, the navigation device 1 uses the retrieved field image.

Next, the navigation apparatus 1 performs image processing such as edge extraction processing for feature object extraction on the visual field image (step 60), and extracts a pattern learned in advance using the image recognition DB 56. Image recognition (pattern recognition) is performed by collating the featured features (step 65).
Next, the navigation apparatus 1 understands what the feature with which the pattern matched (for example, a building, an intersection, etc.) (step 70).
Then, the navigation device 1 performs contour processing by contouring the understood feature (step 75), and stores this by enclosing it in a rectangle (step 80). As described above, the guidance object is extracted as a feature object, contoured, and then surrounded by a rectangle.

FIG. 10 is a flowchart for explaining the overlapping processing in step 20.
The following processing is performed by the line-of-sight recognition function unit 63 (FIG. 3) of the navigation device 1.
First, the navigation device 1 sets a rectangle with the driver's current focus, that is, a rectangle closest to the focus position (step 90).
Next, the navigation device 1 provides guidance information for the reference rectangular guidance object among the plurality of overlapping rectangles (step 95).

For information provision, when the driver is viewing the guidance object, the pull type that provides the guidance information of the guidance object, and when the driver is not viewing the guidance object, There is a push type that provides guidance information of guidance objects.
In the present embodiment, a pull type is provided to provide guidance information corresponding to a rectangle at the focus of the driver's line of sight.

Next, the navigation device 1 extracts another rectangle that overlaps the reference rectangle (step 100), and assigns the positional relationship of the extracted rectangle to the reference rectangle (step 105).
Next, the navigation apparatus 1 sets a determination region (for example, an error circle) corresponding to the overlapping direction centered on the focus of the line of sight while considering the vibration of the vehicle, the distance to the guidance object, and the like (step 110). .

After setting the determination area in this way, the navigation apparatus 1 tracks the movement of the driver's eye position by the line-of-sight camera 32, and determines whether or not the eye position exceeds the range of the determination area (step 120). ).
When the eye position exceeds the determination region (step 120; Y), the navigation device 1 selects a positional relation rectangle corresponding to the eye movement direction (step 125).
If the eye position does not exceed the determination area (step 120; N), the navigation device 1 returns to the main routine (flowchart in FIG. 8).

FIG. 11 is a flowchart for explaining the procedure of the approach processing corresponding to the first method.
The following processing is performed by the line-of-sight recognition function unit 63 (FIG. 3) of the navigation device 1 (hereinafter, the same applies to the second method and the third method).
First, the navigation device 1 selects a plurality of approaching rectangles that are candidates for guidance from the current focus position.
Then, by connecting the rectangular center points of the guidance target candidates that are approaching, a separation direction to be separated by division is set (step 140), and further, a division distance y is calculated (step 145).
Next, the navigation device 1 displays the reaction area on the HUD 45 so as to overlap the rectangle, and starts an animation for separating the reaction area by the separation distance y in the separation direction (step 150).

During the animation, the navigation device 1 observes the driver's line of sight with the line-of-sight camera 32 and determines whether or not the line of sight follows the animation (step 155).
When the line of sight follows the animation (step 155; Y), the navigation device 1 selects the rectangle on the side where the focus of the line of sight has moved (step 160), and returns to the main routine.
On the other hand, when the line of sight does not follow (step 155; N), the navigation device 1 determines that the driver is not looking at the target rectangle, deletes the reaction area and the rectangle (step 170), and returns to the main routine. .

FIG. 12 is a flowchart for explaining the procedure of the approaching process corresponding to the second method.
First, the navigation device 1 sets a reaction region for each approaching rectangle and displays it on the HUD 45, and selects a rectangle with the current focus as a guidance candidate from among them (step 180).

Next, the navigation device 1 determines whether or not the line of sight is on the selected rectangle (step 185).
When the focus is on the selected rectangle (step 185; Y), the navigation apparatus 1 gradually reduces the reaction area of the rectangle (step 190) and measures the gaze time in the rectangle (step 195).

Next, the navigation device 1 determines whether or not the gaze time has reached a predetermined determination time within a predetermined time limit (step 200).
When the predetermined determination time has expired (step 200; Y), the navigation device 1 selects the rectangle as a guidance target and returns to the main routine.
On the other hand, when the predetermined determination time has not expired (step 200; N), the navigation device 1 returns to step 185.

In step 185, if the focus is not in the selected rectangle (step 185; N), the navigation device 1 determines whether the focus has moved to another rectangle (step 210).
When the focus moves to another rectangle (step 210; Y), the navigation apparatus 1 newly selects the other rectangle (step 215) and returns to step 185.
On the other hand, when the focal point has not moved to another rectangle (step 210; N), the navigation device 1 determines that the line of sight has moved to an empty place, erases the reaction area from the HUD 45, and returns to the main routine. To do.

FIG. 13 is a flowchart for explaining the procedure of the approaching process corresponding to the third method. Steps similar to those in the second method are denoted by the same step numbers, and the description is simplified.
First, the navigation device 1 selects a rectangle with the current focus as a guidance candidate from the close rectangles (step 180).

Next, the navigation device 1 determines whether or not the line of sight is on the selected rectangle (step 185).
When the focal point is on the selected rectangle (step 185; Y), the navigation device 1 measures the gaze time in the rectangle (step 195).

Next, the navigation device 1 determines whether or not the gaze time has reached a predetermined determination time within a predetermined time limit (step 200).
When the predetermined determination time has expired (step 200; Y), the navigation device 1 selects the rectangle as a guidance target and returns to the main routine.
On the other hand, when the predetermined determination time has not expired (step 200; N), the navigation device 1 returns to step 185.

In step 185, if the focus is not in the selected rectangle (step 185; N), the navigation device 1 determines whether the focus has moved to another rectangle (step 210).
When the focus moves to another rectangle (step 210; Y), the navigation apparatus 1 newly selects the other rectangle (step 215) and returns to step 185.
On the other hand, if the focal point has not moved to another rectangle (step 210; N), the navigation device 1 determines that the line of sight has moved to a place where there is nothing, and returns to the main routine.

According to the embodiment described above, the following effects can be obtained.
(1) When guidance objects overlap or approach each other, it may be difficult to specify a single guidance object as a guidance object depending on the combination of the recognized object and the line of sight. However, according to the navigation device 1, it is possible to select an appropriate guidance object by performing the overlapping process and the approaching process.
(2) According to the overlapping process, the intention can be determined from the intuitive operation of the driver when the guidance objects overlap.
(3) According to the approach processing (the first method and the second method), when the guidance object is approaching, the reaction area is separated or deformed, so that the driver's intuitive The driver's intention can be determined by detecting a change in the line of sight.
(4) According to the approach processing (third method), the driver's intention can be determined by measuring the gaze time for the guidance object.
(5) The navigation device 1 can appropriately select the guidance object intended by the driver by using the driver's line of sight as the remote control means and by using the overlapping process and the approaching process. . Therefore, the driver's intention and behavior are prefetched and processed, and the driver can acquire information on the outside world more intuitively.

The present embodiment can also be configured as follows.
(1) Configuration 1
Image acquisition means for acquiring an image corresponding to the field of view of the driver of the vehicle;
Recognizing means for recognizing a guidance object in the acquired image;
Gaze detection means for detecting the driver's gaze;
A specifying means for specifying the guidance object in the direction of the detected line of sight by associating the recognized guidance object with the detected line of sight in the acquired image;
Guidance means for guiding the driver's line of sight when there are a plurality of the specified guidance objects;
A line-of-sight change acquisition means for acquiring a change in the line of sight of the driver due to the line-of-sight guidance by the guide means;
Selection means for selecting any one of the identified plurality of guidance objects based on a change in the driver's line of sight due to the line of sight guidance of the guidance means;
Information providing means for providing the driver with information on the selected guidance object;
An information providing apparatus comprising:
(2) Configuration 2
The line-of-sight guidance means guides the driver's line of sight by displaying an area surrounding the plurality of guidance objects in the driver's field of view for each of the guidance objects, and changing the displayed area. To
An information providing apparatus according to Configuration 1, characterized in that:
(3) Configuration 3
The line-of-sight guidance means guides the driver's line of sight by moving in a direction away from the displayed area;
The information providing apparatus according to Configuration 2, characterized by:
(4) Configuration 4
The line-of-sight guidance means continuously moves the region from a position before guidance start to a position after movement,
The information providing apparatus according to Configuration 3, characterized in that:
(5) Configuration 5
The line-of-sight guidance means temporarily selects one of the plurality of guidance objects, displays an area surrounding the provisionally selected guidance object in the driver's field of view, and reduces the displayed area. To guide the driver's line of sight,
The information providing apparatus according to Configuration 1, characterized in that:
(6) Configuration 6
The line-of-sight guidance means sequentially changes guidance objects to be temporarily selected.
The information providing apparatus according to Configuration 5, characterized in that:
(7) Configuration 7
During the driver's line-of-sight guidance by the guide means, for each of the plurality of guidance objects, comprises a measuring means for measuring the time that the driver's line of sight stayed,
The selection means selects a guide object whose line of sight has stayed for a predetermined time or more.
The information providing apparatus according to any one of Configurations 2 to 6, wherein the information providing device is configured as described above.
(8) Configuration 8
An image acquisition function for acquiring an image corresponding to the field of view of the driver of the vehicle;
A recognition function for recognizing a guidance object in the acquired image;
A gaze detection function for detecting the gaze of the driver;
In the acquired image, a specific function for associating the recognized guidance object with the detected line of sight to identify the guidance object in the direction of the detected line of sight;
A guidance function for guiding the driver's line of sight when there are a plurality of the specified guidance objects;
A line-of-sight change acquisition function for acquiring a change in the line of sight of the driver due to the line-of-sight guidance by the guidance function;
A selection function for selecting any one of the identified plurality of guidance objects based on a change in the driver's line of sight due to the line of sight guidance of the guidance function;
An information providing function for providing the driver with information on the selected guidance object;
An information providing program characterized by realizing the above with a computer.

Further, the embodiment can be configured for the following purposes.
(11) Configuration 11
Image acquisition means for acquiring an image corresponding to the field of view of the driver of the vehicle;
Recognizing means for recognizing a guidance object in the acquired image;
Gaze detection means for detecting the driver's gaze;
A specifying means for specifying the guidance object in the direction of the detected line of sight by associating the recognized guidance object with the detected line of sight in the acquired image;
When there are a plurality of the specified guidance objects, a measuring unit that measures the time that the driver's line of sight stays in the plurality of guidance objects for each guidance object;
Selection means for selecting any one of the specified plurality of guidance objects based on the measured time for each guidance object;
Information providing means for providing the driver with information on the selected guidance object;
An information providing apparatus comprising:
(12) Configuration 12
Guidance means for guiding the driver's line of sight when there are a plurality of the specified guidance objects;
The information providing device according to Configuration 11, further comprising:
(13) Configuration 13
The selection means selects a guidance object whose line of sight has stayed for a predetermined time or more during the driver's line of sight guidance by the guidance means.
13. The information providing device according to configuration 12, wherein
(14) Configuration 14
An image acquisition function for acquiring an image corresponding to the field of view of the driver of the vehicle;
A recognition function for recognizing a guidance object in the acquired image;
A gaze detection function for detecting the gaze of the driver;
In the acquired image, a specific function for associating the recognized guidance object with the detected line of sight to identify the guidance object in the direction of the detected line of sight;
When there are a plurality of the specified guidance objects, a measurement function for measuring, for each guidance object, the time that the driver's line of sight stays in the plurality of guidance objects;
A selection function for selecting any one of the specified plurality of guidance objects based on the measured time for each guidance object;
An information providing function for providing the driver with information on the selected guidance object;
An information providing program characterized by realizing the above with a computer.

DESCRIPTION OF SYMBOLS 1 Navigation apparatus 2 Driver | operator 10 Current position detection apparatus 12 Direction sensor 13 Distance sensor 14 GPS receiver 15 Geomagnetic sensor 20 Information processing apparatus 28 Clock 30 Image pick-up device 31 Front camera 32 Line-of-sight camera 40 Input / output device 41 Input device 42 Display 44 Speaker 45 HUD
46 Microphone 50 Storage device 51 Navigation program 52 Map DB
54 Guidance Information DB
55 Traffic Information DB
56 Image recognition database
60 Image Recognition Function Unit 61 Image Capture Unit 62 Image Processing Recognition Unit 63 Gaze Recognition Function Unit 64 Gaze Management Unit 65 Gaze Information Service Unit 66 Voice Recognition Function Unit 67 Voice Recognition Unit 68 Navigation Function Unit 69 Guidance Processing Unit 71 Each Sensor 73 Voice Control Part 81 Field of view 82 Focus 83 Error circle 91-94 Reaction area 101, 103 Focus 107-109 Gaze time

Claims (4)

Image acquisition means for acquiring an image corresponding to the field of view of the driver of the vehicle;
Recognizing means for recognizing a guidance object in the acquired image;
Gaze detection means for detecting the driver's gaze;
A specifying means for specifying the guidance object in the direction of the detected line of sight by associating the recognized guidance object with the detected line of sight in the acquired image;
When there are a plurality of the specified guidance objects and a part of the plurality of guidance objects overlaps with the acquired image, the overlapping direction of the other guidance objects with respect to any one of the guidance objects Overlapping direction setting means for setting
Movement detection means for detecting the movement direction of the position of the driver's eyes;
Selection means for selecting any one of the specified plurality of guidance objects based on the set overlapping direction and the detected movement direction of the eye position;
An information providing apparatus comprising: The movement detection means further detects the movement amount of the eye,
Threshold value setting means for dynamically setting a threshold value for the amount of movement using at least one of the vibration of the vehicle and the distance to the guidance object;
The selecting means selects the guidance object based on the set overlapping direction and the detected moving direction when the detected movement amount of the eye exceeds a predetermined threshold value. The information providing apparatus according to claim 1. The overlapping direction setting means sets the overlapping direction of other guidance countermeasure objects, with the guidance object in front of the plurality of overlapping guidance objects as a reference,
The information providing apparatus according to claim 1 or 2, characterized in that An image acquisition function for acquiring an image corresponding to the field of view of the driver of the vehicle;
A recognition function for recognizing a guidance object in the acquired image;
A gaze detection function for detecting the gaze of the driver;
In the acquired image, a specific function for associating the recognized guidance object with the detected line of sight to identify the guidance object in the direction of the detected line of sight;
When there are a plurality of the specified guidance objects and a part of the plurality of guidance objects overlaps with the acquired image, the overlapping direction of the other guidance objects with respect to any one of the guidance objects Overlap direction setting function to set
A movement detection function for detecting a movement direction of the position of the eyes of the driver;
A selection function for selecting any one of the identified plurality of guidance objects based on the set overlapping direction and the movement direction of the detected eye position;
An information providing program characterized by realizing the above with a computer.

Images