JPH11288341A - Device and method for navigation - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect an object, which is the object of the interest of a user, based on the line-of-sight direction of the user and to output information corresponding to this object. SOLUTION: Map information is stored in a map data storage part 104. A user position detecting part 101 and a user line-of-sight direction detecting part 102 respectively detect the viewpoint position and line-of-sight direction of the user on a map in the map information. When the expansion in the pupil area of the user is detected by a user pupil area detecting part 106, with the detected viewpoint position as a starting terminal, a control part 103 searches and extracts an object on a line segment along with the line-of-sight direction out of the map information. Then, a voice synthesizing part 105 outputs information concerning the extracted object in voice.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a navigation apparatus and method for presenting a description of an object that the user is interested in viewing to the user.

[0002]

2. Description of the Related Art In a conventional navigation apparatus, a user is navigated by displaying a map on a display. In this type of navigation device, the only means of communicating the user's intention is to give an instruction through a touch panel on a display or an operation panel provided separately from the display.

[0003]

Therefore, a display for displaying a map is indispensable. Also, if the user wants to output a description of an object of interest, there is no other way than to use a means such as designating a position corresponding to the object on a map. That is, in the conventional navigation device, it is not possible to immediately output the explanation according to the user's interest or interest in the object that the user is looking at.

The present invention has been made in view of the above problems, and has as its object to detect an object of interest to a user based on the direction of the user's line of sight, and to deal with this. It is an object of the present invention to provide a navigation device and a method capable of outputting information.

[0005]

To achieve the above object, a navigation device according to one aspect of the present invention has, for example, the following configuration. That is, storage means for storing map information, detection means for detecting a user's viewpoint position and a line-of-sight direction on a map based on the map information, and a line segment along the line-of-sight direction starting from the viewpoint position. A search means for searching for and extracting an object located in the map information from the map information, and an output means for outputting information on the object extracted by the search means.

A navigation device according to another aspect of the present invention has, for example, the following configuration. That is,
Storage means for storing map information including information obtained by dividing the map into a plurality of partial areas; and
Detecting means for detecting a viewpoint position and a line-of-sight direction of a user; searching means for searching for and extracting a partial area on a line segment starting from the viewpoint position along the line-of-sight direction from the map information; And output means for outputting a place name corresponding to the partial area extracted in (1).

According to another aspect of the present invention, there is provided a navigation method for achieving the above object, and a storage medium storing a control program for realizing the navigation method by a computer.

[0008]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a block diagram showing a configuration example of the navigation device according to the present embodiment. This navigation device performs navigation by voice synthesis. 10
Reference numeral 1 denotes a user position detection unit which detects a user position by a known method such as GPS. Reference numeral 102 denotes a user's line-of-sight direction detection unit, which detects the user's line-of-sight direction by a known method.
A control unit 103 includes a CPU, a ROM, and a RAM, and implements various controls in the navigation device. A control program for realizing the control described later with reference to the flowchart is stored in the ROM,
Performed by U. The RAM provides a work area when the CPU executes processing based on various control programs.

Reference numeral 104 denotes a map data storage unit, which stores information relating to a spatial region and objects in the spatial region, which will be described later. That is, the map data storage unit 104 stores an area database (FIG. 4), a coordinate database (FIG. 5),
An attribute database (FIG. 6) is registered. The map data storage unit 104 is configured by an external storage device such as a CD-ROM, a floppy disk, or a hard disk. Reference numeral 105 denotes a voice synthesis unit which generates and outputs a voice waveform under the control of the control unit 103. 1
A user pupil area detection unit 06 detects the area of the user's pupil.

FIG. 2 is a diagram showing an example of a field of view observed by a user wearing the navigation device.
Here, there is a group of triangular (triangular pyramid) buildings in front of the user. FIG. 3 is a diagram showing a line-of-sight position and direction of the user's field of view shown in FIG. 2 as viewed from above. Note that the object may be represented using any form of constituent element such as a quadrangular prism other than the triangular pyramid.

In FIG. 3, Si represents a two-dimensional plane Si that divides a space region or space on the earth. A point characterizing an object on the earth (hereinafter, referred to as a space area Si)
If a certain number (for example, 100 or more) of feature points exists, the spatial area Si is divided into a spatial area Si0 and a spatial area Si1. The spatial area Sio is [Vi] in the spatial area Si.
A region that satisfies [X] −Wi ≧ 0 and is a spatial region Si1
Is a region satisfying [Vi] · [X] −Wi <0 in the space region Si. Here, [x] indicates that x is a vector amount. Here, [Vi] = (ai, b
i, ci), [X] = (x, y, z), and [Vi] and Wi are parameters representing the two-dimensional plane Si, and [Vi]
[X] of [X] −Wi ≧ 0 is a spatial area divided by the plane Si. The division of the plane space is performed by
This process is performed until the number of feature points that characterize an object on the earth in i becomes equal to or smaller than a certain number (for example, 100). Also,
The space is divided finely where the density of objects on the earth is high and coarsely where the density is low. As a method of division, for example, division is repeated so that the number of feature points becomes substantially equal.

FIG. 4 is a diagram showing a state in which the objects (feature points) on the earth shown in FIG. 3 are registered in the area database in the map data storage unit 104. Here, if Si is a non-terminal node, the node Si has a plane, [Vi]
Values ([Vi], Wi) characterizing [X] −Wi = 0 are registered. When Si is a terminal node, the three-dimensional position coordinates ([E1], [E2], etc.) of points characterizing an object existing in the area are registered in the node Si.

FIG. 5 is a diagram showing an example of the data structure of a coordinate database indicating what objects on the earth each point registered in the area database of FIG. 4 constitutes. In the coordinate database shown in FIG. 5, the name of the object can be searched using the coordinates as a key. The coordinate database is in the storage unit 104.

FIG. 6 shows an example of the data structure of an attribute database in which a set of triangular patches having the shape of the outer shape of the object is registered for the name of the object. The description of the object is also registered in the attribute database. The attribute database exists in the map data storage unit 104.

FIG. 7 is a diagram showing details of the configuration of the pupil area detection unit 106. The pupil area detection unit 106 includes a subject image input unit 1 including a lens, a half mirror 2 that transmits and reflects light input from the subject image input unit 1, and an intensity of light reflected by the half mirror 2. Image detector 4 for inputting light reflected by the user's eyes and transmitting light transmitted through the half mirror 2 to capture an image of the eye, and an image captured by the pupil image capturing unit 4 Area calculation unit 5 for calculating the pupil area from the pupil area, and pupil area correction for correcting the pupil area data calculated by the pupil area calculation unit 5 using the light intensity data detected by the photodetector 3 and the pupil area correction table 61. The pupil area correction section 6 includes a pupil area correction section 6 and a determination section 7 for determining whether the pupil has been enlarged or reduced from the pupil area data. Note that the above-described pupil area detection unit 106 has a spectacle shape.

The operation of the navigation system according to the present embodiment having the above-described configuration will be described with reference to FIGS. FIG. 8 is a flowchart illustrating the operation of the navigation system according to the present embodiment. In addition,
The processing in this flowchart is performed in the control unit 103. FIG. 9 is a diagram showing a line of sight of the user, an area to be extracted, and an object.

First, in step 701, the user's eyeball position and the user's line of sight are determined. In this process,
The user position detection unit 101 obtains the three-dimensional position coordinates and time of the user's eyeball on the earth using information of radio waves from four satellites by a known GPS method. The three-dimensional position coordinates of the user's eyeball on the earth at the time obtained here are [O]. Note that here, GPS
Is used to detect the position of the user's eyeball, but this method has an error of several tens of meters. Therefore, the configuration may be such that the user's eyeball position is detected by DGPS (Differential Global Positioning System). According to DGPS, the error is several cm or less. Also,
The position may be detected using an antenna or the like fixed on the ground. Further, other position detection methods may be used.

In step 701, the user's line-of-sight direction detecting unit 102
Find [n]. The user's gaze direction may be obtained by any method. The detection of the line of sight direction is disclosed in
No. 3 and JP-A-6-82680 can be applied.

Next, in step 702, a change in the pupil area of the user is detected. As a method for detecting a change in the pupil area, for example, Japanese Unexamined Patent Application Publication No. 7-174541
Can be used. In this method, using the configuration shown in FIG. 7, a factor due to the brightness of the object in the viewing direction of the user is removed, and a correction value of the pupil area due to a mental factor is calculated.

First, the pupil image photographing section 4 photographs a pupil image of the user. Next, the pupil area calculation unit 5 performs image processing to determine the pupil area. The pupil area obtained here includes a factor of the luminance of the subject and a mental factor (interest and interest of the user). Therefore, the pupil area correction unit 6
In, the pupil area calculated by the pupil area calculation unit 5 is corrected based on the light intensity from the photodetector 3 and the pupil area correction table 61, and a change in the pupil area due to a mental factor is detected. Here, the incident light is reflected by the half mirror 2 and detected by the photodetector 3. The light intensity detected by the photodetector 3 is input to the pupil area correction unit 6 and the data of the pupil area correction table 61 is used to remove the influence of the luminance of the subject from the pupil area data and extract only the mental factor. Next, the determination unit 7 determines whether the pupil area has expanded due to a mental factor of the user with respect to the subject. If the pupil area is enlarged due to a mental factor, the determination unit 7 determines that the interest or interest of the user is in the subject, and proceeds to step 703. If no enlargement of the pupil area due to mental factors is detected, the process returns to step 701.

Next, in step 703, it is determined in which direction on the earth the object the user is looking at exists. Here, [Q] = [O] + k [n] k using the three-dimensional position coordinates [O] of the user's eyeball on the earth detected in step 701 and the unit vector [n] in the user's gaze direction. An object viewed by the user exists on the half line Q satisfying> 0.

From step 704 onward, a control procedure for extracting the attributes of the object viewed by the user is shown.

First, in step 704, a search is made in the area database to find in which spatial area the user's eyeball position [O] exists. The user's eyeball may be either left or right. First, regarding the plane S, [V] · [O] −W <
Since it is 0, the user's eyeball exists in the S1 area (FIG. 8).
As for the plane S1, [V1] · [O] −W1 ≧ 0, so that the user's eyeball exists in the S10 region (FIG. 8). Regarding the plane S10, [V10] · [O] −W10 ≧ 0, so the user's eyeball exists in the S100 area (FIG. 8). Regarding the plane S100,
[V100] · [O] −W100 ≧ 0, so the user's eyeball is S1000
Region (FIG. 8). For the plane S1000, [V1
000] · [O] −W1000 ≧ 0, so the user's eyeball is S10000
Region (FIG. 8). Since the area S10000 is the terminal node (FIG. 4), the position of the user's eyeball has been determined. Therefore, the area S10000 is set as the first search area.

Next, in step 705, it is determined whether or not there is an object that intersects the half line [Q] in the search area.

Since there is no object that intersects with the half line [Q] in the area S10000 (FIG. 4), the process proceeds to step 706 to find the next search area. Sibling area S10001 of area S10000
Does not intersect with the half line [Q] (FIG. 9).
There is no object that intersects the ray [Q] (FIG. 9). Next, since the sibling area S1001 of the area S1000 does not intersect with the half line [Q] (FIG. 9), there is no object intersecting with the half line [Q] in the area S100 (FIG. 9). Sibling area S1 of area S100
Since 01 intersects with the half line (FIG. 9), an object which intersects with the half line can exist in the area S101. Child area S1 of area S101
Since 010 intersects with the half line [Q] (FIG. 9), the half line [Q]
May exist, but the child area S1011 is
Since it does not intersect with [Q] (FIG. 9), no object intersects with the ray [Q]. Child areas S10100 and S101 of area S1010
01 intersects the half line [Q] (FIG. 9), but since the area S10101 is closer to the position [O] of the eyeball, the area S10101 is set as the next search area.

Thus, step 705 is executed again. Here, the three-dimensional coordinates of all points registered in the area S10101 are extracted from the area database (FIG. 4). Here, the points [B1], [B2], [B
3] and [B4] are extracted. The coordinate database (FIG. 5) is searched using each of these coordinates as a key, and the name of the object including these points is extracted. Here, the name "B Building" is taken out. next,
The attribute database (FIG. 6) is searched using the name "B Building" as a key, and triangular patch data ([B1], [B2], [B3]) indicating the shape of "B Building", ([B1], [B2], [B4]),
([B1], [B4], [B3]) and ([B4], [B2], [B3]) are extracted, and it is checked whether or not these triangles intersect with the half line [Q].

Here, since these triangles do not intersect with the half line [Q] (FIG. 9), the process proceeds to step 706 to find the next search area. The sibling area S10100 of the area S10101 is
Since it intersects with the half line [Q] (FIG. 9), this is set as the next search area.

At step 705, the three-dimensional coordinates of all the points registered in the area S10100 are extracted from the area database (FIG. 4). Here, point [D1],
[D2] and [D4] are extracted. Using these coordinates as keys, a coordinate database (FIG. 5) is searched to extract the name of the object including these points. Here, the name “D building” is taken out. Next, the attribute database (FIG. 6) is searched using the name “D building” as a key,
Triangular patch data ([D1],
[D2], [D3]), ([D1], [D2], [D4]), ([D1], [D
4], [D3]) and ([D4], [D2], [D3]), and examine whether these triangles intersect with the half line [Q]. Here, the triangles ([D1], [D2], [D4]), ([D1], [D4],
[D3]) intersects the half-line [Q] (Fig. 9), so the eyeball position
Triangle ([D1], [D2], [D4]) closer to [O] and a half-line
The intersection with [Q], that is, the value of k (> 0) is obtained. In addition, by using the distance k from the viewpoint position to the object, for example, “the distance to the building D is
500 meters. "Can be performed.

Next, the process proceeds to step 707, where information relating to the object "D Building" which the user is looking up is extracted from the attribute database (FIG. 6), and "D Building. Tallest in Japan. Height 500m". The voice is output by the voice synthesis unit 105.

As described above, the navigation apparatus according to the present embodiment includes a user's eyeball position detection unit 101, a user's gaze direction detection unit 102, a control unit 103, a map data storage unit 104, a speech synthesis unit 105, A user pupil area detection unit 106 is provided. Then, the user's eyeball position and the user's gaze direction on the earth at each time are detected by the user's eyeball position detection unit 101 and the user's gaze direction detection unit 102, and the user's pupil area detection unit 106 detects the user's pupil area based on mental factors. When the change is detected, the control unit 103 specifies the object viewed by the user using the data in the map data storage unit 104. Then, the attribute data (name, description, etc.) of the object specified in this way is navigated to the user by voice synthesis by the voice synthesis unit 105, so that the user is interested in watching without a device such as a display. You can check what the object is. Further, it is possible to detect a change in the pupil area of the user due to a mental factor and output an explanation immediately according to the user's interest or interest in the object viewed by the user.

In the above embodiment, the direction of the user's line of sight is an instantaneous value at each time, but an average value over a certain period of time (for example, one second) may be used. In addition, the direction of the user's line of sight is kept for a certain period
If a certain number of times (for example, two times) intersects with the triangular patch data of a specific object, it may be determined that the user is looking at the object.

Also, in the above embodiment, what exactly the object the user is looking at is determined. However, using the approximate values of the user's position and the user's line of sight, "there is a Yokohama area" It may be configured to perform rough navigation such as

In the above embodiment, the navigation device is controlled by changing the pupil area. However, the navigation device may be controlled by changing the pupil area changing speed or changing the acceleration.

An object of the present invention is to provide a storage medium storing a program code of software for realizing the functions of the above-described embodiments to a system or an apparatus, and a computer (or CPU) of the system or apparatus.
And MPU) read and execute the program code stored in the storage medium.

In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD
-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also the OS (Operating System) running on the computer based on the instruction of the program code. ) May perform some or all of the actual processing, and the processing may realize the functions of the above-described embodiments.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instruction of the program code, It goes without saying that the CPU included in the function expansion board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

[0040]

As described above, according to the present invention,
It is possible to detect an object of interest of the user based on the direction of the user's line of sight and output information corresponding to the detected object.

[0041]

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a navigation device according to an embodiment.

FIG. 2 is a diagram illustrating an example of a field of view observed by a user wearing the navigation device.

FIG. 3 is a diagram showing a line-of-sight position and direction of the user's field of view shown in FIG. 2 as viewed from above.

FIG. 4 is a diagram showing a state in which an object (feature point) on the earth shown in FIG. 3 is registered in an area database in a map data storage unit 104;

FIG. 5 is a diagram showing an example of a data configuration of a coordinate database indicating what objects on the earth each point registered in the area database of FIG. 4 is;

FIG. 6 is a diagram illustrating an example of a data configuration of an attribute database in which a set of triangular patches having an outer shape of the object is registered with respect to the name of the object.

FIG. 7 is a diagram illustrating details of a configuration of a pupil area detection unit 106.

FIG. 8 is a flowchart illustrating the operation of the navigation system according to the embodiment.

FIG. 9 is a diagram illustrating a line of sight of a user, a region to be extracted, and an object.

Claims (18)

[Claims] A storage means for storing map information; a detection means for detecting a user's viewpoint position and a line-of-sight direction on a map based on the map information; A navigation device comprising: a search unit that searches for and extracts an object on a line segment from the map information; and an output unit that outputs information about the object extracted by the search unit. 2. The map information includes information obtained by dividing a map into a plurality of partial areas, and further includes extraction means for extracting a partial area through which a line segment indicating the line-of-sight direction passes. 2. The navigation device according to claim 1, wherein an object located on the line segment is searched for the partial area extracted by the means. 3. An object in the map information is represented by a plurality of feature points, and is divided so that the number of feature points included in each of the partial regions is equal to or less than a predetermined number. 3. The navigation device according to 2. 4. The storage unit further stores a pair of each object in the map information and a corresponding description, and the output unit stores a description corresponding to the object extracted by the search unit. 2. The navigation device according to claim 1, wherein a sentence is read from said storage means and is output as a voice. 5. The navigation device according to claim 1, wherein the detection of the viewpoint position by the detection unit is performed using a GPS system. 6. A system further comprising: a judging unit for detecting enlargement of a pupil of a user and judging whether or not to output information based on the detection result, wherein the judging unit judges that the information is to be output. The navigation device according to any one of claims 1 to 5, wherein the output unit is operated in a case. 7. The determination means measures the area of the pupil of the user, corrects the measured area based on the light intensity of the observation target, and outputs information based on the corrected area of the pupil. The navigation device according to claim 6, wherein it is determined whether or not the navigation is performed. 8. A storage means for storing map information including information obtained by dividing a map into a plurality of partial areas; a detection means for detecting a viewpoint position and a line of sight of a user on a map based on the map information; Searching means for searching and extracting a partial area on the line segment along the line of sight starting from the viewpoint position from the map information; and outputting means for outputting a place name corresponding to the partial area extracted by the searching means. A navigation device comprising: 9. A navigation method using storage means for storing map information, comprising: a detecting step of detecting a viewpoint position and a line of sight of a user on a map based on the map information; A search step of searching for and extracting an object on a line segment along the line of sight direction from the map information, and an output step of outputting information on the object extracted in the search step. Navigation method. 10. The map information includes information obtained by dividing a map into a plurality of partial areas, and further includes an extraction step of extracting a partial area through which a line segment indicating the line-of-sight direction passes. 10. The navigation method according to claim 9, wherein an object located on the line segment is searched for the partial area extracted in the step. 11. An object in the map information is indicated by a plurality of feature points, and is divided so that the number of feature points included in each of the partial regions is equal to or less than a predetermined number. The navigation method according to claim 10. 12. The storage unit further stores a pair of each object in the map information and a corresponding descriptive sentence, and the output step includes a description corresponding to the object extracted in the search step. 10. The navigation method according to claim 9, wherein a sentence is read from said storage means and is output as a voice. 13. The navigation method according to claim 9, wherein the detection of the viewpoint position in the detection step is performed using a GPS system. 14. The method according to claim 1, further comprising the step of: detecting an enlargement of a pupil of the user and determining whether to output information based on a result of the detection. 14. The navigation method according to claim 9, wherein the output step is operated in a case. 15. The determining step includes measuring an area of a pupil of a user, correcting the measured area based on a light intensity of an observation target, and outputting information based on the corrected area of the pupil. 15. It is determined whether or not it is.
Navigation method described in. 16. A navigation method using storage means for storing map information including information obtained by dividing a map into a plurality of partial areas, wherein a viewpoint position and a line-of-sight direction of a user on a map based on the map information are determined. A detecting step of detecting, a searching step of searching and extracting a partial area on a line segment along the line of sight starting from the viewpoint position from the map information, and corresponding to the partial area extracted in the searching step An output step of outputting a place name to be used. 17. A storage medium for storing a control program for performing a navigation process using storage means for storing map information, the control program comprising: a user's viewpoint position on a map based on the map information. And a code of a detection step of detecting a line-of-sight direction, and a code of a search step of searching for and extracting an object located on a line segment along the line-of-sight direction from the map information, starting from the viewpoint position. And a code for an output step of outputting information on the object extracted in step (a). 18. A storage medium storing a control program for performing a navigation process using storage means for storing map information including information obtained by dividing a map into a plurality of partial areas, the control program comprising: A code for a detecting step of detecting a user's viewpoint position and a line-of-sight direction on a map based on the map information; and searching the map information for a partial area on a line segment starting from the viewpoint position and extending along the line-of-sight direction. A storage medium comprising: a search step code to be extracted; and an output step code to output a place name corresponding to the partial area extracted in the search step.