JPWO2006109527A1 - Navigation device and navigation method - Google Patents

Abstract

  It provides navigation information that enables the synthesis and output of route guidance information in a live-action image by simple data processing. Route search unit (1), route information processing unit (2), photographed image capturing apparatus (3), current position/direction specifying unit (4), posture specifying unit (5), current position plotting unit (6), guidance instruction A mark generation unit (7), a perspective conversion image synthesis unit (8) and an image display device (9) are provided. A real-time image capturing device attached to or incorporated in the main body of the device captures a real-time image of the landscape at that position in real time, and a guidance instruction mark is combined with the actual captured image to provide intuitive guidance. The image is displayed. An L-shaped display is provided for displaying the guidance instruction mark and the intersection or branch point of the image of the route. If the length is longer than a predetermined threshold value, the L-shape is divided and displayed in the middle.

Description

    The present invention relates to a navigation device and a navigation method for combining a guidance image indicating a route from a starting point to a destination with a photographed image and outputting the synthesized image.

  For example, it is often the case that the user wants directions from the nearest station of a railway or other transportation facility or the entrance/exit of an underground mall to a destination desired by a user. In such a case, a navigation method in which a two-dimensional map is displayed on a mobile phone with a GPS (Global Positioning System) for route guidance and a system therefor are generally put into practical use.

  However, it is difficult for the user to immediately and intuitively understand the matching between the actual landscape and the map in the route guidance on the two-dimensional map. For this reason, methods and systems have been proposed in which intuitively easy-to-understand live-action images and actual landscapes are compared to provide route guidance (for example, Patent Documents 1 to 4).

  In such a conventionally proposed technique, the live-action image is stored in the image data recording unit in advance, and when performing the route guidance, the real-image image suitable for the position is read from the image data recording unit and used. There is.

Further, in a so-called car navigation system called a car navigation system, for example, based on a live-action video signal transmitted from a camera fixed to a parking lot, a technique of displaying the situation of the parking lot in the navigation screen as a live-action image is available. Proposed (for example, Patent Document 5)
JP, 2003-194567, A JP, 2002-277275, A JP, 2002-221433, A JP, 2004-226140, A JP, 2001-21368, A

  However, in the conventional method and system as described above, it is necessary to store the photographed image as image data in the image data recording unit in advance. For this reason, for example, even in the same landscape, if the shooting conditions such as the season, the weather, and the time are different, the user can see the images as quite different images. Therefore, including the change in those shooting conditions limits the navigable area. However, it is necessary to prepare image data for different seasons, weather conditions, and times for each image in each position in the area, and in the end, a huge amount of data is impossible to realize in practice. It becomes necessary to capture and store the image data of. Therefore, there is a serious fatal problem that the system as described above becomes unrealizable due to the impracticality of accumulating a huge amount of image data. Alternatively, even if the storage of such a huge amount of image data can be managed somehow, there is an inconvenience in that it is extremely complicated and enormous labor and cost are required.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a navigation device and a navigation method capable of synthesizing and outputting route guidance information to a live-action image by simple data processing.

  A navigation device of the present invention is a route for searching a route from a starting point to a destination from a storage device for storing map information including a route from a starting point to a destination and map information stored in the storage device. The search unit, the route information processing unit that generates and processes image data for displaying the information of the route searched by the route search unit, the photographed image photographing device that photographs the photographed image of the landscape, and the photographed image photographing device. A current position/azimuth identification unit that identifies the current position and orientation, a posture identification unit that identifies the orientation of the photographed image capturing device, a current position plot unit that plots the current position in the searched route information, and a current position Guidance direction mark generation unit for generating a guidance direction mark in the direction from the destination to the destination, a perspective conversion image synthesis unit for performing perspective transformation of the guidance direction mark, and then superimposing and synthesizing the guidance instruction mark, and a perspective conversion image synthesis unit. And an image output device for outputting an image based on the image data combined in (3).

  Further, in the navigation method of the present invention, the map information including the route from the departure point to the destination is acquired from the external map server in the storage device of the portable navigation device by communication and stored in the storage device. A route search step that searches the route from the starting point to the destination from the map information, a route information processing step that generates and processes image data for displaying the information of the route searched by the route search step, and a real scene photograph A real image capturing step of capturing an image by a real image capturing device attached to or incorporated in the body of the portable navigation device, a current position/orientation identifying step of identifying the current position and orientation of the real image capturing device, and a live image. A posture specifying step for specifying the posture of the image capturing apparatus, a current position plotting step for plotting the current position in the searched route information, and a guide instruction mark generation for generating a guide instruction mark in the direction from the current position toward the destination. A step, a perspective conversion image synthesizing step of superimposing and synthesizing the guidance instruction mark on the photographed image after perspective transformation, and an image output step of outputting an image based on the image data synthesized in the perspective conversion image synthesizing step. , Is included.

  In the navigation device or navigation method of the present invention, a real-time image capturing device attached to or incorporated in the main body of the device captures a real-time image of a landscape at that position in real time, and a guidance instruction mark is combined with the live-action image. Then, a guide image that can be intuitively grasped is displayed.

  Here, as map information, it is desirable to use digital map data for pedestrians instead of maps for in-vehicle navigation.

  Further, if the map information in the storage device is acquired from the external map server by communication each time, the memory capacity can be reduced.

  Further, when the current position deviates from the route of the search result by a predetermined distance or more, it is desirable to perform the following processing. In other words, the route search unit re-searches, the route information processing unit re-generates and processes the image data, the real image capturing device re-captures the actual image of the landscape, and the current position/orientation specifying unit captures the actual image. Re-specify the current position and orientation of the device, re-specify the posture of the photographed image capturing device by the posture specifying unit, re-plot the current position in the information of the searched route, and generate a guidance mark. Section regenerates the guidance instruction mark in the direction from the current position to the destination, and the perspective conversion image synthesis section performs perspective transformation of the guidance instruction mark and synthesizes it again in the image of the route information. The image based on the image data combined by the perspective conversion image combining unit is output again.

  By doing so, even if the route guidance information that is temporarily out of the current position of the user is output, it becomes possible to immediately re-output the route guidance information that matches the current position.

  In order to make the output route guidance image easier to see, the guidance instruction mark generation unit is an L-type guidance instruction mark that is displayed up to the intersection or branch point of the route image, and an L-type guidance. A threshold value is predetermined for the length of the instruction mark, and when the L-shaped length is longer than the threshold value, the instruction mark data is generated so that the L-shaped portion is divided and displayed in the middle. It may have any configuration.

  Further, it is possible to provide route guidance information by voice guidance (for example, lane change instruction, destination landmark and destination instruction, related facility instruction, road regulation instruction, etc.) in association with the display of the guidance instruction mark. Good. In this case, it is desirable that the voice guidance is performed according to the priority order of the route guidance information. Further, in addition to the image based on the image data synthesized by the perspective transformation image synthesizing unit, it has a function of displaying at least one of the two-dimensional map image and the three-dimensional CG map image. You may make it switch and output.

  According to the present invention, each time a navigation is performed, a live-action image of a landscape at that position is taken in real time, and a guidance instruction mark is combined with the live-action image to intuitively grasp the route guidance. Image is output (for example, displayed), it is difficult or impossible with a conventional navigation system to realize a navigation device and a navigation method that synthesize and output route guidance information to a live-action video by simple data processing. You can

It is a block diagram showing the composition of the principal part of the navigation device concerning one embodiment of the present invention. FIG. 2 is an external view of a mobile phone incorporating the navigation device of FIG. 1. 9 is a flowchart for explaining a navigation operation. It is a schematic diagram showing the acquisition method of the positional information X. It is a schematic diagram showing an example of the shape of a guidance instruction mark. It is a schematic diagram showing an example when the present location is not largely deviated from the route of the search result. It is a schematic diagram showing an example in case the present location is largely off from the route of the search result. It is a figure showing the spatial geometrical relationship of a world coordinate system and a camera coordinate system. It is a figure showing an example of the perspective transformation from the existing point P'to the point P on a projection surface.

  Hereinafter, the best mode of the present invention will be described with reference to the drawings.

  FIG. 1 shows a block configuration of a navigation device according to an embodiment of the present invention. This navigation device includes a route search unit 1, a route information processing unit 2, a photographed image capturing device 3, a current position/direction specifying unit 4, a posture specifying unit 5, a current position plotting unit 6, a guidance instruction mark generating unit 7, and a perspective conversion. The image synthesizing unit 8, the image display device 9, and the storage device 10 are provided as main parts, and are incorporated in, for example, a camera-equipped mobile phone 100 such as an example shown in FIG.

  The route searching unit 1 searches for a route from a starting point designated by the user to a destination from the map information stored in the storage device 10. As the map information, digital map data for pedestrians is used instead of digital map for vehicles. As a result, route guidance is provided on the sidewalk rather than on the roadway. Note that the map information in the storage device 10 is to be acquired by communication from an external map server each time using the communication function of the camera-equipped mobile phone 100.

  The route information processing unit 2 generates and processes image data for displaying information on the route searched by the route searching unit. The photographed image photographing device 3 is a photographing device including an image pickup device and a video signal processing circuit incorporated in the navigation device, and photographs a photographed image of a landscape at the current position when performing route guidance.

  The current position/direction specifying unit 4 specifies the current position and direction of the navigation device including the photographed image photographing device 3. The posture identifying unit 5 has a function of identifying the posture of the photographed image capturing apparatus 3. The current position plotting unit 6 plots the current position in the searched route information.

  The guide instruction mark generation unit 7 generates a guide instruction mark in the direction from the current position toward the destination. The guide instruction mark generation unit 7 is an L-shaped guide instruction mark that displays a route image up to an intersection or a branch point, and the length of the guide instruction mark has a predetermined threshold value. When the length of the L type generated is longer than the threshold value, it also has a function of generating the data of the guide instruction mark so that the L type is displayed in a state of being divided in the middle.

  The perspective-transformed image synthesizing unit 8 has a function of perspective-transforming the guidance instruction mark and then superimposing and synthesizing the guidance instruction mark on the real image. The image display device 9 displays and outputs an image based on the image data combined by the perspective conversion image combining unit 8.

  Next, the operation of this navigation device will be described with reference to the flowchart of FIG.

  First, in the route searching step S1, map information including a route from a departure point to a destination is acquired from the external map server in the storage device 10 by communication, and the map information stored in the storage device 10 is used to obtain the map from the departure point. Search for a route to your destination. Here, the user specifies the starting point and the destination by inputting a position on the digital map, inputting a telephone number, inputting a zip code, or the like. When the departure place is the current position, the position information of the mobile phone 100 itself in which the navigation device is incorporated can be used. For this route search, for example, the Dijkstra method, which is a general shortest distance route search algorithm, can be used. This Dijkstra method can be used even when inserting a detour on the way. Further, it may be possible to select a sidewalk or search for a route by using only the main street or alley.

  Next, in the route information processing step S2, image data for displaying the information of the route searched in the route searching step S1 is generated and processed. Then, in the photographed image photographing step S3, the photographed image of the landscape at the current position of the user is photographed by the photographed image photographing device 3. Specifically, the user holds the mobile phone 100 incorporating this navigation device over his face and makes the optical axis of the lens system of the photographed image taking device 3 provided in the mobile phone 100 substantially parallel to the ground. , Take landscapes including sidewalks and roads. At this time, the position information (position vector of the position vector) having elements such as latitude, longitude, and altitude is used as information for specifying the current position of the navigation device including the photographed image capturing device 3 by the current position specifying unit 4. Data) Χ is obtained from the GPS 41 and three relay points 42a, 42b and 42c, as schematically shown in FIG. 4, for example. Further, the azimuth information N is obtained by the electronic compass (step S4).

  At the same time, as posture information for the posture specifying unit 5 to specify the posture of the photographed image capturing apparatus 3, for example, a yaw angle, a roll angle and a pitch having three degrees of freedom are provided. Attitude angle information (data of attitude angle vector) Θ consisting of angles is obtained from the gyro sensor (step S5).

  Next, in the current position/azimuth specifying step S6, the current position/azimuth of the photographed image photographing device 3 is specified based on the position information X and the azimuth information N obtained as described above. Then, in the posture specifying step S7, the posture of the photographed image photographing device 3 is specified based on the posture angle information Θ obtained as described above.

  Further, in the current position plotting step S8, the current position in the searched route information is plotted on the digital map. Then, in the guide instruction mark generation step S9, the guide instruction mark M is generated in the direction from the current position toward the destination. The shape and end point of the guide instruction mark M are determined as follows.

(1) As a result of the route search, for example, when there are a plurality of intersections and branch roads along the route in the road network 51 as shown in FIG. For this reason, the route up to the bend position is displayed. That is, the guide instruction mark M is basically L-shaped as shown in FIG. 5A. A threshold is set in advance for this L-shaped length, and if it is longer than that, it is divided in the middle as shown in the example in FIG. 5B (step S9).

(2) As shown in FIG. 6, when the error or deviation of the current position R from the route of the search result is within the allowable range (step S10; Y), the data generated by S9 by that time. Based on, the guidance instruction mark M from the current position to the intersection angle is displayed. However, as shown in FIG. 7, when the current point R deviates from the route of the search result and deviates significantly from the predetermined allowable value (step S10; N), the re-search is performed. In other words, the above steps S1 to S9 are repeated (and in some cases, including the step S11 and subsequent steps as described later). As a result of redoing in such a manner, if the current position does not greatly deviate from the route of the search result (step S10; Y), the process proceeds to the next step. In FIGS. 6 and 7, the “two-dimensional map” is a real image viewed from the satellite, and the “display image” is a real image viewed from the same place on the road, together with the guide instruction mark M. It is a representation.

  In the perspective-transformed image synthesizing step S11, the guide instruction mark is perspective-transformed and synthesized in the image of the route information. More specifically, the perspective transformation here can be achieved by three-dimensional coordinate transformation and perspective transformation as described below.

Now, consider expressing a certain point P=(x, y, z) expressed in the world coordinate system O-XYZ in another camera coordinate system O'-X'Y'Z'. The orthonormal basis of the O-XYZ and O'-X'Y'Z', respectively _{E = (e 1, e 2} , e 3), E'= (e '1, e' 2, e '3) And Here, in the world coordinate system, the Z-axis direction is the height from the ground, and the Y-direction is the true north direction.

  The center position R of the photographed image capturing apparatus (camera) 3 is given by the GPS in the world coordinate system. It is assumed that the given position information has been converted from a latitude/longitude into a plane orthogonal system. The gyro and the electronic compass cause an angle difference θ between the current position and the YZ plane (which direction the map is facing), the inclination φ between the horizontal direction of the camera and the ground at that time, the photographed image capturing apparatus 3 It is assumed that the inclination (depression angle) ρ between the translation direction and the ground is obtained.

At this time, the rotation matrix A for conversion from the world coordinate system O-XYZ to the camera coordinate system O'-X'Y'Z' is represented by the following equation (1).

Now, it is assumed that the point P is given by the following equation (2) in the world coordinate system O-XYZ. At this time, the point P is given by the equation (3) using the camera coordinate system O′-X′Y′Z′.
P=R+x·e ₁ +y·e ₂ +z·e ₃ (2)
P=R+x·e′ ₁ +y·e′ ₂ +z·e′ ₃ (3)

By the way, based on the spatial geometric relationship between the world coordinate system O-XYZ and the camera coordinate system O′-X′Y′Z′ as shown in FIG. 8, E′ is the rotation matrix A in E. It is a product of the multiplications and can be expressed as the following equation (4).
E'=E·A (4)

Here, assuming that e ₁ =[1,0,0] ^T , e ₂ =[0,1,0] ^T T,e ₃ =[0,0,1] ^T , the following equation (5) holds. ..
E'=A (5)

Therefore, the following expression (6) is established from the above.
P-R = [x ', y', z '] T [e' 1, e '2, e' 3] = [x ', y', z '] T A ... (6)

Since A is an orthogonal matrix, since A ⁻¹ =A ^T , P′ can be obtained by the following equation (7).
P′=(P−R)A ⁻¹ =(P−R)A ^T (7)

Then, as schematically shown in FIG. 9, a point P′=(x′, y′, z′) 102 projected from the certain point P′=(x′, y′, z′) 102 to the focus f101 is p(px, py). , 0), the position coordinate of the point p can be expressed by the following equations (8) and (9). This point (px, py) is the desired coordinate value.
px=fx′/(f+y′) (8)
py=fz'/(f+y') (9)

  Finally, in the image display output step S12, an image is displayed and output based on the image data obtained in the perspective transformation image composition step S12.

  In this manner, in the navigation device according to the present embodiment, each time the user designates and inputs the departure place and the destination, a real-time image of the scenery at that position is taken in real time, and the real-image is taken. A guide instruction mark is combined with the guide instruction image, and an image of the route guidance that can be intuitively grasped is displayed and output. Therefore, it is possible to realize a navigation device and a navigation method for synthesizing and outputting route guidance information on a real image by simple data processing, which is difficult or impossible with conventional navigation systems.

  Further, in the above embodiment, an example in which the navigation device of the present invention is incorporated in a mobile phone has been described, but it goes without saying that the application or use of the device configuration and method is not limited to this. Yes. In addition to this, for example, it can be a stand-alone handheld navigation terminal for pedestrians, or can be incorporated into a portable electronic device such as a notebook personal computer, or the like. It can also be applied to a device.

  The output of the route guidance information is not limited to the visual image information (guidance instruction mark M) by the display device, and for example, voice information or character information is added in conjunction with the guidance instruction mark M. You may make it output as information. For example, in an on-vehicle navigation device, route guidance information such as a lane change instruction, a landmark and a destination instruction to turn, a related facility instruction or a road regulation instruction can be given to a driver by a guidance instruction mark and voice guidance linked to it. Can be provided. Here, it is preferable that the voice guidance follows the priority order of information.

  Further, the configuration is not limited to always displaying an image (navigation image of a live-action video) based on the image data synthesized by the perspective transformation image synthesizing unit 8, but a two-dimensional map and a three-dimensional CG map (third order) depending on the scene. The original CG bird view map) or the like may be used together. As a result, for example, a two-dimensional map is used to set the destination, and a three-dimensional CG map is used to list the facilities around the current location, etc., and the navigation image is automatically switched to a live-action image only at the corners of the route or around the destination. Can be displayed.

Further, the respective parts constituting the navigation device do not necessarily have to be incorporated in the same terminal. For example, the route search unit 1, the guidance instruction mark generation unit 7, and the like can be incorporated in an external server or the like to reduce the storage capacity and calculation capacity in the terminal. Specifically, the digital map is held by an external server or the like for route search, or the external server performs guide instruction mark generation and perspective conversion, and transmits the result to the terminal side using communication means. It is a simple form.

Title: Navigation device and navigation method

[0001]
【Technical field】
[0001]
The present invention relates to a portable navigation device and a navigation method for combining a guidance image indicating a route from a departure point to a destination with a real image and outputting the synthesized image.
[Background technology]
[0002]
For example, it is often the case that the user wants directions from the nearest station of a railway or other transportation means or the entrance/exit of an underground mall to the destination desired by the user. In such a case, a navigation method of displaying a two-dimensional map on a mobile phone with a GPS (Global Positioning System) to perform route guidance and a system therefor are generally put into practical use.
[0003]
However, it is difficult for the user to immediately and intuitively understand the matching between the actual landscape and the map in the route guidance on the two-dimensional map. For this reason, methods and systems have been proposed in which intuitively easy-to-understand live-action images are compared with actual landscapes for route guidance (for example, Patent Documents 1 to 4).
[0004]
In such a conventionally proposed technique, the live-action image is stored in the image data recording unit in advance, and when performing the route guidance, the real-image image suitable for the position is read from the image data recording unit and used. There is.
[0005]
In addition, in a so-called car navigation system called a car navigation system, for example, based on a live-action video signal transmitted from a camera fixed to a parking lot, a technique of displaying the situation of the parking lot in the navigation screen as a live-action image is available. Proposed (for example, Patent Document 5)
[Patent Document 1] JP-A-2003-194567 [Patent Document 2] JP-A-2002-277275 [Patent Document 3] JP-A-2002-221433 [Patent Document 4] JP-A-2004-226140 [Patent Document 1] Reference 5: Japanese Patent Application Laid-Open No. 2001-21368 [Disclosure of the Invention]

[0002]
[0006]
However, in the conventional method and system as described above, it is necessary to store the photographed image as image data in the image data recording unit in advance. For this reason, for example, even in the same landscape, if the shooting conditions such as the season, the weather, and the time are different, the user can see the images as quite different images. Therefore, including the change in those shooting conditions limits the navigable area. However, it is necessary to prepare image data for different seasons, weather conditions, and times for each image in each position in the area, and in the end, a huge amount of data is impossible to realize in practice. It becomes necessary to capture and store the image data of. Therefore, there is a serious fatal problem that the system as described above becomes unrealizable due to the impracticality of accumulating a huge amount of image data. Alternatively, even if the storage of such a huge amount of image data can be managed somehow, there is an inconvenience in that it is extremely complicated and enormous labor and cost are required.
[0007]
The present invention has been made in view of the above problems, and an object thereof is to provide a portable navigation device and a navigation method capable of synthesizing and outputting guidance information in a live-action image by simple data processing. is there.
[0008]
A portable navigation device of the present invention searches a route from a starting point to a destination from a storage device for storing map information including a route from a starting point to a destination and the map information stored in the storage device. A route search unit, a route information processing unit that generates and processes image data for displaying information of the route searched by the route search unit, a still image capturing device that captures a still image of a landscape, and a still image capturing A current position/azimuth identification unit that identifies the current position and orientation of the device, a posture identification unit that identifies the orientation of the still image capturing device, and a current position plot unit that plots the current position in the information of the searched route, A guidance instruction mark generation unit that generates a guidance instruction mark in the direction from the current position to the destination, a perspective transformation image synthesis unit that perspectively transforms the guidance instruction mark and then superimposes the guidance instruction mark on a still image, and a perspective transformation image. And an image output device that outputs an image based on the image data combined by the combining unit.
[0009]
Further, in the navigation method of the present invention, the map information including the route from the departure point to the destination is acquired from the external map server in the storage device of the portable navigation device by communication and stored in the storage device. Route from map information to starting point to destination

[0003]
A route search step of searching for a route, a route information processing step of generating and processing image data for displaying information of the route searched by the route search step, and a still image of the landscape are attached to the main body of the portable navigation device or A still image shooting step for shooting with a built-in still image shooting device; a current position/direction specifying step for specifying the current position and direction of the still image shooting device; and a posture specifying step for specifying the attitude of the still image shooting device. , A current position plotting step of plotting the current position in the information of the searched route, a guide instruction mark generating step of generating a guide instruction mark in the direction from the current position to the destination, and a perspective conversion of the guide instruction mark. It includes a perspective transformation image synthesizing step for superimposing and synthesizing on a still image, and an image outputting step for outputting an image based on the image data synthesized in the perspective transformation image synthesizing step.
[0010]
In the portable navigation device or the navigation method of the present invention, a still image of a landscape at that position is captured in real time by a still image capturing device attached to or incorporated in the device body, and a guide instruction mark is added to the still image. Are combined and an intuitively understandable route guidance image is displayed.
[0011]
Here, as the map information, it is desirable to use digital map data for pedestrians instead of maps for in-vehicle navigation.
[0012]
Further, if the map information in the storage device is acquired from the external map server by communication each time, the memory capacity can be reduced.
[0013]
Further, when the current position deviates from the route of the search result by a predetermined distance or more, it is desirable to perform the following processing. That is, the route search unit re-searches, the route information processing unit re-creates and processes image data, the still image capturing device re-captures a still image of the landscape, and the current position/orientation identifying unit captures a still image. The current position and azimuth of the device is re-specified, the posture identification unit re-identifies the posture of the still image capturing device, and the current position plotting unit re-plots the current position in the information of the searched route to generate the guidance instruction mark. Section regenerates the guidance instruction mark in the direction from the current position to the destination, and the perspective conversion image synthesis section performs perspective transformation of the guidance instruction mark and synthesizes it again in the image of the route information. The image based on the image data synthesized by the perspective conversion image synthesizing unit is output again.

[0004]
[0014]
By doing so, even if the route guidance information that is temporarily out of the current position of the user is output, it becomes possible to immediately re-output the route guidance information that matches the current position.
[0015]
In order to make the output route guidance image easier to see, the guidance instruction mark generation unit is an L-type guidance instruction mark that is displayed up to the intersection or branch point of the route image, and an L-type guidance. A threshold value is predetermined for the length of the instruction mark, and when the L-shaped length is longer than the threshold value, the instruction mark data is generated so that the L-shaped portion is divided and displayed in the middle. It may have any configuration.
[0016]
Further, it is possible to provide route guidance information by voice guidance (for example, lane change instruction, destination landmark and destination instruction, related facility instruction, road regulation instruction, etc.) in association with the display of the guidance instruction mark. Good. In this case, it is desirable that the voice guidance is performed according to the priority order of the route guidance information. Further, in addition to the image based on the image data synthesized by the perspective conversion image synthesizing unit, it has a function of displaying at least one of the images by the two-dimensional map and the three-dimensional CG map, You may make it switch and output.
[0017]
According to the present invention, each time a navigation is performed, a still image of a landscape at that position is captured in real time, and a guidance instruction mark is combined with the still image to provide intuitive guidance. Image is output (for example, displayed), it is difficult or impossible with a conventional navigation system to realize a navigation device and a navigation method for synthesizing and outputting route guidance information to a live-action video by simple data processing. You can
[Brief description of drawings]
[0018]
FIG. 1 is a block diagram showing a configuration of a main part of a navigation device according to an embodiment of the present invention.
2 is an external view of a mobile phone incorporating the navigation device of FIG.
FIG. 3 is a flow chart for explaining a navigation operation.

Claims (9)

A storage device for storing map information including a route from a starting point to a destination,
From the map information stored in the storage device, a route search unit for searching a route from a departure point to a destination,
A route information processing unit for generating and processing image data for displaying information on the route searched by the route searching unit;
A live-action image capturing device that captures a live-action image of a landscape,
A current position/azimuth specifying unit that specifies the current position and azimuth of the photographed image capturing device;
A posture specifying unit that specifies the posture of the photographed image capturing device;
A current position plotting unit for plotting the current position in the information of the searched route,
A guide instruction mark generation unit that generates a guide instruction mark in the direction from the current position toward the destination,
A perspective-transformed image synthesizing unit that synthesizes by superimposing the guidance instruction mark on the live-action image after perspective-transforming it;
An image output unit that outputs an image based on the image data combined by the perspective conversion image combining unit. The navigation device according to claim 1, wherein data of a pedestrian digital map is used as the map information. The navigation device according to claim 1 or 2, wherein the map information in the storage device is obtained by communication from an external map server. When the current position deviates from the route of the search result by more than a predetermined distance or range,
The route search unit performs the search again,
The route information processing unit regenerates and processes the image data,
The photographed image capturing apparatus re-photographs the photographed image of the landscape,
The current position/azimuth identification unit re-identifies the current position and orientation of the photographed image capturing device,
The posture specifying unit respecifies the posture of the photographed image capturing apparatus,
The current position plotting unit re-plots the current position in the information of the searched route,
The guide instruction mark generation unit regenerates the guide instruction mark in the direction from the current position toward the destination,
The perspective-transformed image synthesizing unit perspective-transforms the guidance instruction mark to resynthesize the guidance instruction mark into the image of the route information,
The navigation device according to claim 1, wherein the image output device re-outputs an image based on the image data synthesized by the perspective conversion image synthesizing unit. The guide instruction mark generation unit is an L-shaped guide instruction mark that displays up to an intersection or a branch point of the image of the route, and a threshold value is predetermined for the length of the L-shaped guide instruction mark. The navigation device according to claim 1, wherein when the length of the L-shape is longer than the threshold value, the data of the instruction mark is generated so as to display the L-shape by dividing it in the middle. .. The navigation device according to claim 1, wherein route guidance information is provided by voice guidance in association with the display of the guidance instruction mark. The navigation device according to claim 6, wherein the voice guidance is performed according to the priority order of the route guidance information. In addition to the image based on the image data synthesized by the perspective-transformed image synthesizing unit, it has a function of displaying at least one of two-dimensional map and three-dimensional CG map images, and switches each image according to a scene. The navigation device according to claim 1, wherein the navigation device is output. The map information including the route from the starting point to the destination is acquired from the external map server to the storage device of the portable navigation device by communication, and the map information stored in the storage device is used from the starting point to the destination. A route search step for searching the route of
A route information processing step of generating and processing image data for displaying information of the route searched by the route searching step;
A live-action image capturing step of capturing a live-action image of the landscape by a live-action image capturing device attached to or built in the portable navigation device;
A current position/orientation identifying step for identifying the current position and orientation of the photographed image capturing device;
A posture specifying step for specifying the posture of the photographed image capturing apparatus,
A current position plotting step of plotting a current position in the information of the searched route,
A guide instruction mark generating step of generating a guide instruction mark in a direction from the current position toward the destination,
A perspective-transformed image synthesizing step of synthesizing the guidance instruction mark by subjecting it to perspective transformation and then superimposing and synthesizing the image.
An image output step of outputting an image based on the image data synthesized in the perspective transformation image synthesizing step.

Images