JP2011094992A - Navigation device, navigation method and navigation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To pinpoint the present location, even if reception of the radio waves from GPS satellites are not received, and to perform navigation on the basis of the pinpointed present location. <P>SOLUTION: A navigation device is constituted of: an imaged-object detecting section 12A to which images of different photographing magnifications are input from an imaging device 3, and which detects an imaged object from the images; a feature-component extracting section 12B for extracting a feature component of the imaged object; an imaged-object identifying section 12C, which refers to an image recognition database 14 and identifies the imaged object; an imaging-direction predicting section 12D for predicting a photographing direction by detecting an azimuth with an azimuth detector 5; an imaging distance calculating section 12E for calculating the distance from an imaging point up to the imaged object, based on the information on feature component aspect ratio and information on the different photographic magnifications; an imaging point identifying section 12F for retrieving a map information database 15, based on imaged-object identification information, imaging direction information, and imaging distance information, and identifying a current position of a user; and a navigation section 13A for performing route guidance, based on the identified imaging point. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a navigation device, a navigation method, and a navigation program.

  In a navigation device that provides route guidance from a departure place to a destination, portable navigation that incorporates navigation software in a portable terminal such as a mobile phone or a PDA (Personal Digital Assistant) and guides the user of the portable terminal to the destination. The device is known. These portable navigation devices detect radio waves from GPS (Global Positioning System) satellites to detect the current location of the user. However, in such a portable navigation device, in an environment where radio waves from GPS satellites cannot be received, for example, in places where radio waves from GPS satellites such as tunnels, subway stations, and buildings cannot be received, It becomes impossible to grasp the present location.

  Therefore, even in an environment where radio waves from GPS satellites cannot be received, a portable navigation device is known that can detect an accurate moving distance and moving direction using an acceleration sensor or the like and detect an accurate current position. (For example, refer to Patent Document 1). In addition, as a technique for calculating the distance from a target without using radio waves from a GPS satellite, an information processing device that generates an environment map around itself based on an acquired image of a camera is known (for example, Patent Document 2).

JP-A-2005-283386 JP 2008-304268 A

  In the portable navigation device disclosed in Patent Document 1, the position information serving as a reference is obtained by receiving radio waves from GPS satellites. For example, when power is turned on, radio waves from GPS satellites cannot be received. The current location cannot be detected. In addition, if the inclination and orientation of the portable navigation device change frequently, accurate navigation becomes difficult, and the user is forced to use the portable navigation device fixedly. Moreover, if the inclination and direction of the portable navigation device change frequently, the traveling direction must be calculated at all times, which consumes a lot of power.

  The information processing apparatus disclosed in Patent Document 2 only calculates the distance between the user and the target, calculates the distance between the user and the target while moving, and measures the size of the building Nothing is said about. Further, the information processing apparatus disclosed in Patent Document 2 cannot perform navigation.

  Furthermore, a conventional portable navigation device equipped with a GPS reception function consumes a large amount of power if the function of receiving radio waves from GPS satellites is always ON, and receives radio waves from GPS satellites only when used. When the function is turned on, radio waves from GPS satellites are captured each time it is turned on, so it takes time to grasp the current location, and areas where radio waves from GPS satellites cannot be received can be received When a user has a portable navigation device that takes time to grasp the current location to capture radio waves from GPS satellites when moving to a shopping mall or building in addition to the road, the user's There is a problem that it is difficult to map the current location on a map.

  Therefore, in view of the above-described problems, the present invention provides a navigation device, a navigation method, and a navigation program that accurately identify the current location without receiving radio waves from GPS satellites and perform navigation based on the identified current location. The purpose is to provide.

  A navigation device according to one aspect of the present invention includes an image input unit to which a plurality of images having different shooting magnifications are input, and an imaging object that is reflected in an imaging point and an image from images having different shooting magnifications input from the image input unit. And a controller that performs navigation based on the specified imaging point while determining the imaging point based on the distance.

  In addition to the configuration of the navigation device described above, the navigation device according to one aspect of the present invention includes an orientation detection unit that detects an orientation when an image is input by the image input unit, and an object to be identified. An image recognition database in which feature components are registered, and a map information database in which position information is registered, and the above-described control unit captures an imaged object from a plurality of images input by the image input unit An imaging object detection unit that detects the image, a feature component extraction unit that extracts a feature component of the imaging object detected by the imaging object detection unit, and an image from the feature component extracted by the feature component extraction unit with reference to the image recognition database An imaging object specifying unit for specifying an object, a shooting direction prediction unit for predicting a shooting direction from the direction information detected by the direction detection unit, and feature component size ratio information extracted from a plurality of images And an imaging distance calculation unit that calculates the distance from the imaging point to the imaging object based on different imaging magnification information, the imaging object information specified by the imaging object specifying unit, and the imaging direction information predicted by the imaging direction prediction unit The map information database is searched based on the distance information calculated by the imaging distance calculation unit, and the imaging point specifying unit that specifies the position output as the search result as the imaging point is specified by the imaging point specifying unit And a navigation unit that performs route guidance based on the captured imaging point.

  In addition to the configuration of any of the navigation devices described above, the navigation device according to one aspect of the present invention preferably includes a current location search instruction unit that instructs the imaging point specifying unit to search for the user's current location.

  In addition to the configuration of any of the navigation devices described above, the navigation device according to one aspect of the present invention is configured to capture the user's current location every predetermined time after the user's current location is identified by the imaging location identifying unit. It is preferable to have a current location search unit for searching by

  In addition to the configuration of any of the navigation devices described above, the navigation device according to one aspect of the present invention includes the current location information already specified by the imaging point specifying unit and the current location specified by the current location searching unit after a predetermined time has elapsed. It is preferable to have an average moving distance calculation unit that calculates the average moving distance of the user based on the information.

  In addition to the configuration of any of the navigation devices described above, the navigation device according to one aspect of the present invention further includes a search range determination unit that determines a search range for specifying the current location of the user based on the average moving distance information. It is preferable to have.

  In addition to the configuration of any of the navigation devices described above, a navigation device according to one aspect of the present invention includes a designation unit that designates a destination and / or waypoint by an input from a user. The route is generated based on the current location information and the destination and / or waypoint information specified by the specification unit, and the search range determination unit calculates the elapsed time information from the time point searched by the current location search unit and the average travel distance It is preferable to determine a search range for searching the current location of the user based on the average moving distance information of the user calculated by the unit and the route information generated by the navigation unit.

  A navigation method according to one aspect of the present invention is a navigation method executed by a navigation device having an image input unit and a control unit, and includes an image input step in which a plurality of images having different shooting magnifications are input, and an image input The distance between the imaging point and the imaged object captured in the image is obtained from images with different shooting magnifications input from the step, the imaging point is specified based on the distance, and the navigation is performed based on the specified imaging point. And a control step for performing.

  The navigation method according to one aspect of the present invention is the navigation method described above, in which the navigation device includes an orientation detection unit that detects an orientation when an image is input by the image input unit, and an object to be identified. It has an image recognition database in which feature components are registered, and a map information database in which position information is at least registered, and the control unit includes an imaging object detection unit, a feature component extraction unit, an imaging object identification unit, and imaging A direction prediction unit, an imaging distance calculation unit, an imaging point specifying unit, and a navigation unit are included, and the imaging object detection unit detects an imaging object from a plurality of images input in the image input step in the control step. Image-capturing object detection step, and feature-component extraction unit extracts a characteristic component of the image-captured object detected by the image-capturing object detection step The imaging object specifying unit refers to the image recognition database, specifies the imaging object from the feature component extracted in the feature component extraction step, and the orientation information detected by the imaging direction prediction unit by the orientation detection unit. A shooting direction prediction step for predicting a shooting direction from the imaging direction, and an imaging distance calculation unit calculates a distance from the imaging point to the imaged object based on the feature component size ratio information extracted from the plurality of images and the different imaging magnification information. An imaging distance calculating step, an imaging point specifying unit, the imaging object information specified in the imaging object specifying step, the shooting direction information predicted in the shooting direction prediction step, and the distance information calculated in the imaging distance calculation step; An imaging point specifying step for searching the map information database based on the image and specifying the position output as the search result as an imaging point; Navigation unit, a navigation step of performing route guidance based on the imaging point identified by the imaging point specifying step, are intended to be included.

  In addition, a navigation program according to one aspect of the present invention includes a computer that captures an imaging point and an image from an imaging input unit that receives a plurality of images with different shooting magnifications and an image with different shooting magnifications that is input from an image input unit. The distance between the captured object and the imaged object is obtained, and an imaging point is specified based on the distance, and the navigation unit is caused to function as navigation means based on the specified imaging point.

  In the navigation program according to one aspect of the present invention, in the above-described navigation program, the above-described computer has an image recognition database in which feature components for specifying an imaged object are registered, and at least position information is registered. A map information database, and the control means includes an imaging object detection means for detecting an imaging object from a plurality of images input by the imaging input means, and an imaging object detected by the imaging object detection means. Feature component extraction means for extracting feature components, image recognition database, image pickup object specifying means for specifying an image pickup object from the feature components extracted by the feature component extraction means, and orientation when an image is input by the image pickup input means Direction detection means for detecting the shooting direction, shooting direction prediction for predicting the shooting direction from the direction information detected by the direction detection means The imaging object information specified by the imaging object calculating means and the imaging object specifying means for calculating the distance from the imaging point to the imaging object based on the characteristic component size ratio information extracted from the plurality of images and the different imaging magnification information. The map information database is searched based on the shooting direction information predicted by the shooting direction prediction unit and the distance information calculated by the shooting distance calculation unit, and the position output as the search result is specified as the shooting point. An imaging point specifying unit that performs the route guidance based on the imaging point specified by the imaging point specifying unit.

  According to the present invention, it is possible to provide a navigation device, a navigation method, and a navigation program that accurately specify the current location without receiving radio waves from GPS satellites and perform navigation based on the specified current location.

It is a block diagram which shows the whole structure of the navigation apparatus which concerns on the Example of this invention. It is a figure for demonstrating each function which CPU shown in FIG. 1 reads and implement | achieves the various programs stored in the memory | storage device to memory. It is a flowchart which shows the imaging point specific process of the navigation apparatus shown in FIG. FIG. 4 is a diagram for explaining an example of an imaging point specifying process shown in FIG. 3, (A) is a diagram for explaining a process of extracting a specific keyword from an imaged object, and (B) is a user FIG. 6C is a diagram for explaining a process of predicting the direction in which the camera is facing, and FIG. 8C illustrates from the information of the image input by the first shooting and the image input by the second shooting from the user to the captured object. It is a figure for demonstrating the process which calculates the distance of (D), (D) is a figure for demonstrating the process which pinpoints the present location. It is a flowchart which shows the navigation process of the navigation apparatus shown in FIG. It is a flowchart which shows the navigation process of the navigation apparatus shown in FIG. FIGS. 7A and 7B are diagrams for explaining an example of the navigation process shown in FIGS. 5 and 6, and FIG. 7A is a diagram for explaining a first current location search process, and FIG. It is a figure for demonstrating a process, (C) is a figure for demonstrating the 3rd present location search process. FIG. 7 is a diagram for explaining an example of the navigation processing shown in FIGS. 5 and 6, (A) is a diagram for explaining a fourth current location search process, and (B) is a position mismatch. It is a figure for demonstrating the process which re-searches the route | root to the destination in the case of being. It is a flowchart which shows the recovery process from the rest of the navigation apparatus shown in FIG. It is a figure which shows an example of the recovery process shown in FIG.

  Embodiments of the present invention will be described below with reference to FIGS. However, the present invention is not limited to the following examples.

  The navigation apparatus 1 according to the embodiment of the present invention accurately specifies the user's current location without receiving radio waves from a GPS (Global Positioning System) satellite, and also specifies a destination or route specified by the user from the specified current location. It is possible to navigate to the ground.

  FIG. 1 is a block diagram showing an overall configuration of a navigation device 1 according to an embodiment of the present invention. As shown in FIG. 1, this navigation device 1 includes a CPU (Central Processing Unit) 2, an imaging device 3, a graphic processor 4, an orientation detection device 5, an operation unit 6, a display unit 7, a communication device 8, a memory 9, and a video. A memory 10 and a storage device 11 are provided. The storage device 11 stores not only basic software (such as an OS) (not shown) for operating the navigation device 1, but also a current location specifying program 12, a navigation program 13, an image recognition database 14, a map information database 15, and the like. ing.

  Here, a control unit, an imaging object detection unit, a feature component extraction unit, an imaging object identification unit, an imaging direction prediction unit, an imaging distance calculation unit, an imaging point identification unit, a navigation unit, a current location search unit, an average moving distance calculation unit, a search The CPU 2 as a part of each range determination unit is a central processing unit that executes various arithmetic processes in accordance with various programs stored in the storage device 11 or the memory 9 and controls each unit of the navigation device 1. .

  The imaging device 3 serving as an image input unit can capture and input scenery around the current location of the user as a plurality of images with different imaging magnifications. The imaging device 3 is composed of, for example, a small video camera. The photographing process and the input process in the imaging apparatus 3 are performed by operating the operation unit 6 when the object is displayed on the screen while watching the video displayed on the display unit 7 described later. This is done by capturing. Note that an image input by photographing is temporarily stored in the memory 9 and displayed on the display unit 7, and can be stored in the storage device 11 by the user operating the operation unit 6.

  The graphic processor 4 executes drawing processing according to a command list read from the storage device 11 and developed in the video memory 10 described later, converts the obtained image into a video signal, and supplies the video signal to the display unit 7. A drawing processing apparatus to be displayed.

  The direction detection device 5 as the direction detection unit can detect the direction in which the navigation device 1 is facing. The direction detection device 5 is configured by, for example, an electronic compass.

  The current location search instruction unit and the operation unit 6 as an instruction unit are interfaces used when the user operates the navigation device 1. The operation unit 6 includes, for example, numeric buttons, operation key buttons, a determination button, a menu selection button (all not shown), and the like. Further, when the user operates the operation unit 6, a scene around the current location of the user is photographed via the imaging device 3, a photographing magnification is set, a process for specifying the current location is performed, and a navigation process is started. , And so on.

  The display unit 7 is configured by, for example, a liquid crystal display, an organic EL display, and the like, and outputs a result of an instruction from the user via the operation unit 6 to the navigation device 1. For example, the display unit 7 displays an image related to the scenery around the user on the imaging device 3 and displays an image input when a shooting button icon (not shown) displayed on the screen is pressed as a still image. A map image including the current location can be displayed, or a navigation route can be displayed as an image. Note that the display unit 7 may be configured with, for example, a touch panel to display icons that perform the functions of the operation unit 6 described above.

  The communication device 8 is a device for exchanging information with a base station (not shown) via radio waves with an antenna (not shown). For example, an IMT-2000 mobile phone network or a PHS (Personal Handy Phone System). It is a device that exchanges information using a communication system such as.

  The memory 9 is composed of a semiconductor storage device such as a ROM (Read Only Memory) and / or a RAM (Random Access Memory) (not shown), stores a program executed by the CPU 2, and the like to be executed by the CPU 2. Program or data to be stored temporarily.

  The video memory 10 is a semiconductor storage device that temporarily stores a command list to be processed by the graphic processor 4.

  The storage device 11 stores data and programs necessary for operating the navigation device 1. The storage device 11 is composed of a storage medium such as an HDD (Hard Disk Drive) or a flash memory.

  The current location specifying program 12 is a program for specifying the current position of the navigation device 1 by specifying an imaging point from a plurality of input images with different shooting magnifications. The current location specifying program 12 includes an image pickup object detection module, a feature component extraction module, an image pickup object specification module, an image pickup direction prediction module, an image pickup distance calculation module, and an image pickup point specification module. The navigation program 13 includes a navigation module, a current location search module, an average moving distance calculation module, and a search range determination module.

  FIG. 2 is a diagram for explaining each function realized by the CPU 2 shown in FIG. 1 reading various programs stored in the storage device 11 into the memory 9. As illustrated in FIG. 2, the navigation device 1 includes an imaging object detection unit 12A, a feature component extraction unit 12B, an imaging object identification unit 12C, an imaging direction prediction unit 12D, an imaging distance calculation unit 12E, an imaging point identification unit 12F, and a navigation unit 13A. The present location searching unit 13B, the average moving distance calculating unit 13C, and the search range determining unit 13D are provided.

  The photographed object detection unit 12 </ b> A is realized by the CPU 2 reading the imaged object detection module into the memory 9. The imaging object detection module corrects distortion, rotation, and the like due to three-dimensional bias included in two images with different imaging magnifications input by the imaging device 3, and detects and outputs an area estimated to be an imaging object. It is a module to do. Specifically, the imaging object detection module includes characters, signs, buildings, terrain, structure shapes, etc. included in each image (including bridges, signs, road shapes, etc. )) Image feature pattern and shape feature pattern. For example, in order to detect image feature patterns and shape feature patterns, the imaging object detection module is universal in that the density contrast between the character line and the background increases, and the circumscribed rectangle of the character is often close to a square. By utilizing this characteristic, a region with high density contrast and a region where the circumscribed rectangle is close to a square are detected. Note that any detection method other than the above-described detection methods may be used as long as it can detect the shape of characters and structures from the input image in the imaging object detection module. When the imaging object detection module detects a plurality of areas from the input image, the imaging object detection module outputs the plurality of areas.

  The feature component extraction unit 12B is realized by the CPU 2 reading the feature component extraction module into the memory 9. The feature component extraction module extracts and outputs the shape pattern of the region detected by the imaging object detection module or the feature pattern related to the image. When there are a plurality of areas detected by the imaging object detection module, the feature component extraction module extracts and outputs an image feature pattern and a shape feature pattern for each of the areas.

  The imaged object specifying unit 12C is realized by the CPU 2 reading the imaged object specifying module into the memory 9. The captured object specifying module performs pattern matching between the feature pattern registered in the image recognition database 14 (to be described later) and the feature pattern extracted and output by the feature component extraction module, and specifies the captured object from the matching feature pattern. And a character string (character, signboard, building, terrain, name corresponding to the shape of the structure, etc.) corresponding to the specified feature pattern, etc. is output. In addition, when a plurality of image feature patterns and shape feature patterns are output from the feature component extraction module, the captured object specifying module outputs a character string corresponding to each pattern.

  The imaging direction prediction unit 12D is realized by the CPU 2 reading the imaging direction prediction module into the memory 9. The imaging direction prediction module predicts the imaging direction of the user from the azimuth angle detected by the azimuth detection device 5.

  The imaging distance calculation unit 12E is realized by the CPU 2 reading the imaging distance calculation module into the memory 9. The imaging distance calculation module calculates the distance from the user to the imaging object (target object) based on the two images with different imaging magnifications input by the imaging device 3.

  The imaging point specifying unit 12F is realized by the CPU 2 reading the imaging point specifying module into the memory 9. The imaging point identification module acquires the imaging direction information output from the imaging direction prediction module, the character string output from the imaging object identification module, and the distance information output from the imaging distance calculation module, and the map information database 15 To obtain coordinate data as an imaging point. Then, the imaging point specifying module causes the display unit 7 to display the map data together with the map data subjected to map matching on the display unit 7 based on the acquired coordinate data. Note that, when performing the navigation process, the imaging point specifying module outputs the coordinate data as the current location of the user to the navigation module.

  The navigation unit 13 </ b> A is realized by the CPU 2 reading the navigation module into the memory 9. The navigation module generates a navigation route based on the coordinate data that is the current location of the user output from the imaging point specifying module and the coordinate data corresponding to the destination input by the operation unit 6.

  The current location search unit 13B is realized by the CPU 2 reading the current location search module into the memory 9. The current location search module reads the imaging point specifying module included in the current location specifying program 12 into the memory 9 every predetermined time, and performs the current location specifying process. In addition, the imaging location specifying module included in the current location specifying program 12 specifies the search range of the map information database 15 and specifies the current position when the search range is specified from the search range determination module described later. Do. Note that when the navigation processing is started, if two images having different shooting magnifications are not newly input from the imaging device 3, the current location search module captures images in order to input these images. A message prompting the user to display the message may be displayed on the display unit 7.

  The average moving distance calculation unit 13 </ b> C is realized by the CPU 2 reading the average moving distance calculation module into the memory 9. The average moving distance calculation module calculates and outputs the average moving distance of the user based on the plurality of coordinate data specified by the current location specifying process.

  The search range determination unit 13D is realized by the CPU 2 reading the search range determination module into the memory 9. The search range determination module determines a range for searching the current location based on the average travel distance information output from the average travel distance calculation module and the navigation route information output from the navigation module, and outputs the range to the current location search module To do.

  In the image recognition database 14, image patterns and shape patterns of characters, signs, buildings, terrain, and structures are registered. Accordingly, the image recognition database 14 receives a search query for specifying an imaged object from the imaged object specifying module, and matches characters, signboards, buildings, terrain, and structures that match the image pattern and shape pattern included in the search query. The name of the shape of the object can be searched and output.

  The map information database 15 stores information for displaying a map and POI information (Point Of Interest: equivalent to the position information in the claims). Information for displaying a map includes a road layer, a background layer for displaying objects on the map, a character layer for displaying characters such as city names, and a POI icon for displaying a POI icon at a predetermined position of the POI. It consists of layers. The road layer has road link data, node data, and the like, and is used for route search processing and map matching processing. The background layer is divided into one unit divided into a predetermined longitude width and latitude width, and has graphic data representing a planar shape such as a road, a building, a park, a river, and a lake in each unit. The POI icon layer has POI information existing for each unit divided into the same units as the background layer. POI information includes POI position information (longitude / latitude information, coordinate information, etc.), POI icon data, road name facing POI, road link data where POI exists, node data, POI category name, POI name , POI image feature data, and the like. Therefore, the map information database 15 can perform POI information search processing in addition to route search processing and map matching processing. Therefore, the map information database 15 receives a search query for specifying the current location from the imaging point specifying module described above, searches for POI information that matches the character string included in the query, and corresponds to the POI information. Position information can be output.

  Then, the present location specific process of the navigation apparatus 1 of an Example is demonstrated. FIG. 3 is a flowchart showing the current location specifying process of the navigation device 1 shown in FIG.

  START: The CPU 2 starts the current location specifying process in response to the current location specifying processing request generated in accordance with the operation from the operation unit 6 performed by the user. Specifically, the CPU 2 reads each module of the current location specifying program 12 into the memory 8 and realizes each function described above.

  Step S1: In accordance with an operation performed by the user from the operation unit 6, a captured still image is input to the navigation device 1 via the imaging device 3 with the imaging magnification of the imaging device 3 set to X. For example, the CPU 2 may display a message such as “Please photograph nearby characters, signs, and buildings” on the display unit 7 to prompt the user to perform the above-described imaging operation.

  Step S2: The CPU 2 detects the azimuth by the azimuth detecting device 5 in order to obtain the photographing direction in step 1.

  Step S3: A captured still image is input to the navigation device 1 via the imaging device 3 by changing the imaging magnification of the imaging device 3 to 2 × in accordance with an operation from the operation unit 6 performed by the user. For example, the CPU 2 may display a message such as “Please shoot again the object that was shot earlier” on the display unit 7 to prompt the user to perform a second imaging operation. In addition, the shooting magnification is set as twice that at the time of shooting in step S1, but other magnifications (0.5 times, 3 times, etc. at the time of shooting in step S1) are set, or the user performs it. The shooting magnification may be designated according to the operation from the operation unit 6.

  Step S4: The CPU 2 detects the azimuth by the azimuth detecting device 5 in order to obtain the photographing direction in step 3.

  Step S5: The CPU 2 analyzes the images with different photographing magnifications input in Step S1 and Step S3, and extracts image features. Specifically, the CPU 2 extracts image features by the functions of the above-described captured object detection unit 12A and feature component extraction unit 12B.

  Step S6: The CPU 2 generates a search query based on the image feature extracted in step S5, and refers to the image recognition database 14 to search for a character, a signboard, or a building having a matching image feature. To do. Specifically, the CPU 2 searches for matching image features by the function of the above-described captured object specifying unit 12C. If there is a matching image feature (YES in step S6), the CPU 2 proceeds to the process in step S8. On the other hand, if the CPU 2 determines that there is no match in the search process of step S6 according to the operation of the operation unit 6 (NO in step S6), the CPU 2 proceeds to the process of step S7.

  Step S7: The CPU 2 determines again whether or not there is a current location specifying process request. Specifically, a message saying “No match was found. Do you want to identify the current location again?” Is displayed on the display unit 7. If there is no current location specifying process request from the user through the operation from the operation unit 6 (YES in step S7), the CPU 2 ends the current location specifying process (END). On the other hand, when the current location specifying process request is made again (NO in step S7), the CPU 2 returns the process to step S1.

  Step S8: The CPU 2 lists the names of the picked-up object candidates determined to be matched by the search process of step S6 and displays them on the display unit 7. The CPU 2 specifies the imaged object in accordance with the operation of the operation unit 6. The CPU 2 may display a message candidate list on the display unit 7 and display a message such as “Please select a photographed object”, for example. CPU2 will transfer to the process of step S9, if a to-be-photographed object is specified by a user operating the operation part 6. FIG.

  Step S9: The CPU 2 calculates a dimensional ratio with respect to the image size of the characteristic building included in the image of Step S1. Specifically, the CPU 2 calculates the dimensional ratio by the function of the imaging distance calculation unit 12E described above.

  Step S10: The CPU 2 calculates a dimensional ratio with respect to the image size of the characteristic building included in the image of Step S3. Specifically, the CPU 2 calculates the dimensional ratio by the function of the imaging distance calculation unit 12E described above.

  Step S11: The CPU 2 calculates the amount of change in the dimensional ratio calculated in steps S9 and S10. Specifically, the CPU 2 calculates the change amount of the dimensional ratio by the function of the imaging distance calculation unit 12E described above.

  Step S12: The CPU 2 calculates the distance between the building that is the imaged object and the user from the amount of change obtained in step S11 and the magnification ratio of the captured image. Specifically, the CPU 2 calculates the imaging distance by the function of the imaging distance calculation unit 12E described above.

  Step S13: The CPU 2 determines a direction in which the user is assumed to be facing from the azimuth angle detected in steps S2 and S4. Specifically, the CPU 2 predicts the imaging direction by the function of the imaging direction prediction unit 12D described above.

  Step S14: The CPU 2 refers to the map information database 15 using the imaged object information specified in step S8, the imaging distance information calculated in step S12, and the user direction determined in step S13 as a search key. The point corresponding to the current position of is calculated. Specifically, the CPU 2 calculates the position by the function of the imaging point specifying unit 12F described above.

  Step S15: The CPU 2 determines the position calculated in step S14 as the current location of the user and ends the process (END).

  FIG. 4 is a diagram for explaining an example of the current location specifying process shown in FIG. 3, and FIG. 4A is a diagram for explaining a process of extracting a specific keyword from the imaged object, and FIG. FIG. 10 is a diagram for explaining a process of predicting a direction in which the user is facing, and FIG. 8C is a diagram illustrating an image input from the first shooting and an image input from the second shooting from the user. It is a figure for demonstrating the process which calculates the distance to a to-be-photographed object, (D) is a figure for demonstrating the process which pinpoints the present location.

  For example, the user operates the operation unit 6 of the navigation device 1 to photograph twice the same portion around the user from a certain point on the map shown in FIG. Then, the image input by the first shooting is analyzed by the imaging object specifying module of the current location specifying program 12 and the keywords “Honbashi Station”, “RJ”, and “Honbashi Okyo Station” are extracted. From these keyword information, it is expected that the area around the black star in the figure is photographed. Then, as shown in FIG. 4B, the azimuth angle at the time of the first shooting is detected to be 205 degrees toward the imaged object by the imaging direction prediction module of the current location specifying program 12. From this detected azimuth angle, it is expected that the location where the image was taken is in the direction of 205 degrees toward the imaged object. Then, as shown in FIG. 4C, the distance between the imaging object such as a building or a signboard and the user shown in the image input by the second shooting is calculated by the imaging distance calculation module of the current location specifying program 12. . Then, as shown in FIG. 4D, the current location of the user is specified by the imaging point specifying module of the current location specifying program 12 based on the extracted character data, the azimuth angle, and the distance to the imaged object.

  As described above, in the current location specifying processing method of the navigation device 1, the current location peripheral information is acquired from the image captured by the user, and the current location is specified based on the acquired information. Therefore, the navigation device 1 can accurately identify the current location of the user without receiving radio waves from GPS satellites. In addition, since the navigation device 1 does not always need to receive radio waves from GPS satellites, power consumption can be reduced as compared with a method of receiving radio waves from GPS satellites and grasping the current position. In addition, since the navigation device 1 grasps the current location without receiving radio waves from GPS satellites, the current location can never be grasped depending on whether or not radio waves from GPS satellites can be received.

  Next, the navigation process of the navigation device 1 according to the embodiment will be described. 5 and 6 are flowcharts showing the navigation process of the navigation device 1 shown in FIG. The average movement distance k in the following description is calculated by the average movement distance calculation module, is an average movement distance from the point where the navigation process is started, and 0 is set as the initial value.

  START: The CPU 2 starts navigation processing in response to a navigation processing request generated in accordance with an operation from the operation unit 6 performed by the user. Specifically, the CPU 2 reads each module of the navigation program 13 into the memory 8 and realizes each function described above.

  Step S20: The CPU 2 specifies the current location by performing the current location search process shown in FIG. Specifically, the CPU 2 performs current location search processing by the function of the current location search unit 13B described above.

  Step S21: The CPU 2 searches for a route from the current location specified in step S20 to the destination, and starts route guidance to the destination. Specifically, the CPU 2 searches for a route by the function of the navigation unit 13A described above. For the destination and waypoint, for example, a message such as “Please specify the destination and waypoint” may be displayed on the display unit 7 to prompt the user to input. Further, when there are a plurality of search routes, the search routes may be listed and displayed on the display unit 6, and the navigation route may be determined by selection from the user.

  Step S22: The CPU 2 executes the current location search process shown in FIG. 3 every predetermined time (for example, every t minutes) until it reaches the destination. Specifically, the CPU 2 executes current location search processing by the function of the current location search unit 13B described above.

  Step S23: The CPU 2 determines whether or not the average moving distance k is other than zero. When the moving distance k is other than 0, the CPU 2 proceeds to the process of step S24, and when it is 0, the CPU 2 returns to the process of step S22.

  Step S24: The CPU 2 calculates the expected moving distance m based on the average moving distance k. Specifically, the expected moving distance m is calculated by multiplying the value of the average moving distance k by the elapsed time from the time when the current location search process was executed last time.

  Step S25: The CPU 2 calculates the predicted current location P from the predicted moving distance m and the navigation route.

  Step S <b> 26: The CPU 2 detects the azimuth angle θ by the azimuth detection device 5.

  Step S27: In accordance with an operation from the operation unit 6 performed by the user, the captured still image P1 with the shooting magnification of the imaging device 3 set to X is input to the navigation device 1 via the imaging device 3. For example, the CPU 2 may display a message such as “Please photograph characters, signs, and buildings in the vicinity of the traveling direction” on the display unit 7 to prompt the user to perform an imaging operation.

  Step S28: A still image P2 captured by changing the shooting magnification of the imaging device 3 to 2X in accordance with an operation from the operation unit 6 performed by the user is input to the navigation device 1 via the imaging device 3. For example, the CPU 2 may display a message such as “Please shoot again the object that was shot earlier” on the display unit 7 to prompt the user to perform a second imaging operation. In addition, the shooting magnification is set as twice that of step S1, but other shooting magnifications (0.5 times, 3 times, etc.) are set, or shooting is performed according to an operation from the operation unit 6 performed by the user. You may specify the magnification.

  Step S29: The CPU 2 determines whether or not the traveling direction at the predicted current location P matches the azimuth angle θ detected in step S26. If they match (YES in step S29), it is determined that there is a high possibility of being on the navigation route, and the process proceeds to step S30. If they do not match (NO in step S29), the route may not be on the navigation route. Is determined to be high, and the process proceeds to step S31.

  Step S30: The CPU 2 searches the POI information around the predicted current position P (for example, the area within the radius α km with the predicted current position P as the center) and analyzes the images P1 and P2 input in step S28 and step S29. Then, an image feature is extracted to specify an imaged object. Specifically, the CPU 2 analyzes each image by each function of the search range determination unit 13D, the imaging object detection unit 12A, the feature component extraction unit 12B, and the imaging object identification unit 12C, extracts an image feature, and selects an imaging object. Identify.

  Step S31: The CPU 2 searches the POI information around the predicted current position P (for example, the area within the radius β km around the predicted current position P) and analyzes the images P1 and P2 input in step S28 and step S29. Then, an image feature is extracted to specify an imaged object. Note that the relationship between α in step S30 and β in step S31 is set to satisfy α <β.

  Step S32: The CPU 2 determines the distance between the imaged object specified in step S30 or step S31 and the user based on the imaging magnification ratio of the captured images P1 and P2 and the size ratio of the imaged object. Is calculated.

  Step S33: The CPU 2 predicts the direction of the user predicted from the azimuth angle θ detected in step S26, the imaged object information specified in step S30 or step S31, the imaging distance information calculated in step S32, The position is calculated with reference to the map information database 15 using as a search key. In addition, the direction which the user faces predicted from the azimuth angle θ detected in step S26 is predicted to be an approximate direction such as east, west, south, and north.

  Step S34: The CPU 2 determines the position calculated in step S33 as the current location of the user.

  Step S35: The CPU 2 calculates an average moving distance k for t minutes after starting the route guidance. Specifically, the CPU 2 calculates the average movement distance k by the function of the average movement distance calculation unit 13C described above.

  Step S36: The CPU 2 performs a current location search process to determine whether or not the vehicle is on the navigation route (on the route). If it is determined that the CPU 2 is on the navigation route (YES in step S36), the process proceeds to step S37. On the other hand, when it is determined that the CPU 2 is not on the navigation route (NO in step S36), the CPU 2 performs a reroute search (step S38).

  Step S37: The CPU 2 determines whether or not the destination has been reached. If it is determined that the user is at the destination (YES in step S37), the CPU 2 ends the navigation process (END). On the other hand, when it is determined that the destination has not been reached (NO in step S37), the CPU 2 proceeds to the process of step S22.

  FIG. 7 is a diagram for explaining an example of the navigation processing shown in FIGS. 5 and 6, (A) is a diagram for explaining the first current location search processing, and (B) is 2 It is a figure for demonstrating the present location search process of the time, (C) is a figure for demonstrating the present location search process of the 3rd time. FIG. 8 is a diagram for explaining an example of the navigation processing shown in FIG. 5 and FIG. 6, (A) is a diagram for explaining the fourth current location search processing, and (B) is a position diagram. It is a figure for demonstrating the process which re-searches the route | root to the destination when does not correspond.

  As shown in FIG. 7A, the user at the start point causes the navigation apparatus 1 to start the navigation process in order to guide the route to the goal point. Then, the navigation device 1 performs the current location search process to identify the current location of the user and starts route guidance from the identified current location to the goal point that is the destination. Then, as shown in FIG. 7B, the navigation device 1 performs a second current location search process t minutes after the start. The navigation device 1 performs the same process as the first current position search process for the current position search process, but stores the average moving distance k that has traveled for t minutes from the start. Subsequently, as shown in FIG. 7C, the navigation device 1 performs a third current location search process 2t minutes after the start. In the third current location search process, the predicted point on the route guidance is specified in consideration of the average moving distance k1 stored in the second current location search process and the direction in which the user is facing. The navigation device 1 compares the traveling direction at the specified predicted current position with the azimuth angle detected at present, and since it is the same or substantially the same direction, the current position search is limited to a predetermined range from the predicted position. Process. The navigation device 1 also stores the average travel distance k2 that has traveled from t minutes to 2t minutes even in the third current location search process. Further, as shown in FIG. 8A, the navigation device 1 performs a fourth current location search process 3t minutes after the start. In the fourth current location search process, the same processing as the third current location search process is performed using the average moving distances k1 and k2. However, in the fourth current location search process, the traveling direction at the identified predicted point and the azimuth angle detected at present are not the same or substantially the same direction, so the third current location search from the predicted point The current location search process is performed only within a predetermined range that is wider than the range. As a result, as shown in FIG. 8B, since it is specified that the user is at a position off the route, the navigation is continued by performing a process of re-searching the route to the destination.

  As described above, in the navigation method of the navigation device 1, the current location search process is repeated to accurately identify the current location of the user who is the imaging point without receiving radio waves from the GPS satellites, and the user is determined based on the identified current location. Route guidance can be performed on In the second and subsequent current location search processing, the average movement distances k1 and k2 of the user are calculated. Therefore, the search range that is the target of the current location search processing is limited, and the amount of calculation is reduced, thereby reducing power consumption. It becomes possible.

  Next, the recovery process from the pause of the navigation device 1 according to the embodiment will be described. FIG. 9 is a flowchart showing a recovery process from the pause of the navigation apparatus 1 shown in FIG. Note that the term “pause” here refers to a case where the power of the navigation device 1 is turned off or a case where the navigation process is temporarily interrupted by a user operation.

  START: The CPU 2 pauses once during the navigation process of the navigation device 1 and starts the recovery process when it is resumed. In the following processing, it is assumed that the average moving distance k is already a value other than 0, which is an initial value.

  Step S40: The CPU 2 calculates the pause time z of the navigation device 1. The pause time z can be calculated by storing the time at the previous current location search process in advance and comparing it with the current time.

  Step S41: The CPU 2 calculates the expected moving distance m using the average moving distance k and the pause time z. Specifically, a value obtained by multiplying the average moving distance k by the pause time z is set as the expected moving distance m.

  Step S42: The CPU 2 determines the search area Y from the expected moving distance m. Specifically, an area within a predetermined range (for example, a radius of 50 m) is defined as a search area Y centered on a point that has moved by an estimated movement distance m from the previously specified point.

  Step S43: The CPU 2 performs a current location specifying process from the search area Y. Specifically, when searching with reference to the map information database 15, the current location specifying process is performed using the POI information included in the search area Y as a search target.

  Step S44: The CPU 2 calculates an average moving distance k for t minutes after starting the route guidance.

  Step S45: The CPU 2 performs a current location specifying process to determine whether or not the vehicle is currently on the navigation route (on the route).

  Step S46: If it is determined that the user is on the navigation route (YES in step S45), the CPU 2 proceeds to the process of step S48, restarts the navigation process, and ends the recovery process (END). If it is determined that the route is not on the navigation route (NO in step S45), the CPU 2 performs route search processing again (step S47).

  FIG. 10 is a diagram illustrating an example of the recovery process illustrated in FIG. 9. As shown in FIG. 10, when the navigation apparatus 1 is turned off and restarted after being turned on again in a situation where the current location search process is performed at least twice, the pause time z is set to 3t minutes. The predicted current location is calculated by multiplying the added time by the average moving distance k calculated at the time of the second current location search process, and the current location search is limited to a predetermined range centered on the calculated predicted current location. Yes.

  Such a recovery process of the navigation device 1 takes more time than a process of receiving radio waves from GPS satellites and specifying the current location even if the power is turned off during route guidance. Therefore, power consumption can be suppressed. In addition, by limiting the search range of the current location search process in consideration of the downtime, it is possible to reduce the calculation amount of the current location search process.

  In the embodiment described above, the current location is specified from information included in a plurality of images with different shooting magnifications taken by the user. Therefore, it is possible to identify the current location and perform navigation based on the identified current location without receiving radio waves from GPS satellites.

(Other embodiments)
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention. For example, an image around the current location of the user is captured in accordance with the operation of the operation unit 6 and an image is captured and input each time. May be controlled so that the navigation apparatus 1 inputs a plurality of captured images.

  In the above-described embodiment, two images having different shooting magnifications are input to the navigation device 1. In addition, three or four images having different shooting magnifications are input to the navigation device 1. The captured object included in the image is identified, the dimensional ratio of the captured object to the image size is determined, the distance from the user (imaging location) to the position of the captured object is determined from the shooting magnification ratio, etc., and the current location specifying process is performed It may be.

  When a plurality of different image features are obtained from three or four images having different shooting magnifications by the above-described method, a candidate for an image that satisfies all combinations of the plurality of image features, or these image features The degree of matching of the character strings obtained from the above satisfies a predetermined criterion (for example, when a majority or half of these image features are to output the same character string, it is determined that the criterion is satisfied, and In the following cases, it may be determined that only the imaged object candidate is displayed on the display unit 7.

  In the above-described embodiment, the image recognition database 14 and the map information database 15 are built in the navigation device 1, but the image recognition database 14 and the map information are also connected to a server (not shown) that can be connected via a network. The database 15 may be built in and the captured image may be transmitted from the navigation device 1 to the server. In this case, the server that has received the captured image is configured to transmit the results of the above-described current location specifying process, navigation process, and recovery process to the navigation apparatus 1. Such a navigation apparatus 1 can identify the current location as an imaging point without performing the image recognition database 14 and the map information database 15 by itself, and can perform navigation processing based on the identified current location.

  Further, only the image recognition database 14 described above is built in the navigation device 1, and a search query including a character string that is a recognition result, distance information from the imaged object, and azimuth angle information is built in the map information database 15. You may make it transmit to a server. In that case, the navigation device 1 is configured so that the current location specifying process result is returned from the server. Such a navigation device 1 can identify the current location as an imaging point without performing the map information database 15 itself, and can perform navigation processing based on the identified current location.

  Alternatively, only the map information database 15 described above may be incorporated in the navigation device 1 and the captured image may be transmitted to a server incorporating the image recognition database 14. In this case, the server that receives the captured image is configured to return to the navigation device 1 the result of calculating the captured object specified from the image and the distance to the captured object. Such a navigation apparatus 1 can identify the current location as an imaging point without performing the image recognition database 14 itself, and can perform navigation processing based on the identified current location.

  In addition, the above-described programs (current location specifying program 12, navigation program 13) are recorded in advance on a computer-readable recording medium, and the above-described program is installed in another computer, whereby the navigation device 1 of the embodiment is used. You may make it function as a means to implement | achieve each function of these.

DESCRIPTION OF SYMBOLS 1 ... Navigation apparatus, 2 ... CPU (a control part, an imaging object detection part, a characteristic component extraction part, an imaging object specific | specification part, an imaging | photography direction estimation part, an imaging distance calculation part, an imaging point specific | specification part, a navigation part, a present location Search unit, average moving distance calculation unit, part of search range determination unit), 3 ... imaging device (image input unit), 4 ... graphic processor, 5 ... direction detection device (direction detection unit) , 6... Operation unit (current location search instruction unit, designation unit), 7 display unit, 8 communication device, 9 memory, 10 video memory, 11 storage device , 12... Current location specifying program (control unit, imaging object detection unit, feature component extraction unit, imaging object specification unit, imaging direction prediction unit, imaging distance calculation unit, part of imaging point specification unit),.・ Navigation program (control unit, navigation ® emission unit, current position search unit, the average moving distance calculating unit, the search range determining section each part), 14 ... image recognition database, 15 ... map information database

Claims (11)

An image input unit for inputting a plurality of images with different shooting magnifications;
The distance between the imaging point and the imaged object shown in the image is obtained from images with different shooting magnifications input from the image input unit, the imaging point is specified based on the distance, and the specified imaging point And a control unit that performs navigation based on the navigation device. The navigation device according to claim 1,
An azimuth detector that detects an azimuth when the image is input by the image input unit;
An image recognition database in which characteristic components for specifying the imaged object are registered;
A map information database in which position information is registered, and
The controller is
An imaging object detection unit for detecting the imaging object from the plurality of images input by the image input unit;
A feature component extracting unit that extracts a feature component of the imaged object detected by the imaged object detecting unit;
An imaging object specifying unit that specifies the imaging object from the feature component extracted by the feature component extraction unit with reference to the image recognition database;
A shooting direction prediction unit that predicts the shooting direction from the direction information detected by the direction detection unit;
An imaging distance calculation unit that calculates a distance from the imaging point to the imaged object at the time of shooting based on feature component size ratio information extracted from the plurality of images and different shooting magnification information;
The map information database based on the imaged object information specified by the imaged object specifying unit, the shooting direction information predicted by the shooting direction predicting unit, and the distance information calculated by the imaging distance calculating unit. An imaging point specifying unit that specifies the position output as the search result as the imaging point,
A navigation unit that provides route guidance based on the imaging point identified by the imaging point identification unit;
A navigation device comprising: The navigation device according to claim 2,
A navigation apparatus comprising: a current location search instructing unit that instructs the imaging point specifying unit to search the current location of the user. The navigation device according to claim 2 or 3,
A navigation apparatus, comprising: a current location searching unit that searches the current location of the user by the imaging location specifying unit every predetermined time after the current location of the user is specified by the imaging location specifying unit. The navigation device according to claim 4,
An average moving distance calculating unit that calculates the average moving distance of the user based on the current position information already specified by the imaging point specifying unit and the current position information specified after the predetermined time has elapsed by the current position searching unit; A navigation device characterized by that. The navigation device according to claim 5,
A navigation apparatus comprising: a search range determining unit that determines a search range for specifying the current location of the user based on the average moving distance information. The navigation device according to claim 6,
A designation unit for designating a destination and / or waypoint by input from the user;
The navigation unit generates a route based on the current location information of the user and the destination and / or waypoint information specified by the specifying unit,
The search range determination unit includes elapsed time information from the time point searched by the current location search unit, the average travel distance information of the user calculated by the average travel distance calculation unit, and the route generated by the navigation unit And a search range for searching the current location of the user based on the information. A navigation method executed by a navigation device having an image input unit and a control unit,
An image input step in which a plurality of images having different shooting magnifications are input to the image input unit;
The control unit obtains the distance between the imaging point and the imaged object shown in the image from the images with different shooting magnifications input from the image input step, and specifies the imaging point based on the distance. A control step for performing navigation based on the captured imaging point;
A navigation method characterized by comprising: The navigation method according to claim 8, wherein
The navigation device
An azimuth detector that detects an azimuth when the image is input by the image input unit;
An image recognition database in which characteristic components for specifying the imaged object are registered;
A map information database in which position information is registered, and
The control unit includes an imaging object detection unit, a feature component extraction unit, an imaging object identification unit, an imaging direction prediction unit, an imaging distance calculation unit, an imaging point identification unit, a navigation unit,
The control step includes
The imaging object detection unit detects the imaging object from the plurality of images input in the image input step; and
A feature component extraction step in which the feature component extraction unit extracts a feature component of the imaged object detected in the imaged object detection step;
The captured object specifying unit refers to the image recognition database and specifies the captured object from the feature component extracted in the feature component extraction step;
A shooting direction prediction step in which the shooting direction prediction unit predicts the shooting direction from the direction information detected by the direction detection unit;
An imaging distance calculation step in which the imaging distance calculation unit calculates a distance from the imaging point to the imaging object at the time of imaging based on feature component size ratio information extracted from the plurality of images and different imaging magnification information. When,
The imaging point specifying unit, the imaging object information specified in the imaging object specification step, the shooting direction information predicted in the shooting direction prediction step, the distance information calculated in the imaging distance calculation step, An image capturing point specifying step for searching the map information database based on the image and specifying the position output as the search result as an image capturing point;
A navigation step in which the navigation unit performs route guidance based on the imaging point identified by the imaging point identification step;
The navigation method characterized by including. Computer
Imaging input means for inputting a plurality of images having different shooting magnifications;
The distance between the imaging point and the imaged object shown in the image is obtained from images with different shooting magnifications input from the image input unit, the imaging point is specified based on the distance, and the specified imaging point Control means for performing navigation based on
Navigation program to function as. A navigation program according to claim 10, wherein
The computer
An image recognition database in which characteristic components for specifying the imaged object are registered;
A map information database in which position information is registered, and
The control means includes
An imaging object detection means for detecting the imaging object from the plurality of images input by the imaging input means;
Feature component extraction means for extracting feature components of the imaged object detected by the imaged object detection means;
An imaging object specifying means for specifying the imaging object from the feature components extracted by the feature component extraction means with reference to the image recognition database;
Azimuth detecting means for detecting the azimuth when the image is input by the imaging input means;
Shooting direction prediction means for predicting the shooting direction from the direction information detected by the direction detection means;
An imaging distance calculation means for calculating a distance from the imaging point to the imaged object at the time of imaging based on feature component size ratio information extracted from the plurality of images and different imaging magnification information;
The map information database based on the imaged object information specified by the imaged object specifying unit, the shooting direction information predicted by the shooting direction predicting unit, and the distance information calculated by the imaging distance calculating unit. Imaging point specifying means for searching for and specifying the position output as the search result as an imaging point,
Navigation means for performing route guidance based on the imaging point specified by the imaging point specifying unit;
A navigation program characterized by including: