JP6384898B2 - Route guidance system, method and program - Google Patents

Description

The present invention, route guidance system for guiding a route from the current position to the destination, relates to a method and a program, particularly relates to a route guidance system, method and program that can automate updates with presenting an object for measuring the position .

  As a method for guiding the route from the current location to the destination, the following methods are disclosed.

  In route guidance by GPS (Global Positioning System), it is possible to measure the current position by receiving radio waves from multiple satellites and provide route information to the destination. In route guidance by wireless LAN, it is possible to receive radio waves from a plurality of wireless access points, measure the current position by triangulation or the like, and provide route information to the destination.

  In Patent Document 1, by reading an IC tag installed at a fixed location, the position of the terminal is known, and the IC tag is also installed on the mobile terminal body so that the positional relationship between the mobile terminals can be acquired. Propose that. In addition to the positional relationship with the fixed IC tag, by analyzing the positional relationship between mobile terminals that move, the position of the mobile terminal can be specified with higher accuracy.

  In Patent Document 2, the point at which the image is photographed is identified by recognizing a character such as a signboard or a sign based on text information to be photographed and position information associated with the text information registered in advance. Proposed method to do.

JP 2012-137296 A JP 2013-145176

  In the route guidance by GPS, there is a problem that it is difficult to accurately measure the current position when radio waves from a satellite cannot be received due to a shield such as an indoor or high-rise building.

  In the route guidance by wireless LAN, the current position can be measured indoors by installing a plurality of wireless access points, so the problem of route guidance by GPS can be solved. However, there is a problem that it is difficult to accurately measure the current position when there are a plurality of radio wave interferences and multipaths. In addition, there is a problem that a maintenance cost after starting operation is incurred, for example, it is necessary to update a database for storing installation information every time a wireless access point is added, deleted, or changed.

  In patent document 1, the problem in the route guidance by GPS can be solved by installing an IC tag holding position information. In general, IC tags have a narrower communication range than wireless LAN, so the measurement accuracy of the current position is high, and part of the problem of route guidance by wireless LAN can be solved. However, since it is necessary to install IC tags extensively in order to improve measurement accuracy, there remains a problem that the installation is costly. Therefore, in Patent Document 1, a portable terminal is used as an amplifier for a fixed IC tag in order to achieve both reduction in the installation cost of the IC tag and higher measurement accuracy. However, a sufficient number of portable terminals exist in the vicinity of the IC tag. In such a situation, there is a problem that the position cannot be measured at all.

  Since Patent Literature 2 uses existing signs and signs, it can solve the problem of route guidance by GPS and the installation cost of Patent Literature 1. However, if the same character string is present in a plurality of different places (such as different stores in the same series) such as a convenience store signboard, the current position may be misidentified. In the first place, the information provided to the user is only the current position on the planar map, and does not provide a means for letting the user know the object to be imaged and recognized. First, there is a problem that the user does not know what to shoot, and second, how the user should move toward the destination in relation to the scenery that is actually visible. There is a problem of not knowing.

The present invention has been proposed in view of the above circumstances, and provides a route guidance system, method, and program capable of performing intuitive route guidance for a user by presenting an object for measuring a position. This is the primary purpose. Furthermore, the second object is to facilitate the update of the object to be presented.

  In order to achieve the above object, the present invention provides a route guidance system for performing route guidance from a current location where a user exists to a destination input by the user, and acquires a scene at the current location where the user exists as a captured image. An imaging unit that recognizes a predetermined imaging target from the captured image, a recognition unit that determines the current location, and a search unit that searches for a route from the determined current location to a destination input by the user; A presentation unit for presenting information to a user, and presentation information for route guidance according to the searched route, including presentation information including a scene image on the route. A control unit for controlling, a predetermined image feature amount for the recognition unit to recognize a predetermined imaging target, and a predetermined imaging target for the recognition unit to determine the current location Presentation information including position information, route structure information as an object for the search unit to search for the route, and a scene image associated with position information as an object to be presented to the presentation unit under the control of the control unit And a storage unit that stores the first feature.

In addition to the first feature described above, the present invention further includes, when the recognizing unit succeeds in determining the current location, the captured image at the time of the success is used as position information of the determined current location. A scene image as a target to be presented to the presenting unit in the linked position information or the scene, in which the captured image at the time of success is not stored in the storage unit and stored in the storage unit. The second feature is that the change over time of the scene image is reflected as the latest recognition object and presentation object by storing the image as an image candidate.

A third feature of the present invention is a route guidance program that causes a computer to function as the route guidance system.

Furthermore, the present invention is a route guidance method provided in a route guidance system that causes a computer to function by software so as to perform route guidance from a current location where a user exists to a destination input by the user, wherein the presence of the user An imaging step of acquiring a scene at the current location as a captured image, a recognition step of determining the current location by recognizing a predetermined imaging target from the captured image, and a destination input by the user from the determined current location To a presentation device that presents information to a user as a presentation step for searching for a route that leads to the user, and presentation information for route guidance according to the searched route, including a scene image on the route. A control step for controlling to present the image, wherein the recognition step recognizes a predetermined imaging target. A predetermined image feature amount for determining the current position and the position information corresponding to a predetermined imaging target for determining the current location, and the search step refers to route structure information as a target for searching for the route. The control step is executed with reference to presentation information including a scene image associated with position information, and the presentation information includes the recognition step for the captured image acquired in the imaging step. If the current location is determined successfully, the captured image acquired in the successful imaging step is stored in the storage unit in association with the determined location information of the current location, and stored if unsuccessful. A fourth feature is that it is a route guidance method that reflects the temporal change of the scene image by not doing so.

According to the first, third, or fourth feature, intuitive route guidance can be performed by presenting a scene image on a route that is also an imaging target to the user.

  According to the second feature, a captured image that has been successfully identified as the current location is stored as a scene image or a candidate thereof, so that the presentation target can be easily updated.

It is a functional block diagram of the route guidance system concerning one embodiment. It is a figure which shows one Embodiment of a structure in the case of implement | achieving a route guidance system with a route guidance server and a portable terminal. It is a figure which shows the data structure of the information which the memory | storage part has memorize | stored. It is a figure which shows notionally the data structure of the information which the memory | storage part has memorize | stored from a viewpoint as map information. It is a figure for demonstrating the guidance information according to 2nd embodiment of the image information to present, and the inclination and / or direction of a captured image. It is a flowchart of operation | movement of the route guidance system which concerns on one Embodiment.

  FIG. 1 is a functional block diagram of a route guidance system according to an embodiment of the present invention. The route guidance system 10 includes an input unit 1, an imaging unit 2, a recognition unit 3, a search unit 4, a control unit 5, a presentation unit 6, and a storage unit 7.

  In the route guidance system 10, all of the functional units may be realized by a mobile terminal such as a smartphone held by a user who uses the route guidance, or a part of the functional units is not a mobile terminal but a separate route guide. It may be realized by a server. FIG. 2 is a diagram showing an embodiment of a configuration in a case where the route guidance system is realized by sharing the roles of functional units with a route guidance server and a mobile terminal.

  In FIG. 2, the mobile terminals 30A, 30B, etc. held by each user have an input unit 1 and an imaging unit 2 as the minimum configuration for realizing input / output required on the user side using the route guidance. In addition, an example is shown in which the route guidance server 20 includes a recognition unit 3, a search unit 4, a control unit 5, and a storage unit 7, which are the remaining components. In FIG. 2, two examples of the mobile terminals 30A and 30B are shown, but an arbitrary number corresponding to the number of users can be taken. Various information necessary for carrying out the present invention may be exchanged between the portable terminals 30A, 30B and the like and the route guidance server 20 by well-known communication via a network or the like.

  As a modification of the configuration of FIG. 2, the mobile terminal 30A, 30B, or the like may include all or part of the recognition unit 3, the search unit 4, the control unit 5, and the storage unit 7 on the route guidance server 20 side. It is. In addition, if at least the storage unit 7 is provided in one route guidance server 20 and the storage unit 7 is shared by a plurality of users, the benefits of update information as described later can be obtained by the existence of a plurality of users. Can be obtained more effectively.

  The function of each part of the route guidance system 10 shown in FIG. 1 (or FIG. 2) is as follows.

  The input unit 1 accepts a manual input of a destination by a user. For input means, character input using a keyboard or touch panel, selection input from a destination list via a menu screen, selection input from a map location and / or place name via a map screen, etc. can be used. The input unit 1 provides the input means to the user. Options for route guidance, such as distance, cost, required time, number of branching routes, height difference when walking, etc. It may be entered as a setting. The input destination and priority route type are output to the search unit 4 as destination information and priority route type information.

  The input unit 1 also accepts an input for the user to operate the display information displayed by the presentation unit 6 and outputs it to the control unit 5 as operation information.

  The storage unit 7 stores various information necessary for the function of the route guidance system 10 in advance by manual setting by an administrator or the like, and each information is stored as necessary for each functional unit. For reference. In particular, the storage unit 7 provides necessary information to the recognition unit 3, the search unit 4, and the control unit 5 for reference.

  FIG. 3 is a diagram showing a data structure of information stored in the storage unit 7 for reference to the above-described units. In FIG. 3, [1] is mainly used for reference of the recognition unit 3, [2] is mainly used for reference of the search unit 4, and [3] is a data structure of information mainly used for reference of the control unit 5. Show. Note that the pieces of information having the data structures [1] to [3] are not independent of each other, but are associated with each other by each ID (imaging target ID and node ID). The storage unit 7 stores each piece of information including the association. Hereinafter, [1] to [3] will be described.

  The information of the data structure shown in [1] is mainly provided for the recognition unit 3 to recognize the imaging target and acquire the current location information of the captured image, and for each imaging target, the image The feature amount is associated with the map arrangement information. That is, the image feature quantity 102 and the map arrangement information 103 are associated with the imaging object ID 101 that is an identifier of an individual imaging object. The map arrangement information 103 is information for specifying the arrangement of the imaging target on the map, and includes a node ID 104 and position information 105 as a subordinate structure.

  Here, the node ID 104 is the same type as the node ID 204 of [2] (and the node ID 304 of [3]), and will be described later in the description of [2]. The position information 105 is information for specifying the physical position of the imaging target, and can be configured to include, for example, longitude 106, latitude 107, floor 108, and orientation 109 as subordinate structures. The floor 108 is information on the floor (how many floors) when the imaging target is arranged in a building or the like. The azimuth 109 is information that identifies which azimuth the imaging target looks when used as a landmark in route guidance.

  The position information 105 can be various embodiments. The position information 105 may be given in a form in which a part of the substructure shown in [1] is omitted, changed, and / or other attributes are added. For example, instead of the longitude 106 and the latitude 107, it may be given as a predetermined xy coordinate value. If the floor does not exist, the floor 108 may be replaced with altitude information that takes a continuous value. Further, the position information 105 may be given in a graph structure representing a spatial arrangement, an image, a three-dimensional structure such as a 3D model, or other formats.

  The information of the data structure shown in [2] is mainly provided for search processing in the search unit 4. Of the information, a part having a known graph structure by associating a corresponding adjacent node ID list 121 with each node ID 204 is referred to as path structure information. The route structure information is used for search processing in the search unit 4.

  As is well known, a node set 204 and a node set 204 can be obtained by combining a structure in which an edge is provided between a node of one node ID 204 and each node of its adjacent node ID list 121 for all nodes to which the node ID 204 is given. A graph structure consisting of a set of edges in between is obtained. In addition, the nodes and edges are set in advance so that the edges between the nodes correspond to paths used when actually going between positions on the map represented by the respective nodes.

  Further, in [2], the imaging target ID list 122 and the position information 205 corresponding to the node are further associated with the node ID 204, but these are information separate from the path structure information, and are searched. It is not required in the search process in part 4. The list 122 will be described later with reference to FIG. The position information 205 is the same type of information as the position information 105 of [1], and specifies the position of the node.

  The information on the data structure shown in [3] is displayed on the presentation unit 6 under the control of the control unit 5 and used for route guidance, and is displayed at the time of the route guidance. Each of the scene images to be associated with the scene image ID 131, the scene image 132 as its actual content, the time 133 when the image was acquired, and the map arrangement information 303 corresponding to the image Is stored in the storage unit 7. The map arrangement information 303 is information for specifying a position on the map with respect to the scene image 132, has the same configuration as the map arrangement information 103 for the imaging target shown in [1], and has a corresponding node ID 304 and position. Information 305 is included as a substructure. The position information 305 has the same configuration as the position information 105.

  FIG. 4 shows the concept of [1], [2] of the information stored in the storage unit 7 whose data structure has been explained with reference to FIG. It is a figure shown by a typical example. In FIG. 4, (A) is an area (part of) an area as an example of an actual route guidance target, and [B] is a node set in advance in a manual or the like in the storage unit 7 for the area. An example of the imaging target is shown in [C] as an example of a graph structure configured for the city area and set in advance in the storage unit 7 manually or the like.

  In [A], a city area divided into passages S1 and S2 extending north and south and passages S3 and S4 extending east and west is shown, and the positions of the intersections of the passage S1 and the passages S3 and S4, respectively, Nodes N1 and N2 are set, and the positions of the intersections of the path S2 and the paths S3 and S4 are set to the nodes N3 and N4, respectively. Buildings B1 to B4 exist in the vicinity of each of the nodes N1 to N4, and a building B5 exists in the central section of each of the nodes N1 to N4.

  [B] shows an example in which an imaging target existing in the vicinity of each node is registered in advance. For example, for a node N1, three predetermined objects (for example, specific signs placed in buildings B1 and B5) that are visible at the position of the node N1, that is, at the intersection of the passages S1 and S3, are imaged. Registered as targets O11, O12, O13. Further, for each of the imaging targets O11, O12, and O13, as shown in [1] of FIG. 3, the image feature amount and the map arrangement information are registered. At the time of the registration, information indicating that the imaging targets O11, O12, and O13 are associated with the node N1 is set in the node ID 104 in the map arrangement information 103, and the positions of the imaging targets O11, O12, and O13 Are set in the position information 105, respectively.

  Similarly, the imaging targets O21 to O24 are registered in the node N2, the imaging targets O31 and O32 are registered in the node N3, and the imaging targets O41 to O44 are registered in the node N4. The feature amount and map arrangement information are registered. As described above, in [1] of FIG. 3, the data structure has been described mainly for each imaging target. However, one or more imaging targets associated with each node exist based on the data structure, This can be explicitly managed and registered as the “corresponding imaging target ID list 122” shown in [2] of FIG. (That is, although information of one or more imaging targets corresponding to each node can be obtained from [1] alone, the list 122 of [2] can be obtained as information that is actually tabulated and listed.)

  [C] shows a graph structure of each of the nodes N1 to N4 as a specific example of the data elements 204 and 121 of [2] in FIG. 3, that is, the path structure information. Adjacent relationships exist between the nodes N1, N2, N1, N3, N2, N4, and N3, N4, and edges E12, E13, E24, and E34 exist, respectively. Here, each edge corresponds to each element portion of the route indicated by the route guidance, and the result of the route guidance according to the present invention is given as a route on the graph in the route structure information.

  In the example of FIG. 4, the description has been given assuming that an object existing in the vicinity of the position of each node is set as an imaging target. In general, an arbitrary object can be set as an imaging target as long as it can be seen from the position of each node (capable of imaging) even if it does not necessarily exist in the vicinity, and the imaging set as the position of each node The physical distance from the target position may be greatly separated. The relationship between the corresponding node position and the position of the imaging target can be included in the position information 105 shown in [1] of FIG.

  The information stored in the storage unit 7 has been described mainly from a formal viewpoint with reference to FIGS. Details regarding the use of the information will be described together with the description of the recognition unit 3, the search unit 4, the control unit 5, and the like.

  In response to a user operation, the imaging unit 2 captures a scene that the user is viewing at a point where the user actually exists, obtains the captured image, and passes the captured image to the recognition unit 3. Here, in the operation of the present invention, it is basically assumed that the scene to be captured includes an imaging target stored in advance in the storage unit 7. Specifically, the imaging object stored in advance is one of various predetermined objects previously installed on the street or in a building or the like, such as an existing sign, a QR code (registered trademark), a marker, a poster, etc. Part or all can be used.

  Imaging in the imaging unit 2 may be performed at every timing desired by the user by a user operation, or may be continuously performed at a predetermined sampling period. As the imaging unit 2, a digital camera or the like that is provided as a standard in a mobile terminal can be used.

  The recognizing unit 3 extracts an image feature amount from the captured image obtained by the imaging unit 2, and stores an image feature amount for each predetermined imaging target (image feature amount 102 of [1] in FIG. 3) stored in the storage unit 7. 3 is recognized, and the imaging target included in the captured image is recognized, and the positional information and node information associated with the recognized imaging target (position information 105 and node ID 104 in [1] in FIG. 3). Is obtained as current location information where the captured image is captured and passed to the search unit 4.

  In the recognition unit 3, when the imaging target is recognized from the captured image and the current location information is successfully obtained, the recognition unit 3 sends the captured image to the storage unit 7 in order to prompt the update or the like to represent the latest situation at the current location. May be transmitted. The storage unit 7 assigns a new ID and time stamp (scene image ID 131 and time 133 of [3] in FIG. 3) to the received captured image as the latest image at the time, and newly uses a scene. It can be saved as an image (scene image 132 of [3] in FIG. 3) or as a candidate for it. At this time, the contents of the map arrangement information 303 in FIG. 3 may be the contents of the map arrangement information 103 in the recognized imaging target.

  When the current location information cannot be obtained by the recognition unit 3, the corresponding captured image is not transmitted to the storage unit 7. That is, when obtaining the current location information fails, the process of saving the captured image as a scene image or its candidate in the storage unit 7 is not performed.

  In the recognition of the imaging target by the recognition unit 3, a known method using an image feature amount can be used. That is, first, the image feature amount extracted from a predetermined imaging target is stored in advance in the storage unit 7 in association with the position information of the imaging target by a manual operation such as an administrator as a preparatory work. At this time, information such as [1] in FIG. 3 is prepared in advance by adding map layout information by manual work or the like. Next, the corresponding imaging target is identified by comparing the image feature amount extracted by the recognition unit 3 from the captured image captured by the imaging unit 2 with the image feature amount stored in the storage unit 7, and the identification What is necessary is just to acquire the positional information linked | related with the acquired imaging target.

  Here, it is desirable that the image feature amount has a robust property to enlargement / reduction, rotation, and the like of the image. For example, SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), Ferns, etc. can be used. In addition, for matching image feature amounts, an optimal combination that minimizes the difference is selected from the combination of the image feature amount of the captured image and the image feature amount of the storage unit 7. For example, in SIFT, a dot product of feature amounts is obtained, and the most similar feature amount and the second most similar feature amount are selected. When the ratio between the similarity with the former and the similarity with the latter exceeds a preset threshold, the former is estimated as an image feature quantity in a correspondence relationship. Finally, position information associated with the imaging target for which a certain number or more of correspondences are obtained is acquired.

  In addition, since the captured image generally includes one or more imaging targets, the recognition unit 3 identifies all imaging targets as much as possible by the above method, and corresponds to the best matching node as a whole. The position information is output as the current position. At this time, normalization according to the number of imaging targets corresponding to the nodes may be performed to determine the most matching node. For example, as a recognition result of an imaging target for a certain captured image, in the example of [B] in FIG. 4, all the imaging targets O31 and O32 in the node N3 are also two, but actually half of all four When the imaging targets O41 and O42 at the node N4 that are only the nodes are obtained, the current position of the captured image is output as the position of the node N3 where a plurality of imaging targets match at a higher rate. (In other words, the node N3 matches at a rate of 2/2 = 100%, and the node N4 matches at a rate of 2/4 = 50%.) The best matching node may be determined by other methods. .

  As described above, in the present invention, by registering a plurality of imaging targets corresponding to each node in the storage unit 7 in advance, when some imaging targets are changed (for example, only one signboard) Even when the location information is changed), the position information corresponding to the node can be obtained as the current location information. In addition, by transmitting the captured image at this time to the storage unit 7, even when a part of the imaging target is changed, it is possible to automatically acquire the changed scene image. Therefore, at the time of the transmission, it is preferable to transmit information for distinguishing between a plurality of imaging targets that have been successfully recognized and those that have failed.

  The search unit 4 receives the current location information obtained by the recognition unit 3, the destination information obtained from the input unit 1 as a user input, and the route structure information stored in the storage unit 7 as input. , The route information from the current location to the destination is obtained by searching and output to the control unit 5.

  Specifically, the search unit 4 reads the route structure information ([2] in FIG. 3) stored in the storage unit 7 and searches for a route from the current location to the destination. Since the route structure information is configured as a graph such as [C] in FIG. 4, the search unit 4 defines the current node and the destination node in the graph, and then the existing A * algorithm or the like. Is applied to determine the route as the shortest route. Alternatively, an exhaustive search may be performed. In order to determine the shortest path, a physical distance for moving the corresponding actual path to the edge in the path structure information in the storage unit 7 may be set in advance.

  If there are multiple routes as a result of the search, according to the priority route type input at the input unit 1 as user input, select a route with few branches or a route with few stairs in addition to the shortest route. Also good. The characteristics of the priority route types such as the number of steps and the number of branches may be converted into numerical values on the basis of a predetermined standard as the weights of edges and nodes in the route structure information and set in the storage unit 7 in advance.

  The control unit 5 reads the presentation information corresponding to the route information searched by the search unit 4 and the current position information obtained by the recognition unit 3 from the storage unit 7 and passes it to the presentation unit 6. Then, the presentation unit 6 provides the presentation information to the user and realizes route guidance. As the presentation information read and presented by the presentation unit 6, some or all of the image information, map information, character information, and voice information related to the searched route can be used. It is preferable to present at least image information and other information as auxiliary information.

  Each information related to the route for presenting is as follows (1) to (4). Note that when a plurality of pieces of the information are presented, the information may be presented using a predetermined layout corresponding to the type of information to be presented.

  (1) The image information related to the route arrives next to the current location on the route among the scene images registered in advance in the storage unit 7 described in [3] of FIG. 3 as the first embodiment. A scene image corresponding to the node can be used. That is, from the user's standpoint, the place where the same scene as the scene image can be seen is the place to go to next, and the user can intuitively grasp the route by presenting the image in advance.

  As the second embodiment, as the image information related to the route, it is possible to use scene images registered in advance in the storage unit 7 in each node on the route in the order of the route. Assume that nodes 1, 2,..., N are obtained as a result of the route search. That is, assume that node 1 is the current location, the user moves in the order of nodes 2, 3,... According to the route, and node n is the destination. In this case, image information is prepared on the assumption that scene images 1, 2,..., N corresponding to the nodes 1, 2,. Alternatively, the image information may be prepared by omitting the image of the current node 1 and arranging the scene images 2,..., N corresponding to the nodes 2,. Also in the second embodiment, since the user is presented with scene images of places to be sequentially headed on the route, it is possible to intuitively grasp the route.

  Here, when arranging a plurality of scene images, various well-known layout techniques and graphic techniques for displaying a plurality of images can be used. In addition to simply arranging them, the control unit 5 accepts input of user operations via the input unit 1, focusing only on what the user particularly wants to see among a plurality of images, The presentation unit 6 can be made to present it under the control of the control unit 5.

  In the second embodiment, the image information is prepared using only the scene image 2 of the node 2 in the second embodiment.

  FIG. 5 is a diagram schematically showing image information presented in the second embodiment or the like. In FIG. 5, a series of scene images G1, G2, G3, G4, and G5 are displayed as thumbnails on the lower side in the display screen D1 of the presentation unit 6 in a manner that the image sizes change while overlapping each other. An image G30 obtained by enlarging and highlighting the scene image G3 designated by the user via the input unit 1 is shown on the upper side.

  In the example of FIG. 5, the user can move according to the route using the images G1 to G5 as a guide display by sequentially moving portions where the same scene as the images G1 to G5 can be seen. At this time, an image to be particularly confirmed can be enlarged and displayed by the user, and the image G3 specified by the user is enlarged and displayed as an image G30 in FIG. Note that an arrow A30 is further superimposed in the display screen D1, but this is for explaining an example separate from the second embodiment and will be described later. (FIG. 5 is used together to explain the two different examples.)

  (2) For the map information related to the route, well-known information in the route guidance such as a route that is searched for on the map image data and a current location can be displayed. In general, a map is an image, but the term “presentation information” in the present invention particularly indicates that the “image information” in (1) does not include the map image itself, and other than the map itself (added on the map) (Including information) is used as a term meaning an image for realizing route guidance.

  (3) Character information related to the route includes information on how to reach the node that should be reached next to the current location on the route (for example, “Please turn right at the XX intersection”), and addresses along the route. Well-known information in route guidance, such as information on landmarks and landmarks, can be used. (4) As voice information related to the route, voice information of the character information can be used. Note that the map information, character information, and voice information are also registered in advance in the storage unit 7 by manual setting of an administrator or the like.

  In addition, in the presentation of the character information of (3) and the audio information of (4), as described in the second embodiment in the image information of (1), the presentation is performed according to the order of the routes. You may do it. As described above, the scene image in the image information (1) is set in advance in the storage unit 7 as data structure information such as [3] in FIG. 3, but the character information (3) and ( The voice information 4) is stored in advance in the storage unit 7 as information having the same data structure. That is, the scene image 132 in [3] of FIG. 3 is stored as replaced with character information or voice information.

  As described above, (1) to (4), each type of presentation information read from the storage unit 7 and presented to the presentation unit 6 under the control of the control unit 5 for providing a route guidance service to the user has been described. . When presenting the various types of presentation information, it is possible to further devise to realize appropriate route guidance according to the situation where the user is placed.

  FIG. 6 is a flowchart of the operation of the route guidance system 10 according to an embodiment. Hereinafter, each step in the operation will be described with reference to FIG. 6, and the implementation of route guidance using appropriate presentation information according to the user's situation will be described.

  In step S1, an administrator or the like builds information and registers it in the route guidance system 10. Specifically, the information described in FIG. 3 is acquired by a manual or the like and stored in the storage unit 7. The step S1 corresponds to a preparation stage for enabling the route guidance system 10 to perform route guidance, and the subsequent step S2 and subsequent steps are actual route guidance stages.

  In step S2, the route guidance system 10 receives a destination input from the user at the input unit 1, and acquires a captured image of the current location viewed by the user at the imaging unit 2 by the operation of the user. In step S3, the route guidance system 10 performs a route search by causing the recognition unit 3 and the search unit 4 to function using the destination and the captured image obtained in step S2. In step S4, the route guidance system 10 presents presentation information for route guidance to the user by causing the control unit 5 and the presentation unit 6 to function according to the route search result in step S3.

  As described above, after the steps S2 to S4, in order to newly use the route guidance, after the destination input and the acquisition of the captured image, the first route guidance information is obtained as the first route guidance information. The presentation information corresponding to the route is presented. After step S5, the user continues to use the route guidance system 10 while moving to reach the destination, and continuously receives route guidance information according to the current situation of the user.

  In step S5, the route guidance system 10 determines whether or not the user has newly acquired a captured image of the current location by the imaging unit 2, and if it is acquired, provides the latest route guidance information at the time of acquisition. Therefore, the process proceeds to step S6 and subsequent steps. If not acquired, step S5 is repeated until it is acquired.

  That is, in the present invention, the route guidance system 10 realizes the route guidance at that time point, triggered by the user's acquisition of the captured image at that time point. Note that, as described above in the description of the imaging unit 2, the user may acquire the captured image by the user performing an imaging operation at each time point desired by the user. It may be configured such as being attached to a wearable terminal, and captured images may be continuously acquired at a predetermined frame rate.

  In step S6, the recognition unit 3 identifies the current location using the captured image acquired in step S5, and the process proceeds to step S7. The current location is specified as a node. Further, in the identification in step S6, even if the current location cannot be identified because the predetermined imaging target cannot be recognized from the captured image, etc., the process proceeds to step S7 after obtaining that it cannot be identified as a result.

  In step S7, the recognizing unit 3 determines whether or not the current location specified in step S6 has deviated from the route. If deviating, the process proceeds to step S8, and if not deviating, the process proceeds to step S9. Specifically, if the node that is the current location is not a node that constitutes the latest route information that is to be presented to the user at the time, it is determined that the node has deviated, and otherwise Is determined not to deviate.

  In step S8, the search unit 4 re-searches the route from the current location deviating from the route to the destination accepted in step S2, so that the latest route information as the target to be presented to the user is obtained. The corresponding presentation information is presented to the presentation unit 6 under the control of the control unit 5, and the process returns to step S5.

  As the presentation information in step S8, in addition to the image information and the like described as (1) to (4) above, information that deviates from the previously presented route (information that alerts the deviation) and the like Information indicating that the route has been reset due to deviation may be included. The image of the deviation can be presented in various forms such as images, characters, sounds, and the like.

  As an example of step S8, for example, route guidance is performed in the field shown in FIG. 4, and the route information initially presented to the user is “N1 → N2 → N4” at the node shown in [C]. It is assumed that the vehicle headed from the point N1 to the destination N4 via the route N2. In this case, if the current location is specified as N3 by the captured image acquired by the user while moving, the route is deviated from the route, so the route is searched again and a new route “N3 → N4 "is set as the presentation target to the user.

  In step S9, the recognizing unit 3 confirms whether or not the current location specified in step S6 is the destination.If the current location is the destination, the process proceeds to step S10. Otherwise, the process proceeds to step S11. move on. In step S10, the presentation information indicating that the destination has been reached is presented to the presentation unit 6 under the control of the control unit 5, and the flow of FIG. 6 ends. The presentation of the arrival at the destination can be in various forms such as images, characters, voices, and the like.

  In step S11, the presentation information corresponding to the current location on the route information at the time is presented to the presentation unit 6 under the control of the control unit 5, and the process returns to step S5. Note that there are cases where the current location can be identified as a node in step S6 and cases where the current location cannot be identified in step S6.

  Here, when it cannot be specified, it is determined as a determination on the side of the route guidance system 10 that the user is correctly moving along the route but the current location cannot be specified. And as presentation information, what is necessary is just to use the thing when the present location can be specified in the latest past. Note that the current location could not be specified because the current location is a position between nodes on the route and there is no imaging target, but the current location is the node position (due to the effect of shooting direction, etc.) Is not imaged.

  On the other hand, if it can be specified, in addition to the image information described as (1) to (4) above, information corresponding to the time point can be presented as presentation information. The following various embodiments are possible for the presentation.

  In one embodiment, when the current location is node i and the next node on the route is i + 1, information for reaching node i + 1 from node i along the route (hereinafter referred to as guidance information) May be presented according to the current location. The guidance information is set in the storage unit 7 in advance. The guidance information can be configured in the modes (1) to (4) above.

  Further, the guidance information may be configured as an image and may be presented to the user as presentation information by superimposing it on the captured image used when specifying the current location. In this case, since the information is directly superimposed on the scene that the user is actually viewing, very intuitive route guidance is possible.

  For example, the guidance information as an image to be superimposed on the captured image indicates the inclination (tilt with respect to the horizontal direction) and / or direction (direction on the horizontal plane) of the position of the next node i + 1 when viewed from the current location. When the information is displayed as an image, it is displayed as a relative change from the tilt and / or direction of the captured image. Note that the tilt and direction of the captured image may be acquired together with the captured image as the tilt and direction of the imaging unit 2 by using a known sensor (such as a gravity sensor or an orientation sensor).

  An arrow A30 in FIG. 5 is an example of guidance information corresponding to the inclination and / or direction of the captured image. In the inclination and / or direction of the captured image, the inclination and / or to advance to the next node i + 1. The direction is expressed as an image of arrows arranged in space. With the arrow A30, the user can intuitively understand that the user should proceed to the ticket gate of the station. In the above description, the image G30 is described as an enlarged view of the scene image G3 designated by the user, but the image G30 on which the arrow A30 is superimposed is a captured image. (In FIG. 5, in order to explain two examples, the role of the image G30 is used together.)

  In one embodiment, image information for route guidance is stored in the storage unit 7 in each node constituting the map structure information, and position information is also defined in the image information, and the current location The node i may be presented by being superimposed on the captured image at the position of the defined position information.

  Specifically, it can be presented superimposed on a captured image using a technique similar to a known AR (augmented reality) tag. However, in the present invention, the AR tag itself arranged in the real space is not necessary, and the same information that is superimposed by the AR tag is presented by superimposing position information as predefined image information. For this purpose, image information that defines relative position information viewed from the node position may be prepared for each node i.

  About the content of the image information superimposed by the method similar to the said AR tag, it can be set as the predetermined content relevant to a path | route. For example, the direction of traveling along the route can be indicated by a planar or spatial arrow or other image, or by characters. For example, in the example of the field of FIG. 4, when the route “N1 → N2 → N4” is guided, when the user arrives at N2 from the N1 side and acquires the captured image, the path on the captured image It is possible to present on the S1 the image information as a signboard indicating that it should turn right.

  Further, the content of the image information can indicate which of the plurality of routes is visible by visually displaying the area. For example, in the above example, when performing the guidance of the route “N1 → N2 → N4”, when the user reaches N2 from the N1 side and acquires the captured image, the portion of the path S4 on the side leading to N4 May be displayed in a frame.

  In addition, a plurality of pieces of image information in the same method as the above AR tag are set in each node, and only those that match the inclination and / or direction in the captured image acquired by the user are superimposed on the captured image. It may be. Note that the tilt and / or direction in the captured image is acquired by a sensor or the like as described above.

  For example, in the example of the field of FIG. 4, when the captured image is acquired by reaching N2 along the route “N1 → N2 → N4”, and N2 is reached along the route “N4 → N2 → N1”. Even when the captured image is acquired, the direction is different even in the same place N2, and thus different types of images may be superimposed.

  In addition, when superimposing only those that match the tilt and / or direction, only image information having position information within a predetermined range determined from the tilt and / or direction is superimposed on the captured image. You may make it show. The predetermined range may be set as a predetermined range that is assumed to be visible in the inclination and / or direction.

  For example, the image information may be superimposed only within a horizontal range determined according to the tilt (horizontal tilt) of the captured image, and the captured image is narrowed and only the nearby ground is captured. If the captured image is wide in the horizontal direction, the image information may be superimposed in the wide horizontal range, limited to the ground in the vicinity.

  In addition, when superimposing image information of the same technique as the AR tag on a captured image, in the present invention, in particular, instead of position information in a three-dimensional space generally required for an AR tag, Only the inclination and / or direction when viewed from the corresponding node may be set. Then, only image information that matches the inclination and / or direction in the captured image acquired in the node within a predetermined range may be selected as a superimposition target.

  Further, image information of the same technique as the above AR tag may be displayed at a predetermined position by the AR tag. In this case, information such as a feature amount for recognizing the AR tag needs to be stored in the storage unit 7 instead of needing position information. The AR tag may be realized by an imaging target that is recognized by the recognition unit 3.

  Hereinafter, supplementary matters in the present invention will be described.

  (1) The scene image stored in advance in the storage unit 7 suffices if the user can recognize the scene. In addition to what is actually acquired as a natural image, an image obtained by processing the natural image (important land) It is also possible to use illustrations, CG (computer graphics), etc.

  (2) The recognition unit 3 may be configured as follows instead of the embodiment in which the image location is extracted from a predetermined imaging target and the current location is specified as described above. That is, position information may be embedded in blink information by a transmitter such as visible light communication or infrared light communication, and the recognition unit 3 may acquire the current location by analyzing the blink information. In this case, it is not necessary to set a predetermined imaging target and its image feature amount in the storage unit 7, but it is necessary to install a communication device having the above function in advance in the field to be guided. is there. In addition, in order to obtain flicker information in the imaging unit 2, it is necessary to perform imaging as a moving image over a predetermined period. However, a captured image on which information for route guidance is superimposed is obtained within the predetermined period. What is necessary is just to use.

  (3) A scene image to be presented to the user for route guidance is prepared for each node on the map structure information, but as additional attributes, prepared for each time zone, every inclination and / or direction, every weather In this regard, an object that matches the attribute when the user acquires the captured image may be the presentation target. For example, a time zone attribute may be provided so that different scene images corresponding to the case where the user has acquired the captured image are daytime and nighttime may be presented. Generally, since the scene is different between daytime and night (unless indoors or the like), appropriate route guidance can be realized by providing this distinction.

  (4) When the recognition unit 3 succeeds in recognizing the current location, a corresponding captured image is stored in the storage unit 7. After the storage, the latest captured image is always automatically used as a corresponding scene image. You may make it utilize, and you may continue using what was initially set. Even in the case of continuous use, since the latest captured image is stored in the storage unit 7 at any time, an administrator or the like can manually select an appropriate image and set it as a new scene image.

  (5) In the description of FIG. 6, it has been described that various presentation information corresponding to the current time point in the current location is presented in Step S11. However, Step S4 in which a route search is performed for the same various information for the first time. May also be presented.

  DESCRIPTION OF SYMBOLS 10 ... Route guidance system, 1 ... Input part, 2 ... Imaging part, 3 ... Recognition part, 4 ... Search part, 5 ... Control part, 6 ... Presentation part, 7 ... Memory | storage part

Claims (20)

A route guidance system that provides route guidance from a current location where a user exists to a destination input by the user,
An imaging unit that acquires a scene in the current location where the user exists as a captured image;
A recognition unit that determines the current location by recognizing a predetermined imaging target from the captured image;
A search unit that searches for a route from the determined current location to a destination input by the user;
A presentation unit for presenting information to the user;
A control unit for controlling the presentation unit to present presentation information including a scene image on the route, which is presentation information for route guidance according to the searched route;
A predetermined image feature amount for the recognition unit to recognize a predetermined imaging target, position information corresponding to the predetermined imaging target for the recognition unit to determine the current location, and the search unit to search for the route. A storage unit that stores route structure information as a target and presentation information including a scene image associated with position information as a target to be presented to the presenting unit under the control of the control unit;
When the recognition unit succeeds in determining the current location, the captured image at the time of success is stored in the storage unit in association with the position information of the determined current location, and is not stored in case of failure. A route guidance system that reflects a temporal change of a scene image as the latest recognition target.   The route guidance according to claim 1, wherein the predetermined type of imaging target includes some or all of existing signs, QR codes (registered trademark), markers, and posters installed in advance. system.   The storage unit stores the captured image at the time of success as a scene image as a target to be presented to the presenting unit in the associated position information or a candidate for the scene image, and changes over time of the scene image Is reflected as the latest presentation information. The route guidance system according to claim 1, wherein: The user inputs a route priority type in addition to the destination,
4. The route guidance system according to claim 1, wherein the search unit searches for a route in consideration of the inputted route priority type.   The control unit controls the presentation unit to present presentation information including all or part of image information, map information, character information, and voice information corresponding to the searched route. The route guidance system according to any one of claims 1 to 4.   The control unit causes the presentation unit to present presentation information including all or part of image information, character information, and voice information in each part of the searched route in an order corresponding to the searched route. 5. The route guidance system according to claim 1, wherein the route guidance system is controlled as follows.   The control unit controls the presentation unit to present presentation information including a scene image of a place to be reached on the searched route next to the determined current location. The route guidance system according to any one of 1 to 6.   The control unit controls the presentation unit to present a presentation image including scene images arranged in order of arrival at scenes to be reached along the searched route. The route guidance system according to any one of claims 1 to 7.   The control unit controls the presentation unit to present presentation information including guidance information to a place to be reached on the searched route next to the determined current location. Item 9. A route guidance system according to any one of Items 1 to 8.   The route according to claim 1, wherein the control unit controls the presentation unit to present presentation information including the guidance information superimposed on the captured image. Guidance system.   In the captured image, the guidance information to be superimposed indicates the inclination and / or direction to the place to be reached on the searched route next to the determined current location from the inclination and / or direction of the captured image. The route guidance system according to claim 10, wherein the route guidance system is shown as a relative change in the distance.   The control unit includes image information corresponding to the determined current location, and image information for allowing a user to recognize the searched route by being superimposed on the captured image so as to be distinguished from other routes. The route guidance system according to any one of claims 1 to 11, wherein presentation information is controlled to be presented to the presentation unit.   The control unit recognizes the searched route by superimposing on the captured image, which is image information corresponding to the determined current location and the inclination and / or direction of the captured image. The route guidance system according to claim 1, wherein presentation information including image information to be displayed is controlled to be presented to the presentation unit. A plurality of image information is prepared in advance by being distinguished by position information and inclination and / or direction,
The control unit only superimposes a plurality of the position information that matches the position information of the determined current location and the inclination and / or direction within a predetermined range with respect to the inclination and / or direction of the captured image from the plurality. The route guidance system according to claim 13, wherein   When the current location determined by the recognizing unit deviates from the route already searched by the search unit, the search unit re-searches a route from the determined current location to the input destination. The route guidance system according to any one of claims 1 to 14, wherein   The route guidance system according to claim 15, wherein when the re-search is performed, the user is notified of the fact. The scene image is prepared for each attribute,
The control unit selects a scene image prepared for each attribute that matches the attribute of the captured image, and controls the presentation unit to present presentation information including the selected scene image. The route guidance system according to any one of claims 1 to 16, wherein   The route guidance system according to claim 17, wherein the attribute is a time zone.   A route guidance program for causing a computer to function as the route guidance system according to any one of claims 1 to 18. A route guidance method provided in a route guidance system that causes a computer to function by software so as to perform route guidance from a current location where a user exists to a destination input by the user ,
An imaging step of acquiring a scene in the current location where the user exists as a captured image;
A recognition step of determining the current location by recognizing a predetermined imaging target from the captured image;
A search step for searching for a route from the determined current location to a destination input by the user;
A control step of controlling the presentation information for route guidance according to the searched route, including presentation information including a scene image on the route to be presented to a presentation device that presents information to the user; With
The recognition step is executed with reference to a predetermined image feature amount for recognizing a predetermined imaging target and position information corresponding to the predetermined imaging target for determining the current location, and the searching step includes The control step is executed with reference to presentation information including a scene image associated with the position information, and is executed with reference to route structure information as a target for searching for a route.
When the recognition step succeeds in determining the current location for the captured image acquired in the imaging step, the presentation information indicates the captured current image acquired in the successful imaging step as the determined current location. The route guidance method is characterized in that the positional information of the scene image is linked and stored in the storage unit, and the change with time of the scene image is reflected by not storing the location information when it fails .