JP2022055218A - Route guide device, route guide system, and program - Google Patents

Abstract

To provide a route guide device capable of providing route guidance in various spaces including indoors, and a route guidance system and a program.SOLUTION: A route guide device is configured to perform the steps of: acquiring a piece of map information in which a position where a key image is displayed is specified by the map coordinate system; detecting a key image from image data which is picked up from the vicinity of the route guide device; and acquiring relative position information between the key image in a given virtual space coordinate system and the route guide device on the basis of the detected key image. The route guide device is further configured to acquire position information in the map of the map coordinate system of the route guide device using the acquired map information and the acquired relative position information, and provide the acquired position information in the map for predetermined processing to search and guide the route.SELECTED DRAWING: Figure 1

Description

The present invention relates to a route guidance device, a route guidance system, and a program.

In recent years, a route guidance system using a GPS (Global Positioning System) has become widespread, and a route guidance system using an application program or the like running on a smartphone or the like is widely used in places where GPS signals can be received, such as outdoors.

Japanese Unexamined Patent Publication No. 2020-034531

With this conventional route guidance system, it is not possible to perform route guidance in a place where GPS signals cannot be received, such as indoors. Therefore, for example, a technique for guiding a route using a beacon signal, such as the technique disclosed in Patent Document 1, has been considered.

However, in the route guidance technology using a beacon signal, information on the relative position such as the direction of the user who is going to receive the guidance with respect to the source of the beacon signal cannot be acquired, so that the guidance method is limited.

The present invention has been made in view of the above circumstances, and one of the objects thereof is to provide a route guidance device, a route guidance system, and a program capable of performing route guidance in various spaces including indoors. do.

One aspect of the present invention that solves the problems of the above-mentioned conventional example is a route guidance device that guides a route in which edges are combined, and predetermined key images are posted at a plurality of places in the vicinity of the edges. The map information acquisition means for acquiring the map information defined by the map coordinate system at least at the position where the key image is posted, which is the map information defining the edge by the predetermined map coordinate system, and the above. A predetermined virtual space based on an imaging means that captures an image of the surroundings of a route guidance device and outputs the captured image data, a key detecting means that detects the key image from the image data, and the detected key image. The map of the route guidance device using the key image in the coordinate system, the relative position acquisition means for acquiring the relative position information of the route guidance device, and the acquired map information and the acquired relative position information. It has an in-map position information acquisition means for acquiring in-map position information of a coordinate system, and uses the acquired in-map position information for a predetermined process of searching and guiding a route along the edge. It was done.

According to the present invention, it is possible to provide route guidance in various spaces including indoors.

It is a block diagram which shows the example of the route guidance system which concerns on embodiment of this invention. It is explanatory drawing which shows the content example of the map information used in the route guidance system which concerns on embodiment of this invention. It is a functional block diagram which shows the example of the route guidance apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows the example of the coordinate conversion of the route guidance apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows the example of the route guidance screen displayed by the route guidance apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows another example of the coordinate transformation by the route guidance apparatus which concerns on embodiment of this invention. It is a flowchart which shows the operation example of the route guidance apparatus which concerns on embodiment of this invention. It is a flowchart which shows the outline of the route guidance processing by the route guidance apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows still another example of the coordinate transformation by the route guidance apparatus which concerns on embodiment of this invention.

An embodiment of the present invention will be described with reference to the drawings. The route guidance system 1 including the route guidance device 10 according to the embodiment of the present invention is used in a facility or the like in which a predetermined key image is posted in the vicinity of the edge, with the route that the user can pass through such as a passage as an edge. ..

Here, the key image is an image represented on a rectangular plane, and may include a computer-readable code image such as a bar code or a two-dimensional bar code. However, the key image is not limited to these examples, and the key image may be, for example, a store advertisement or a signboard itself. In the example of this embodiment, the key image is supported by a support member such as a frame for posting the key image on a wall surface or the like near the edge. In the following, this key image and the support member will be referred to as a key image bulletin board.

As illustrated in FIG. 1, the route guidance system 1 of the present embodiment has a route guidance device 10 carried by a user and map information connected to the route guidance device 10 so as to be accessible via a network. It is configured to include the server 20.

Further, in the example of the present embodiment, the administrator of the route guidance system 1 generates map information in advance. As illustrated in FIG. 2A, this map information includes a map image of the facility or area where route guidance is performed (a rasterized image or a vector image may be used) and the map image. It defines edges and nodes (edge endpoints) along the routes that users can pass through in the facilities and areas shown. In the example of this embodiment, the edges and nodes are defined by the coordinates in the predetermined map coordinate system.

Further, in this map information, it is assumed that the position and direction in which the key image is posted are defined as the coordinates or vectors of this map coordinate system. Here, in the map coordinate system, for example, an X'Z'orthogonal coordinate system is set on a plane parallel to the ground surface (assuming that the ground surface is approximately a plane) with a predetermined position on the ground surface as the origin, and the ground surface is set. The X'Y'Z'cartesian coordinates of the right-handed system with the Y'axis in the vertical direction (the vertical direction may be the positive direction) may be used. At this time, the positive direction of the X'axis or the positive direction of the Z'axis may be set to the north.

Specifically, in the example of FIG. 2A, a plurality of edges Ea, Eb ... And nodes Ma, Mb ..., Which are end points of the edges E, are set on the passage P through which the user can pass. Further, in the example of FIG. 2, the posting position Q of the key image in the map coordinate system is recorded in the vicinity of the edge E in association with the information S for specifying the key image posted at the posting position.

Further, as shown in FIG. 2B, the map information includes information representing the posting position Q of the key image in the map coordinate system and the map coordinate system of the key image in association with the information S for specifying the key image. Information indicating the direction (which may be the normal direction with respect to the surface of the key image) Qd and the actual size Qs of the key image is recorded and included.

In addition to these, the map information may be preset with a position that can be set as a destination by the user, such as the position of a store in the facility (represented by a map coordinate system).

Here, the information for specifying the key image may be decoded data obtained by decoding the coded image when the key image includes the coded image. Further, when the key image does not include the coded image, the information necessary for searching the key image from the captured image data such as the key image itself or the feature amount obtained based on the key image (from the image). As a method for searching an image portion that is a predetermined key, various well-known methods can be adopted, so detailed description here is omitted).

Since the case where there are multiple floors connected through escalators, elevators, stairs, etc. is shown here, map information corresponding to each floor and information specifying nodes M corresponding to each other are set on each floor. ing. As an example, in FIG. 2, map information A on the first floor and map information B on the second floor are shown, and node Mx (intermediate point of the escalator) at the corresponding positions is set in both. It shows the state of being.

In the present embodiment, the map information is stored in the map information server 20. Specifically, the map information server 20 holds the map information in association with the information that identifies the facility, building, or the like corresponding to the map information. The map information server 20 accepts a request for map information as well as information for identifying a facility, a building, or the like via a network. In response to the request for the map information, the map information server 20 reads out the map information corresponding to the facility or building specified by the information received together with the request, and transfers the read map information to the requesting smartphone or the like. On the other hand, it is sent via the network.

Further, the route guidance system 1 of the present embodiment includes a route guidance device 10 carried by the user. The route guidance device 10 may be, for example, a smartphone or the like, and specifically, as illustrated in FIG. 1, includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, an image pickup unit 15, and a communication unit 16. Consists of.

The control unit 11 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12. In the present embodiment, the control unit 11 performs processing for fulfilling the original function of a smartphone or the like that operates as the route guidance device 10, and also accesses the map information server 20 to acquire map information and next. Processes the route guidance of.

That is, the control unit 11 of the route guidance device 10 according to the example of the present embodiment is a process for receiving image data obtained by capturing an image of the periphery of the route guidance device 10 itself from the image pickup unit 15 and detecting a key image from the image data. To execute. When the key image is detected here, the control unit 11 acquires information representing the relative position information between the key image in the predetermined virtual space coordinate system and the route guidance device 10 based on the detected key image. ..

Further, the control unit 11 acquires the position information in the map of the map coordinate system of the route guidance device 10 by using the acquired map information and the relative position information, and uses the acquired position information in the map along the edge. It is used for a predetermined process of searching for and guiding the route. The specific contents of the processing for the route guidance of the control unit 11 will be described later.

The storage unit 12 is a memory device or the like, and holds a program or the like executed by the control unit 11. This program may be provided stored in a computer-readable and non-temporary recording medium, copied to the storage unit 12, or downloaded via a network. Further, in the present embodiment, the storage unit 12 also operates as a work memory of the control unit 11.

The operation unit 13 is a touch panel or the like, and receives an instruction operation by the user and outputs information indicating the content of the instruction operation to the control unit 11. The display unit 14 is a display or the like stacked below the touch panel, and displays and outputs information according to an instruction input from the control unit 11.

The image pickup unit 15 is a camera or the like, and sequentially acquires images captured around the route guidance device 10, and sequentially outputs the data of the acquired images to the control unit 11.

The communication unit 16 includes a wireless network interface. The communication unit 16 outputs the information received wirelessly via the network to the control unit 11. Further, the communication unit 16 transmits information via the network according to the instruction input from the control unit 11. If the route guidance device 10 is a smartphone, for example, the communication unit 16 may have a configuration for performing communication via a mobile phone network.

Next, the content of specific route guidance processing by the control unit 11 of the route guidance device 10 will be described. As illustrated in FIG. 3, the control unit 11 that performs this processing includes a map information acquisition unit 21, a key detection unit 22, a relative position acquisition unit 23, a map position information acquisition unit 24, and a route guidance processing unit. The 25 and the information output unit 26 are functionally included.

In the following description, the control unit 11 arranges XZ Cartesian coordinates on a surface parallel to the actual floor surface, and a virtual space having a vertical direction (the upward direction is a positive direction) as the Y axis on this surface. A coordinate system is set, and processing is performed using this virtual space coordinate system and the map coordinate system, which is the coordinate system of map information. Further, here, the direction of the X-axis or the Z-axis of the virtual space coordinate system is, for example, the direction in which the route guidance device 10 is facing when the user activates the route guidance device 10 (the angle of view imaged by the image pickup unit 15). The XYZ coordinate system of the right-handed system when the (center direction) is the positive direction of the X axis may be the virtual space coordinate system. However, the method of setting the virtual space coordinate system is not limited to this, and for example, if the north direction can be detected by a geomagnetometer or the like built in the route guidance device 10, the north direction may be the positive direction of the X-axis.

The map information acquisition unit 21 accesses the map information server 20 to acquire map information. In an example of this embodiment, the user inputs the facility name of the facility where he / she is located. The map information acquisition unit 21 requests the map information server 20 for map information corresponding to the facility with the input facility name. When the map information acquisition unit 21 receives the map information from the map information server 20, the map information acquisition unit 21 stores the map information in the storage unit 12.

During this route guidance process, the key detection unit 22 controls the image pickup unit 15 to sequentially image the periphery of the route guidance device 10 and output the image data obtained by the image pickup. The key detection unit 22 receives the image data that the image pickup unit 15 sequentially captures and outputs, and detects the key image from the image data. The key detection unit 22 outputs information indicating the position (may be a position in the XZ plane) of the detected key image in the virtual space coordinate system and its direction (normal direction with respect to the plane of the key image).

Here, when the key image includes a computer-readable code image, the key detection unit 22 detects a computer-readable code image, which is a key image, from the captured image data. As for such a process of detecting a coded image from image data, a widely known method can be adopted, and thus detailed description thereof will be omitted here.

The key detection unit 22 decodes the coded image included in the detected key image to obtain the decoded data. Then, using the decoded data as a key, the information indicating the position of the detected key image on the map coordinates is searched and acquired from the map information acquired by the map information acquisition unit 21.

Then, the key detection unit 22 has a position on the map coordinates of the acquired key image, a position in the virtual space coordinate system (may be a position in the XZ plane), and its direction (normal direction with respect to the plane of the key image). Outputs information that represents.

When the key image does not include the coded image, the key detection unit 22 uses the information for identifying the key image included in the map information acquired by the map information acquisition unit 21 to obtain the key image from the captured image data. Attempt to detect. Specifically, the key detection unit 22 will execute a process (image search process) for detecting a portion matching the key image included in the acquired map information.

Then, when any of the key images included in the map information is detected, the key detection unit 22 determines the position of the detected key image on the map coordinates (acquired from the map information) and the virtual space coordinate system. (It may be a position in the XZ plane) and information indicating its orientation (normal direction with respect to the plane of the key image) are output.

The relative position acquisition unit 23 acquires the relative position information between the key image in the virtual space coordinate system and the route guidance device 10 based on the position and orientation information of the key image detected by the key detection unit 22. .. Specifically, the relative position acquisition unit 23 first sets the information of the region where the key image is captured in the image data acquired by the image pickup unit 15 and the position of the key image ((x2, y2, z2)). ) And the orientation information, the position in the virtual space coordinate system of the imaging unit 15, that is, the position (x1, y1, z1) of the route guidance device 10 in the virtual space coordinate system is estimated. As this estimation method, a widely known method using virtual reality technology (so-called AR technology) can be adopted. An example of this method is, for example, Hirokazu Kato, et.al., "Augmented Reality System Based on Marker Tracking and Its Calibration", Journal of the Virtual Reality Society of Japan, Vol4, No4, pp.607-616, (1999). ) Etc. corresponds to the method of finding the camera coordinate system. The coordinates y1 and y2 in the height direction may be set in advance as a value h indicating the line-of-sight height.

The position information acquisition unit 24 in the map represents the position of the route guidance device 10 in the map coordinate system based on the relative position between the captured key image and the route guidance device 10 obtained by the relative position acquisition unit 23. Generate information. As an example, the position information acquisition unit 24 in the map has information representing the relative position between the key image in the virtual space coordinate system and the route guidance device 10.
x2-x1
y2-y1
z2-z1
To get.

と表される。また、図４（ａ）に示したように、点Ｂを通りＺ軸に平行な線分上の点（Ｚ軸正の方向にある点とする）をＣとして、角ＣＢＡは、

で表される（ただしｎは０以上の整数）。ここで、
・Ｃ点がＢ点よりもＡ点に近く、かつ、ｚ1≧ｚ2のときには、ｎ＝０
・Ｃ点がＢ点よりもＡ点に近く、かつ、ｚ2＞ｚ1のときには、ｎ＝１
・Ｃ点がＡ点よりもＢ点に近く、かつ、ｚ1≧ｚ2のときには、ｎ＝１
・Ｃ点がＡ点よりもＢ点に近く、かつ、ｚ2＞ｚ1のときには、ｎ＝０とする。 Here, as illustrated in FIG. 4A, the position of the key image in the XZ plane of the virtual space coordinate system is B (x2, z2), and the position of the route guidance device 10 is A (x1, z1). do. The distance d (AB) between these ABs is

It is expressed as. Further, as shown in FIG. 4A, the angle CBA is defined by C being a point on a line segment that passes through point B and is parallel to the Z axis (assumed to be a point in the positive direction of the Z axis).

It is represented by (where n is an integer greater than or equal to 0). here,
・ When point C is closer to point A than point B and z1 ≧ z2, n = 0.
-When point C is closer to point A than point B and z2> z1, n = 1
-When point C is closer to point B than point A and z1 ≧ z2, n = 1
-When point C is closer to point B than point A and z2> z1, n = 0 is set.

In the following description, the angle of the angle is changed depending on whether the XZ plane is measured clockwise or counterclockwise when viewed from the positive direction of the Y axis. That is, the angle CBA = − angle ABC.

が成り立つ。なお、角ＣＢＤは、設定から既知である。 Further, let D be a point on a line segment that passes through point B and is parallel to the direction of the key image (normal direction of the surface of the key image) (assumed to be a point closer to point A than point B).

Is true. The angle CBD is known from the setting.

が成立する。 Since the information about this angle is also valid in the actual space, the position of the key image is eventually changed to B'even in the X'Z'plane in the map coordinate system X'Y'Z', as illustrated in FIG. 4 (b). (X'2, z'2), the position of the route guidance device 10 is A'(x'1, z'1), and the direction of the key image (normal to the surface of the key image) is further passed through the point B'. When a point on a line parallel to the direction) (assumed to be a point closer to the A'point than the B'point) is D'.

Is established.

Here, s is the scale ratio of the map coordinate system to the virtual space coordinate system, and the point C'is a point (Z'axis positive) on a line segment passing through the point B'and parallel to the Z'axis in the map coordinate system. The point is in the direction of). The scale ratio s is the size of the key image in the virtual space coordinate system (for example, its width may be sufficient, and is obtained as the difference between the coordinate values in the virtual space coordinate system at both ends in the width direction) and the key in the map coordinate system. It can be obtained by the ratio with the size of the image (corresponding direction, for example, the difference between the coordinate values in the map coordinate system at both ends in the width direction in the above example).

を得て、地図座標での経路案内装置１０の位置Ａ′の座標（ｘ′1，ｚ′1）を、
ｘ′1＝ｘ′2＋Δｘ′，
ｚ′1＝ｚ′2＋Δｚ′
として求めることができる。 Also, from this equation (6),

The coordinates (x'1, z'1) of the position A'of the route guidance device 10 in the map coordinates are obtained.
x'1 = x'2 + Δx',
z'1 = z'2 + Δz'
Can be obtained as.

The position information acquisition unit 24 in the map obtains the values of the equations (1) to (3) based on the information on the relative position between the captured key image and the route guidance device 10, and also obtains the scale ratio separately. Using s and these obtained values, Δx ′ and Δz ′ are obtained by the equations (4) to (8). Then, the position information acquisition unit 24 in the map uses the position coordinates (x'2, z'2) of the captured key image in the map coordinates.
x'1 = x'2 + Δx',
z'1 = z'2 + Δz'
, The coordinates (x'1, z'1) of the position A'of the route guidance device 10 in the map coordinates are obtained.

The route guidance processing unit 25 accepts the setting of the destination from the user and performs processing for guiding the route. Specifically, the route guidance processing unit 25 uses the position of the route guidance device 10 in the map coordinates (that is, the position of the user who carries it) acquired by the position information acquisition unit 24 in the map to provide map information. Above, the point (point representing the current position) CU on the edge closest to the position is obtained.

Then, the route guidance processing unit 25 obtains a route from the current position to the destination by using the point DE on the edge of the set destination and the point CU on the edge representing the current position. Since such a method of finding a route on the edge from the current position to the destination based on these two points on the edge is widely known, detailed description here will be omitted.

The route guidance processing unit 25 instructs the information output unit 26 to output the route information obtained here.

The information output unit 26 is realized by using library software for general virtual reality (AR) processing. Such library software includes ARKit (Apple Computer, USA) and ARCore (Google, USA).

The information output unit 26 uses such library software to display the image data captured by the image pickup unit 15 on the display unit 14 and superimpose the image data on the image data to guide the route. Display the guidance image of. Specifically, as illustrated in FIG. 5, this guide image has a straight line E (virtual straight line) representing an edge defined by map information and an arrow P (virtual arrowhead) representing the direction in which the user should go. It is assumed to be a virtual reality image (three-dimensional image) including the figure).

In such a guide image, edges defined by a plurality of points on the map coordinates in the map information are converted into points in a virtual space coordinate system that can be superimposed on the image data obtained by the image pickup unit 15. , It is obtained by drawing a virtual straight line connecting the converted points as an image seen from the position of the route guidance device 10 in the virtual space coordinate system.

Specifically, the conversion from the map coordinate system to the virtual space coordinate system is performed in the same manner as the example of the conversion from the virtual space coordinate system to the map coordinate system exemplified above. Specifically, when the route guidance device 10 is located at the A'point (x'1, z'1) on the map coordinate system exemplified in FIG. 6A, the end points (nodes) M'1 and M'of the edge E12. The position of 2 in the virtual space coordinate system is obtained as follows.

First, the information output unit 26 performs the following processing for each of the points M'1 and M'2 in the map coordinates. That is, the information output unit 26 passes through the point M'(either M'1 or M'2, M'1 is taken as an example in FIG. 6), and the point in the positive direction of the Z'coordinate is set as C'. Further, a point D'that passes through this point M'and is on a line segment orthogonal to the edge E12 and is closer to the point A'than the point M'is set. In the following, the coordinates of the point M'are expressed as (x'3, z'3).

とし、また、

を求める。ここでｎは０以上の整数であり、
・Ｃ′点がＭ′点よりもＡ′点に近く、かつ、ｚ′1≧ｚ′3のときには、ｎ＝０
・Ｃ′点がＭ′点よりもＡ′点に近く、かつ、ｚ′2＞ｚ′1のときには、ｎ＝１
・Ｃ′点がＡ′点よりもＭ′点に近く、かつ、ｚ′1≧ｚ′2のときには、ｎ＝１
・Ｃ′点がＡ′点よりもＭ′点に近く、かつ、ｚ′2＞ｚ′1のときには、ｎ＝０とする。なお、角Ｃ′Ｍ′Ｄ′は、設定から既知である。 Then, the information output unit 26 sets the distance d (A'M') between the point A'and the point M'.

And also

Ask for. Here, n is an integer of 0 or more,
When the point C'is closer to the point A'than the point M'and z'1 ≥ z'3, n = 0.
When the C'point is closer to the A'point than the M'point and z'2>z'1, n = 1
When point C'is closer to point M'than point A'and z'1 ≥ z'2, n = 1
When the point C'is closer to the point M'than the point A'and z'2>z'1, n = 0 is set. The angle C'M'D'is known from the setting.

と、求める。ここでｓは既に説明した縮尺比ｓと等しいものである。 Since the angles are saved before and after the conversion, the information output unit 26 determines the relationship between the angles of each point and the distance.

I ask. Here, s is equal to the scale ratio s already described.

を得て、これから点Ｍの仮想空間座標系での座標値（ｘM，ｚM）を、
ｘM＝ｘ′3＋Δｘ
ｚM＝ｚ′3＋Δｚ
として求める。なお、Ｙ座標値は、規定値ｈに設定してもよい。 And the information output unit 26

From now on, the coordinate values (xM, zM) of the point M in the virtual space coordinate system are calculated.
xM = x'3 + Δx
zM = z'3 + Δz
Ask as. The Y coordinate value may be set to the specified value h.

The information output unit 26 obtains the coordinate values in the virtual space coordinate system for each of the points M'1 and M'2 in the map coordinates by the conversion using the equations (9) to (16). Then, a virtual straight line connecting the coordinate values is drawn and rendered as an image seen from the position of the route guidance device 10 in the virtual space coordinate system. This process is similar to the process of drawing in virtual reality (AR).

Further, the information output unit 26 draws a virtual arrow in the direction of the point M'1 or the point M'2 (the direction of the node to which the user should go) by superimposing it on the virtual straight line drawn here. You may. This process can be performed, for example, by drawing a conical figure.

[motion]
This embodiment basically has the above configuration and operates as follows. When a user carrying the route guidance device 10 visits a facility where key images are posted in various places, the route guidance device 10 is activated and one of the key images is imaged, and the route guidance device 10 is next. The process illustrated in FIG. 6 is started. In the following example, the key image includes a code image such as a two-dimensional bar code, and the information that identifies the map information (map information identification information) and the information that identifies the key image are encoded in the code image. It shall be represented. Here, the map information includes information such as edge information representing a route that the user can pass through and information such as the position of a key image, and it is assumed that these information are represented by a map coordinate system.

First, the route guidance device 10 recognizes a key image from the captured image data and extracts a code image included in the key image. The route guidance device 10 decodes the extracted coded image to obtain decoded data. As described above, the decoded data includes map information specific information and information for specifying a key image.

As illustrated in FIG. 7, the route guidance device 10 acquires the map information specified by the map information specific information included in the decoded data from the map information server 20 (S1). Further, the route guidance device 10 detects the position and orientation information of the key image included in the captured image data (S2), arranges XZ Cartesian coordinates on a surface parallel to the actual floor surface, and is vertical on this surface. (S3), the key image in the virtual space coordinate system with the Y-axis in the above direction (the upward direction is the positive direction) and the relative position information with the route guidance device 10 are acquired (S3).

Next, the route guidance device 10 generates information indicating the position of the route guidance device 10 in the map coordinate system based on the relative position between the captured key image and the route guidance device 10 (S4). Information on the position of the route guidance device 10 in the map coordinate system can be obtained by the method described above. That is, the route guidance device 10 acquires the position and orientation information in the map coordinates of the key image specified by the information for specifying the key image in the decoded data from the map information acquired in step S1. Further, the route guidance device 10 is based on the relative position information between the key image in the virtual space coordinate system acquired in step S3 and the route guidance device 10, and the position and orientation information of the key image in the map coordinates. Obtain information on the position of the route guidance device 10 in the map coordinate system.

This process is also performed by converting the marker coordinate system using the key image as a marker and a given virtual space coordinate system, and the conversion between the marker coordinate system and the camera coordinate system which is the coordinate system seen from the route guidance device 10. , Obtain information indicating the position of the route guidance device 10 in the marker coordinate system, obtain a conversion formula between the position and orientation of the corresponding key image in the map coordinate system and the marker coordinate system, and use the map coordinate system. Information indicating the position of the route guidance device 10 may be acquired.

The route guidance device 10 also receives information on coordinates in the map coordinate system representing the location of the destination from the user (S5). Then, the route guidance device 10 finds a point DE on the edge closest to the input destination location and sets the destination (S6). In addition, in the input of the destination in step S5, for example, referring to the destination database in which the store name and the position information of the entrance in the map coordinate system are associated in advance, the store name included in the destination database is input. This may be done by displaying the list and accepting the selection of the store from the list from the user. In this example, the route guidance device 10 performs the process of step S6 using the position information in the map coordinate system associated with the selected store name as the coordinates of the destination.

The route guidance device 10 obtains the point CU on the edge closest to the coordinates in the map coordinate system represented by the information generated in step S4, sets the current position (S7), and sets the current position (S7), and the point CU representing the current position through the edge. A route from to the point DE representing the destination is generated and the route guidance process is executed (S8). The route generated here can be represented, for example, as a sequence of edges and nodes to be passed before reaching the destination, and the processing of the generation can be processed in a well-known navigation system.

Further, an example of the route guidance process in step S8 is as illustrated in FIG. 8 (a). It should be noted that the route guidance device 10 repeats the process of capturing an image by the image pickup unit 15 during this route guidance process.

As a route guidance process, the route guidance device 10 synthesizes and displays an image drawn by the image data captured by the imaging unit 15 as a line segment in the virtual space coordinate system with an edge including the point CU (S11). ). Further, in step S11, the route guidance device 10 may draw an image of a straight line representing an edge and an image of an arrow pointing in the direction of the node to be passed next.

Specifically, in this example, the route guidance device 10 obtains the coordinates of the end points of the edge including the point CU in the virtual space coordinate system. In this process, the route guidance device 10 converts the position of the end point of the edge (represented by the map coordinate system in the map information) into the information of the virtual space coordinate system, and draws a straight line connecting the converted points. It is drawn as an image seen from the position of the route guidance device 10 in the virtual space coordinate system. Here, the process of superimposing the straight line connecting each point of the virtual space coordinate system set based on the captured image data on the image data and drawing the drawing uses the library software for AR processing. Can be realized.

Then, when the user moves with reference to the image displayed on the route guidance device 10, the route guidance device 10 moves together with the user by motion tracking processing using the library software for AR processing. After that, the position of the route guidance device 10 in the virtual space coordinate system is acquired (S12). Then, the route guidance device 10 returns to step S11, superimposes the image data finally captured by the imaging unit 15, and serves as a guidance image from the position updated in step S12, such as a virtual straight line representing an edge or a virtual straight line. Next, a guide image including a virtual arrow indicating the direction of the node to be passed is newly drawn. The route guidance device 10 performs the processing of steps S11 and S12 until the user instructs to stop the processing, or until the distance between the position acquired in step S12 and the destination falls below a predetermined threshold value. repeat.

[Position correction during movement]
Further, when the user moves during the route guidance process by the route guidance device 10, not only the motion tracking process is performed, but also when a new key image can be detected from the image data captured by the image pickup unit 15, the new key image can be detected. The position information of the route guidance device 10 in the map coordinate system may be corrected based on the information such as the position and orientation of the newly detected key image.

As another example of the route guidance process in step S9, the route guidance device 10 of this example sequentially receives the image data obtained by the image pickup unit 15 sequentially taking images as illustrated in FIG. 8 (b). Then, the image in which the edge including the point CU is drawn as a line segment in the virtual space coordinate system is synthesized and displayed with respect to the accepted image data (S21). This process is the same as the process of step S11 shown in FIG. 8 (a). Further, also in this step S21, the route guidance device 10 may draw an image of a straight line representing an edge and an image of an arrow pointing in the direction of the node to be passed next.

Further, the route guidance device 10 attempts to detect a key image from the image data captured by the image pickup unit 15 (S22). Here, when the route guidance device 10 detects the position and orientation information of the key image included in the captured image data (S22: Yes), the virtual space is used by using the position and orientation information of the detected key image. The key image in the coordinate system and the relative position information of the route guidance device 10 are acquired (S23).

Then, the route guidance device 10 generates information indicating the position of the route guidance device 10 in the map coordinate system based on the relative position between the captured key image and the route guidance device 10 (S24). Since the information on the position of the route guidance device 10 in the map coordinate system is the same as the method already described, the repeated description will be omitted. As a result, the route guidance device 10 acquires the position of the route guidance device 10 in the virtual space coordinate system after moving with the user. The position information acquired in step S24 is the position information corrected by the newly detected key image, and is expected to be relatively accurate as compared with the position obtained by the motion tracking process.

On the other hand, if the key image cannot be detected from the captured image data in step S22 (S22: No), the route guidance device 10 uses the library software for AR processing to perform motion tracking processing. Acquires the position of the route guidance device 10 in the virtual space coordinate system after moving with the user (S25). Then, the route guidance device 10 returns to step S21, superimposes the image data finally captured by the imaging unit 15, and represents an edge as a guidance image from the position updated in step S24 or step S25. A new guide image including a straight line and a virtual arrow indicating the direction of the node to be passed next is drawn.

According to this example of the present embodiment, the error of the position information accumulated by the motion tracking process due to the movement of the user can be corrected based on the position information relative to the newly detected key image.

[Presentation of guidance information]
Further, the route guidance device 10 of the present embodiment determines whether or not a predetermined cancellation condition is satisfied when displaying information including a guidance image, and at least guides the user while the cancellation condition is satisfied. The display of the image may be stopped.

Here, the stop condition may be a condition based on the current position of the route guidance device 10 and a predetermined area (hereinafter referred to as a stop condition area) in the map information.

In this example, the administrator of the route guidance system 1 sets the stop condition area in the map information in advance. This stop condition area can be represented as a set of three or more points (stop condition area specified points) surrounding the stop condition area, and here, the coordinates of the stop condition area specified points are represented by a map coordinate system. It shall be.

The route guidance device 10 determines the position of the route guidance device 10 at that time when displaying information including a guide image, for example, when performing the process of step S11 or step S21 in FIGS. 8A and 8B. It is checked whether or not the point CU in the represented map coordinate system is inside the stop condition area. When the route guidance device 10 determines that the point CU in the map coordinate system representing the position of the route guidance device 10 is inside the stop condition region, at least a straight line representing an edge or the next passage is passed. Stop displaying the guide image including the arrow indicating the direction of the power node.

At this time, the route guidance device 10 may stop displaying not only the guide image but also the image data last captured by the image pickup unit 15. Further, when the display of the guidance image or the like is stopped in this way, the route guidance device 10 may display a message or the like indicating that the user is in the area designated to stop the display of the guidance of the route.

Further, the route guidance device 10 may display an area on the map image in which the cancellation condition is satisfied, together with or in place of this message. At this time, the route guidance device 10 may superimpose the map image included in the map information on the map image and draw a polygon surrounded by the stop condition area defined point to represent the stop condition area. ..

The administrator of the route guidance system 1 sets, for example, an area including an escalator boarding / alighting position, an intersection, and other places where the user of the route guidance device 1 is determined not to gaze at the screen as a stop condition area. be able to.

The discontinuation condition is not limited to the example described here, and the image captured by the image pickup unit 15 is, for example, a condition that the image data captured by the image pickup unit 15 includes a predetermined key image. The conditions may be based on data, and various conditions such as time zone, date, and day of the week can be adopted.

Further, in the example described here, at least the display of the guide image is stopped while the cancellation condition is satisfied, but the present embodiment is not limited to this. That is, the route guidance device 10 of the present embodiment determines whether or not the predetermined implementation conditions are satisfied when displaying the information including the guidance image, and only while the implementation conditions are satisfied. The guide image may be displayed. In this example, if the implementation conditions are not satisfied, at least the display of the guide image is stopped.

Also in this example, the implementation conditions may be conditions based on the current position of the route guidance device 10 and a predetermined region (hereinafter referred to as an implementation condition region) in the map information. That is, the administrator of the route guidance system 1 may set the implementation condition area in the map information in advance. Here, the implementation condition area can also be set by defining three or more points (implementation condition area regulation points) surrounding the implementation condition area, as in the case of the stop condition area. The coordinates of this implementation condition area regulation point are also represented by the map coordinate system.

The route guidance device 10 determines the position of the route guidance device 10 at that time when displaying information including a guide image, for example, when performing the process of step S11 or step S21 in FIGS. 8A and 8B. It is examined whether or not the point CU in the represented map coordinate system is inside the above-mentioned implementation condition area. When the route guidance device 10 determines that the point CU in the map coordinate system representing the position of the route guidance device 10 is not inside the implementation condition region, at least a straight line representing an edge or the next passage is passed. Stop displaying the guide image including the arrow indicating the direction of the power node.

Further, when the route guidance device 10 determines that the point CU in the map coordinate system representing the position of the route guidance device 10 is inside the implementation condition region, the image finally captured by the image pickup unit 15 is obtained. A guide image including a virtual straight line representing the edge seen from the position CU and a virtual arrow indicating the direction of the node to be passed next is drawn by superimposing the data.

When the route guidance device 10 determines that the point CU in the map coordinate system representing the position of the route guidance device 10 is not inside the implementation condition region, not only the guide image but also the image pickup unit 15 finally determines. The display of the captured image data may also be stopped. Further, when the display of the guidance image or the like is stopped in this way, the route guidance device 10 may display a message or the like indicating that the user is in the area designated to stop the display of the guidance of the route.

Further, the route guidance device 10 may display an area on the map image in which the implementation conditions are satisfied, together with or in place of this message. At this time, the route guidance device 10 may superimpose the map image included in the map information on the map image and draw a polygon surrounded by the implementation condition area defining points to represent the implementation condition area. ..

The administrator of the route guidance system 1 includes, for example, the vicinity of the guide plate, a floor map, and other areas including a place where it is judged that there is no problem even if the user of the route guidance device 1 looks at the screen. It can be set as an area.

Further, the implementation conditions are not limited to the conditions based on the area set on the map. As an example of another implementation condition, for example, a condition that the image data captured by the image pickup unit 15 includes a key image, or a condition determined based on the image data captured by the image pickup unit 15. good. Further, conditions such as the date and time and the day of the week may be used as implementation conditions.

[Other examples of guidance images]
In the description so far, the guide image uses a virtual straight line or a figure displayed by superimposing the image data captured by the image pickup unit 15, but the present embodiment is limited to this. not. That is, the route guidance device 10 replaces or with a virtual straight line or figure displayed as a guide image superimposed on the image data captured by the image pickup unit 15, and together with the map image included in the map information. , An image including a marker indicating the position of the route guidance device 10 on the map image may be displayed.

[Modification example related to key image detection]
In the present embodiment, the route guidance device 10 needs to identify the information of the position where the captured key image is posted. Therefore, in the above description, the key images are different from each other or have a reference numeral. Although it was supposed that the images were combined, the present embodiment is not limited to this.

For example, if the route guidance device 10 can recognize an absolute direction (for example, northward) with respect to the ground surface, that is, if it is equipped with a geomagnetic measuring instrument or the like, the posted key image is the same or similar to the image itself. Even if they exist, they can be identified if the directions of the notices (angles from the north direction) are different from each other.

As illustrated in FIG. 9A, a point D in the direction of the key image (from a predetermined point B in the surface of the key image to the normal direction of the surface), and the predetermined points B to Z of the key image. Using the point C in the axis (Z-axis of the virtual space coordinate system) direction and the north-facing N-axis, the angle of the N-axis from the Z-axis is θ (θ is the clockwise direction when the floor surface is viewed from above. Is positive, and −π <θ ≦ π), and when the point n is taken from the point B in the direction parallel to the N axis, the angle θ is subtracted from the angle CBD to obtain the angle nBD.

Further, as illustrated in FIG. 9B, a point D'in the direction of the key image in the map coordinate system (from the point B'corresponding to the point B in the plane of the key image to the normal direction of the plane). , Point C'in the direction of the Z'axis (Z'axis of the map coordinate system) from the point B'in the key image, and the north-facing N'axis, and the angle of the N'axis from the Z'axis is θ'. (As for θ', the clockwise direction when the floor surface is viewed from above is positive, and −π <θ ′ ≦ π), and the point n ′ (point n ′) in the direction parallel to the N ′ axis from the point B ′. The angle n'B'D'can be obtained by subtracting the angle C'B'n'= θ'from the angle C'B'D'.

In this example, in the map information, the value of the above angle n'B'D'(this value is a known value that can be obtained in advance) is set for each key image as the orientation feature amount of the key image. Associate and record.

The route guidance device 10 obtains an angle nBD between the direction (vector BD) of the key image detected from the captured image data and the north direction (vector BN) separately detected. Further, the route guidance device 10 searches for a key image included in the map information in association with an image matching the key image detected from the captured image data. Here, since it is assumed that a plurality of the same or similar key images are included in the map information, a plurality of key images (candidates) are found as a result of this search.

The route guidance device 10 has an angle n'B'D'recorded in association with the key image candidate found by the search (to distinguish each of them below, the angle n'B'D'for each key image candidate. Is taken as the angle αi (i = 1, 2, ...; i corresponds to each key image candidate)) is acquired from the map information.

The route guidance device 10 obtains the absolute value βi = | αi − angle nBD | of the difference between the angle αi for each acquired key image candidate and the obtained angle nBD, and finds the smallest βi among the βi. Then, the route guidance device 10 identifies a key image related to the angle αi corresponding to the minimum βi, and determines that the specified key image is a key image included in the captured image data. Then, the route guidance device 10 obtains information for identifying the determined key image.

The route guidance device 10 uses the information for specifying the key image obtained here, acquires the position and orientation information in the map coordinate system for the key image specified by the information for specifying the key image, and obtains the route. It is used for processing such as calculating the position of the guide device 10 in the map coordinate system.

[Floor identification]
Further, as already described, in the present embodiment, the map information may be set across a plurality of floors, but in this case, the route guidance device 10 is located on which floor. Need to be identified.

Therefore, information for identifying the floor on which the key image is posted is included in the map information in association with the key image, and the floor where the route guidance device 10 is located when the key image is detected from the captured image data is included in the map information. It may be specified.

However, in this case, immediately after moving between floors by an escalator, an elevator, or the like, the route guidance device 10 may misidentify which floor the route guidance device 10 is located on. Therefore, the key image may be posted on the floor surface or wall surface that is first imaged when the floor is reached, such as the escalator for each floor or the exit of the elevator.

Further, when the route guidance device 10 has a built-in barometer, it may be detected that the floor has been moved by using the atmospheric pressure difference or the like.

[Effect of embodiment]
According to the example of the present embodiment, when the route guidance device 10 detects an AR marker which is a key image arranged in various places of the facility by using virtual reality (AR) technology, the AR coordinate system of the AR marker (AR coordinate system). For example, the position and orientation in (may match the camera coordinate system) and the position and orientation of the corresponding AR marker in the map coordinate system fixed to the ground surface in advance as map information are used. The position and orientation of the route guidance device 10 in the map coordinate system are specified.

Further, as a result, the route guidance device 10 in the specified map coordinate system uses the information of the edges and nodes (represented by the map coordinate system) included in the network of the passable route set as the map information. Navigation by so-called navigation technology is realized by using position and orientation information. Therefore, it is possible to provide route guidance in various spaces including indoors.

Further, in the present embodiment, the AR technology is used to superimpose the image data of the passage in the facility imaged by the route guidance device 10 on the screen of the route guidance device 10, and to form a straight line corresponding to the edge. Draws and displays a guide image such as a figure of the above and a figure showing the direction of the node to be passed next. This makes it possible to clearly indicate the moving direction to the user.

Then, the route guidance device 10 cancels the display of the guidance image in the area satisfying the cancellation condition predetermined by the administrator or the like. Alternatively, the guidance display is performed only in the area (so to speak, the zone where AR display is permitted) that satisfies the implementation conditions set in advance by the administrator or the like.

As a result, it is possible to suppress a decrease in usage etiquette due to watching the guidance display at an arbitrary place.

1 route guidance system, 10 route guidance device, 11 control unit, 12 storage unit, 13 operation unit, 14 display unit, 15 imaging unit, 16 communication unit, 20 map information server, 21 map information acquisition unit, 22 key detection unit, 23 Relative position acquisition unit, 24 Map position information acquisition unit, 25 Route guidance processing unit, 26 Information output unit.

Claims (9)

It is a route guidance device that guides a route that combines edges.
A predetermined key image is posted in a plurality of places in the vicinity of the edge.
Map information that defines edges by a predetermined map coordinate system, and at least the position where the key image is posted is a map information acquisition means for acquiring map information defined by the map coordinate system.
An imaging means that images the periphery of the route guidance device and outputs the captured image data.
A key detecting means for detecting the key image from the image data,
Based on the detected key image, a relative position acquisition means for acquiring relative position information between the key image in a predetermined virtual space coordinate system and the route guidance device, and
It has an in-map position information acquisition means for acquiring in-map position information of the map coordinate system of the route guidance device by using the acquired map information and the acquired relative position information.
A route guidance device that performs a predetermined process of searching and guiding a route along the edge of the acquired position information in a map. The route guidance device according to claim 1.
The image pickup means takes an image of the periphery of the route guidance device while guiding the route, and outputs the captured image data.
The key detecting means detects a key image from the image data and obtains a key image.
When the key image is detected from the image data, the relative position acquisition means acquires the relative position information between the key image in the predetermined virtual space coordinate system and the route guidance device based on the detected key image. death,
When the relative position information is acquired, the map position information acquisition means acquires the map position information of the map coordinate system of the route guidance device.
The route guidance device is
It further has a correction means for correcting the position information of the route guidance device based on the acquired position information in the map. The route guidance device according to claim 1 or 2.
A route guidance device further comprising a display means for superimposing and displaying an image captured by the image pickup means and a guide image for guiding the route during route guidance. The route guidance device according to claim 3.
The display means is a route guidance device that stops the display of at least the guidance image while the predetermined stop condition is satisfied. The route guidance device according to claim 3.
The display means is a route guidance device that displays the guidance image only while the predetermined implementation conditions are satisfied. The route guidance device according to claim 4.
The discontinuation condition is a condition based on the position of the route guidance device and the area in the map.
The display means is a route guidance device that displays an area where the discontinuation condition is satisfied. The route guidance device according to claim 5.
The implementation condition is a route guidance device which is a condition determined based on the image data captured by the image pickup means. A key image bulletin board installed by the administrator and posting a predetermined key image at multiple positions near the edge where the user can pass, and
An information providing device that provides map information that defines an edge by a predetermined map coordinate system, and at least the position where the key image is posted provides the map information defined by the map coordinate system.
A route guidance system that includes a route guidance device carried by the user.
The route guidance device
The map information acquisition means for acquiring the map information and
An imaging means that images the periphery of the route guidance device and outputs the captured image data.
A key detecting means for detecting the key image from the image data,
Based on the detected key image, a relative position acquisition means for acquiring relative position information between the key image in a predetermined virtual space coordinate system and the route guidance device, and
It has an in-map position information acquisition means for acquiring in-map position information of the map coordinate system of the route guidance device by using the acquired map information and the acquired relative position information.
A route guidance system that performs a predetermined process of searching and guiding a route along the edge of the acquired position information in a map. A program that causes a computer to guide a route in which a predetermined key image is a combination of edges posted in multiple places in the vicinity thereof.
Map information acquisition means for acquiring map information defined by the map coordinate system at least at a position where the key image is posted, which is map information defining the edge by a predetermined map coordinate system.
An imaging means that images the periphery of the route guidance device and outputs the captured image data.
A key detecting means for detecting the key image from the image data,
Based on the detected key image, a relative position acquisition means for acquiring relative position information between the key image in a predetermined virtual space coordinate system and the route guidance device, and
Using the acquired map information and the acquired relative position information, an in-map position information acquisition means for acquiring the in-map position information of the map coordinate system of the route guidance device, and an in-map position information acquisition means.
To function as
A program that causes the computer to perform a predetermined process of searching and guiding a route along the edge of the acquired position information in a map.

Images