JP7146542B2 - Route guidance program, route guidance device, and route guidance system - Google Patents

Description

The present invention relates to a route guidance program, a route guidance device, and a route guidance system that provide route guidance to a user of a terminal device (typically a mobile terminal device such as a mobile phone) using augmented reality (AR) technology.

2. Description of the Related Art In recent years, many application software (hereinafter abbreviated as “apps”) using maps have been developed as mobile terminal devices such as mobile phones have become more sophisticated and have higher resolutions. As one of such applications, an application called a "route guidance application" or the like for guiding a user of a mobile terminal device (hereinafter referred to as "user") to a destination is known. The route guidance application searches for a route from the current position of the user to the destination, and displays the result together with a map on the screen of the mobile terminal device. By using such a route guidance application, the user can easily reach the destination even if there is no map at hand.

By the way, "meeting" is an act that is performed on a daily basis as a human act. Meetings are generally held at specific locations. Therefore, the route guidance application described above is also used by the user when meeting. However, people often feel stress when the other party does not show up at the meeting place when the meeting time arrives. Also, in such cases, people often want to know the current location of the other party. Therefore, in recent years, services have been provided that allow multiple users of mobile terminal devices to share location information with each other. According to this service, the user can check the current location of another person on the map when meeting.

In addition, in Japanese Patent Application Laid-Open Nos. 2000-348286 and 2010-192994, it is possible to search and display a route from one's own current position to the other's current position on a one-to-one basis (display on a two-dimensional map). ) is disclosed.

JP-A-2000-348286 JP 2010-192994 A

According to the conventional technology described above, information on the current position of the other party is displayed on a two-dimensional map (hereinafter referred to as a "2D map"). Therefore, it is possible to grasp the two-dimensional relationship between one's own current position and the other party's current position at the time of meeting. However, depending on the environment of the meeting place, it often takes a considerable amount of time for the two to meet even though they are close to each other on the 2D map. In this regard, factors such as those described below may prevent the two parties from discovering each other at the time of meeting.
1: The meeting place is crowded (for example, in front of the ticket gate of a major station, in front of an event venue).
2: The meeting place is spatially large and there are no suitable landmarks (for example, a huge live venue, a huge theme park).
3: The structure of the building used as the meeting place is complicated (for example, underground malls at major stations, huge commercial facilities such as department stores and shopping malls).

In view of the above points, the present invention is a meeting (meeting with two people), regardless of the environment (degree of congestion, size of space, complexity of structure such as building) of the place where both meet. It is an object of the present invention to provide a route guidance program, a route guidance device, and a route guidance system that enable a person to meet a person.

A first invention provides a computer included in a mobile terminal device having a function of displaying a captured image on a display unit,
location information acquisition means for acquiring information on the current location of the device at any time;
Location information sharing means for sharing current location information between the device itself and another device;
Based on the current position information of the own device acquired by the position information acquisition means and the current position information of the other device shared by the position information sharing means, from the current position of the own device to the other device a route search means for searching for a travel route to the current position; and
Functioning as a route display means for displaying an augmented reality image indicating the traveling route obtained by the route search means, superimposed on the captured image ,
If the straight line distance between the current position of the own device and the current position of the other device is equal to or greater than a predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the distance of the other device. A route along the road to the current position is obtained,
If the straight line distance between the current position of the own device and the current position of the other device is less than the predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the other device. It is a route guidance program characterized by obtaining a straight route to the current position of .

A second invention is based on the first invention,
said computer,
sharing request means for requesting the other device to share current location information; and
Further functioning as response means for notifying the other device whether or not to accept the request when the other device requests sharing of current location information,
Current position by the location information sharing means when there is a notification to the effect that the request by the sharing request means is accepted from the other device and when the response means accepts the request from the other device of information is shared.

A third invention is, in the second invention,
said computer,
Further functioning as an approval time setting means for setting an approval time for allowing sharing of current location information when the other device notifies that the request by the sharing request means is accepted,
The current position information is shared by the position information sharing means for a period from a predetermined timing until the approval time set by the approval time setting means elapses.

A fourth invention is, in the third invention,
said computer,
It is characterized by further functioning as approval time resetting means for resetting the approval time when the approval time set by the approval time setting means has passed from the predetermined timing.

A fifth invention is any one of the first to fourth inventions,
The route display means displays an augmented reality image showing the traveling direction of the other device in addition to the augmented reality image showing the traveling route.

A sixth invention is any one of the first to fifth inventions,
The route display means is characterized by displaying an augmented reality image showing the orientation of the holder of the other device in addition to the augmented reality image showing the traveling route.

A seventh invention is any one of the first to sixth inventions,
said computer,
Captured video request means for requesting transmission of captured video from the other device;
a photographed image transmission means for transmitting a photographed image of the own device to the other device when a request for transmission of the photographed image is received from the other device;
It is characterized by further functioning as other device image display means for displaying the captured image on the display unit when the other device transmits the captured image in response to the request by the captured image requesting means.

An eighth invention is a route guidance device having a display unit,
a captured image display means for displaying a captured image on the display unit;
a location information acquisition means for acquiring information on the current location of the device itself at any time;
location information sharing means for sharing current location information between the device itself and another device;
Based on the current position information of the own device acquired by the position information acquisition means and the current position information of the other device shared by the position information sharing means, from the current position of the own device to the other device a route search means for searching for a travel route to the current position;
route display means for superimposing and displaying an augmented reality image showing the travel route obtained by the route search means on the captured image ;
If the straight line distance between the current position of the own device and the current position of the other device is equal to or greater than a predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the distance of the other device. A route along the road to the current position is obtained,
If the straight line distance between the current position of the own device and the current position of the other device is less than the predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the other device. A straight path to the current position of is obtained .

A ninth invention is a route guidance system realized by two mobile terminal devices having a display unit, a first mobile terminal device and a second mobile terminal device,
Each mobile terminal device
a captured image display means for displaying a captured image on the display unit;
a location information acquisition means for acquiring information on the current location of the device itself at any time;
location information sharing means for sharing current location information between the device itself and another device;
Based on the current position information of the own device acquired by the position information acquisition means and the current position information of the other device shared by the position information sharing means, from the current position of the own device to the other device a route search means for searching for a travel route to the current position;
route display means for superimposing and displaying an augmented reality image showing the travel route obtained by the route search means on the captured image ;
If the straight line distance between the current position of the own device and the current position of the other device is equal to or greater than a predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the distance of the other device. A route along the road to the current position is obtained,
If the straight line distance between the current position of the own device and the current position of the other device is less than the predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the other device. A straight path to the current position of is obtained .

According to the first aspect, information on the current position is shared between the device itself and the other device. Then, the route searching means searches for a travel route from the current position of the device itself to the current position of the other device, and an augmented reality image showing the travel route is displayed superimposed on the captured image. Since the direction of the current position of the meeting partner is indicated using the augmented reality image, the meeting partner can be easily found regardless of the surrounding environment when the user and the meeting partner approach each other. As described above, when meeting (meeting with two people), both parties can easily meet regardless of the environment of the place where they meet (degree of congestion, size of space, complexity of structure such as building). becomes possible. In addition, since the traveling route from the current position of the own device to the current position of the other device is indicated, it is unnecessary to determine a specific meeting place at the time of meeting. Furthermore, since the route is indicated using an augmented reality image, the user can intuitively grasp the direction in which he/she should proceed, and the user can meet the other party simply by following the augmented reality image showing the route. can be done.
Moreover, when the self and the other party are separated, it becomes easy to move toward the other party, and when the self and the other party are in a close position, it becomes easy to search for the other party.

According to the second aspect of the present invention, since information on the current position is shared between mutually authenticated persons, infringement of privacy due to unauthorized sharing of the information on the current position is prevented.

According to the third aspect, it is possible to prevent invasion of privacy due to unintended continuation of the shared state of current position information for a long period of time.

According to the fourth aspect of the invention, since the approval time can be reset, when the approval time elapses, route guidance for meeting can be provided without redoing the process of mutual authentication between two users. can continue.

According to the fifth aspect, it is possible to confirm whether or not the meeting partner is moving closer to the user.

According to the sixth aspect, for example, when meeting a person for the first time, it is easy to identify the person even if the place where the two meet is crowded.

According to the seventh aspect, it is possible to confirm the scenery that the meeting partner is looking at, so that the current position of the meeting partner can be grasped in detail.

According to the eighth invention, the same effects as those of the first invention are obtained.

According to the ninth invention, the same effects as those of the first invention are obtained.

1 is a block diagram showing a device configuration for realizing a route guidance system according to one embodiment of the present invention; FIG. 2 is a block diagram showing the hardware configuration of a mobile terminal device in the embodiment; FIG. 2 is a block diagram showing a detailed functional configuration of a mobile terminal device that functions as a route guidance device in the embodiment; FIG. FIG. 10 is a flow chart showing a processing procedure of a mobile terminal device on a request transmission side in the embodiment; FIG. FIG. 10 is a flow chart showing a processing procedure of a mobile terminal device on the request receiving side in the embodiment; FIG. FIG. 4 is a diagram showing an example of a shot image displayed on the display unit in the embodiment; FIG. 11 is a diagram showing a display example of an augmented reality image showing a traveling route in the embodiment; FIG. 11 is a diagram showing another display example of an augmented reality image showing a traveling route in the above embodiment. FIG. 11 is a diagram showing a display example of an augmented reality image showing the traveling direction of the other party in the above embodiment. FIG. 11 is a diagram showing a display example of an augmented reality image showing the orientation of the other party in the above embodiment. FIG. 11 is a diagram showing an example of an augmented reality image showing which direction the other party is facing in relation to the self in the above embodiment. FIG. 11 is a diagram showing a display example of an augmented reality image showing the opponent's orientation and the opponent's traveling direction in the above embodiment. FIG. 10 is a diagram showing a display example when a captured image from another device is displayed on the display unit in the above embodiment. It is a figure showing an example of the warning screen to the effect that it is moving away from the other party's position in the above-mentioned embodiment. FIG. 10 is a diagram showing a display example of the time required for the user and the other party to meet in the above embodiment. FIG. 10 is a diagram showing an example of changing the width of the route image according to the distance between the player and the opponent in the embodiment; FIG. 11 is a diagram showing an example in which an augmented reality image is superimposed on an actual captured image and displayed in the above embodiment. FIG. 11 is a diagram showing an example in which an augmented reality image is superimposed on an actual captured image and displayed in the above embodiment. FIG. 11 is a diagram showing an example in which an augmented reality image is superimposed on an actual captured image and displayed in the above embodiment.

An embodiment of the present invention will be described below with reference to the accompanying drawings.

<1. Overall configuration>
FIG. 1 is a block diagram showing a device configuration for realizing a route guidance system according to one embodiment of the present invention. This route guidance system is implemented by a server 20 and two mobile terminal devices 10 . The server 20 and the mobile terminal device 10 are connected via a communication line such as the Internet. In this embodiment, a route guidance application is installed in the mobile terminal device 10 in order to realize this route guidance system. The route guidance application in the present embodiment is software for presenting a route from one's current position to the current position of the meeting partner when meeting up with a person. A user can start the route guidance application by a predetermined operation. Note that the route guidance application may have a function of setting a fixed position as a destination and presenting a route from the current position to the destination, but such a function is not included in the present invention. is not directly related, so its explanation is omitted.

<2. Hardware Configuration of Mobile Terminal Device>
FIG. 2 is a block diagram showing the hardware configuration of the mobile terminal device 10. As shown in FIG. The mobile terminal device 10 has a CPU 11 , a flash ROM 12 , a RAM 13 , a communication control section 14 , an image capturing section (camera) 15 , an input operation section 16 , a display section 17 , a GPS sensor 18 and a geomagnetic sensor 19 . The CPU 11 performs various arithmetic processing and the like in order to control the mobile terminal device 10 as a whole. The flash ROM 12 is a writable non-volatile memory, and stores various programs and various data that should be retained even when the power of the mobile terminal device 10 is turned off. The RAM 13 is a writable volatile memory that temporarily stores programs, data, and the like being executed. The communication control unit 14 controls data transmission to the outside and data reception from the outside. The image capturing unit (camera) 15 captures the scenery seen from the current position based on the user's operation. The input operation unit 16 is, for example, a touch panel, and receives input operations by the user. The display unit 17 displays images based on commands from the CPU 11 . The GPS sensor 18 acquires position information (latitude and longitude information) for identifying the user's current position based on radio waves received from GPS satellites. The geomagnetic sensor 19 acquires azimuth information indicating the azimuth to which the mobile terminal device 10 is directed (for example, the azimuth to which the display unit 17 is directed). Note that although an example in which the GPS sensor 18 and the geomagnetic sensor 19 are provided in the mobile terminal device 10 as sensors is described here, sensors other than these (for example, an acceleration sensor) are provided in the mobile terminal device 10 . Sometimes it is

In the mobile terminal device 10 , a route guidance program that implements the route guidance application is stored in the flash ROM 12 . When the user instructs to start the route guidance application, the route guidance program stored in the flash ROM 12 is read to the RAM 13, and the CPU 11 executes the route guidance program read to the RAM 13. Accordingly, the mobile terminal device 10 functions as a route guidance device that provides the user with various functions for route guidance.

<3. Functional configuration>
FIG. 3 is a block diagram showing a detailed functional configuration of the mobile terminal device 10 functioning as a route guidance device. As shown in FIG. 3, the mobile terminal device 10 functionally includes a sharing establishment means 100, an information sharing means 110, an approval time setting means 120, an approval time resetting means 130, an orientation detection means 140, a current position detection means. It has means 150 , route searching means 160 , video shooting means 170 , AR route display means 180 and video reference means 190 . Sharing establishment means 100 includes sharing request means 102 and response means 104 . Information sharing means 110 includes location information sharing means 112 . The image referencing means 190 includes a captured image requesting means 192 , a captured image transmitting means 194 , and another device image displaying means 196 .

In the following description, of the two mobile terminal devices 10 that implement the route guidance system, the mobile terminal device 10 of interest here will be referred to as the "own device", and the other mobile terminal device 10 will be referred to as the "other device". It says. Exchange of data and the like between the device itself and other devices is performed via the server 20 . However, data may be exchanged directly between the device itself and another device.

The sharing establishment means 100 establishes a state in which the current position information is shared between the device itself and another device. Sharing request means 102 requests sharing of current location information from other devices. The response means 104 notifies the other device whether or not to accept the request when the other device requests sharing of the current position information. When there is a notification from the response means 104 of the other device to the effect that the request by the sharing request means 102 of the own device is accepted, and when the response means 104 of the own device responds to the request by the sharing request means 102 of the other device When the device notifies that it accepts the request, a state is established in which the current location information is shared between the device itself and the other device. When a request is made by the sharing request means 102 and a response is made by the response means 104, a screen prompting for passcode input is displayed, and two users enter predetermined passcodes. The information on the current location may be shared only when it is received.

The information sharing means 110 maintains a state in which various types of information including current position information are shared between the own device and other devices via the server 20 . In other words, the information sharing means 110 transmits the information of its own device to the server 20 and acquires the information of the other device from the server 20 so that the other device can refer to the information of its own device. Positional information sharing means 112 shares current positional information between its own device and other devices.

Approval time setting means 120 sets an approval time for allowing sharing of current location information when receiving a notification from response means 104 of another device to the effect that the request is accepted in response to a request by sharing request means 102 of own device. (length of time). The approval time resetting means 130 resets the approval time when the approval time set by the approval time setting means 120 has passed.

The azimuth detecting means 140 detects the azimuth to which the mobile terminal device 10 is facing. This azimuth detection means 140 is implemented by the geomagnetic sensor 19 (see FIG. 2) as hardware. Current position detection means 150 detects the current position of the user based on radio waves received from GPS satellites. This current position detection means 150 is implemented by the GPS sensor 18 (see FIG. 2) as hardware. Although position detection using GPS is described here as an example, the present invention is not limited to this. For example, the current position may be detected using indoor positioning technology such as Wi-Fi (registered trademark) and beacons, or VIO (Visual-Inertial Odometry) technology (position recognition using camera images and inertial sensors technology) may be used to detect the current position.

The route searching means 160 calculates the current position of the own device based on the current position information of the own device detected by the current position detecting means 150 and the current position information of the other devices shared by the position information sharing means 112. to the current position of the other device. Road information and the like downloaded from the server 20 at appropriate timings are used when searching for a travel route by the route search means 160, but this is not directly related to the present invention, so a description thereof will be given below. are omitted. The image capturing means 170 captures a landscape image. The image capturing means 170 is implemented by the image capturing unit (camera) 15 (see FIG. 2) as hardware. A photographed image (landscape image) as a result of photographing by the image photographing means 170 is displayed on the display section 17 . The AR route display means 180 displays an augmented reality (AR) image showing the traveling route obtained by the route searching means 160 so as to be superimposed on the captured image. As a result, the captured image in which the augmented reality image is embedded is displayed on the display unit 17 .

The image reference unit 190 realizes display of images shot by another device on its own device and display of images shot by its own device on another device. The photographed image requesting means 192 requests transmission of a photographed image (a photographed image as a result of photographing by the image photographing unit 170 of the other device) to the other device. The photographed image transmitting means 194 transmits the photographed image (the photographed image which is the result of photographing by the image photographing unit 170 of the own device) to the other device when there is a request for the transmission of the photographed image from another device. The other device video display means 196 displays the captured video on the display unit 17 when a captured video is transmitted from another device in response to the request from the captured video requesting means 192 .

In this embodiment, the current position detection means 150 implements the position information acquisition means, and the AR route display means 180 implements the route display means.

<4. Processing procedure>
Next, a process procedure of route guidance (guidance of a traveling route from one's current position to the other party's current position) in this embodiment will be described. Here, first, the processing procedure (see FIG. 4) of the side requesting the sharing of the current location information (hereinafter referred to as the "request sending side") will be described, and then the side requesting the sharing of the current location information (see FIG. 4) will be described. Hereinafter, the processing procedure (refer to FIG. 5) of the "request receiving side") will be described.

<4.1 Processing procedure of request sender>
FIG. 4 is a flowchart showing a processing procedure when the own device becomes the mobile terminal device 10 on the request transmission side. After a waiting state (step S100) after activation of the route guidance application, the user of the own device transmits a command requesting sharing of current position information to other devices (step S110). In other words, the sharing request means 102 of the own device requests the other device to share the current position information.

After that, based on the notification transmitted from the other device, it is determined whether or not the request is permitted (step S120). As a result, if the above request is not granted, the process returns to step S100. On the other hand, if the above request is permitted, the approval time setting means 120 performs processing for setting a time (approval time) during which sharing of information on the current location is permitted (step S130). In this step S130, for example, a screen for inputting the approval time is displayed on the display unit 17 of the device itself, and the user inputs an appropriate length of time from the screen. As a result, the following steps S140 to S160 are repeated until the approval time set in step S130 elapses. In step S120, if no notification is sent from the other device within a certain period of time after sending the command requesting the sharing of the current position information, it may be determined that the request has not been permitted.

In step S140, acquisition of current position information of the own device and other devices, which is position information required for route search, is performed. More specifically, information on the current position of the device is acquired by the current position detection means 150 (GPS sensor 18) based on radio waves received from GPS satellites, and information on the current position of other devices held in the server 20 is obtained. Acquired by the location information sharing means 112 . After that, based on the position information acquired in step S140, the route searching means 160 searches for a traveling route from the current position of the device itself to the current position of the other device (step S150).

After searching for the travel route, an augmented reality image showing the travel route is displayed on the display unit 17 of the device itself, superimposed on the captured video (video captured by the video capturing means 170) (step S160). In this embodiment, a route along a road (a route that can actually be traveled) from the current position of the device to the current position of the other device is displayed as a travel route using augmented reality technology. In this regard, it is assumed that, for example, a photographed video as shown in FIG. 6 is displayed on the display unit 17 before the augmented reality image is displayed. In this case, in step S160, the display unit 17 displays an augmented reality image within the captured image as shown in FIG. In the example shown in FIG. 7, as the augmented reality image, a route image 41 whose line width is narrowed as the distance from the current position increases, and an arrow image 42 representing the direction of travel are displayed superimposed on the captured image.

For example, as shown in FIG. 8, a linear route from the current position of the device itself to the current position of the other device may be displayed as a progress route using augmented reality technology. Also, the display mode of the augmented reality image showing the traveling route may be changed according to the linear distance between the current position of the own device and the current position of the other device. For example, if the straight line distance is equal to or greater than a predetermined threshold, a route along a road from the current position of the device itself to the current position of the other device is displayed as an augmented reality image as shown in FIG. If the distance is less than a predetermined threshold, as shown in FIG. 8, a linear route from the current position of the device itself to the current position of the other device may be displayed as an augmented reality image as a traveling route.

After the augmented reality image is displayed, it is determined whether or not the approval time set in step S130 has elapsed (determination whether or not the approval time has elapsed from a predetermined timing such as when the approval time has been set). (Step S170). As a result, if the approval time has not passed, the process returns to step S140, and if the approval time has passed, the process proceeds to step S180.

In step S180, it is determined whether or not the approval time has been reset. As a result, if the approval time has been reset, the process returns to step S130, and if the approval time has not been reset, the process proceeds to step S190. In this step S180, first, a screen for asking the user whether or not to extend the approval time is displayed on the display unit 17 of the own device. On that screen, the user selects whether or not to extend the approval time. When the user selects to extend the approval time, a screen for inputting the approval time again is displayed on the display unit 17 of the device itself, and the user inputs an appropriate length of time from the screen. . When the approval time is reset, information on the approval time after reset is sent to the server 20, and when the approval time is not reset, information to that effect is sent to the server 20. be.

In step S190, the sharing of the position information and the like between the own device and the other device ends. As a result, the route guidance from the current position of the own device to the current position of the other device ends.

By the way, by repeating the processing of steps S140 to S160 as described above, the captured video/augmented reality image in the display unit 17 changes at any time according to changes in the current position/orientation of the self and changes in the current position of the other party. do. The user should proceed according to the augmented reality image showing the travel route.

In this embodiment, the approval time can be reset, but the present invention is not limited to this, and when the initially set approval time elapses, the current location information sharing state is forcibly canceled. You can do so.

<4.2 Processing procedure on the request receiving side>
FIG. 5 is a flowchart showing a processing procedure when the own device becomes the mobile terminal device 10 on the request receiving side. After a waiting state (step S200) after activation of the route guidance application, a command requesting sharing of current position information is received from another device (step S210). That is, the sharing establishing means 100 receives a command (a command requesting sharing of current position information) transmitted from another device. The user decides whether or not to allow requests from other devices. That is, the response means 104 notifies the other device whether or not to permit the request from the other device. Then, it is determined whether or not the request from the other device is permitted (step S220). As a result, if the request is not granted, the process returns to step S200. On the other hand, if the request is permitted, the process proceeds to step S230.

When the user approves the request from the other device, the above-described approval time is set in the other device. Information on the approval time is transmitted from the other device to the server 20 . Then, in step S230, information on the approval time is acquired from the server 20 by the information sharing means 110. FIG. As a result, the following steps S240 to S260 are repeated until the approval time elapses.

In step S240, information on the current positions of the own device and other devices, which is position information required for route search, is acquired. These pieces of information are obtained in the same manner as in step S140. Then, based on the position information acquired in step S240, the route searching means 160 searches for a traveling route from the current position of the device itself to the current position of the other device (step S250).

After searching for the travel route, an augmented reality image showing the travel route is displayed on the display unit 17 of the device itself (step S260). The display of the augmented reality image in step S260 is performed in the same manner as in step S160.

After displaying the augmented reality image, it is determined whether or not the approval time has passed (step S270). As a result, if the approval time has not passed, the process returns to step S240, and if the approval time has passed, the process proceeds to step S280.

When the approval time has passed, the user of the other device determines whether or not to reset the approval time. In step S280, the own device determines whether or not the approval time has been reset by another device. As a result, if the approval time has been reset, the process returns to step S230, and if the approval time has not been reset, the process proceeds to step S290. When the approval time is reset, information on the approval time after the reset is obtained from the server 20 by the information sharing means 110 .

In step S290, the sharing of position information and the like between the own device and other devices ends. As a result, the route guidance from the current position of the own device to the current position of the other device ends.

The processing of steps S240 to S260 is repeated on the request receiving side as well as steps S140 to S160 on the request transmitting side. As a result, the captured video/augmented reality image in the display unit 17 changes at any time according to changes in the current position/orientation of the user and changes in the current position of the other party. The user should proceed according to the augmented reality image showing the travel route.

<4.3 Summary>
As described above, first, the user on the request transmitting side and the user on the request receiving side mutually authenticate each other. After that, information on the current position of both parties is shared, and the progress route from the current position of the user on the request sending side to the current position of the user on the request receiving side is displayed on the display unit 17 of the mobile terminal device 10 on the request sending side in augmented reality. The progress route from the current position of the user on the request receiving side to the current position of the user on the request sending side is displayed using an augmented reality image on the display unit 17 of the mobile terminal device 10 on the request receiving side. be done. Both proceed according to an augmented reality image that shows the travel route. At that time, the augmented reality images displayed on the display units 17 of the mobile terminal devices 10 of both are updated as at least one of the two moves. Also, if the two do not meet within the approval time, the approval time is reset as necessary. As described above, both of them finally meet by proceeding according to the augmented reality image showing the advancing route.

<5. Display screen>
In the above, as an example of the augmented reality image displayed superimposed on the captured image, the augmented reality image showing the traveling route (see FIGS. 7 and 8) was exemplified. However, an augmented reality image showing additional information may be displayed in addition to the augmented reality image showing the traveling route. This will be explained below.

When meeting using this route guidance application, information on the traveling direction of the other party is useful information. For example, if it is known that the other party is moving away from the user, it is possible to tell the other party to check the direction of travel by, for example, calling the other party. Therefore, an augmented reality image showing the traveling direction of the other party (the traveling direction of the other device) may be displayed at a predetermined position on the display unit 17, as indicated by the dotted line portion denoted by reference numeral 43 in FIG. In the example shown in FIG. 9, the user can grasp the traveling direction of the other party by an augmented reality image composed of human marks and arrows.

In addition, it is preferable to be able to grasp the orientation of the other party so that the other party can be easily identified even in a crowded place, especially when meeting with a person whom the person has met for the first time. Therefore, the portable terminal device 10 is provided with a function for specifying the direction of the user based on the direction information detected by the direction detection means 140 (the geomagnetic sensor 19), and as shown in the dotted line portion denoted by reference numeral 44 in FIG. , an augmented reality image showing the orientation of the other party (the owner of the other device) may be displayed at a predetermined position on the display unit 17 . Regarding this, for example, a plurality of images (augmented reality images) as shown in FIG. It may be displayed at a predetermined position of the section 17 .

Note that an augmented reality image composed of a face mark and an arrow indicating the direction may be displayed at a predetermined position on the display unit 17, as indicated by the dotted line portion denoted by reference numeral 45 in FIG. This enables the user to grasp the orientation of the other party and the traveling direction of the other party.

When meeting someone, there are many times when I want to know what the other person is looking at so that I can know their current location. Therefore, this route guidance application is provided with a function of displaying an image captured by another device on the display unit 17 of the own device. This function is realized by the video reference means 190 (see FIG. 3). In this embodiment, when the user selects a menu for executing this function, the photographed image requesting means 192 requests another device to transmit the photographed image. On the other hand, the photographed image transmitting means 194 of the other device transmits the photographed image (still image in this case) which is the result of photographing by the image photographing means 170 of the other device. It should be noted that the request for transmission of the captured image may be rejected. In the own device, when a captured image is transmitted from another device in response to a request from the captured image requesting means 192 , the captured image is displayed on the display section 17 by the other device image display means 196 . As a result, an image as shown in FIG. 13, for example, is displayed on the display unit 17 of the own device. In the example shown in FIG. 13, an image captured by the own device (including an augmented reality image) is displayed in the portion denoted by reference numeral 51, and an image captured by another device is displayed in the portion denoted by reference numeral 52. In this way, the scenery viewed by the other party can be confirmed, so that the current position of the other party can be grasped in detail.

When making an appointment using this route guidance application, it is preferable to notice the direction error as early as possible when traveling in the wrong direction. Therefore, in such a case, a screen (typically a pop-up screen) is displayed on the display unit 17 to warn that the opponent is moving away from the position of the other party, such as the screen denoted by reference numeral 46 in FIG. Also good.

When meeting someone, people often feel stress when they do not know how long it will take to meet with the other person. Therefore, as indicated by the dotted line portion denoted by reference numeral 47 in FIG. 15, the time required for the user and the other party to meet may be displayed at a predetermined position on the display unit 17. FIG. This allows the user to know how long it will take to meet with the other party, thereby reducing the user's stress at the time of meeting. In order to visually grasp the degree of approach between the self and the other party, the route image 41 is displayed with a wider width as the distance between the self and the other party becomes shorter, as shown in FIG. 16, for example. You can do it.

17 to 19 show examples in which an augmented reality image is superimposed on an actual captured image. In the examples shown in FIGS. 17 and 18, in addition to the augmented reality image showing the traveling route, an augmented reality image showing the direction and an augmented reality image showing the current position of the other party (image displayed as "G") are displayed. ing. In the example shown in FIG. 19, in addition to the augmented reality image showing the traveling route, the augmented reality image showing the direction and the distance from the current position to the next turning position are displayed. As described above, in addition to the augmented reality image showing the traveling route, various augmented reality images showing additional information may be displayed.

<6. Effect>
According to this embodiment, the current position information is shared between the own device and the other device. Then, the route search means 160 searches for a travel route from the current position of the device itself to the current position of the other device, and the display unit 17 displays an augmented reality image showing the travel route superimposed on the captured image. Since the direction of the current position of the meeting partner is indicated using the augmented reality image, the meeting partner can be easily found regardless of the surrounding environment when the user and the meeting partner approach each other. As described above, according to the present embodiment, when meeting (meeting with two people), regardless of the environment (degree of congestion, size of space, complexity of structure such as building) where can easily meet.

In addition, since the traveling route to the current position of the other party is displayed on the display units 17 of the mobile terminal devices 10 of both parties, it is not necessary to determine a specific meeting place at the time of meeting.

Furthermore, since the route is shown using an augmented reality image, the user can intuitively grasp the direction in which he or she should proceed, and the user can meet the other party simply by following the augmented reality image showing the route. can be done. In this way, meeting up becomes significantly easier compared to the prior art using a 2D map.

Furthermore, mutual current location information is shared only when the other device accepts a request from one device (request for sharing current location information). That is, information on the current location is shared between mutually authenticated persons. This prevents invasion of privacy due to unauthorized sharing of current location information. In addition, since the length of time for which the current location information is shared (approval time) can be set, it is possible to prevent the infringement of privacy due to unintentional continuation of the current location information sharing state for a long time. . In addition, since the approval time can be reset, route guidance for meeting can be continued without redoing the process of mutually authenticating each other when the approval time has passed.

<7. Others>
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 10... Portable terminal device 20... Server 100... Share establishment means 102... Share request means 104... Response means 110... Information sharing means 112... Location information sharing means 120... Approval time setting means 130... Approval time reset means 140... Orientation detection Means 150 Current Position Detecting Means 160 Route Searching Means 170 Image Shooting Means 180 AR Route Displaying Means 190 Image Referencing Means 192 Photographed Image Requesting Means 194 Photographed Image Transmitting Means 196 Other Apparatus Image Displaying Means

Claims (9)

a computer included in a mobile terminal device having a function of displaying captured images on a display unit;
location information acquisition means for acquiring information on the current location of the device at any time;
Location information sharing means for sharing current location information between the device itself and another device;
Based on the current position information of the own device acquired by the position information acquisition means and the current position information of the other device shared by the position information sharing means, from the current position of the own device to the other device a route search means for searching for a travel route to the current position; and
Functioning as a route display means for displaying an augmented reality image indicating the traveling route obtained by the route search means, superimposed on the captured image ,
If the straight line distance between the current position of the own device and the current position of the other device is equal to or greater than a predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the distance of the other device. A route along the road to the current position is obtained,
If the straight line distance between the current position of the own device and the current position of the other device is less than the predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the other device. A route guidance program characterized in that a straight route to the current position of is obtained . said computer,
sharing request means for requesting the other device to share current location information; and
Further functioning as response means for notifying the other device whether or not to accept the request when the other device requests sharing of current location information,
Current position by the location information sharing means when there is a notification to the effect that the request by the sharing request means is accepted from the other device and when the response means accepts the request from the other device 2. The route guidance program according to claim 1, wherein the information of is shared. said computer,
Further functioning as an approval time setting means for setting an approval time for allowing sharing of current location information when the other device notifies that the request by the sharing request means is accepted,
3. The route according to claim 2, wherein information on the current location is shared by the location information sharing means for a period from a predetermined timing until the approval time set by the approval time setting means has passed. guidance program. said computer,
4. The path according to claim 3, further functioning as approval time resetting means for resetting the approval time when the approval time set by said approval time setting means has passed from said predetermined timing. guidance program. 5. The method according to any one of claims 1 to 4, wherein said route display means displays an augmented reality image showing a traveling direction of said other device in addition to said augmented reality image showing said traveling route. Directions program as described. 6. Any one of claims 1 to 5, wherein said route display means displays an augmented reality image showing the direction of the owner of said other device in addition to said augmented reality image showing said traveling route. Route guidance program as described in section. said computer,
Captured video request means for requesting transmission of captured video from the other device;
a photographed image transmission means for transmitting a photographed image of the own device to the other device when a request for transmission of the photographed image is received from the other device;
The apparatus further functions as other device image display means for displaying the captured image on the display unit when the other device transmits the captured image in response to the request from the captured image requesting means. 7. The route guidance program according to any one of 1 to 6. A route guidance device having a display unit,
a captured image display means for displaying a captured image on the display unit;
a location information acquisition means for acquiring information on the current location of the device itself at any time;
location information sharing means for sharing current location information between the device itself and another device;
Based on the current position information of the own device acquired by the position information acquisition means and the current position information of the other device shared by the position information sharing means, from the current position of the own device to the other device a route search means for searching for a travel route to the current position;
route display means for superimposing and displaying an augmented reality image showing the travel route obtained by the route search means on the captured image ;
If the straight line distance between the current position of the own device and the current position of the other device is equal to or greater than a predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the distance of the other device. A route along the road to the current position is obtained,
If the straight line distance between the current position of the own device and the current position of the other device is less than the predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the other device. A route guidance device characterized in that a straight route to the current position of is obtained . A route guidance system implemented by a first mobile terminal device and a second mobile terminal device, which are two mobile terminal devices each having a display unit,
Each mobile terminal device
a captured image display means for displaying a captured image on the display unit;
a location information acquisition means for acquiring information on the current location of the device itself at any time;
location information sharing means for sharing current location information between the device itself and another device;
Based on the current position information of the own device acquired by the position information acquisition means and the current position information of the other device shared by the position information sharing means, from the current position of the own device to the other device a route search means for searching for a travel route to the current position;
route display means for superimposing and displaying an augmented reality image showing the travel route obtained by the route search means on the captured image ;
If the straight line distance between the current position of the own device and the current position of the other device is equal to or greater than a predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the distance of the other device. A route along the road to the current position is obtained,
If the straight line distance between the current position of the own device and the current position of the other device is less than the predetermined threshold value, the route searching means selects the traveling route from the current position of the own device to the other device. A route guidance system characterized in that a straight route to the current position of is obtained .

Images