JP5346115B6 - Portable movement support device - Google Patents

Abstract

Provided is a portable movement support apparatus that functions as a pacemaker and portable navigation system in walking and the like using augmented reality technology, and functions as a portable navigation system for evacuation guidance when a disaster occurs.
A mobile terminal or a terminal composed of an optical transmission type HMD or the like, which is superposed on a real space and guides a user such as an electronic pet for leading a user, or in addition, the user and the guide. Display means for displaying leads to be connected, user movement speed detection means, guide movement speed determination means for determining an assumed movement speed in the real space of the guide according to the user's heart rate, etc., and a size for displaying the guide A change means for changing the sheath position, or a lead slack amount determination means in addition thereto, so that the user can intuitively recognize and adjust his / her movement speed by moving so as to follow the guide. Portable movement support device.
[Selection] Figure 4

Description

The present invention is an augmented reality (AR) that can be used as a pacemaker or a portable navigation system in a walking exercise.
The present invention relates to a portable mobility support device using Augmented Reality) technology.
In addition, the movement in this invention shall include the movement by use of a bicycle etc. besides the movement by walking (walking, jogging, etc.).

In recent years, fat burning effect by aerobic exercise is considered to be very effective not only for diet but also for health promotion. This is because oxygen taken into the blood by aerobic exercise promotes fat burning. Moreover, it is because a virtuous cycle of further burning fat is generated by increasing muscles by continuous exercise and increasing basal metabolism. There are various methods for aerobic exercise, such as jogging and swimming, but walking is particularly popular because it can be easily tackled. However, in walking, there is a risk of traffic accidents due to oversight of signals when crossing a pedestrian crossing. In particular, elderly people suffer from a decline in their ability to detect danger, such as a reduction in their visual acuity, so the need for a technology that can support their movement is increasing as the population ages rapidly.

From the viewpoint of efficiently burning fat by aerobic exercise, it is most important to take oxygen into the blood, and care must be taken not to become anaerobic exercise due to over pace. Therefore, it is first required to know the load due to exercise. Since the load due to exercise has individual differences due to physical strength, etc., it is not an objective exercise amount, but a heart rate (pulse rate) that is personal biological information is a guide Is effective. This is why many athletes train while watching their heart rate. The heart rate suitable for fat burning is generally about 70% of the maximum heart rate, around 120, although it depends on the age and the like.

For this reason, various portable terminals capable of measuring biological information such as heart rate have been proposed. For example, Patent Document 1 describes a technology related to an information communication terminal such as a mobile phone and a biological information system that can measure biological information such as a heart rate.
In this technique, biological information that cannot be measured by the information communication terminal can be stored and managed in the information communication terminal.

Patent Document 2 relates to a portable biological information monitor, and in particular, a portable biological information monitor using a blood flow meter that measures information related to blood flow in a target biological tissue using scattered light from the biological tissue. The technology about is described.

A wristwatch having a heart rate function has also been proposed. For example, Patent Document 3 describes a technique related to a motion measurement device that is of a wristwatch type and can measure exercise intensity and heart rate while worn on an arm.
This technique measures a heart rate based on an electrocardiogram by a pair of electrodes, and the same principle can be applied to a portable terminal.

A movement support apparatus equipped with a heart rate monitor has also been proposed. For example, Patent Document 4 describes a technology related to a portable navigation system that can be carried, and in particular, a technology that provides information that can contribute to health promotion.
According to this technology, calorie consumption calculated from the heart rate can be associated with map data and displayed for each section of the road, and as a result, outdoor fitness activities can be supported more enjoyably and effectively. , "Which area on the map contributed to calorie consumption, how hard it was" was displayed, and was used to evaluate calorie consumption due to exercise afterwards. Moreover, it displays a map and does not use augmented reality technology as in the present invention.

On the other hand, a portable terminal device using augmented reality technology has also been proposed. For example, Patent Document 5 describes a technique related to an annotation display system and method that can change the amount of annotation information displayed on a mobile terminal in accordance with the moving speed of the mobile terminal.
The amount of annotation information is changed according to the moving speed of the mobile terminal, but guides such as electronic pets and virtual persons are not displayed. Further, the present invention is a technique for changing the amount of annotation information, and the present invention of following or guiding a user by moving a guide is characterized by the technical idea of the invention (the problem to be solved, (Solution, effect).

The portable movement support apparatus according to the present invention can be used as a portable navigation system. In this regard, although the augmented reality technology is not used, as a portable navigation system, for example, Patent Document 6 includes a mobile terminal device such as a mobile phone and a route search server, and is started by the route search server. The present invention relates to a navigation system that searches for an optimum guide route from a place to a destination and displays a current position and a guide route on a map on a display unit of the mobile terminal device, and provides a route guide, and particularly, a mobile terminal device. And a route search server for a pedestrian navigation system, a technique related to a navigation system is described in which an average speed is obtained from a pedestrian walking history and automatically set as a pedestrian walking speed. .
This technology is designed to automatically set the “walking speed” as the route search condition setting. As in the present invention, the user can adjust his / her movement speed while intuitively recognizing his / her movement speed. It wasn't a technique to make it. Also, the average speed is obtained based on the user's past walking history (log) stored in the server, and the user's walking speed is automatically set. There wasn't. Furthermore, the guide route is displayed on the mobile terminal, and guides such as an electronic pet and a virtual person are not displayed on the mobile terminal using augmented reality technology.

On the other hand, as a technique that can be used in the event of a disaster as in the present invention, Patent Document 7 describes a technique relating to a disaster prevention information providing system and a disaster prevention information distribution server that use augmented reality technology.
This technology displays disaster information and AR tags that indicate evacuation routes, and was able to provide appropriate disaster prevention information in real time to users in a small area. It is displayed with an AR tag with an arrow, and displays a guide such as an electronic pet or a virtual person. Furthermore, this guide is displayed away or close according to the user's heart rate or moving speed, The present invention, which allows the user to intuitively recognize the evacuation route and the optimum moving speed, is different in the characteristics of the technical idea of the invention (problems to be solved, solution means, and effects).

Japanese Patent No. 4119862 Japanese Patent No. 4658705 Japanese Patent No. 4822208 JP 2007-292505 A JP 2011-123807 A Japanese Patent No. 4776716 JP 2011-186681 A

However, the prior art has the following problems.

First, the problems when using as a pacemaker will be described.
Like the techniques described in Patent Document 1 to Patent Document 3, the method of displaying biological information such as heart rate numerically is not suitable for the user to intuitively recognize his / her own pace, In order to control the exercise load by feeding back the measured biological information to the user, the measurement result must be continuously confirmed. For this reason, not only does it take a long time for the user to visually check the display unit, but unlike the case of using augmented reality technology as in the present invention, the field of view is blocked by the mobile terminal, so It may lead to an unexpected accident due to carelessness.

Further, even if the biological information such as heart rate can be confirmed, if the user tries to speed up or slow down his / her pace based on the result, it is described in Patent Documents 4 to 7. In this method, it is difficult to intuitively recognize in real time whether the moving speed of the self is fast or slow, or how fast or slow it is.

Also, moving while looking at a mobile terminal is not pleasant and lacks fun. On the other hand, walking with pets and other people is more enjoyable and easier than walking alone, but raising real pets is difficult, and when walking with other people, It is difficult to maintain a self-pace suitable for aerobic exercise when there is a difference in physical fitness.

Next, problems when used as a portable navigation system will be described.
The method of displaying a map as in the technique described in Patent Document 4 or Patent Document 6, or the method of displaying an arrow or the like as in the technique described in Patent Document 5 or Patent Document 7, When it is lost, an operation for returning to the guidance route is required.
Also, in the case of the method of displaying the self position on the map, it is difficult for a child who has insufficient map interpretation ability or a visually impaired person to intuitively recognize the traveling direction.
Furthermore, when moving a long distance, pace distribution or a marching plan, such as moving in which section and in what time, is indispensable for reaching the target point without frustration on the way. However, in the conventional navigation system, the guide does not lead or guide the user as in the present invention, and the optimum moving pace or the planned moving speed cannot be presented in real time.

Therefore, the present invention solves the problems of the prior art, and guides the user such as an electronic pet or a virtual person using augmented reality technology, so that aerobic exercise can be enjoyed during normal times. It is an object of the present invention to provide a portable movement support apparatus that functions as a pacemaker for continuing the movement and functions as a portable navigation system for evacuation guidance when a disaster occurs.

In order to solve the above-mentioned problem, a portable movement support apparatus according to claim 1 is a portable terminal including a camera (smart phone (multifunction mobile phone) / mobile phone / PHS, tablet terminal, video transmission (Video
See-Through) head-mounted display devices, portable game consoles, etc.), augmented reality (AR:
Using Augmented Reality) technology, a guide (electronic pet, virtual person, etc.) for leading the user is displayed on the display of the mobile device superimposed on the image (still image or moving image) captured by the camera Display means for changing and a changing means for changing the size and position for displaying the guide, and the user can intuitively recognize his / her moving speed by moving so as to follow the guide. The feature is that it can be adjusted.

In the present invention, by displaying a virtual guide using augmented reality technology, the user can perceive as if the guide exists in the real space. In the present invention, the position in the real space where the guide looks as if it exists is defined as “assumed position”, and the moving speed in the real space where it appears as if the guide is moving is defined as “assumed moving speed”. To do.

The portable terminal used in the present invention includes a display capable of displaying an image (still image or moving image) taken by a camera. This display is not limited to displays such as smartphones and tablet devices, but video transmission (Video
This includes displays that are worn on the head or glasses, such as See-Through) head-mounted displays. In other words, any display can be used as long as it can use augmented reality technology.

In the case of using the head mounted display, since both hands become hands-free, the user can move using a bicycle or the like, and can use it not only for walking but also for cycling and the like. At this time, if the guide is a virtual person, it is desirable to move it using a bicycle or the like as in the case of the user. In the case of an electronic pet, it is desirable to model an animal having the necessary exercise ability. The guide may be a virtual animal, a real animal such as a dog, or an imaginary character.

First, the display means displays a guide such as an electronic pet or a virtual person for leading the user on the display of the portable terminal. Guides can be generated in a variety of ways, from simple icons that imitate characters such as electronic pets and virtual characters, to three-dimensional computer graphics animations that enhance reality.

Note that the augmented reality technique in the present invention is not a technique using markers or a technique using feature points extracted by image processing.

Next, the changing means changes the size and position of the guide displayed according to biological information such as the user's heart rate or the moving speed or moving pace of the user.

When changing according to biometric information such as the user's heart rate, for example, when the heart rate is high and the exercise load is large, the guide should be enlarged so that the position close to the user becomes the assumed position of the guide. When displayed at the bottom, when the heart rate is low and the exercise load is small, the guide is displayed small at the top of the display so that the position far from the user is the assumed position of the guide.
In addition to the heart rate, biological information such as blood pressure and body temperature may be used.

A heart rate monitor or the like for measuring the heart rate may be integrated with the portable terminal, or may be externally attached to the portable terminal, built in a wristwatch, or worn on the chest with a band, and the portable terminal. May be connected using a cable, a power-saving short-range wireless technology, or the like. Moreover, the pulse of a head can also be measured by integrating with a head-mounted display or a head-up display.

When changing the size or position of the guide according to the heart rate, for example, the user's heart rate suitable for aerobic exercise is set as the target heart rate. When the heart rate of the user who is walking is smaller than the target heart rate, the guide is displayed small on the top so that the position far from the user becomes the assumed position of the guide. On the other hand, when it is larger than the target heart rate, the guide is displayed largely at the center or at the bottom so that the position close to the user is the assumed position of the guide. This allows the user's heart rate to follow the guide to be maintained in the vicinity of the target heart rate. The movement speed can be adjusted more intuitively than when walking while watching the heart rate displayed on a wristwatch type heart rate monitor or the like. In addition, it is possible to adjust the moving speed while confirming that the distance between the user and the guide is getting closer or closer, so that the user can approach the target heart rate slowly, not suddenly while looking at the numbers. Can be adjusted.

In addition, the present invention visualizes the invisible internal state such as heart rate as a number, and differs from general biofeedback that uses it for training, as a concrete presence that becomes the target of the user The user's guide is presented in front of the user, the assumed position of the guide is changed according to the user's heart rate, and the user can follow the guide so that the user's heart rate can be controlled. It also has the technical significance of providing a new method.

When the size and position of the guide display are changed according to the movement speed and movement pace of the user, for example, when the movement speed of the user is high and the exercise load is large, the position close to the user becomes the assumed position of the guide. In this way, the guide is displayed at the center or at the bottom of the display. On the contrary, when the movement speed of the user is slow and the exercise load is small, the guide is displayed small on the upper part of the display so that the position far from the user becomes the assumed position of the guide. The movement speed and pace of the user is determined by GPS (Global Positioning
System) or pedometer. When traveling by bicycle, this can also be detected from the number of rotations of the wheel.

As a result, the user can intuitively recognize whether the guide is faster or slower than the guide based on the size and position of the guide, and the user can follow his / her movement to move his / her own movement. The load can be adjusted.
In addition to changing the display size and position of the guide, various expression methods can be added, such as slowing or moving the guide foot.

On the other hand, as a wearable computer configuration technology, optical transmission (Optical
See-Trough head mounted display (HMD)
Head-Mounted Display) or head-up display (HUD)
Head-Up Display) is becoming popular. Since these displays are hands-free unlike mobile terminals, they are not only suitable for exercise, but also have high safety because the real space can be seen with the naked eye. Video transparency
Unlike the head-mounted display of See-Through), there is no worry that the field of view will be blocked by system trouble.

Therefore, the portable movement support apparatus according to claim 2 is an optical see-through head mount display (HMD:
Head-Mounted Display (HUD)
Head-Up Display), augmented reality (AR:
Augmented Reality) technology, a display means for displaying a guide such as an electronic pet or a virtual person for guiding the user in superposition on the real space, and a change means for changing the size and position for displaying the guide The user can intuitively recognize and adjust his / her moving speed by moving so as to follow the guide.

In addition, if the user can see the real space in front of the eyes and the guide at the same time, the optical transmission type head-mounted display and head-up display can be configured in the form of glasses or goggles. It doesn't matter.

First, the display means is an electronic pet or virtual person for leading a user similar to the portable movement support apparatus according to claim 1 by superimposing the display means on a real space visible through an optically transmissive head-mounted display or head-up display. Etc. Guide is displayed.

Next, as the changing means for changing the size and position for displaying the guide, the same means as the portable movement support apparatus according to claim 1 is employed.

As a result, the user can see the real space with the naked eye and can move more safely because there is no fear that the field of view is blocked by the system trouble. Moreover, since both hands become hands-free, the user can move using a bicycle or the like, and can also be used in cycling or the like.

In the portable movement support apparatus according to claim 1 and 2, when the guide is displayed on the display regardless of the position / posture / orientation of the camera, it is displayed at an assumed position that cannot originally exist. May end up. For example, when the camera is pointed at the sky, the guide may be displayed as if it is floating in the air, or when the camera is pointed at the river, the guide may be displayed as if it is present on the water.
Therefore, in order to solve this problem and display a more realistic feeling as if the guide exists in the real space, first, a reasonable position where the assumed position of the guide in the real space can exist. After determining the position, it is necessary to determine the size and position that would be seen if the guide was taken in the position, posture, and orientation of the camera.

For this reason, the portable movement support apparatus according to claim 3 is a portable terminal such as a smartphone provided with a camera, and is augmented reality (AR:
Augmented Reality) display means for displaying a guide such as an electronic pet or a virtual person for leading the user on the display of the mobile terminal superimposed on the image taken by the camera; A positional relationship between the user's position and the assumed position of the guide in the real space; a user movement speed detecting means for detecting a moving speed; a guide moving speed determining means for determining an assumed movement speed of the guide in the real space; And changing means for changing the size and position for displaying the guide, so that the user can intuitively recognize and adjust his / her moving speed by moving so as to follow the guide. It is characterized by doing so.

The augmented reality technology used in the portable movement support apparatus according to claim 3 is not a technology using markers or a technology using feature points extracted by image processing.
This technology uses position information obtained by sensors such as Positioning System.

First, user movement speed detection means for detecting the user's movement speed includes a method of automatically detecting from the user's step count measured using a mobile terminal such as a smartphone with a pedometer function, an acceleration sensor, a geomagnetic sensor, and a gyroscope. A method using various sensors such as a sensor, a method using GPS positioning, and the like can be used.

Next, as the guide moving speed determining means for determining the assumed moving speed in the real space of the guide, the simplest method is a method in which the user can manually set in advance. For this purpose, the user may be able to select and set the speed by operating the touch panel or the like, or the movement speed for each section on the movement route may be set as a movement plan in advance.

It is also possible to automatically determine the assumed moving speed of the guide. For example, it can be automatically determined based on biological information such as the user's heart rate.

When deciding according to biological information such as the user's heart rate, for example, when the heart rate is high and the exercise load is large, the assumed exercise speed of the guide is reduced to reduce the exercise load of the user following the guide It can be so. On the other hand, when the heart rate is low and the exercise load is small, the assumed movement speed of the guide is increased so that the exercise load of the user following the guide can be weighted.
In addition to the heart rate, it is also possible to use biological information such as blood pressure, body temperature, and respiratory rate.

When determining the guide movement speed according to the movement speed or movement pace of the user, for example, when the user movement speed is high and the exercise load is large, the guide movement speed is slowed down to follow the guide. The user's exercise load can be reduced. On the contrary, when the movement speed of the user is slow and the exercise load is small, the assumed movement speed of the guide is increased so that the exercise load of the user following the guide can be weighted.

Furthermore, the assumed moving speed of the guide can be increased or decreased based on terrain data such as a slope. For example, since the exercise load is large on an uphill, the exercise load on the user following the guide can be reduced by reducing the assumed movement speed of the guide. On the contrary, since the exercise load is small on the downhill, the exercise load of the user following the guide can be weighted by increasing the assumed moving speed of the guide.

By using at least one of the above, it is possible to automatically determine the assumed moving speed of the guide.

The heart rate monitor may be integrated with the mobile terminal, or it may be externally attached to the mobile terminal, built into the wristwatch, or worn on the chest with a band, and connected to the mobile terminal with a cable or power-saving short distance. You may connect using radio | wireless technology etc. Moreover, the pulse of a head can also be measured by integrating with a head-mounted display or a head-up display.

When determining the assumed moving speed of the guide according to the user's heart rate, for example, the user's heart rate suitable for aerobic exercise is set as the target heart rate. When the heart rate of the user who is walking is smaller than the target heart rate, the assumed moving speed of the guide is increased, and when it is larger, the assumed moving speed of the guide is decreased. This allows the user's heart rate to follow the guide to be maintained in the vicinity of the target heart rate. The movement speed can be adjusted more intuitively than when walking while looking at a wristwatch type heart rate monitor. In addition, by changing the assumed moving speed of the guide, the moving speed can be adjusted while confirming that the distance between the user and the guide is getting closer or closer. Instead, it can be adjusted to approach the target heart rate gently.

In addition, the present invention visualizes the invisible internal state such as heart rate as a number, and differs from general biofeedback that uses it for training, as a concrete presence that becomes the target of the user The guide of the user is presented in front of the user, the assumed moving speed of the guide is changed according to the user's heart rate, and the user can follow the guide so that the user's heart rate can be controlled. It also has the technical significance of providing a new method of feedback.

In addition, the present invention, when a disaster occurs, for example, by maximizing the assumed movement speed of the guide with respect to the approach of the tsunami, or by increasing or decreasing the assumed movement speed of the guide according to the physical condition of the user, Since it can support the user to move quickly and safely, it can also help to ensure safety of life and body.

Next, the changing unit changes the size and position of the guide displayed based on the position / posture / azimuth of the mobile terminal and the assumed position of the guide in the real space.
The position of the mobile terminal is detected by using GPS.

The attitude of the mobile terminal is detected by using an acceleration sensor or a gyro sensor.

The orientation of the portable terminal is detected by using a geomagnetic sensor. Further, the accuracy can be improved by combining with an acceleration sensor. Furthermore, since the geomagnetic sensor may be affected by the surrounding magnetic field, it is conceivable to use a gyro sensor in combination.

The assumed position of the guide can also be determined by adding a distance obtained by integrating the assumed moving speed with time, using the initial position of the guide as a base point. The initial position of the guide is set at or near the user's start position. The traveling direction of the guide is made to coincide with the traveling direction of the user. When used as a portable navigation system to be described later, the traveling direction of the guide is set to a direction along the searched movement route.

Further, based on the position of the user, that is, the position of the mobile terminal, the assumed position of the guide may be determined based on the difference between the movement speed of the user and the assumed movement speed of the guide. In addition to this, various methods are conceivable as long as the method can intuitively recognize whether the own moving speed is faster or slower than the assumed moving speed of the guide.

In particular, in the method of determining the assumed position using a value obtained by integrating the assumed moving speed of the guide with time, a case where the guide is too far from the user is assumed. Therefore, in such a case, an upper limit is set for the separation distance between the two, and when determining the assumed movement speed of the guide, an attenuation coefficient is multiplied according to the separation distance between the user and the guide, etc. Can be considered. Another possible method is to redisplay the guide that is too far away as if it is recalled.

When the guide functions as a pacemaker, the means for displaying the guide in the direction in which the user proceeds is as described above. On the other hand, when the guide functions as a navigator, the guide is displayed in the direction in which the user should travel along the route searched by the navigation system.

Based on the position / posture / azimuth of the portable terminal and the assumed position of the guide obtained by the above method, the size and position of the guide are changed. As the display means, the same means as the portable movement support apparatus according to claim 1 is employed.

This makes it possible for the user to intuitively recognize whether the guide is faster or slower than the guide from the size and position of the guide that appears to exist in real space. By trying to follow the guide, his / her exercise load can be adjusted.
In addition to changing the display size and position of the guide, various expression methods can be added, such as slowing or moving the guide foot.

On the other hand, as a wearable computer configuration technology, optical transmission (Optical
See-Trough head mounted display (HMD)
Head-Mounted Display) and HUD: Head-Up
Display) is spreading. Since these displays are hands-free unlike mobile terminals, they are not only suitable for exercise but also allow the real space to be seen with the naked eye. Video transparency
Unlike the head-mounted display of See-Through), there is no worry that the field of view will be blocked by system trouble.

Therefore, the portable movement support device according to claim 4 is an optical see-through head mount display (HMD:
Head-Mounted Display (HUD)
Head-Up Display), augmented reality (AR:
Augmented Reality) technology, a display means for displaying a guide such as an electronic pet or a virtual person for guiding the user in superposition on the real space, and a user movement speed detection means for detecting the movement speed of the user The guide moving speed determining means for determining the assumed moving speed of the guide in the real space, and the size and position of the guide based on the positional relationship between the position of the user and the assumed position of the guide in the real space. And a changing means for changing the speed of the user so that the user can intuitively recognize and adjust his / her moving speed by moving so as to follow the guide.

The augmented reality technology in the present invention is not a technology using markers or a technology using feature points extracted by image processing.
This technology uses position information obtained by sensors such as Positioning System.

Optical See-Trough head-mounted display (HMD:
Head-Mounted Display (HUD)
For the head-up display, the same device as the portable movement support apparatus according to claim 2 is adopted. As the display means, the same means as the portable movement support apparatus according to claim 2 is adopted. As the user movement speed detection means, the guide movement speed determination means, and the change means, the same means as the portable movement support device according to claim 3 is adopted. However, it is assumed that the various sensors are integrated with the head-mounted display or the head-up display, or are separately carried and connected to the system by a cable or a power-saving short-range wireless technology.

This not only improves the sense of reality for the display of the guide, but also allows the user to see the real space with the naked eye and to move more safely because there is no fear that the field of view will be blocked by system trouble. It becomes like this. In addition, since both hands become hands-free, the user can move using a bicycle or the like, and can use it for cycling or the like in addition to walking.

The distance between the guide and the user can be intuitively recognized from the size and position of the guide, but for example, if the user's moving speed is slow, the guide will be far away from the shooting range of the camera. It is assumed that it will end up. In addition, when the traveling direction is changed along the guidance route, it is assumed that the camera is out of the shooting range. Furthermore, it is desirable to be able to recognize the own moving speed more intuitively for the purpose of more efficient aerobic training.

Accordingly, the portable movement support apparatus according to claim 5 is the portable movement support apparatus according to any one of claims 1 to 4, wherein the display means displays a guide, and the user and the guide are displayed. A lead slack amount determining means for displaying a lead to be connected and determining a slack amount of the lead based on a positional relationship between the position of the user and an assumed position in the real space of the guide; By moving while following the guide while observing the amount of looseness of the lead, the movement speed of the self can be intuitively recognized and adjusted.

The display means for displaying the lead connecting the user and the guide on the display of the portable terminal deforms the three-dimensional computer graphics model of the lead in real time from a method of preparing several shape models according to the slackness of the lead. Various methods can be considered.

The lead slack amount determining means for determining the lead slack amount can be determined by calculating the positional relationship between the user and the guide. In addition, it can also be determined by taking into consideration the lead material and stretchability. Furthermore, if you give a sense of being pulled by the guide using the vibration function by the vibration motor described later, or if you visually express the elasticity by changing the thickness of the lead etc., it will be more realistic. Can be increased.

As a result, the user can move while visually recognizing the moving speed of the user by looking at the amount of looseness of the lead.

Even in the case where the guide is out of the shooting range of the camera, the direction in which the guide exists can be roughly estimated using the lead. The guide can be displayed again on the display of the mobile terminal by redirecting the camera of the mobile terminal in the estimated direction.

Even if the lead connecting the user and a guide such as an electronic pet is displayed, and there is no guide in the shooting range of the camera and the guide is not displayed on the display, one end of the lead is always visible from the user side. By displaying it as extended, it is possible to immediately know the position where the guide is supposed to exist by relying on the lead. In addition, the distance between the user and the guide can be roughly estimated based on vibrations caused by the vibration function, lead thickness, and the like.

One end of the user side of the lead can be displayed as extending from the hand when a portable terminal such as a smartphone is held in the hand. In addition, when wearing a head-mounted display, etc., by connecting a wristwatch, a mobile terminal such as a smartphone, a controller, etc., which is located at the user's hand, to the system using cables, power-saving short-range wireless technology, etc. The lead can be displayed as extending from the user's hand, or can be displayed as extending from a fixed point in the air immediately before the user. Alternatively, a marker-type augmented reality technique can be used to recognize the position of the user with the marker and to display a lead connecting the guide and the guide on the head-mounted display.
Then, by directing the mobile terminal in the direction in which the lead extends, the guide easily enters the shooting range of the camera and is displayed on the display.

Thus, even when the guide leading the user is out of the display range of the display, it can be found immediately.

In order to obtain a fat burning effect by aerobic exercise including walking, it is necessary to continue exercise. By displaying a guide such as an electronic pet or a virtual person in front of the user using augmented reality technology, the enjoyment is increased and the exercise can be continued without getting tired.
In addition, as a measure for continuing to enjoy walking, it is recommended to walk while enjoying the surrounding scenery and seasonal changes. In the case of taking a real pet for a walk, the pet often stops showing interest in the flowers on the roadside.

Therefore, the portable movement support apparatus according to claim 6 is the portable movement support apparatus according to any one of claims 1 to 4, wherein the object in the image photographed by the camera is recognized and recognized. The guide is responsive to an object so that the voice and movement of the guide change. It is assumed that the portable movement support apparatus according to claim 6 is provided with a camera.

Image recognition is performed by first correcting noise removal, edge enhancement, enlargement / reduction, rotation, etc. on the captured digital image, extracting the features, and then performing pattern matching with the standard pattern on the extracted feature patterns. Do. In order to make the guide react to a moving object, it is effective to perform feature extraction using a difference image between frames.

As a result, guides such as electronic pets and virtual people can display the situation where they are interested in the flowers on the roadside and stop, and users can also walk while enjoying the surrounding scenery and seasonal changes. It becomes like this.

In order to increase the fat burning effect, aerobic exercise is required while taking oxygen into the blood, and there is a heart rate suitable for it. Although there are individual differences depending on age, physical strength, etc., it is desirable to maintain the heart rate at about 120. In this respect, a heart rate monitor of a wristwatch type or a type attached to the chest with a band is commercially available so that the user can exercise while constantly checking the heart rate. However, it is difficult to intuitively understand the measurement result with the method of displaying the heart rate numerically. Also, while checking the heart rate, it is dangerous because the field of view is blocked. Furthermore, it is difficult to control the moving speed by checking the increase or decrease of the heart rate.

Therefore, a portable movement support apparatus according to claim 7 is the portable movement support apparatus according to any one of claims 1 to 4, further comprising a heart rate monitor, wherein the change means measures the user with the heart rate monitor. The size and position of the guide display can be changed according to the heart rate.

As the heart rate monitor, not only one integrated with a mobile terminal such as a smartphone but also one built in a wristband or a band attached to the chest can be used. In this case, it is assumed that the system is connected to the system by a cable or power-saving short-range wireless technology. Also, the heart rate monitor can be held separately, and any form can be used as long as it can measure the heart rate and transmit the measurement result to the system. In the case of a head-mounted display or a head-up display, the pulse of the head can be measured by a heart rate meter (pulse meter) configured integrally therewith.

At this time, it is possible to change the assumed moving speed of the guide and to make the guide speak. For example, if it is an electronic pet, it will make a cry, or if it is a virtual person, it will say “faster”, “more slowly”, “let's increase the pace”, “let's reduce the pace”, etc. Or make it emit.

This makes it possible to intuitively recognize the moving speed for maintaining the optimum heart rate, unlike when the movement speed is controlled by confirming the increase / decrease of the heart rate with numbers.

In the case of an optically transmissive head-mounted display or a head-up display, the user can move while looking forward, so that he or she can notice a person or a car located in front. In addition, even when moving while looking at a mobile terminal such as a smartphone, there is no danger of obstructing the field of view because an actual image is displayed on the display by augmented reality technology. However, depending on the orientation of the mobile terminal, there may be a case in which a person or a car located in front cannot be noticed, resulting in danger.

Accordingly, a portable movement support apparatus according to claim 8 is the portable movement support apparatus according to any one of claims 1 to 4, wherein an object detection sensor such as an infrared sensor, an ultrasonic sensor, or a millimeter wave radar sensor is used. The object detection sensor detects a person or a car positioned in front of the user, and a guide reacts to the detected person or the car so that the voice or movement of the guide changes. And

For example, when the guide reacts to the detected person, an operation of the guide moving forward so as to avoid the person is displayed in front of the user. You can also say "There are people in the front. Please be careful."

As a result, safety such as walking exercise performed outdoors can be increased.

The user can see a real image projected by augmented reality technology, so that the traffic signal at the pedestrian crossing can be identified. However, when the user is an elderly person or a blind person, it may be difficult to identify traffic signals.

Therefore, the portable movement support apparatus according to claim 9 is the portable movement support apparatus according to any one of claims 1 to 4, wherein the estimated position of the guide is matched with the map information, or a traffic signal or the like. The guide is temporarily stopped in front of a pedestrian crossing by image recognition.

The necessity of the temporary stop is determined by comparing the assumed position in the real space of the guide with the map information. At this time, the guide stops before the pedestrian crossing with sufficient time to alert the user. When a virtual person is displayed as a guide, the user can move his hand sideways as if to stop the user. Furthermore, if the vibration function of the portable terminal is used in combination, it is possible to call attention more effectively.

Image recognition recognizes that the traffic signal for pedestrians has stopped, recognizes the traffic of cars and the presence of roads. Moreover, the collation accuracy with map information can also be improved. Furthermore, it is possible to identify the fact that the traffic signal for pedestrians is in the advanced state by image recognition, and to prompt the start after the temporary stop. However, there is a risk of misrecognition, so it is desirable to always set a temporary stop before crossing the road.

In addition to being able to move forward while looking forward with augmented reality technology without being blocked by the mobile device, the guide pauses in front of the pedestrian crossing to alert and stop the user. So it will help prevent accidents when crossing the road.

In recent years, with the advancement of social networking services, communication utilizing data communication has become possible. However, there are concerns about various harmful effects caused by the bias toward communication in cyberspace.

Accordingly, a portable movement support apparatus according to claim 10 is the portable movement support apparatus according to any one of claims 1 to 4, wherein other data communication such as infrared communication or short-range wireless communication is used. Information on the assumed position information of the guide leading the user in the real space, information on the form of the guide, and the like are acquired, and the guide leading the other user can be displayed.

Information about the guide leading the other user is acquired and displayed on the display of the user's portable movement support apparatus as if it exists at the assumed position in the real space. Information regarding the assumed position of the guide in the real space and the type / form / direction of the guide is shared with the other users.

As a result, the users of the portable movement support apparatus can share information about the guide that leads them and communicate via the augmented reality space. As a result, the enjoyment of a walking exercise or the like that tends to be monotonous can be increased. In addition, it is possible to provide a sound social networking service that is not biased toward cyberspace.

When a disaster occurs, it is important to guide the evacuation by a calm guide such as a fire brigade member. In particular, it is assumed that children and elderly people are not good at map interpretation, so it is difficult to evacuate alone, and someone needs to lead and guide.

Accordingly, the portable movement support apparatus according to claim 11 is the portable movement support apparatus according to any one of claims 1 to 4, and acquires information related to an evacuation route at the time of disaster using data communication. The guide can guide the user according to the information on the evacuation route.

The evacuation route searches for the route by the navigation system or acquires the latest information by data communication. Then, a guide is displayed to guide and guide the user along the evacuation route. The guide is moved as fast as possible in consideration of the moving speed and exercise load of the user.

As a result, the latest evacuation route can be evacuated at an optimum speed.

When displaying the lead connecting the user and the guide, it becomes more intuitive if the voice and movement of the guide can be changed according to the amount of slack of the lead.

Therefore, the portable movement support apparatus according to claim 12 is characterized in that, in the portable movement support apparatus according to claim 5 , the voice and movement of the guide change according to the amount of looseness of the lead. To do.

The voice or movement of the guide is changed only when the lead is in tension, or is changed stepwise according to the degree of tension of the lead.

This makes it possible to recognize and adjust the moving speed of the user more intuitively.

When displaying the lead connecting the user and the guide, it becomes more intuitive if the tension state can be presented by the vibration function of the mobile terminal or the like.

Accordingly, the portable movement support apparatus according to claim 13 is the portable movement support apparatus according to claim 5 , wherein the vibration motor is used to allow the user to recognize the state where the lead is in tension. Features.

The vibration of the vibration motor is made to vibrate only when the lead is in tension, or is changed stepwise depending on the degree of tension of the lead.

This makes it possible to recognize and adjust the moving speed of the user more intuitively.

By displaying the lead connecting the user and the guide such as an electronic pet, it is possible to intuitively recognize the direction of the assumed position of the guide, but furthermore, the voice of the guide can be heard from the assumed position of the guide If possible, the camera can be pointed more easily at the assumed position of the guide.

Accordingly, a portable movement support apparatus according to claim 14 is the portable movement support apparatus according to any one of claims 1 to 4, comprising stereo headphones or earphones, and the user's voice is guided by a stereo sound. The user can intuitively recognize the direction of the assumed position of the guide in the real space.

Since it is not preferable for traffic safety that the surrounding sound is blocked, a bone conduction speaker or a super directional speaker is used as a stereo headphone or earphone so that the surrounding sound can be heard simultaneously. In the case of using a head mounted display or a head up display, these speakers can be provided in the frame part to be put on the ear.

Assuming the assumed position of the guide in the real space as the sound source position, the voice of the guide is generated electronically.

As a result, the camera can be easily pointed in the direction in which the voice of the guide can be heard, that is, in the direction of the assumed position of the guide.

In particular, in order to reach a destination in a long-distance movement, it is necessary not only to instruct a direction with an arrow or the like but also to guide at an optimum moving speed.

Therefore, a portable movement support apparatus according to claim 15 is the portable movement support apparatus according to any one of claims 1 to 4, wherein the guide position in advance is compared with map information in advance. The guide is able to guide the user to the specified destination.

Instead of setting the moving speed as a condition for the route search, the assumed moving speed of the guide that guides the user along the searched route is presented in real time and intuitively. The assumed movement speed of the guide is determined based on the movement speed of the user, biological information such as the heart rate of the user, the road gradient at the current position of the user, the movement plan, and the like.

Thereby, it is possible to arrive at the destination without frustration while maintaining the pace distribution.

On days when calorie intake is high, it may be possible to increase calorie consumption by increasing exercise load.

Accordingly, the portable movement support apparatus according to claim 16 is the portable movement support apparatus according to any one of claims 1 to 4, wherein the changing means guides according to a user's calorie intake and exercise amount. The size and position of the display can be changed.

About the calorie intake of the user, by operating the touch panel etc., the user directly inputs the daily calorie value, or by shooting the actual meal eaten by the camera and processing the image, The calorie intake is automatically calculated. And the daily calorie intake is accumulated and stored, and when the calorie intake is excessive, the assumed position of the guide in the portable movement support apparatus according to claim 1 or 2 is farther than usual. Thus, the assumed movement speed of the guide in the portable movement support apparatus according to claim 3 or 4 is increased. On the other hand, when the calorie intake is small, the assumed position of the guide in the portable movement support apparatus according to claim 1 or 2 is set closer to the normal position, and according to claim 3 or claim 4. The assumed movement speed of the guide is reduced in the portable movement support apparatus.

As for the amount of exercise of the user, the user inputs the daily exercise amount directly by operating a touch panel or the like, or automatically detects the number of steps and the amount of exercise that the user has walked by commuting. And these daily exercise amounts are accumulated and stored, and when exercise is insufficient, the assumed position of the guide in the portable movement support device according to claim 1 or 2 is made farther than usual, The assumed movement speed of the guide in the portable movement support apparatus according to claim 3 or 4 is increased. On the other hand, when the amount of exercise is sufficient, the assumed position of the guide in the portable movement support apparatus according to claim 1 or 2 is set closer than usual, and claim 3 or claim 4 is provided. The assumed movement speed of the guide in the portable movement support apparatus described in (1) is reduced.

By reflecting calorie intake and exercise amount on the body shape of the guide, the user can have an opportunity to review himself objectively.

Accordingly, the portable movement support apparatus according to claim 17 is the portable movement support apparatus according to any one of claims 1 to 4, wherein the body shape of the guide changes according to the calorie intake and exercise amount of the user. It was made to do.

The user's caloric intake and exercise amount are obtained by the same method as in the sixteenth aspect.
To change the body shape of the guide, it is possible to perform complicated calculations based on the actual caloric intake and exercise amount of the actual animal, but the simplest method is proportional to the caloric intake. It is a method of making it fat little by little, and making it thin gradually in proportion to the amount of exercise.

As a result, the body shape of the electronic pet reflects the daily amount of meal and exercise of the user. By looking at the body shape of the electronic pet, the user can have an opportunity to review their lifestyles objectively. You will be able to pay attention to excessive calories and exercise.

In a mobile terminal such as a smartphone or a terminal composed of an optically transmissive head-mounted display or a head-up display, a program is necessary to execute a series of processes necessary for the mobile movement support apparatus.

Therefore, first, the program according to claim 18 is executed on an augmented reality (AR: AR) in a mobile terminal such as a smartphone provided with a camera.
Augmented Reality) technology, a display process for displaying a guide such as an electronic pet or a virtual person for guiding the user on the display of the mobile terminal superimposed on an image taken by the camera, and the guide For causing the computer to execute change processing for changing the size and position to be displayed so that the user can intuitively recognize and adjust his / her movement speed by moving so as to follow the guide. It is a program.

Thus, a series of processes necessary for the portable movement support apparatus according to claim 1 can be executed.

Next, the program according to claim 19 is an optical transmission (Optical transmission).
See-Trough head mounted display (HMD)
Head-Mounted Display (HUD)
Augmented reality (AR) on a terminal that is made up of Head-Up Display
Using Augmented Reality) technology, display processing for displaying guides such as electronic pets and virtual people to guide the user overlaid in real space, and change processing for changing the size and position for displaying the guides This is a program for causing a computer to execute so that the user can intuitively recognize and adjust his / her moving speed by moving so as to follow the guide.

Thus, a series of processes necessary for the portable movement support apparatus according to claim 2 can be executed.

Next, the program according to claim 20 is an augmented reality (AR: AR) in a portable terminal such as a smartphone provided with a camera.
Augmented Reality) technology, a display process for displaying a guide such as an electronic pet or a virtual person for leading the user on the display of the mobile terminal superimposed on the image captured by the camera; Positional relationship between the user movement speed detection process for detecting the movement speed, the guide movement speed determination process for determining the assumed movement speed of the guide in the real space, and the position of the user and the assumed position of the guide in the real space The user can intuitively recognize and adjust his / her movement speed by moving the change process for changing the size and position of the guide so as to follow the guide. A program for causing a computer to execute.

Thus, a series of processes necessary for the portable movement support apparatus according to claim 3 can be executed.

Finally, the program according to claim 21 is an optical transmission (Optical transmission).
See-Trough head mounted display (HMD)
Head-Mounted Display (HUD)
Augmented reality (AR) on a terminal that is made up of Head-Up Display
Augmented Reality) technology, a display process for displaying a guide such as an electronic pet or a virtual person to guide the user in superposition on the real space, and a user movement speed detection process for detecting the movement speed of the user , Size and position of the guide to be displayed based on a guide moving speed determination process for determining an assumed moving speed of the guide in the real space and a positional relationship between the position of the user and the assumed position of the guide in the real space. A program for causing a computer to execute a change process for changing the information so that the user can intuitively recognize and adjust his / her moving speed by moving in accordance with the guide.

Thus, a series of processes necessary for the portable movement support apparatus according to the fourth aspect can be executed.

You can enjoy walking while watching electronic pets and virtual people. This makes it easier to continue aerobic exercise that tends to be monotonous, leading to the maintenance and promotion of modern human health. And in an aging society, it can contribute to the suppression of increase in medical expenses and the activation of society.

The guide that guides and guides the user can intuitively inform the user of danger at an intersection or the like, which can contribute to traffic safety. In particular, it is effective in preventing signal oversight due to a reduction in visual acuity such as the elderly.

By guiding the evacuation route at the time of a disaster in an easy-to-understand manner, it is easy for children to the elderly to intuitively recognize the evacuation route and the moving speed when evacuating. Can contribute to life and physical safety, and the locality of damage.

FIG. 1 is a block diagram illustrating an example of a hardware configuration of a portable movement support apparatus. FIG. 2 is an explanatory diagram showing an example of display of a guide (a virtual person at a short distance). FIG. 3 is an explanatory diagram illustrating an example of display of a guide (a virtual person at a long distance). FIG. 4 is an explanatory diagram showing an example of a display of a guide (an electronic pet stopped before crossing a pedestrian crossing). FIG. 5 is an explanatory diagram illustrating an example of display of a guide (electronic pet crossing a pedestrian crossing). FIG. 6 is an explanatory diagram showing an example of a display in a state where the lead connecting the guide (electronic pet walking on the sidewalk) is loose. FIG. 7 is an explanatory diagram showing an example of a display in a state where the lead connecting the guide (electronic pet walking on the sidewalk) is in tension. FIG. 8 is an explanatory diagram showing an example of a display in a state where the lead connecting the guide (electronic pet walking on the sidewalk) is pulled and extended. FIG. 9 is an explanatory diagram illustrating an example of a state where the guide (electronic pet walking on the sidewalk) is out of the shooting range of the camera and is no longer displayed on the display. FIG. 10 is an explanatory diagram illustrating an example of a state where a guide (an electronic pet walking on a sidewalk) is displayed again by changing the direction of the camera depending on the lead. FIG. 11 is an explanatory diagram illustrating an example of a state in which a guide (an electronic pet walking on a sidewalk) reacts to a flower blooming on a roadside. FIG. 12 is an explanatory diagram showing an example of a state in which the guide (electronic pet walking on the sidewalk) is reacting to the guide (electronic pet) leading other users. FIG. 13 is an explanatory diagram illustrating an example of display of a guide (a virtual person at a short distance) when an optical transmission type head mounted display is used. FIG. 14 is a flowchart illustrating an example of a processing procedure executed on the portable movement support apparatus.

Examples of the portable movement support apparatus according to the best mode of the present invention will be described below. In addition, as much as possible, the same code | symbol is attached | subjected to the same part and the overlapping description is abbreviate | omitted.

FIG. 1 is a block diagram schematically illustrating an example of the most basic hardware configuration of the portable movement support apparatus.
The CPU 7 loads and executes a control program stored in a ROM (Read-Only Memory) 4 using a RAM (Random-Access Memory) 5 as a work area.
User interface (User
Interface) 1 is composed of a touch panel, a remote control switch, a voice recognition device, and the like, and is used for initial setting, activation / termination, and the like of the application.
A camera (Camera) 2 is a digital system, and a guide such as a virtual person or an electronic pet is superimposed on a real image captured by the camera (Camera) 2 and displayed on a display 6 using augmented reality technology. Further, when an optically transmissive head-mounted display or head-up display is used as in the portable movement support apparatus according to claim 2 or 4, a guide such as a virtual person or an electronic pet is provided in the user's field of view. The images are superimposed and displayed on the display 6.
For this purpose, it is necessary to detect the position, orientation, and orientation of a portable terminal, an optical transmission type head-mounted display, or the like. Therefore, the position is detected by the GPS 3, and the acceleration sensor (Accelerometer) 10 and the gyro sensor (Gyro
The posture is detected by a sensor 11 and a magnetic sensor (magnetic)
Field sensor) 12 detects the orientation.
Data storage device (Data
The storage 8 stores data related to guides such as virtual persons and electronic pets, data related to standard patterns used for image recognition, map data, and the like. It is also possible to save the route searched by the navigation system, the data related to the moved route, and the like.
Heart rate (Heart rate
monitor) 9 is used to determine the assumed position of the guide. When the user's heart rate is higher than the target heart rate, the guide is displayed closer, and when it is lower, the guide is displayed far away. . As a result, when the guide is displayed at a distance, the user raises the walking pace to catch up with the guide, and as a result, the heart rate increases. On the contrary, when the guide is displayed in front of you, the heart rate decreases as a result of lowering the walking pace without hitting it. As a result, the user's heart rate can be adjusted in the vicinity of the target heart rate. Further, when the assumed position is calculated from the assumed movement speed of the guide as in the portable movement support device according to claim 3 or 4, the heart rate monitor 9 is used to determine the assumed movement speed, and the user If the heart rate of the guide is higher than the target heart rate, the assumed moving speed of the guide is decelerated, and if it is lower, the assumed moving speed of the guide is accelerated.

FIG. 1 shows an example of the most basic hardware configuration. For example, when a data communication network is used like the portable movement support apparatus according to claim 10 or 11, a network interface is used. Add the required configuration. Data communication makes it possible to share information about guides with other users.

FIG. 2 is an explanatory diagram illustrating an example of a display of a guide (virtual person) at a short distance. In a smartphone 13 provided with a camera, an image is captured by the camera on the display 14 of the smartphone 13 using augmented reality technology. A state is shown in which a guide (virtual person at a short distance) 16 representing a virtual person leading the user 15 is displayed superimposed on a real image in front of the user 15. There is a pedestrian crossing in front of the user 15, and the guide displayed by augmented reality technology is in the state of the moment about to cross the pedestrian crossing, and the traffic signal for pedestrians at this time is in the advanced state. The guide is stopped in front of the pedestrian crossing by comparing the assumed position with the map information and recognizing the image when the traffic signal for pedestrian is stopped. In addition, an automobile that travels is detected by an object detection sensor such as an infrared sensor, an ultrasonic sensor, or a millimeter wave radar sensor, and a voice reacts to the detected automobile so that voice and movement change.

There are two types of augmented reality technology, one that synthesizes a virtual image and one that adds information to an image captured by a camera. The display means of the present invention uses the former.

In the use of the portable movement support device, the guide guides the user along the movement path in advance, and the guide leads the user along the movement direction selected by the user without setting the movement path in advance. There is.

When the movement route is determined in advance, the position of the guide is determined based on the position of the user. Specifically, when the user's moving speed is delayed compared to the assumed moving speed of the guide, the guide is displayed as if it is at a long distance, and when it is moving, the guide is displayed at a short distance. Display as it exists. This display method is intuitive and easy to understand, and is effective in walking when deciding a movement route in advance and evacuation guidance at the time of disaster.

When the user moves while freely selecting a movement route without setting a movement route in advance, it is necessary to discontinuously move the position of the guide as the user changes the traveling direction. In this case, for example, immediately after the user changes the direction of travel, if the guide rushes and moves to the front of the user for display, the user can be continuously led in a natural manner. When the user's movement speed is behind the guide's assumed movement speed, the guide is displayed as if it is at a long distance, and when it is moving, it is displayed as if the guide is at a short distance. This is the same as the case where the movement route is determined in advance. This display method is particularly effective in walking when a movement route is not set in advance.

The traveling direction of the user 15 is detected by a method of averaging the directions detected by combining the geomagnetic sensor and the acceleration sensor, a method of obtaining the direction from the coordinates of two points measured by GPS, or a method of combining these methods. .

The moving speed of the user 15 is detected using the GPS 3 or the acceleration sensor 10. In the method using the GPS 3, position information of the user 15 that can be determined by the GPS 3 mounted on the smartphone 13 is used. Specifically, the moving speed is detected by locating the self-position of the user 15 at two points at a certain time interval by the GPS 3 and dividing the distance between the two points by the certain time. Also, in places such as buildings and tunnels where GPS satellites cannot be captured, the pedometer function of the smartphone 13 is used to multiply the step number of the user 15 obtained by the installed acceleration sensor by the step length of the user 15 input in advance. By detecting the moving speed. In order to improve the detection accuracy, it is conceivable to use the gyro sensor 11 in combination.

As the moving speed determination means of the guide 16, a method of setting a speed programmed in advance or a method of setting a speed determined based on the heart rate of the user 15 or the like is used. Specifically, a method of setting a constant speed in advance, a method of automatically determining from a gradient distribution on a moving path using map information, a method of increasing or decreasing the moving speed of the user 15 according to the heart rate, or The assumed moving speed of the guide is determined by a method combining them.

The changing means for changing the size and position of the guide 16 changes the size and position at which the guide 16 is displayed based on the position / posture / orientation of the mobile terminal 13 and the assumed position of the guide 16 in the real space.
The position of the mobile terminal 13 is detected by using GPS. The posture of the mobile terminal 13 is detected by using an acceleration sensor or a gyro sensor. The orientation of the mobile terminal 13 is detected by using a geomagnetic sensor. Further, the accuracy can be improved by combining with an acceleration sensor. Furthermore, since the geomagnetic sensor may be affected by the surrounding magnetic field, it is conceivable to use a gyro sensor in combination.

The assumed position of the guide 16 is determined by adding the distance obtained by time integration of the assumed moving speed with the initial position of the guide as a base point. The initial guide position is set to the user's start position. The traveling direction of the guide is set along the searched movement route.

In the method of determining the assumed position using the value obtained by integrating the assumed moving speed of the guide with time, a case where the guide is too far from the user is assumed. Therefore, an upper limit is set for the separation distance between the two, and an attenuation coefficient can be multiplied according to the separation distance between the user and the guide when the assumed moving speed of the guide is determined.

FIG. 3 is an explanatory diagram showing an example of the display of a guide (a virtual person at a long distance). Following the state of FIG. 2, the guide resumes moving forward after the traffic signal changes to advance, and before the pedestrian crossing. In order to increase the user's heart rate, which has been lowered due to being stopped at, the assumed movement speed of the guide is increased, and as a result, the guide is displayed so as to exist at a position away from the user.
The size and position for displaying the guide (virtual person at a long distance) 17 are determined based on the position / posture / azimuth of the mobile terminal 13 and the assumed position of the guide (virtual person at a long distance) 17 in the real space.

FIG. 4 is an explanatory diagram showing an example of the display of a guide (an electronic pet stopped before crossing a pedestrian crossing), and a user 15 together with a guide (an electronic pet stopped before a crosswalk) 18. And a lead 19 (a slack state) 19 connecting the guide 18 (an electronic pet stopped before crossing the pedestrian crossing) 18 is displayed. One end on the user side of the lead (sagging state) 19 is in the center of the lowermost part of the display 14 and is displayed as if extending from the hand of the user 15 holding the mobile terminal.

FIG. 5 is an explanatory view showing an example of a display of a guide (an electronic pet crossing a pedestrian crossing). Following the state of FIG. To increase the user's heart rate, which was low due to being stopped before the sidewalk, the guide's estimated speed of movement increases, and as a result, the guide appears to be far away from the user .
The lead (tensioned state) 21 connecting the user 15 and the guide (electronic pet crossing the pedestrian crossing) 20 is displayed thinly in a straight line or the like, as if the guide is tensioned by pulling the user 15. indicate. At this time, if the vibration function of the portable terminal 13 is used, the user 15 can experience the tension state of the lead in a tactile sense.

6 to 10, when used as a portable navigation system, the direction in which the guide is present can be roughly estimated by using the lead even when the guide is out of the shooting range of the camera. The display of a series of states in which the guide can be displayed again on the display of the portable terminal by redirecting the camera in the estimated direction will be described.

First, FIG. 6 is an explanatory view showing an example of a display in which the lead connecting the guide (electronic pet walking on the sidewalk) is slack, and the moving speed of the user 15 is the guide (electronic pet walking on the sidewalk). 22 is the same or faster than the assumed moving speed.

Next, FIG. 7 is an explanatory diagram showing an example of a display in a state where the lead connecting the guide (electronic pet walking on the sidewalk) is in tension, and the assumed moving speed of the guide (electronic pet walking on the sidewalk) 22 The guide has moved away from the user due to the increase in the speed or the movement speed of the user has decreased, and the lead (strained state) 21 is in tension.

Next, FIG. 8 is an explanatory view showing an example of a display in which the lead connecting the guide (electronic pet walking on the sidewalk) is pulled and extended, and further, the lead (pulling) is pulled by moving the guide away from the user. Part of the head of the guide 22 (electronic pet walking on the sidewalk) 22 is out of the shooting range of the camera and cannot be displayed on the display 14.

Next, FIG. 9 is an explanatory view showing an example of a state in which the guide (electronic pet walking on the sidewalk) is out of the shooting range of the camera and is no longer displayed on the display. The guide (display out of the shooting range of the camera) is shown. Electronic pet) 24 that is no longer displayed on the display 14 can no longer be displayed on the display 14. However, in the lead (a state in which the lead is pulled to the left front and a part is not displayed on the display) 25, one end on the user side is fixed to the center of the lowermost part of the display 14, so that the guide is extended in the extending direction of the lead. It can be estimated that it exists.

FIG. 10 is an explanatory diagram illustrating an example of a state where a guide (an electronic pet walking on a sidewalk) is displayed again by changing the direction of the camera depending on the lead. The guide (electronic pet that has been displayed again on the display) 26 is connected to the user 15 by a lead (a state in which the entire display is again displayed on the display) 27 and can guide the user 15 in a direction to proceed. .

FIG. 11 is an explanatory diagram illustrating an example of a state in which a guide (an electronic pet walking on a sidewalk) reacts to a flower blooming on a roadside. A flower 28 on the roadside photographed by the camera is recognized as an image, and a preprogrammed action 29 (electronic pet reacting to the flower blooming on the roadside) 29 is performed. At this time, by registering the height at which the recognition target generally exists, the position of the recognition target in the real space can be estimated from the position / posture / orientation of the mobile terminal. This makes it possible to express a more realistic operation based on the positional relationship between the estimated position of the recognition target and the assumed position of the guide. For example, a guide 29 (an electronic pet reacting to a flower blooming on the roadside) 29 can approach the flower 28 blooming on the roadside and express an action of smelling the nose.

FIG. 12 is an explanatory diagram showing an example of a state in which the guide (electronic pet walking on the sidewalk) is reacting to the guide (electronic pet) leading other users. Using data communication such as infrared communication or short-range wireless communication, information on the guide (electronic pet) 31 leading other users 30 is acquired, as if it exists at the assumed position in the real space, It is displayed on the display 14 of the portable movement support apparatus of the user 15. By sharing information on the assumed position in the real space of this guide and the type, form, direction, etc. of the guide with other users 30, the guide (the electron reacting to the guide (electronic pet) leading other users) The operation of the pet) 32 can be displayed.

FIG. 13 is an explanatory diagram showing an example of the display of the guide (virtual person at a short distance) 16 when the optical transmission type head-mounted display 33 is used. Glasses with a pulse meter. It also has various sensors built-in. A guide (a virtual person at a short distance) 16 is displayed as if it actually exists, superimposed on the real world seen through the glasses. The pulse of the user's head is measured, and the guide 16 is moved away when it is lower than the target heart rate, and is approached when it is higher. When the user follows the guide 16 and walks, the user's exercise load changes, and the heart rate (pulse rate) approaches the target heart rate.

Further, when the GPS is provided, the user's own position is measured, and the guide is estimated based on the assumed position of the guide estimated based on the measured position, and the position / posture / orientation of the optical transmission type head-mounted display 33. The size and position for displaying 16 can be determined. As a result, it is possible to prevent the guide from being displayed at a position where it cannot exist in the air or on the water, so that more realistic display is possible.

FIG. 14 is a flowchart illustrating an example of a basic processing flow of the portable movement support apparatus.

First, at the point where the guidance / guidance is started, the position (at the user's initial position and the initial assumed position of the guide is in the vicinity thereof), posture, and orientation are detected (S1). Next, the position / size and the like for displaying the guide on the display are determined (S2), and the guide is displayed on the display so as to be superimposed on the actual image (S3).
Next, the position and moving speed of the user are detected (S4).
Next, the user's heart rate (H) is detected (S5). Then, H> target heart rate is determined (S6). If the result of S6 is YES, the assumed moving speed of the guide is decreased (S7). If the result of S6 is NO, the assumed moving speed of the guide is increased. (S8).
Next, the estimated position of the guide is calculated and changed by using a value obtained by integrating the estimated moving speed of the guide with time (S9). At this time, when used as a portable navigation system, the guide moves in a direction along the searched route.
Next, the position / posture / orientation of the mobile terminal is detected (S10), and the position / size, etc., for displaying the guide is changed based on the positional relationship between the user's position and the assumed position of the guide (S11). The guide is redisplayed on the display so as to be superimposed on the video (real space) (S12).
Finally, it is determined whether or not the guidance / guidance by the guide is to be terminated (S13). If the result of S13 is NO, the process is repeated from S4, and if the result of S13 is YES, the process is terminated.

The field of application of Augmented Reality (AR) technology has expanded, and with the spread of smartphones and head-mounted displays, there is a possibility of creating new industries.

1 User interface (User
Interface)
2 Camera
3 GPS (Global Positioning
System)
4 ROM (Read-Only Memory)
5 RAM (Random-Access Memory)
6 Display
7 CPU (Central Processing Unit)
8 Data storage device (Data
storage)
9 Heart rate monitor
monitor)
10 Accelerometer
11 Gyro sensor (Gyro
Sensor)
12 Magnetic sensor (Magnetic
field sensor)
13 A mobile terminal such as a smartphone equipped with a camera 14 A display 15 of a mobile terminal 16 A user 16 Guide (a virtual person at a short distance)
17 Guide (a virtual person at a distance)
18 Guide (Electronic pet stopped before crossing the pedestrian crossing)
19 Lead (Loose)
20 guides (electronic pets crossing a pedestrian crossing)
21 Lead (strained)
22 Guide (electronic pet walking on the sidewalk)
23 Lead (in a stretched state)
24 Guide (an electronic pet that is out of the camera's shooting range and no longer appears on the display)
25 Lead (Pulled to the left front and part of it is not displayed on the display)
26 Guide (an electronic pet that is now shown on the display again)
27 Lead (when the whole is displayed on the display again)
28 Flowers blooming on the roadside 29 Guide (electronic pets reacting to flowers blooming on the roadside)
30 Other users 31 Guide to lead other users (electronic pets)
32 Guide (Electronic pet reacting to guide (electronic pet) leading other users)
33 Optically transmissive head-mounted display

Claims (21)

A mobile terminal such as a smartphone equipped with a camera,
Using Augmented Reality (AR) technology,
On the display of the mobile terminal,
Overlaid on the image taken with the camera,
Display means for displaying a guide such as an electronic pet or a virtual person for leading the user;
Changing means for changing the size and position of displaying the guide based on biological information such as the user's heart rate, or the moving speed or moving pace of the user ,
A portable movement support apparatus comprising: An optical see-through (HMD) head-mounted display (HMD) or head-up display (HUD) terminal,
Using Augmented Reality (AR) technology,
Overlaid on the real space,
Display means for displaying a guide such as an electronic pet or a virtual person for leading the user;
Changing means for changing the size and position of displaying the guide based on biological information such as the user's heart rate, or the moving speed or moving pace of the user ,
A portable movement support apparatus comprising: A mobile terminal such as a smartphone equipped with a camera,
Using Augmented Reality (AR) technology,
On the display of the mobile terminal,
Overlaid on the image taken with the camera,
Display means for displaying a guide such as an electronic pet or a virtual person for leading the user;
User moving speed detecting means for detecting the moving speed of the user;
Guide moving speed determining means for determining an assumed moving speed in the real space of the guide based on biological information such as the user's heart rate or the moving speed or moving pace of the user ;
Based on the positional relationship between the position of the user and the assumed position in the real space of the guide, changing means for changing the size and position for displaying the guide;
A portable movement support apparatus comprising: An optical see-through (HMD) head-mounted display (HMD) or head-up display (HUD) terminal,
Using Augmented Reality (AR) technology,
Overlaid on the real space,
Display means for displaying a guide such as an electronic pet or a virtual person for leading the user;
User moving speed detecting means for detecting the moving speed of the user;
Guide moving speed determining means for determining an assumed moving speed in the real space of the guide based on biological information such as the user's heart rate or the moving speed or moving pace of the user ;
Based on the positional relationship between the position of the user and the assumed position in the real space of the guide, changing means for changing the size and position for displaying the guide;
A portable movement support apparatus comprising: Display means
Along with the guide
Display the lead connecting the user and the guide,
5. The lead slack amount determining means for determining a slack amount of the lead based on a positional relationship between the position of the user and an assumed position in the real space of the guide. The portable movement support apparatus according to any one of the above. 5. An object in an image photographed by a camera is image-recognized, and a guide reacts to the recognized object so that a voice or movement of the guide changes. A portable movement support apparatus according to claim 1. 5. The apparatus according to claim 1, further comprising: a heart rate monitor, wherein the changing unit can change a size and a position where the guide is displayed according to a heart rate of the user measured by the heart rate monitor. A portable movement support apparatus according to claim 1. An object detection sensor such as an infrared sensor, an ultrasonic sensor, or a millimeter wave radar sensor is provided. The object detection sensor detects a person or a car located in front of the user, and a guide reacts to the detected person or the car. The portable movement support apparatus according to any one of claims 1 to 4, wherein voice and movement of the guide are changed. 5. The guide according to claim 1, wherein the guide is temporarily stopped in front of the pedestrian crossing by collating the assumed position of the guide in the real space with map information or by recognizing a traffic signal or the like. The portable movement support apparatus according to any one of the above. The assumed position information in the real space of the guide leading the other user is acquired by using data communication, and the guide leading the other user can be displayed. The portable movement support apparatus according to any one of the above. 5. The information on the evacuation route at the time of disaster occurrence is acquired using data communication, and the guide can guide the user according to the information on the evacuation route. Portable mobility support device. 6. The portable movement support apparatus according to claim 5 , wherein the voice and movement of the guide change according to the amount of looseness of the lead. 6. The portable movement support apparatus according to claim 5 , wherein a user can recognize a state in which the lead is in tension by using a vibration motor. With stereo headphones or earphones, portable mobile support device as claimed in claim 1, characterized in that the guide voice so that the user can listen to the stereo sound to claim 4. 5. The mobile phone according to claim 1, wherein the guide can guide the user to a destination designated in advance by collating the assumed position in the real space of the guide with map information. Type movement support device. 5. The portable movement support according to claim 1, wherein the changing means is configured to change a size and a position of displaying the guide according to a calorie intake amount and an exercise amount of the user. apparatus. The portable movement support apparatus according to any one of claims 1 to 4, wherein the body shape of the guide is changed in accordance with the calorie intake and exercise amount of the user. In a mobile terminal such as a smartphone equipped with a camera,
Using Augmented Reality (AR) technology,
On the display of the mobile terminal,
Overlaid on the image taken with the camera,
Display processing to display guides such as electronic pets and virtual people to lead users,
Based on the biological information such as the user's heart rate, or the moving speed or moving pace of the user, a change process for changing the size and position for displaying the guide,
The program for making the portable movement assistance apparatus of Claim 1 perform. In a terminal consisting of an optical see-through head mount display (HMD) or a head-up display (HUD),
Using Augmented Reality (AR) technology,
Overlaid on the real space,
Display processing to display guides such as electronic pets and virtual people to lead users,
Based on the biological information such as the user's heart rate, or the moving speed or moving pace of the user, a change process for changing the size and position for displaying the guide,
The program for making the portable movement assistance apparatus of Claim 2 perform. In a mobile terminal such as a smartphone equipped with a camera,
Using Augmented Reality (AR) technology,
On the display of the mobile terminal,
Overlaid on the image taken with the camera,
Display processing to display guides such as electronic pets and virtual people to lead users,
A user moving speed detection process for detecting the moving speed of the user;
Guide moving speed determination processing for determining an assumed moving speed in the real space of the guide based on biological information such as the user's heart rate or the moving speed or moving pace of the user ;
Based on the positional relationship between the position of the user and the assumed position in the real space of the guide, a change process for changing the size and position of displaying the guide,
The program for making the portable movement assistance apparatus of Claim 3 perform. In a terminal consisting of an optical see-through head mount display (HMD) or a head-up display (HUD),
Using Augmented Reality (AR) technology,
Overlaid on the real space,
Display processing to display guides such as electronic pets and virtual people to lead users,
A user moving speed detection process for detecting the moving speed of the user;
Guide moving speed determination processing for determining an assumed moving speed in the real space of the guide based on biological information such as the user's heart rate or the moving speed or moving pace of the user ;
Based on the positional relationship between the position of the user and the assumed position in the real space of the guide, a change process for changing the size and position of displaying the guide,
The program for making the portable movement assistance apparatus of Claim 4 perform.