WO2019021447A1

WO2019021447A1 - Wearable terminal display system, wearable terminal display method and program

Info

Publication number: WO2019021447A1
Application number: PCT/JP2017/027351
Authority: WO
Inventors: 俊二菅谷
Original assignee: 株式会社オプティム
Priority date: 2017-07-28
Filing date: 2017-07-28
Publication date: 2019-01-31

Abstract

[Problem] To identify a prediction subject from an image of the field of view of a wearable terminal, and display a future prediction calculated according to the prediction subject, as augmented reality on the display of the wearable terminal. [Solution] A wearable terminal display system which displays future predictions of a prediction subject on the display of a wearable terminal, the wearable terminal display system comprising: an image acquisition means which acquires an image of the prediction subject which has entered the field of view of the wearable terminal; an identifying means which identifies the prediction subject by performing image analysis on the image; a calculating means which calculates a future prediction for the prediction subject; and a future prediction display means which displays on the display of the wearable terminal, as augmented reality, the future prediction for the prediction subject which is visible through the display.

Description

Wearable terminal display system, wearable terminal display method and program

The present invention relates to a wearable terminal display system, a wearable terminal display method, and a program for displaying a calculated future prediction as an augmented reality on a display target of a wearable terminal with respect to a prediction target seen through the display panel.

In recent years, the use of IT for forecasting the future has advanced. For example, an apparatus for identifying and extracting a document that predicts the future has been provided (Patent Document 1).

JP, 2016-206751, A

However, the system of Patent Document 1 has a problem that it can not provide future prediction only by looking at the prediction target.

In view of the above problems, the present invention identifies a prediction target from an image of a field of view of a wearable terminal, and displays a future prediction calculated according to the prediction target on a display plate of the wearable terminal as an augmented reality, wearable terminal It aims at providing a terminal display method and program.

The present invention provides the following solutions.

An invention according to a first feature is a wearable terminal display system for displaying a future prediction of a prediction target on a display board of the wearable terminal, and an image acquisition unit for acquiring an image of the prediction target within the view of the wearable terminal An identifying means for analyzing the image to identify the prediction target, a calculating means for calculating a future prediction of the prediction target, and the prediction that appears to be transmitted through the display plate to the display plate of the wearable terminal And a future prediction display means for displaying the future prediction as an augmented reality for the object.

An invention according to a first feature is a wearable terminal display method for displaying a future prediction of a prediction target on a display board of the wearable terminal, and an image acquisition step of acquiring an image of the prediction target within the view of the wearable terminal Identifying the prediction target by analyzing the image, calculating the future prediction of the prediction target, calculating the future prediction of the prediction target, and displaying the prediction on the display plate of the wearable terminal. Providing a target with a future prediction display step of displaying the future prediction as an augmented reality.

The invention according to the first aspect is characterized in that the computer has an image acquisition step of acquiring an image of an object to be predicted which has come into view of the wearable terminal, an image analysis of the image to specify the object to be predicted, and Calculating the future prediction of the prediction target, and displaying the future prediction as the augmented reality on the display target of the wearable terminal on the display target of the wearable terminal. Provide a program to make

The future prediction according to the prediction object can be displayed on the display board of the wearable terminal simply by putting the prediction object in the view of the wearable terminal.

FIG. 1 is a schematic view of a wearable terminal display system. FIG. 2: is an example which calculated and displayed future prediction on the display board of a wearable terminal.

The best mode for carrying out the present invention will be described below. This is merely an example, and the technical scope of the present invention is not limited to this.

The wearable terminal display system according to the present invention is a system for displaying the calculated future prediction as an augmented reality on a display target of the wearable terminal with respect to a prediction target seen through the display panel. A wearable terminal is a terminal with a view such as a smart glass or a head mounted display.

An outline of a preferred embodiment of the present invention will be described based on FIG. FIG. 1 is a schematic view of a wearable terminal display system according to a preferred embodiment of the present invention.

As shown in FIG. 1, the wearable terminal display system includes an image acquisition unit, an identification unit, a calculation unit, and a future prediction display unit, which are realized by the control unit reading a predetermined program. Although not shown, similarly, determination means, change means, detection means, action result display means, position direction acquisition means, estimation means, guideline display means, selection acceptance means may be provided. These may be application based, cloud based or otherwise. Each means described above may be realized by a single computer or may be realized by two or more computers (for example, in the case of a server and a terminal).

An image acquisition means acquires the image of the prediction object which entered into the view of a wearable terminal. An image captured by the camera of the wearable terminal may be acquired. Or even if it is except a wearable terminal, if such an image can be acquired, it does not matter. The image may be a moving image or a still image. In order to display future predictions in real time, real-time images are preferable.

The identification means analyzes the image to identify the prediction target. For example, it is specified whether the forecast target is Kagoshima black pig, Yubari melon, or land in Minato-ku, Tokyo. The prediction target can be identified from the color, shape, size, features, and the like. Of course, the prediction target is not limited to these. In addition, if it takes time to identify all the predicted objects that have appeared, only the predicted object in the center of the view of the wearable terminal may be identified. By identifying only the prediction object in the center of the view, the time required for identification can be greatly reduced. Machine learning may improve the accuracy of image analysis. For example, machine learning is performed using the past image to be predicted as teacher data.

A calculation means calculates the future prediction according to prediction object. With future prediction, for example, Kagoshima black pig grows up to what size, how much sugar content of Yubari melon will become, land of Minato-ku, Tokyo, overseas 1-chome land is likely to be sold at unit price, etc. It is. Of course, future prediction is not limited to these. The future prediction according to the target of prediction may be calculated with reference to a database in which the future prediction is registered in advance. Also, the Web content pre-linked to the prediction target may be accessed to calculate the future prediction. For example, it can be calculated from Web content by assigning a URL etc. that links the prediction target and the future prediction. Further, the future prediction may be calculated from the Web content searched by searching the prediction target via the Internet. For example, since there is a case where past information is posted on an information site etc., it can be calculated from an internet search. Alternatively, there are cases where it is possible to calculate future predictions from social networking services (SNS) or word-of-mouth sites.

The future prediction display means displays the future prediction as an augmented reality on the prediction object which is seen through the display plate on the display plate of the wearable terminal. For example, as shown in FIG. 2, on the display board of the wearable terminal, the future prediction drawn with a broken line is displayed as an augmented reality with respect to the prediction target drawn with a solid line seen through the display board. Here, for the sake of understanding, solid lines are real and broken lines are augmented reality. By displaying the future prediction as an augmented reality on the prediction target drawn as a solid line that can be seen through the display board, it is possible to visually grasp what kind of future prediction is in the prediction target. Although the future prediction to be displayed as augmented reality may be displayed so as to overlap with the prediction target seen through the display board, it becomes difficult to see the prediction target, so even if the display of future prediction can be switched ON / OFF. Good.

The determination means determines whether or not the displayed future prediction has been browsed. By acquiring the image being browsed and analyzing the image, it may be determined whether the future prediction has been browsed. In addition, it may be determined from the sensor information of the wearable terminal, the sensor information worn by the viewer, etc. whether or not the future prediction is browsed. For example, a sensor that detects a sight line, a motion sensor, an acceleration sensor, and the like.

The change means changes the degree of attention so that the future prediction is changed to viewed when it is determined that the view has been viewed, and the future prediction is viewed when it is determined that the view is not viewed. By doing this, it is possible to visually grasp which future prediction has been viewed and has not been viewed. For example, it may be regarded as already viewed by putting a check in the check box of the future prediction. For example, a stamp may be pressed on the future prediction to make the view already viewed. Moreover, the change of attention level may change the color and the size of the future prediction, or may stamp stamps so that the future prediction may stand out.

The detection means detects an action on the displayed future prediction. The action is, for example, a gesture, hand movement, gaze movement, or the like. By acquiring an image being browsed and performing image analysis, it is possible to detect an action on future prediction. Further, the action on the future prediction may be detected from the sensor information of the wearable terminal, the sensor information attached to the viewer, or the like. For example, a sensor that detects a sight line, a motion sensor, an acceleration sensor, and the like.

The action result display means displays the result according to the action as the augmented reality on the prediction target viewed through the display plate on the display plate of the wearable terminal. For example, the display of the future prediction may be turned off when an action to delete the future prediction is detected. For example, the link may be opened upon detecting an action to open the link attached to the future prediction. For example, when an action to turn a page of a future prediction is detected, the page may be turned. Of course, other actions may be used.

The position / direction unit acquires the terminal position and the imaging direction of the wearable terminal. For example, the terminal position can be acquired from the GPS (Global Positioning System) of the wearable terminal. For example, the imaging direction can be acquired from the geomagnetic sensor or the acceleration sensor of the wearable terminal when imaging is performed by the wearable terminal. You may acquire from other than these.

An estimation means estimates the target position of prediction object based on a terminal position and an imaging direction. If the terminal position and the imaging direction are known, the target position of the imaged prediction target can be estimated.

In addition, the specifying unit may specify the prediction target from the target position and the image analysis. Specific accuracy can be improved by utilizing position information. For example, if it is possible to improve the accuracy of specifying that the location is Sensoji, the reliability of the future prediction to be displayed is also improved.

The guideline display means displays a guideline for imaging a prediction target as an augmented reality on the display board of the wearable terminal. For example, guidelines such as a frame or a cross may be displayed. It will be easier to analyze the image by having the image taken according to the guidelines.

In addition, the acquisition unit may acquire an image captured along a guideline. By acquiring and analyzing only the image captured along the guideline, it is possible to efficiently specify the prediction target.

The selection receiving unit receives selection of a selection target for the prediction target viewed through the display plate of the wearable terminal. For example, the selection target selection may be accepted by looking at the prediction target seen through the display plate of the wearable terminal for a certain period of time. For example, the selection of the selection target may be received by touching the prediction target which is seen through the display plate of the wearable terminal. For example, the selection of the selection target may be received by positioning the cursor on the prediction target that is seen through the display plate of the wearable terminal. For example, a sensor that detects a sight line, a motion sensor, an acceleration sensor, and the like.

In addition, the future prediction display means may display the future prediction as an augmented reality on the display plate of the wearable terminal in accordance with only the selection object that is seen through the display plate. Since the future prediction is displayed as an augmented reality according to only the selected selection object, it is possible to pinpoint the future prediction. If future predictions are displayed on all of the identified prediction targets, the display screen may be cumbersome.
[Description of operation]

Next, the wearable terminal display method will be described. The wearable terminal display method according to the present invention is a method of displaying the calculated future prediction as an augmented reality on a prediction target that is seen through the display plate on a display plate of the wearable terminal.

The wearable terminal display method includes an image acquisition step, a identification step, a calculation step, and a future prediction display step. Although not shown in the drawings, similarly, a determination step, a change step, a detection step, an action result display step, a position direction acquisition step, an estimation step, a guideline display step, and a selection acceptance step may be provided.

The image acquisition step acquires an image of a prediction target that has come into view of the wearable terminal. An image captured by the camera of the wearable terminal may be acquired. Or even if it is except a wearable terminal, if such an image can be acquired, it does not matter. The image may be a moving image or a still image. In order to display future predictions in real time, real-time images are preferable.

In the identification step, the image is analyzed to identify a prediction target. For example, it is specified whether the forecast target is Kagoshima black pig, Yubari melon, or land in Minato-ku, Tokyo. The prediction target can be identified from the color, shape, size, features, and the like. Of course, the prediction target is not limited to these. In addition, if it takes time to identify all the predicted objects that have appeared, only the predicted object in the center of the view of the wearable terminal may be identified. By identifying only the prediction object in the center of the view, the time required for identification can be greatly reduced. Machine learning may improve the accuracy of image analysis. For example, machine learning is performed using the past image to be predicted as teacher data.

The calculation step calculates a future prediction according to the prediction target. With future prediction, for example, Kagoshima black pig grows up to what size, how much sugar content of Yubari melon will become, land of Minato-ku, Tokyo, overseas 1-chome land is likely to be sold at unit price, etc. It is. Of course, future prediction is not limited to these. The future prediction according to the target of prediction may be calculated with reference to a database in which the future prediction is registered in advance. Also, the Web content pre-linked to the prediction target may be accessed to calculate the future prediction. For example, it can be calculated from Web content by assigning a URL etc. that links the prediction target and the future prediction. Further, the future prediction may be calculated from the Web content searched by searching the prediction target via the Internet. For example, since there is a case where past information is posted on an information site etc., it can be calculated from an internet search. Alternatively, there are cases where it is possible to calculate future predictions from social networking services (SNS) or word-of-mouth sites.

The future prediction display step displays the future prediction as an augmented reality for the prediction target that is seen through the display plate on the display plate of the wearable terminal. For example, as shown in FIG. 2, on the display board of the wearable terminal, the future prediction drawn with a broken line is displayed as an augmented reality with respect to the prediction target drawn with a solid line seen through the display board. Here, for the sake of understanding, solid lines are real and broken lines are augmented reality. By displaying the future prediction as an augmented reality on the prediction target drawn as a solid line that can be seen through the display board, it is possible to visually grasp what kind of future prediction is in the prediction target. Although the future prediction to be displayed as augmented reality may be displayed so as to overlap with the prediction target seen through the display board, it becomes difficult to see the prediction target, so even if the display of future prediction can be switched ON / OFF. Good.

The determination step determines whether the displayed future prediction has been viewed. By acquiring the image being browsed and analyzing the image, it may be determined whether the future prediction has been browsed. In addition, it may be determined from the sensor information of the wearable terminal, the sensor information worn by the viewer, etc. whether or not the future prediction is browsed. For example, a sensor that detects a sight line, a motion sensor, an acceleration sensor, and the like.

The changing step changes the degree of attention so that the future prediction is changed to viewed if it is determined to have been viewed, and the future prediction is to be viewed if it is determined that it has not been viewed. By doing this, it is possible to visually grasp which future prediction has been viewed and has not been viewed. For example, it may be regarded as already viewed by putting a check in the check box of the future prediction. For example, a stamp may be pressed on the future prediction to make the view already viewed. Moreover, the change of attention level may change the color and the size of the future prediction, or may stamp stamps so that the future prediction may stand out.

The detecting step detects an action on the displayed future prediction. The action is, for example, a gesture, hand movement, gaze movement, or the like. By acquiring an image being browsed and performing image analysis, it is possible to detect an action on future prediction. Further, the action on the future prediction may be detected from the sensor information of the wearable terminal, the sensor information attached to the viewer, or the like. For example, a sensor that detects a sight line, a motion sensor, an acceleration sensor, and the like.

The action result display step displays, on the display board of the wearable terminal, the result corresponding to the action as the augmented reality for the prediction target seen through the display board. For example, the display of the future prediction may be turned off when an action to delete the future prediction is detected. For example, the link may be opened upon detecting an action to open the link attached to the future prediction. For example, when an action to turn a page of a future prediction is detected, the page may be turned. Of course, other actions may be used.

The position direction step acquires the terminal position and the imaging direction of the wearable terminal. For example, the terminal position can be acquired from the GPS (Global Positioning System) of the wearable terminal. For example, the imaging direction can be acquired from the geomagnetic sensor or the acceleration sensor of the wearable terminal when imaging is performed by the wearable terminal. You may acquire from other than these.

The estimation step estimates a target position of a prediction target based on the terminal position and the imaging direction. If the terminal position and the imaging direction are known, the target position of the imaged prediction target can be estimated.

In addition, the identification step may identify the prediction target from the target position and the image analysis. Specific accuracy can be improved by utilizing position information. For example, if it is possible to improve the accuracy of specifying that the location is Sensoji, the reliability of the future prediction to be displayed is also improved.

The guideline display step displays a guideline for imaging a prediction target as an augmented reality on a display plate of the wearable terminal. For example, guidelines such as a frame or a cross may be displayed. It will be easier to analyze the image by having the image taken according to the guidelines.

In addition, the acquisition step may acquire an image captured along a guideline. By acquiring and analyzing only the image captured along the guideline, it is possible to efficiently specify the prediction target.

The selection receiving step receives selection of a selection target for the prediction target viewed through the display board of the wearable terminal. For example, the selection target selection may be accepted by looking at the prediction target seen through the display plate of the wearable terminal for a certain period of time. For example, the selection of the selection target may be received by touching the prediction target which is seen through the display plate of the wearable terminal. For example, the selection of the selection target may be received by positioning the cursor on the prediction target that is seen through the display plate of the wearable terminal. For example, a sensor that detects a sight line, a motion sensor, an acceleration sensor, and the like.

Further, the future prediction display step may display the future prediction as the augmented reality on the display board of the wearable terminal in accordance with only the selection target that is seen through the display board. Since the future prediction is displayed as an augmented reality according to only the selected selection object, it is possible to pinpoint the future prediction. If future predictions are displayed on all of the identified prediction targets, the display screen may be cumbersome.

The above-described means and functions are realized by a computer (including a CPU, an information processing device, and various terminals) reading and executing a predetermined program. The program may be, for example, an application installed on a computer, or a SaaS (software as a service) provided from a computer via a network, for example, a flexible disk, a CD It may be provided in the form of being recorded in a computer readable recording medium such as a CD-ROM or the like, a DVD (DVD-ROM, DVD-RAM or the like). In this case, the computer reads the program from the recording medium, transfers the program to the internal storage device or the external storage device, stores it, and executes it. Alternatively, the program may be recorded in advance in a storage device (recording medium) such as, for example, a magnetic disk, an optical disk, or a magneto-optical disk, and may be provided from the storage device to the computer via a communication line.

As a specific algorithm of the above-mentioned machine learning, nearest neighbor method, naive Bayes method, decision tree, support vector machine, reinforcement learning, etc. may be used. In addition, deep learning may be used in which feature quantities for learning are generated by using a neural network.

As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. Further, the effects described in the embodiments of the present invention only list the most preferable effects resulting from the present invention, and the effects according to the present invention are limited to those described in the embodiments of the present invention is not.

Claims

A wearable terminal display system for displaying a future prediction of an object to be predicted on a display board of the wearable terminal,
An image acquisition unit that acquires an image of a prediction target within the field of view of the wearable terminal;
Image analysis of the image to specify the prediction target;
Calculation means for calculating the future prediction of the prediction target;
Future prediction display means for displaying the future prediction as augmented reality for the prediction target seen through the display plate on the display plate of the wearable terminal;
A wearable terminal display system comprising:
The wearable terminal display system according to claim 1, wherein the specifying unit specifies only a prediction target in the center of the field of view of the wearable terminal.
The wearable terminal display system according to claim 1, wherein the calculation means calculates the future prediction of the prediction target with reference to a database in which a future prediction is registered in advance.
The wearable terminal display system according to claim 1, wherein the calculation means calculates the future prediction by accessing Web content previously linked to the prediction target.
The wearable terminal display system according to claim 1, wherein the calculation unit searches the prediction target via the Internet and calculates the future prediction from the searched Web content.
Determining means for determining whether the displayed future prediction has been viewed;
Changing means for changing the future prediction to viewed when it is determined that the page has been browsed;
The wearable terminal display system according to claim 1, comprising:
Determining means for determining whether the displayed future prediction has been viewed;
Changing means for changing the degree of attention so that the future prediction is browsed if it is determined that the browsing is not performed;
The wearable terminal display system according to claim 1, comprising:
Detection means for detecting an action on the displayed future prediction;
Action result display means for displaying the result according to the action as the augmented reality on the prediction target viewed through the display plate on the display plate of the wearable terminal;
The wearable terminal display system according to claim 1, comprising:
Position direction acquisition means for acquiring a terminal position and an imaging direction of the wearable terminal;
Estimating means for estimating a target position of the prediction target based on the terminal position and the imaging direction;
Equipped with
The wearable terminal display system according to claim 1, wherein the specifying unit specifies the prediction target from the target position and the image analysis.
The display board of the wearable terminal is provided with a guideline display means for displaying a guideline for imaging the prediction object as an augmented reality,
The wearable terminal display system according to claim 1, wherein the acquisition unit acquires the image captured along the guideline.
And a selection receiving unit configured to receive a selection of a selection target for the prediction target viewed through the display plate of the wearable terminal.
The wearable terminal display system according to claim 1, wherein the future prediction display means displays the future prediction as an augmented reality on the display plate of the wearable terminal in accordance with only the selection object seen through the display plate. .
A wearable terminal display method for displaying a future prediction of an object to be predicted on a display board of the wearable terminal,
An image acquisition step of acquiring an image of a prediction target that has come into view of the wearable terminal;
Image analysis of the image to identify the prediction object;
Calculating the future prediction of the prediction object;
A future prediction display step of displaying the future prediction as augmented reality for the prediction target viewed through the display plate on the display plate of the wearable terminal;
A wearable terminal display method comprising:
On the computer
An image acquisition step of acquiring an image of a prediction target that has come into view of the wearable terminal;
Image analysis of the image to identify the prediction object;
Calculating the future prediction of the prediction object;
A future prediction display step of displaying the future prediction as augmented reality for the prediction target viewed through the display plate on the display plate of the wearable terminal;
A program to run a program.