JP6714632B2 - Information display terminal and information display method - Google Patents

Description

The present invention relates to an information display terminal and an information display method.

Various methods have been proposed as information display methods for so-called wearable information display terminals that display predetermined information using a goggle-type or eyeglass-type head-mounted display.

For example, Patent Document 1 describes a method of switching the information display mode between a summary mode and a detailed mode according to the user's situation. Further, for example, in Patent Document 2, the information given by the user via the input means with respect to the current position or the situation in the real world specified by the object information is combined with the information picked up by the image pickup means. The method of displaying is described.

JP, 2007-163634, A JP, 2011-28763, A

Wearable information display terminals such as eyeglasses are different from mobile terminals such as smartphones in that when a user acquires predetermined information, the user must actively see the contents displayed by the mobile terminal on the mobile terminal. On the other hand, in a wearable information display terminal, the user can passively check the information display in the field of view in the operating state. However, in the wearable information display terminal, since the display of information is started and ended regardless of the user's intention, the information is displayed even when it is not particularly necessary. There is.

The conventional technique described in Patent Document 1 switches the display mode of information according to the situation such as whether the user is walking, gazing at the displayed image, or uttering. , No particular consideration was given to how to determine when the user wants to display information about an arbitrary object and how to start or end displaying information about the object. ..

Further, the prior art described in the above-mentioned Patent Document 2 displays the information about the object given by the user by combining it with the information captured by the image capturing means. No particular consideration was given to whether to start or end at an effective timing.

The present invention has been made in view of these problems, and in a wearable information display terminal or the like, display of information regarding an object is started or ended in a timely manner, and the information is displayed in a format that is easily recognized by the user. An object is to provide an information display terminal and an information display method.

The above object can be achieved by, for example, the invention described in the claims.

ADVANTAGE OF THE INVENTION According to this invention, the display of the information regarding an object can be started or ended in a timely manner, and information can be displayed in a format which a user can easily recognize, for example, improving a user's convenience in a wearable information display terminal. There is an effect that can be.

The block diagram which shows an example of a structure of an information display terminal. The sketch which shows an example when a user equips an information display terminal. The operation|movement flowchart in 1st Embodiment. The 1st figure which shows an example of the relevant information display in 1st Embodiment. The 2nd figure which shows an example of the relevant information display in 1st Embodiment. The block diagram which shows the system configuration on a network. The operation|movement flowchart in 2nd Embodiment. The figure which shows the example of a display at the time of display trigger registration. The 1st figure which shows an example of the relevant information display in 2nd Embodiment. The 2nd figure showing an example of related information display in a 2nd embodiment. The operation|movement flowchart in 3rd Embodiment. The 1st figure which shows an example of the relevant information display in 3rd Embodiment. The 2nd figure which shows an example of the relevant information display in 3rd Embodiment. The operation|movement flowchart in 4th Embodiment. The 1st figure which shows an example of the relevant information display in 4th Embodiment. The 2nd figure which shows an example of the relevant information display in 4th Embodiment.

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an example of the configuration of the information display terminal 101. The first embodiment will be described using this.

Reference numeral 101 is an information display terminal, 102 is a control unit that controls the entire information display terminal 101, 103 is an imaging unit, 104 is an image processing unit, 105 is a display control unit, 106 is a voice input/output unit corresponding to a microphone or earphone, 107 Is a voice processing unit, 108 is a communication processing unit, 109 is a communication input/output unit, 110 is a position detecting unit such as GPS (Global Positioning System), 111 is a sensor input/output unit, and 112 is various sensors.

The communication input/output unit 109 is connected to the network by wireless communication. In this embodiment, a case where the information display terminal 101 is directly connected to the Internet or the like to acquire information will be described. However, as a configuration in which the information terminal performs communication with another device such as an information terminal (smartphone or the like) by short-range wireless communication such as Bluetooth (registered trademark), and the information terminal performs connection with the Internet or the like and main processing. Good.

The information display terminal 101 is a head mounted display including at least the image pickup unit 103 and the display control unit 105. As the various sensors 112, a sensor such as an inclination sensor or an acceleration sensor that detects the user's body posture or movement, a sensor that detects the user's physical condition such as a line-of-sight sensor or a temperature sensor, or a user instruction input such as a pressure sensor or a capacitance sensor. It is possible to mount a plurality of sensors including those used as an input/output I/F for detecting the.

FIG. 2 is a sketch drawing showing an example when the user wears the information display terminal 101. The same components as those in FIG. 1 are designated by the same reference numerals.

Reference numeral 201 is a user, 202 is a schematic view of the area of the first image captured by the image capturing unit 103, and 203 is a schematic view of the area of the second image displayed by the display control unit 105. .. The area of the second image is not limited to the size or range indicated by 203, and may cover a wide field of view. Further, the form of the information display terminal 101 is capable of being imaged in the visual field direction of the user 201, and the display control unit 105 that is mounted on the head and displays information by projecting light according to the information into the visual field of the user 201, for example. What is necessary is just to arrange the imaging unit 103. For example, it may be a spectacle type, goggles type, one-eye type, two-eye type, etc., and is not limited to the shape shown as an example in FIG.

The first image of the area 202 captured by the image capturing unit 103 is analyzed by the image processing unit 104, and a predetermined object included therein is detected. Information indicating the detected characteristics of the object, for example, information indicating the characteristics related to the pattern described below is transmitted to the network through the communication processing unit 108, and the information about the object acquired from the network in response to the information is It is supplied to the image processing unit 104. The image processing unit 104 generates the second image based on the information and supplies the second image to the display control unit 105. The display control unit 105 displays the second image in the area 203.

The display control unit 105 displays an image on, for example, a half mirror provided at a position corresponding to the area 203, or directly projects and displays a second image on the retina of the user 201. In the case of a mold and the second image is not displayed, the area 203 does not hide the visual field of the user 201.

FIG. 3 is an operation flow diagram in the first embodiment. FIGS. 4A and 4B are diagrams showing an example of related information display in the first embodiment, showing a user's visual field.

The operation of the information display terminal 101 in this embodiment will be described in detail with reference to FIGS. 3, 4A, and 4B. In the present embodiment, a case will be described in which the information display terminal 101 displays the related information regarding a predetermined object as the second image, particularly when the user searches for a product in a store or the like.

First, in step 301, the process is started. The entire process may be started by touch input of a pressure sensor or the like in the sensor 112 or user's voice input from the voice input/output unit 106. Further, it may be automatically performed by recognizing that the user is in a specific store based on the position information detected by the position detection unit 110 or the image captured by the image capturing unit 103.

Next, in step 302, a predetermined object is detected. Specifically, the image processing unit 104 performs image recognition on an image continuously captured by the image capturing unit 103, and a plurality of clipped objects are compared with a predetermined pattern shape, color, or its deformation predicted value. A predetermined object is detected by matching or the like. The predetermined pattern may be a default pattern (for example, a genre such as "clothes") according to store information, or the pattern may be acquired from the Internet or the like in association with the store information.

In addition, when the user has decided on a desired product, the product name, model number, genre, etc. are designated in advance, and a predetermined pattern is acquired from a server on the Internet or other public information to detect a more specific predetermined object. It is also possible to do so. The predetermined object detected in step 302 is not limited to a single object, and a plurality of predetermined objects may be detected simultaneously in the imaging area.

When the predetermined object is detected, related information about the predetermined object is acquired in step 303. Specifically, the communication processing unit 108 acquires information about the predetermined object from the Internet or the like via the communication input/output unit 109. When the product name, model number, etc. can be determined by the product tag or the like from the image of the predetermined object imaged by the imaging unit 103, it is also possible to acquire the related information based on them.

Then, in step 304, the display trigger is determined. The display trigger is a specific object for starting the display of the second image based on the related information, and in this case, the "hand" is used as the display trigger. Even if the predetermined object is detected in step 302 and the related information of the predetermined object is acquired in step 303, unless the display trigger is captured by the image capturing unit 103 and recognized by the image processing unit 104, the second image of the second image is detected. Not displayed. The method of recognizing the “hand” by the image display unit 104 is the same as the method of recognizing the object in step 302.

For example, as shown in FIG. 4A, even if 402 to 405 are detected as the predetermined objects in the image pickup area 401, the second image is not displayed because the display trigger does not exist in the image pickup area 401. (No in step 304).

On the other hand, as shown in FIG. 4B, when the specific object 407 that is the display trigger is recognized in the imaging area 401 and it is determined that the object is close to the predetermined object (Yes in step 304 ), the specific object 407 is identified in step 305. The second image based on the related information of the predetermined object 402 is generated by the image processing unit 104. Further, the second image is displayed by the display control unit 105 in a form that can be recognized as information related to the predetermined object 402.

In this case, the content displayed as the second image 406 is product information regarding the predetermined object 402. Here, word-of-mouth information and price information about the product are displayed in a format that is easily recognized by the user. Note that the displayed content may be any information useful to the user, and is not particularly limited. Further, the broken line of the imaging region 401 is shown only for the sake of convenience, and is not displayed within the visual field of the user.

In step 306, the determination of the display trigger is continuously performed, and the related information is displayed while the display trigger is recognized. When the position of the display trigger is changed and the predetermined object is changed, the display image is switched to the second image regarding the changed predetermined object (Yes in step 306). When the specific object 407, which is a display trigger, cannot be recognized in the imaging region 401, the display of the second image is ended in step 307, and the process returns to the detection of the predetermined object in step 302 (step 306). No).

FIG. 5 is a block diagram showing a system configuration on the network. Acquisition of the related information via the communication processing unit 108 will be briefly described with reference to FIG.

The information display terminal 101 acquires the related information of the predetermined object 402 from the Internet 501 with the above-described configuration. The information is transmitted by an unspecified third party via the Internet 501 using, for example, the personal computer 502, the mobile terminal 503, the information display terminal 504, etc., and is stored in the storage 506 by the server 505, for example. The information display terminal 101 transmits information for identifying the predetermined object 402 detected from the image captured by the image capturing unit 103, and the server 505 provides information related to the predetermined object 402 among the information accumulated in the storage 506. It is provided to the display terminal 101.

The server 505 collects information about a predetermined store, a predetermined area, and the like in advance and provides the information to the information display terminal. On the contrary, when the specific server 505 does not collect related information, the information display terminal 101 searches for and acquires information related to the predetermined object 402 from the information disclosed by a plurality of servers connected to the Internet. You can also do it.

As described above, according to the first embodiment of the present invention, in the wearable information display terminal, regarding the related information of the predetermined object, the display start of the second image is started by using the display trigger of bringing the hand close to the object, for example. Alternatively, it ends. As a result, it is possible to display the second image at a timing and a period in which the user's intention or action (for example, picking up a product = wanting to see information related to a product) is reflected. The user can freely use both hands while wearing the information display terminal, and there is no need to operate the information display terminal 101 itself when selecting an object, which improves convenience. In addition, the second image is not displayed at unnecessary timings, or the information for a plurality of objects is not displayed at the same time, and thus it is possible to prevent the display from being bothersome to the user.

Next, a second embodiment of the information display terminal 101 will be described with reference to FIGS. 6 to 8B. Since the configuration of the information display terminal 101 is basically the same as that in the case of FIG. 1 described above, description will be given using the numbers given to the respective constituent elements of FIG. Similarly to the first embodiment, a case will be described in which the information display terminal 101 displays the related information about a predetermined object as the second image, especially when the user colors the product in the store or the like.

FIG. 6 is an operation flow chart in the second embodiment.

FIG. 7 is a diagram showing a display example when a display trigger is registered.

The feature of this embodiment is that the user can register the above-mentioned display trigger. FIG. 6 shows an operation flow when registering a display trigger, and FIG. 7 shows a field of view of a user when registering a display trigger.

First, in step 601, the process is started. The process can be started by a user input by touch input or voice input of a pressure sensor or the like. Next, in step 602, a frame for registering a display trigger is displayed. Specifically, the image processing unit 104 generates an image representing the registration frame, and the display control unit 105 displays the registration frame 702 in the range of the image capturing area 701 of the image capturing unit 103, that is, in the user's visual field direction as shown in FIG. 7. To do.

In step 603, the user positions the specific object to be registered as the display trigger so as to fit in the registration frame 702. In this case, the “palm” is the specific object 703. Subsequently, in step 604, when the image processing unit 104 recognizes that the specific object 703 exists within the range of the registration frame 702 from the image captured by the image capturing unit 103, the user determines whether to register the object as a display trigger. Contact The inquiry to the user may be made by displaying characters as shown in FIG. 7, or may be made by voice output from the voice input/output unit 106.

When the registration instruction is given by the user's instruction input (Yes in step 604), the specific object 703 is stored in the memory (not shown) in the information display terminal 101 as a display trigger. The memory may be provided so as to be accessible from the control unit 102 or the image processing unit 104, or may be built therein. Then, in step 606, the process ends. When the image processing unit 104 recognizes the existence of the specific object 703 in step 604, an image representing the outline of the specific object 703 is generated and displayed by the display control unit 105, and a voice is output from the voice input/output unit 106. It is also possible to notify the user that the specific object 703 has been recognized.

8A and 8B are diagrams illustrating an example of related information display according to the second embodiment. Here, as in the case of the first embodiment, in the case where the information display terminal 101 displays the related information about the predetermined object as the second image when the user searches for the product in the store or the like. This will be described with reference to FIGS. 8A and 8B. Here, it is assumed that “palm” is registered as the specific object 703 in step 605.

Since the operation flow is basically the same as that in the case of FIG. 3, detailed description will be omitted. In the determination of the display trigger in step 304 described above, the predetermined object 802 exists in the imaging area 801 as shown in FIG. 8A, and the related information is acquired from the Internet or the like, but the specific object 803 is the “back of the hand”. , Related information is not displayed because it is not recognized as a registered display trigger. The matching determination with the registered display trigger is performed by the image processing unit 104 by matching with the shape and color of the object or their predicted deformation values. In FIG. 8B, the specific object 804 is a “palm” and is determined as a registered display trigger in step 605 of FIG.

Further, in step 305, the related information of the predetermined object 802 specified by the display trigger is generated as the second image 805 by the image processing unit 104. Further, the second image 805 is displayed by the display control unit 105 in a form that can be recognized as information related to the predetermined object 802.

Similar to the above, the content displayed as the second image 805 may be any information useful for the user as product information regarding the predetermined object 802, and is not particularly limited. Further, the broken lines of the imaging regions 701 and 801 are shown only for convenience, and are not displayed within the user's visual field.

As described above, according to the second embodiment of the present invention, in the wearable information display terminal, by registering a specific object as a display trigger, display start of the second image according to the user's intention or The termination can be done. As in the case of the first embodiment, the user can freely use both hands while wearing the information display terminal, there is no need to operate the information display terminal 101 itself when selecting an object, and Since it is possible to change the display trigger as needed, the convenience is improved.

As a modified example of the present embodiment, an embodiment in which the processing start in step 301 of FIG. 3 is performed by the registered display trigger can be considered. That is, as the operation of the information display terminal 101, first in step 301, only the determination of the display trigger is continuously performed. Then, when it is determined from the image captured by the image capturing unit 103 that the object recognized by the image recognizing unit 104 matches the registered display trigger, the operation after step 302 is performed. In the example of the present embodiment, a series of operations can be performed by showing the “palm” in the imaging area 401. Since the operation can be started easily, there is an effect that the convenience of the user can be further improved.

Next, a third embodiment of the information display terminal 101 will be described with reference to FIGS. 9, 10A and 10B. Since the configuration of the information display terminal 101 is basically the same as that in the case of FIG. 1 described above, description will be given using the numbers given to the respective constituent elements of FIG. Also in the present embodiment, as in the above-described embodiments, the information display terminal 101 displays related information as a second image for a predetermined object, especially when the user is looking for a product in a store or the like. The case will be described.

FIG. 9 is an operation flow chart in the third embodiment.

FIG. 10A and FIG. 10B are diagrams showing an example of related information display in the third embodiment, and show the field of view of the user.

The operations of steps 901 to 903 are the same as the operations of steps 301 to 303 in FIG. 3 described above, and detailed description thereof will be omitted. When the processing is started in step 901, a predetermined object is detected in step 902, and in step 903, related information regarding the predetermined object is acquired from the Internet or the like. The predetermined object detected in step 902 is not limited to a single object, and a plurality of predetermined objects may be detected simultaneously in the imaging area.

Subsequently, in step 904, the line of sight of the user is determined. The gaze point of the user wearing the information display terminal 101 is detected by the gaze sensor included in the sensor 112. Regarding the line-of-sight sensor, for example, a method using infrared rays described in Patent Document 1 and a method using a CMOS (Complementary Metal-Oxide Semiconductor) sensor are known. When the gazing point is not within the range of the detected predetermined object (No in step 904), the second image related to the related information is not displayed. For example, as shown in FIG. 10A, even if 1002 to 1005 are respectively detected as the predetermined objects in the imaging region 1001, if the gazing point 1006 does not reach any one of the predetermined objects, the second information related to the related information is displayed. No image is displayed.

On the other hand, as shown in FIG. 10B, when the gazing point 1006 is on the predetermined object 1002 and the state continues for a predetermined time (for example, 3 seconds) or more (Yes in step 904), the predetermined object 1002 is determined in step 905. Related information of is generated as a second image 1007 by the image processing unit 104, and is displayed by the display control unit 105 in a form that can be recognized as information related to the predetermined object 1002. The content displayed as the second image 1007 is the product information regarding the predetermined object 1002 as in the above-described embodiment, and the word-of-mouth information and the price quote information regarding the product are displayed in a format that the user can easily recognize. ing.

Note that the displayed content may be any information useful to the user, and is not particularly limited. Further, the broken line of the imaging region 1001 and the mark of the gazing point 1006 are shown for convenience only, and are not displayed in the visual field of the user. However, with respect to the mark of the gazing point 1006, it is possible to provide a mode in which it is displayed as an image so that the user can recognize it. In that case, the shape of the mark is not limited to the shape of the gazing point 1006.

In step 906, the line-of-sight determination is continuously performed, and the second image 1007 is displayed while the gazing point 1006 is within the range of the predetermined object 1002 or the second image 1007 (Yes in step 906). This is because the gazing point 1006 moves to the second image 1007 side when the second image 1007 is displayed. If the gazing point 1006 is out of the range of the predetermined object 1002 or the second image 1007, or if it is out of the range and continues for a predetermined time (for example, 3 seconds) or more, the display of the second image 1007 is terminated (step (No in 906), the process returns to the detection of the predetermined object in step 902.

As described above, according to the third embodiment, in the wearable information display terminal 101, with respect to the related information of the predetermined object, by starting or ending the display of the second image according to the user's gaze point, the user can It is possible to display an image at a timing and a period in which the intention or action (for example, paying attention to a product = wanting to see information related to a product) is reflected. As in the case of the above-described embodiment, the user can freely use both hands while wearing the information display terminal 101, and it is not necessary to operate the information display terminal 101 itself when selecting an object, which is convenient. The property is improved. In addition, the second image is not displayed at unnecessary timings, or the information for a plurality of objects is not displayed at the same time, and thus it is possible to prevent the display from being bothersome to the user.

Next, a fourth embodiment of the information display terminal 101 will be described with reference to FIGS. 11, 12A and 12B. Since the configuration of the information display terminal 101 is basically the same as that in the case of FIG. 1 described above, description will be given using the numbers given to the respective constituent elements of FIG. Also in the present embodiment, as in the above-described embodiments, the information display terminal 101 displays related information as a second image for a predetermined object, especially when the user is looking for a product in a store or the like. The case will be described.

FIG. 11 is an operation flow chart in the fourth embodiment.

FIG. 12A and FIG. 12B are diagrams showing an example of related information display in the fourth embodiment, and show the field of view of the user.

The operations from step 1101 to step 1103 are the same as the operations from step 301 to step 303 in FIG. 3 described above, and detailed description thereof will be omitted. When the processing is started in step 1101, a predetermined object is detected in step 1102, and in step 1103 related information about the predetermined object is acquired from the Internet or the like. In the present embodiment, a case will be described in which a plurality of predetermined objects are simultaneously detected in the imaging area in step 1102.

When a plurality of predetermined objects are detected and the related information is acquired, candidate marks are displayed in step 1104. The candidate mark is a mark displayed when a valid one of the detected predetermined objects can be acquired as the related information. For example, FIG. 12A shows a case where 1202 to 1205 are detected as predetermined objects in the imaging region 1201 in step 1102, and valid related information about 1202, 1204, and 1205 can be acquired in step 1103. The candidate marks 1206 to 1208 are generated as an image by the image processing unit 104, and are displayed by the display control unit 105 so as to be superimposed on a part of the corresponding predetermined object. Note that the shape of the candidate mark is not limited to that shown in FIG. 12A as long as the candidate mark is large enough to be recognized by the user and it can be discriminated to which predetermined object it is related.

Then, in step 1105, the display trigger is determined. In this case, the "hand" is used as the display trigger as in the above-described embodiment. In FIG. 12A, since there is no display trigger in the imaging area 1201, only the candidate marks 1206, 1207, and 1208 are displayed, and the second image related to the related information of each predetermined object is not displayed.

On the other hand, as shown in FIG. 12B, when the specific object 1209, which is a display trigger, is recognized in the imaging area 1201 and is determined to be close to the predetermined object 1202, the predetermined object 1202 specified by the display trigger is determined in step 1106. Related information of is generated as a second image 1210 by the image processing unit 104, and is displayed by the display control unit 105 as a substitute for the candidate mark 1206.

In this case, the predetermined object 1202 is close to the other objects 1203 to 1205. Therefore, the second image 1210 is displayed so as to overlap a part of the predetermined object 1202. This makes it clear which object the second image 1210 relates to. In addition, also in other embodiments, in order to clarify which object the second image relates to, it is preferable to display the second image such that a part of the second image overlaps the related object. ..

The content displayed as the second image 1210 is the product information regarding the predetermined object 1202, as in the above-described embodiment, and the word-of-mouth information and the price quote information regarding the product are displayed in a format that the user can easily recognize. ing. Note that the displayed content may be any information useful to the user, and is not particularly limited. Further, the candidate marks 1207 and 1208 are continuously displayed as the candidate marks because the corresponding object is not selected. Note that the broken line of the imaging region 1201 is shown only for the sake of convenience, and is not displayed within the user's visual field.

In step 1107, the determination of the display trigger is continuously performed, and the related information is displayed while the display trigger is recognized (Yes in step 1107). When the position of the display trigger is changed and the predetermined object near the display trigger is changed, the candidate mark of the changed predetermined object is switched to the display of the second image related to the related information. For the predetermined object before the change, the second image related to the related information is switched to the candidate mark display again. When the specific object 1209 which is the display trigger cannot be recognized in the imaging area 1201 (No in step 1107), the second image display related to the related information is ended in step 1108, and the predetermined object of step 1102 is displayed. Return to detection. However, the candidate marks already displayed in the image pickup area 1201 continue to be displayed.

As described above, according to the fourth embodiment, by displaying the candidate mark on the wearable information display terminal 101 prior to displaying the related information of the predetermined object, the user can select which object can display the related information in advance. It becomes possible to recognize what. With regard to the related information itself, the display trigger is used to start or end the display of the second image, so that it is possible to display the second image at a timing and period that reflects the user's intention or action. The user can freely use both hands while wearing the information display terminal 101, and can display the second image while recognizing which object the related information can be displayed. Furthermore, there is no need to operate the information display terminal 101 itself when selecting an object, which has the effect of improving convenience.

As a modified example of this embodiment, an embodiment in which the sizes or colors of the candidate marks 1206 to 1208 in FIG. 12A are made different according to the validity of the content of the relevant information concerned can be considered. Since each related information has been acquired at the time of displaying the candidate mark, for example, when the amount of additional information is large, or when the product is specified by the user in advance, the display mark is enlarged or the red frame is displayed. To do. This makes it easier for the user to recognize in advance.

In the present embodiment, the operation of displaying the candidate mark has been described as being combined with the first embodiment, but the operation may be combined with other embodiments.

In all of the embodiments described above, in the case where the information display terminal displays the related information as the second image for a predetermined object, especially when the user is looking for the product in the store, particularly in the clothing store. Has been explained. However, the application scene is not limited to this, and the invention can be applied to stores in other categories, information display outdoors, and the like. Further, although the case where the “hand” is mainly used as the display trigger has been described, other ones can be designated and used as the display trigger.

Further, the present invention is not limited to the above-described embodiment, and various modifications are included. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Further, the above-mentioned respective functions, processing units, etc. may be realized partially or entirely by hardware, or may be realized by software by the processor interpreting and executing a program for realizing each function. Good. It should be noted that the control lines and information lines shown are those that are considered necessary for explanation, and not all the control lines and information lines are shown.

101, 504... Information display terminal, 102... Control section, 103... Imaging section, 104... Image processing section, 105... Display control section, 106... Voice input/output section, 107... Voice processing section, 108... Communication processing section, 109 ... communication input/output unit, 110... position detection unit, 111... sensor input/output unit, 112... various sensors, 201... user, 202, 401, 701, 801, 1001, 1201... imaging region, 203... image region, 402, 802, 1002, 1202... Predetermined object, 406, 805, 1007, 1210... Related information image, 407, 804, 1209... Display trigger, 501... Internet, 502... Personal computer, 503... Mobile terminal, 505... Server, 506 ... Storage, 702... Registration frame, 1006... Gaze point, 1206, 1207, 1208... Candidate mark.

Claims (4)

An information display terminal for displaying information in the direction of the user's field of view,
A communication processing unit that performs communication for acquiring information from the network,
An image pickup unit for picking up the visual field direction,
An image processing unit that detects a first object from a first image captured by the image capturing unit and generates related information of the first object acquired from the network via the communication processing unit as a second image. When,
A display controller that displays the second image in the viewing direction;
A line-of-sight sensor that detects the gazing point of the user,
A control unit for controlling the operation of the components of the information display terminal,
The control unit is
Causing the image processing unit to generate a predetermined mark image when the related information of the first object is acquired from the network,
Controlling the display control unit to superimpose and display the predetermined mark image on a part of the first object from which the related information is acquired;
It is determined whether the gazing point of the user detected by the line-of-sight sensor has been present for a predetermined time or more at a position overlapping the first object,
If it is determined that it exists, the display control unit is controlled so as to display the second image instead of the predetermined mark image that is superimposed and displayed on a part of the first object,
An information display terminal, characterized in that the display control unit is controlled so that the display of the second image is continued while the gazing point of the user exists in a range overlapping with the second image. The information display terminal according to claim 1,
The control unit is
The information display terminal, wherein the display control unit controls the display of the second image so as to be superimposed on a part of the first object. A method for displaying information in the direction of the user's field of view,
Image the viewing direction,
Detecting the first object from the first image obtained by imaging,
Acquiring related information about the first object from the network,
Generating a predetermined mark image when the related information of the first object is acquired from the network,
The predetermined mark image is displayed by being superimposed on a part of the first object from which the related information is acquired,
Generate a second image relating to the relevant information,
Detecting the user's gazing point,
It is determined whether or not the user's gazing point exists at a position overlapping the first object for a predetermined time or more,
When it is determined that the second image is present, the second image is displayed in place of the predetermined mark image that is superimposed and displayed on a part of the first object,
The information display method, wherein the display of the second image is continued while the gazing point of the user exists in a range overlapping with the second image. In the information display method according to claim 3,
The information display method, wherein the display of the second image is performed by being superimposed on a part of the first object.

Images