JP2006153630A - Information terminal device and information display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information terminal device capable of effectively providing information to a user. <P>SOLUTION: The information terminal device is provided with an image display 50, an image display controller for indicating information on the image display 50, a combiner 60 transmitting an actual view field, projecting information indicated on the image display 50 as optically virtual image and indicating overlapping in the actual view field, and a mode detector for detecting through the combiner 60 whether the present mode is a direct view mode for directly providing information indicated on the image display 50 or the virtual image mode for providing information indicated on the image display 50 as the virtual image. The image display controller indicates different information on the image display according to the detection result of the mode detector. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an information terminal device that displays predetermined information as a virtual image in a real field of view.

2. Description of the Related Art Conventionally, information terminal devices that have increased added value by providing more information to users have been developed.
For example, Japanese Unexamined Patent Application Publication No. 2003-130678 (Patent Document 1) discloses a navigation technique that provides a user with an optimum route to a destination.
In the above-mentioned Patent Document 1, when information on a destination is received from an information terminal carried by the user, an optimum route is searched based on the current position of the information terminal and the position information of the destination, and the search result is stored in the mobile terminal. A technique for delivering to is disclosed.
Japanese Patent Laid-Open No. 2002-278583 (page 1-3, FIG. 1-4)

In the technique disclosed in Patent Document 1, map information for guiding a user to a destination is displayed on a display screen of an information terminal carried by the user. Thereby, the user can reach the destination while alternately checking the map information displayed on the display screen and the surrounding situation (for example, a road or a building).
On the other hand, depending on the content of the information, if the information can be displayed superimposed on the surrounding situation such as a landscape, the information may be easily discriminated.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide an information terminal device that can provide information to a user more effectively.

In order to achieve the above object, the present invention provides the following means.
The present invention provides an image display unit, an image display control unit that displays information on the image display unit, and light transmitted from a real field of view in a user's line-of-sight direction, and information displayed on the image display unit. A semi-transparent optical element that is displayed as an optical virtual image in a superimposed manner in the real field of view, and the current mode is a direct-view mode that directly provides information displayed on the image display unit, or the half-view A mode detection unit that detects whether the virtual display mode provides the information displayed on the image display unit as a virtual image through a transmission optical element, and the image display control unit is configured to detect the mode detection unit according to a detection result of the mode detection unit. Provided is an information terminal device that varies the information displayed on the image display unit.

According to the above configuration, in the direct view mode, the information displayed on the image display unit is directly provided to the user. On the other hand, in the virtual vision mode, the information displayed on the image display unit is optically superimposed on the information in the real field of view transmitted through the semi-transmissive optical element by the semi-transmissive optical element and provided to the user. Is done.
Whether the current mode is the direct viewing mode or the virtual viewing mode is detected by the mode detection unit. Then, information according to the detection result of the mode detection unit is input to the image display control unit, and information according to the mode, in other words, information that is different between the direct viewing mode and the virtual vision mode is displayed on the image display. The image is displayed on the image display unit by the control unit.
Thus, since the image to be displayed on the image display unit varies depending on the mode, in the case of the direct view mode, for example, the user is provided with useful information as long as it can be displayed on the image display unit, In the virtual vision mode, it is possible to display optimal information using information from the outside world as a virtual image.

  In the information terminal device of the present invention, the positional relationship between the image display unit and the semi-transmissive optical element differs depending on the mode, and the mode detection unit is configured to have a positional relationship between the image display unit and the semi-transmissive optical element. Depending on the current mode, the current mode may be detected.

According to the above configuration, the positional relationship between the image display unit and the semi-transmissive optical element is different between the direct viewing mode and the virtual vision mode. It is possible to easily detect the current mode.
For example, in the direct view mode, the image display unit and the semi-transmissive optical element are arranged so as to overlap each other, so that information displayed on the image display unit is provided via the semi-transmissive optical element superimposed on the image display unit. Is done. On the other hand, in the virtual vision mode, the image display unit and the semi-transmissive optical element are opened at a predetermined angle with the image display unit and the main body on which the semi-transmissive optical element is supported, and are disposed substantially opposite to each other. The information displayed on the image display unit is reflected by the transflective optical element and provided as a virtual image. Therefore, in this case, the mode detection unit can detect the current mode by detecting whether or not the image display unit and the semi-transmissive optical element are overlapped. .

  The information terminal device of the present invention may include an operation unit for switching the mode, and the mode detection unit may detect the mode based on a signal from the operation unit.

  According to the above configuration, switching from the virtual vision mode to the direct vision mode or switching from the direct vision mode to the virtual vision mode is performed by operating the operation unit. A predetermined signal is output. Therefore, the current mode can be easily detected by detecting the signal output from the operation unit by the mode detection unit.

  The information terminal device of the present invention may include a posture detection unit that detects the posture of the main body, and the mode detection unit may detect the current mode according to the detection result of the posture detection unit.

According to the above configuration, the posture of the main body is detected by the posture detection unit, and the detection result is input to the mode detection unit. Since the posture of the main body is different between the direct-view mode and the virtual-vision mode, the mode detection unit can easily detect the current mode based on the detection result of the posture detection unit.
The posture detection unit is, for example, a gyro sensor.

  The information terminal device of the present invention includes an information acquisition unit that acquires information in an actual visual field via the semi-transmissive optical element in the virtual vision mode, and the image display control unit is acquired by the information acquisition unit. Information related to the information in the actual visual field may be displayed on the image display unit.

According to the above configuration, information in the real field of view through the transflective optical element is acquired by the information acquisition unit. For example, in an art museum or the like, an exhibit existing in a real field of view is acquired as information by the information acquisition unit. Further, for example, when moving in an urban area, a building or a house existing in the real field of view is acquired as information.
Information in the real field of view acquired by the information acquisition unit is input to the image display control unit. Then, the image display control unit displays information related to the information in the actual visual field on the image display unit. For example, if the information in the real field of view is an exhibit, the information related to the exhibit is the information related to the building, if the information in the actual field of view is a building, for example, the name of the building and the building The tenant and the like are displayed on the image display unit.
This makes it possible to superimpose information such as the name of the exhibit and the author as a virtual image on information existing in the real field of view, such as an exhibit.

  In the information terminal device of the present invention, the information acquisition unit captures an object that is present in the real field of view based on an imaging unit that captures a landscape in the direction of the real field of view and an image acquired by the imaging unit. An object specifying unit for specifying, and in the virtual vision mode, the image display control unit is configured such that information about the object appears as a virtual image in the vicinity of the target specified by the target specifying unit. The information related to the object may be displayed on the image display unit.

According to the above configuration, a landscape in the direction of the real field of view visually recognized through the semi-transmissive optical element is imaged by the imaging unit. And the target object which exists in a real visual field is specified by the target object specific | specification part based on the image acquired by the imaging part. In the virtual vision mode, the image display control unit displays information related to the object on the image display unit so that the information related to the object existing in the real field of view appears as a virtual image in the vicinity of the object. Is done.
This makes it possible to superimpose information on the target object as a virtual image in the vicinity of the target object existing in the real field of view, so that it is possible to provide information to the user more easily.

  The information terminal device of the present invention includes an input unit for inputting a destination, a position information acquisition unit for acquiring position information of the current location, a direction detection unit for detecting the direction of the real field of view, and the destination from the current location. A route acquisition unit that acquires a route to the destination, and a traveling direction specifying unit that specifies a traveling direction to the destination based on the location information of the current location, the direction of the real field of view, and the route to the destination In the virtual vision mode, the image display control unit may display information indicating the traveling direction on the image display unit.

According to the above configuration, for example, the destination is input by operating the input unit by a user or the like. The location information of the current location is acquired by the location information acquisition unit. The route from the current location to the destination is acquired by the route acquisition unit. Then, based on the position information of the current location, the direction of the real field of view, and the route to the destination, the traveling direction determination unit identifies the traveling direction to the destination. In the virtual vision mode, the traveling direction is displayed on the image display unit by the image display control unit. As a result, it is possible to superimpose the traveling direction as a virtual image on the actual landscape, so that it is very easy to understand and effectively guide the user to the destination.
The position information acquisition unit is, for example, a GPS (Global Positioning System).
The route acquisition unit may include, for example, map information, and may be configured to acquire a route to a destination based on the map information, or for route guidance provided outside. A configuration in which information on the current position and the destination is transmitted to the server, and a route is searched on the server, and a route to the destination is acquired by receiving a search result from the server. It may be.

  The present invention provides an image display unit, an image display control unit that displays information on the image display unit, and transmits light from a real field of view in a user's line-of-sight direction, and optically displays information displayed on the image display unit. A semi-transmissive optical element that is superimposed and displayed as a virtual image in the real field, and the current mode is a direct-view mode that directly provides information displayed on the image display unit, or the semi-transmissive optical element through the semi-transmissive optical element. An information terminal device having a mode detection unit that detects whether the information displayed on the image display unit is a virtual vision mode that provides a virtual image, information associated with the direct viewing mode, and a virtual vision mode An information storage unit for storing information, and in the direct viewing mode, displays information associated with the direct viewing mode on the image display unit, and in the virtual viewing mode, Providing an information display system for superimposing displaying the in actual field information associated with the said imaginary vision mode displayed on the image display unit as an optically virtual image via an over-optical element.

According to the above configuration, the information associated with the direct viewing mode and the information associated with the virtual vision mode are stored in the information storage unit, and the information terminal device, for example, displays information corresponding to the mode as information. The information acquired from the storage unit is displayed on the image display unit. Thereby, in the case of the direct viewing mode, information suitable for the direct viewing mode displayed on the image display unit is directly provided to the user, and in the case of the virtual viewing mode, the virtual vision displayed on the image display unit is provided. Information suitable for the mode is provided as a virtual image through the transflective optical element.
The information storage unit may be, for example, an external storage device that is directly electrically connected to an input terminal such as a USB terminal included in the information terminal device, and is connected to the information terminal device through an electric line such as a LAN (Local Area Network). The server may be provided.

  According to the information terminal device of the present invention, the information to be displayed on the image display unit varies depending on whether the mode is the direct view mode or the virtual vision mode, so that it is possible to provide information to the user more effectively. Play.

Hereinafter, an embodiment of an information terminal device according to the present invention will be described in the order of [First Embodiment] and [Second Embodiment] with reference to the drawings.
[First Embodiment]
The information terminal device according to the first embodiment of the present invention functions as a navigator system that provides route guidance from the current location to the destination. Details will be described below.
1 and 2 are external perspective views showing a configuration example of the information terminal device according to the first embodiment of the present invention.
As shown in FIGS. 1 and 2, an information terminal device 10 according to the present embodiment includes a display device 30, an imaging device (outside) outside a housing (main body) 20 containing various control mechanisms and a control board described later. An imaging unit) 40 and an operation unit 70 are provided.
The display device 30 includes an image display unit 50 and a combiner 60. The image display unit 50 and the combiner 60 are housed in a space part of the opening 21 that penetrates the housing 20 (see FIG. 2) in a retracted state, and each side of the short side that is rectangular is housing 20. Opening and closing operation by swinging is enabled.

  The image display unit 50 includes, for example, a so-called thin display (hereinafter referred to as “display surface”) 51 such as a liquid crystal display panel (LCD) or an organic EL display panel, and an image display control unit 75 (see FIG. 4) described later. The output information is displayed on the display surface 51. The image display unit 50 is fixed to, for example, a support member 52 that is pivotally supported by the housing 20, and the display surface 51 is a surface that faces the combiner 60 described later in a stored state, in other words, the housing 20 is directed to the inside.

The combiner 60 reflects the light from the image display unit 50, that is, the light of information displayed on the display surface 51 of the image display unit 50, and transmits the light from the real visual field in the user's line-of-sight direction. It is an optical element. In other words, the combiner 60 has a free-form surface (three-dimensional aspheric surface) on at least one surface, transmits an external field image that is a real field of view, and displays information (for example, for example, on the display surface 51 of the image display unit 50). This is a transflective optical element that superimposes and displays an image or text as a virtual image on a real image in a real field of view.
In the present embodiment, the combiner 60 is a substantially rectangular plate-like member made of a transparent material such as acrylic or polycarbonate, and a reflective layer such as a metal film is formed on one side thereof. The surface of the reflective layer becomes a reflective surface 61 that reflects the light incident from the image display unit 50 on the surface.
In addition to the above-described transflective optical element, for example, a half mirror, a hologram element, or the like can be adopted as the combiner 60, and one of these is appropriately selected and used according to the purpose. It is possible.

In the direct view mode according to the present embodiment, as shown in FIG. 2, the image display unit 50 and the combiner 60 are accommodated in the opening 21, and therefore, the image display unit 50 and the combiner 60 are overlapped. The information displayed on the image display unit 50 is provided directly to the user via the combiner 60.
On the other hand, in the virtual vision mode according to the present embodiment, as shown in FIG. 1, the reflecting surface 61 of the combiner 60 and the display surface 51 of the image display unit 50 are substantially opposed to each other and are substantially Z-shaped together with the housing 20. Open to form a shape.
And as shown in FIG. 3, the side surface 22 used as the rear-end part side of the housing | casing 20 is hold | maintained so that it may press against a user's temporal region. In this state, the information light displayed on the display surface 51 of the image display unit 50 is emitted obliquely forward through the opening 21, and reflected toward the rear end side of the housing 20 by the reflection surface 61. The Thereby, the information displayed on the image display unit 50 is visually recognized as a virtual image in front of the combiner 60, that is, farther than the combiner 60 as viewed from the user.

The imaging device 40 mainly functions as a part of an information acquisition unit that acquires information in the actual visual field via the combiner 60. For this reason, as shown in FIG. 3, the imaging device 40 is provided at a position where an actual visual field landscape can be captured via the combiner 60 that spreads in the user's line-of-sight direction in the above-described virtual vision mode. In the virtual vision mode, the imaging device 40 captures a landscape in the direction of the user's line of sight and outputs the acquired image to a processing unit 86 (see FIG. 4) described later.
As the imaging device 40, for example, an analog imaging device using a photosensitive member such as a photosensitive film may be used, or a digital imaging device using an imaging device such as a CCD element may be used. In the present embodiment, a digital imaging device is used as the imaging device 40.
The operation unit 70 functions as a man interface (input unit) including a plurality of numeric keys, buttons, and the like, and outputs a signal according to the operation content by the user to a processing unit 86 (see FIG. 4) described later.

Next, an electrical configuration built in the housing 20 will be described with reference to FIG.
FIG. 4 is a block diagram showing an outline of the electrical configuration of the information terminal device according to the present embodiment. In this figure, the same components as those in FIGS. 1 to 3 are denoted by the same reference numerals.
As shown in FIG. 4, the housing 20 includes a GPS receiving unit 81, a direction detecting unit 82, a communication unit 83, a mode detecting unit 84, a storage unit 85, a processing unit 86, an image display control unit 87, and the like. ing. Each of these units is electrically connected to the processing unit 86.
The GPS receiver 71 functions as a location information acquisition unit that acquires location information of the current location. The GPS receiving unit 71 acquires position information of the current location from GPS satellites, and outputs the acquired position result to the processing unit 86.
For example, the direction detection unit 72 detects the direction of the real field of view through the combiner 60 in the virtual vision mode, and outputs the detected direction to the processing unit 86.
The communication unit 73 communicates with an external server based on an instruction from the processing unit 86 and outputs information acquired from the server to the processing unit 86. The communication unit 73 has a function as a route acquisition unit of the present invention that acquires a route from the current location to the destination.
The mode detection unit 74 detects the positional relationship between the image display unit 50 and the combiner 60 and outputs the detection result to the processing unit 86. In the information terminal device according to the present embodiment, in the direct view mode, as illustrated in FIG. 2, the image display unit 50 and the combiner 60 are arranged to overlap each other. On the other hand, in the case of the virtual vision mode, as shown in FIG. 1, the image display unit 50 and the combiner 60 are substantially opposed to each other and are opened so as to form a substantially Z shape together with the housing 20.
Therefore, the mode detection unit 74 is configured by, for example, a switch that mechanically detects the state when each of the image display unit 50 and the combiner 60 is opened at a predetermined angle or more with respect to the housing 20. Different signals are output to the processing unit 86 when the positional relationship between the image display unit 50 and the combiner 60 is in the state shown in FIG.

In the storage unit 85, various image templates and character templates are registered in advance, and information acquired from an external server via the communication unit 83 is temporarily stored.
The processing unit 86 executes various types of processing based on various types of information input from the above-described units, and causes the image display unit 50 to display information corresponding to the various types of processing results. For example, the processing unit 86 includes a CPU (Central Processing Unit) (not shown), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. A series of processing procedures for realizing various functions to be described later (for example, a traveling direction specifying function) is recorded in a ROM or the like in the form of a program, and the CPU reads the program into the RAM or the like to process and process information. Various functions described later are realized by executing the arithmetic processing.

The operation of the information terminal device 1 according to this embodiment configured as described above will be described.
First, when a destination is input by operating the operation unit 70 by the user, the operation unit 70 outputs an electrical signal corresponding to the destination to the processing unit 86. When detecting that the destination has been input (step SA1 in FIG. 5), the processing unit 86 activates the GPS receiving unit 81 (step SA2). Thereby, the position information of the current location is acquired from the GPS satellite by the GPS receiving unit 81, and this position information is input to the processing unit 86 (step SA3).

  Subsequently, the processing unit 86 sends the destination information input from the operation unit 70 and the current position information acquired from the GPS receiving unit 81 to the route search server installed outside via the communication unit 83. Send. Thereby, the route to the destination, the map information to the destination, and the like are searched by the route search server, and these pieces of information are transmitted from the route search server to the communication unit 83. A route search server having a route search function and a map function is disclosed in, for example, Japanese Patent Application Laid-Open No. 2003-130678, and can be easily realized by using these well-known techniques.

  The communication unit 83 outputs the received route to the destination and map information to the destination to the processing unit 86. When the processing unit 86 acquires the route to the destination and the map information to the destination, the processing unit 86 stores this in the storage unit 85 (step SA4).

  Subsequently, the processing unit 86 acquires information on the mode detected by the mode detection unit 84, that is, a signal corresponding to the current mode, and based on this signal, determines whether the current mode is the direct-view mode or the false mode. It is determined whether the viewing mode is selected (step SA5). As a result, when the current mode is the direct view mode, the processing unit 86 acquires the position information of the current location from the GPS receiving unit 81, the route information from the storage unit 85 to the destination, and the vicinity of the current location. Get map information.

Based on these pieces of information, an image in which the route to the destination is superimposed on the map information near the current location is created and output to the image display control unit 87 (step SA6).
The image display control unit 87 causes the image display unit 50 to display the map information indicating the route to the destination input from the processing unit 86 (step SA7). Thereby, for example, map information as shown in FIG. 6 is displayed on the image display unit 50, and this map information is directly displayed to the user via the combiner 60 arranged so as to be superimposed on the image display unit 50. Will be visually recognized.

  On the other hand, in the above-described step SA5, when the current mode is the virtual vision mode, the processing unit 86 acquires the position information of the current location from the GPS receiving unit 81 and from the direction detecting unit 82 via the combiner 60. Get the direction of the real field of view. Further, the processing unit 86 acquires route information from the storage unit 85 to the destination and map information near the current location.

Then, based on the position information of the current value, the direction of the real field of view, and the route information to the destination, the traveling direction, in other words, the direction in which the user should proceed at the present time is specified (step SA10). Then, the processing unit 86 acquires an image of an arrow corresponding to the direction from the storage unit 85 and outputs the acquired image to the image display control unit 87.
The image display control unit 87 displays the arrow input from the processing unit 86 on the image display unit 50 (step SA9). Thereby, for example, as shown in FIG. 7, an arrow is displayed on the image display unit 50. And the information (refer FIG. 9) superimposed on this arrow as a virtual image in the real visual field through the combiner 60 as shown in FIG. 8 will be visually recognized by the user.

Subsequently, as described above, when display according to the current mode is performed, the processing unit 86 determines whether or not the destination has been reached (step SA10). This is determined based on whether or not the current location position information acquired by the GPS receiver 81 and the destination location information substantially match.
As a result, if it is determined that the destination has not been reached, the processing from step SA3 to step SA10 is repeated until it is determined in step SA10 that the destination has been reached.

As described above, according to the information terminal device according to the present embodiment, in the direct view mode, for example, as shown in FIG. 6, a map near the current location showing the route to the destination is displayed. In the case of the visual mode, for example, as shown in FIG. 9, an external image that the user visually recognizes through the combiner 60, that is, a state around the current location is displayed. The arrow indicating the route to the destination is superimposed and displayed as a virtual image.
As described above, since the image displayed on the image display unit 50 is different depending on the mode, in the case of the direct viewing mode, for example, the user can provide useful information as long as it can be displayed on the image display unit 50, In other words, detailed information that cannot be obtained in the virtual vision mode is displayed, and in the virtual vision mode, the route is displayed in the information of the outside world, so that various information such as the route is provided to the user in an easily understandable manner. be able to.

[Second Embodiment]
Next, an information terminal device according to the second embodiment of the present invention will be described. Note that the mechanical structure and the electrical structure are the same as those of the information terminal device 10 according to the first embodiment.
In the present embodiment, the content of the processing performed by the processing unit 72 is different from that in the first embodiment described above. That is, in the first embodiment described above, guidance to the destination is performed based only on the current position information received by the GPS receiver 81.
In the present embodiment, more detailed road guidance is provided by taking in the information of the outside world that spreads in the direction of the real field of view captured by the imaging device 40 (see FIG. 1) and using this information to guide to the destination. make it happen.
Details of the information terminal device according to the present embodiment will be described below with reference to the drawings. Note that details of the processing contents similar to those in the first embodiment described above are omitted.

  First, as in the first embodiment described above, when a destination is input by operating the operation unit 70 by the user (step SB1 in FIG. 10), the processing unit 86 includes the GPS receiving unit 81. The location information of the current location is acquired from the location information, and the location information of the current location and the destination are transmitted via the communication unit 83 to a route search server installed outside. The route search server searches for a route to the destination, extracts map information to the destination, and further displays detailed images around the destination (hereinafter referred to as “detailed peripheral images”), building arrangements, Detailed peripheral information such as building information is acquired, and the detailed peripheral information is transmitted to the information terminal device. The detailed peripheral information is, for example, image information used for image matching described later, and is used for acquiring more accurate position information by using this image information.

These pieces of information transmitted from the route search server are input to the processing unit 86 via the communication unit 83. The processing unit 86 stores these pieces of information in the storage unit 85 (step SB2).
Subsequently, the processing unit 86 calculates the distance to the destination based on the route to the destination, the current location information, and the destination location information (step SB3). Subsequently, it is determined whether or not this distance is equal to or less than a predetermined reference distance (for example, 50 m) set in advance (step SB4).
As a result, if the distance to the destination exceeds the reference distance (“NO” in step SB4), that is, if the user has not reached the periphery of the destination, the first embodiment described above is used. Similar processing, that is, processing from step SA3 to step SA10 in FIG. 5 is performed, and the process returns to step SB3.

  On the other hand, if the distance to the destination is within the reference distance (“YES” in step SB4), that is, if the user has reached the periphery of the destination, the position information based on the current position by GPS In addition to acquisition, more accurate position information is acquired by specifying the current location by image matching. That is, in this case, the processing unit 86 extracts the detailed peripheral image of the position information from the storage unit 85 based on the position information of the current location acquired by the GPS receiving unit 81, and the extracted detailed peripheral image And the image of the real field of view output from the imaging device 40 are subjected to image matching, thereby specifying the current position information more accurately (step SB6).

If the current mode is the direct view mode, the processing unit 86 extracts a peripheral map based on the position information specified in step SB6 from the storage unit 85, creates an image showing a route on the peripheral map, This is output to the image display control unit 87.
On the other hand, if the current mode is the virtual vision mode, the processing unit 86, based on the current position information acquired in step SB6, the direction of the real field of view acquired from the direction detection unit 82, and the position information of the destination, The traveling direction is specified, an arrow based on the traveling direction is acquired from the storage unit 85, and this is output to the image display control unit 87.

The image display control unit 87 displays the image input from the processing unit 86 on the image display unit 50 (step SB7). As a result, it is possible to provide the user with information as shown in FIG. 6 in the direct viewing mode and with information as shown in FIG. 9 in the virtual vision mode.
Subsequently, as described above, when display according to the current mode is performed, the processing unit 86 determines whether or not the destination has been reached (step SB8). As a result, if it is determined that the destination has not been reached, the processing from step SB3 to step SB8 is repeated until it is determined in step SB8 that the destination has been reached.

  As described above, according to the information terminal device according to the present embodiment, when the vicinity of the destination is reached, not only the positional information by GPS but also the image of the real field of view acquired by the imaging device 40, Since the current position information is specified by performing image matching with the detailed peripheral image around the destination, navigation can be performed based on more accurate position information. Thereby, highly accurate navigation can be realized.

In the present embodiment, the detailed peripheral image used for image matching only needs to be stored in the storage unit 85 until the user reaches the periphery of the destination, and the acquisition timing within that period is not particularly limited. .
Further, when the user reaches the vicinity of the destination, the detailed information of the destination is displayed on the image display unit 50 in addition to the map information or instead of the map information in the direct view mode. good.
For example, when the destination is a tenant in a building, information such as building entrance information, elevator information, and the number of floors of the tenant may be displayed on the image display unit 50. In this way, it is possible to provide more detailed information to the user by enhancing the information.

In addition, when the user has reached the vicinity of the destination, that is, when the user is in the immediate vicinity of the destination, the route guidance by the direct view mode, that is, the route is shown on the surrounding map and presented. In some cases, it is preferable to present the direction of guidance in the virtual mode, that is, the direction of travel is indicated by an arrow. In order to deal with such a case, when the distance to the destination is within a predetermined distance and the current mode is the direct view mode, the processing unit 86 makes an imaginary to the image display control unit 87. Information for instructing switching to the viewing mode may be output.
As a result, a message instructing to switch to the virtual mode is displayed on the image display unit 50, so that the user who has confirmed this message can grasp that the user is approaching the destination. By switching to the virtual vision mode according to the message, it is possible to be guided by more preferable information and reach the destination.

[Third Embodiment]
Next, an information terminal device according to a third embodiment of the present invention will be described. Note that the mechanical structure and the electrical structure are the same as those of the information terminal device 10 according to the first embodiment.
In this embodiment, position information is not acquired using GPS, but position information is specified using the image matching demonstrated in 2nd Embodiment mentioned above.
In the present embodiment, as an example, a case where a user standing at the entrance of a building is guided to a desired tenant in the building will be described.

In this case, for example, as shown in FIG. 11, the floor map in the building is divided into a plurality of blocks in advance, and each block is associated with the position information and registered in the route search server.
The processing unit 86 of the information terminal device 10 transmits information on the current location (here, information on the entrance of the building) and information on the destination to the route search server via the communication unit 83. The route search server extracts four surrounding blocks (see FIG. 11) around the current location based on the received current location information, searches for the route to the destination, and uses the searched route as the information terminal device. 10 to send.
When the processing unit 86 of the information terminal device 10 acquires the image information (blocks indicated as “DL” in FIG. 11) and the route information to the destination via the communication unit 83, Information is stored in the storage unit 85.

Subsequently, the processing unit 86 acquires information on the real field of view captured by the imaging device 40, and based on the image of the real field of view and the image information of the four surrounding blocks stored in the storage unit 85, Identify. Then, the traveling direction is identified based on the identified current position, the direction of the real field of view acquired by the direction acquisition unit 82, and the route information to the destination stored in the storage unit 85, and according to the traveling direction. The image display control unit 87 causes the image display control unit 87 to display the displayed arrow.
When the user moves and the block to which the current position belongs moves as shown in FIG. 12, the processing unit 86 has information about four blocks around the moved block (FIG. 12). (Blocks indicated as “DL” in FIG. 12) from the server for route search and information of the four blocks acquired previously stored in the storage unit 85 (blocks indicated as “erase” in FIG. 12) Is deleted. Then, by performing such processing, guidance is provided to the destination, that is, the tenant in the building.

Note that the processing unit 86 obtains information on the entire 8 blocks around the current location block from the route search server, as shown in FIGS. 13 and 14, instead of the above-described 4 surrounding blocks. Also good.
As described above, according to the information terminal device according to the present embodiment, the current location is specified by image matching. Therefore, the road within a limited relatively narrow range such as a guide in a building or a guide in a station. Even if guidance is necessary and the position information by GPS has low resolution and effective route guidance is difficult, it is possible to perform route guidance with sufficient accuracy.

[Fourth Embodiment]
The information terminal device according to the fourth embodiment of the present invention functions as a work navigator system that provides detailed explanation of exhibits in a museum or the like. The mechanical structure of the information terminal device according to the present embodiment is the same as that of the information terminal device 10 according to the first embodiment, but the electrical structure is different. In addition, about the component similar to 1st Embodiment mentioned above, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

FIG. 15 is a block diagram showing an outline of the electrical configuration of the information terminal device according to the present embodiment.
Similar to the first embodiment described above, the information terminal device according to the present embodiment includes the imaging device 40, the operation unit 70, the mode detection unit 84, and the image display control unit 87, the exhibit detection unit 91, and the storage unit. 95, a processing unit 96, and the like. Each of these units is electrically connected to the processing unit 96.
The exhibit detection unit 91 functions as an information acquisition unit that acquires information in the real field of view via the combiner 60. In the present embodiment, identification information for identifying an exhibit existing in the real field of view via the combiner 60 is detected.
For example, when an oscillator that transmits radio waves with high directivity according to identification information for specifying the exhibit is attached in the vicinity of the exhibit displayed in the museum, the exhibit detection unit 91 is configured as a receiver for receiving radio waves from this oscillator.
When the exhibit detection unit 91 acquires the identification information of the exhibit, the exhibit detection unit 91 outputs this information to the processing unit 96.

The storage unit 95 stores an exhibition table in which information on the direct viewing mode and information on the virtual viewing mode are registered for each piece of identification information of the exhibit. For example, in the direct view mode, the name of the exhibit, the author, the country, the age, and the detailed description of the exhibit are registered. In the virtual vision mode, the name and author are registered.
The processing unit 96 executes various types of processing based on the various types of information input from the above-described units, and causes the image display unit 50 to display information corresponding to the various types of processing results. For example, the processing unit 96 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. (not shown). A series of processing procedures for realizing various functions to be described later is recorded in a ROM or the like in the form of a program, and the CPU reads this program into the RAM or the like and executes information processing / calculation processing. Various functions to be described later are realized.

The operation of the information terminal device 10 according to the present embodiment configured as described above will be described.
First, when exhibit identification information is acquired by the exhibit detection unit 91, the identification information is input to the processing unit 96. When the processing unit 96 acquires the identification information of the exhibit (step SC1 in FIG. 16), based on the mode information detected by the mode detection unit 84, the current mode is the direct view mode or the virtual mode. It is determined whether or not there is (step SC2). As a result, when the current mode is the direct view mode, the processing unit 96 is information corresponding to the identification information input from the storage unit 95 from the exhibit detection unit 91 and is associated with the direct view mode. Information is extracted (step SC3), and this information is output to the image display controller 87.
The image display control unit 87 displays the information input from the processing unit 96 on the image display unit 50 (step SC4). As a result, for example, information on an exhibit as shown in FIG. 17 is displayed on the image display unit 50, and this information is directly displayed to the user via the combiner 60 arranged to be superimposed on the image display unit 50. Will be visually recognized.

On the other hand, in the above-described step SC2, if the current mode is the virtual vision mode, the processing unit 96 is information corresponding to the identification information input from the storage unit 95 from the exhibit detection unit 91, Information associated with the virtual vision mode is extracted (step SC5).
Subsequently, the processing unit 86 acquires an image of a landscape with a real field of view from the imaging device 40, and identifies an exhibit in the real field of view by analyzing the image (step SC6). Then, an image is created so that the information acquired in step SC4 is projected as a virtual image in the vicinity of the exhibit so as not to overlap the exhibit, and this image is output to the image display control unit 87.
The image display control unit 87 displays the information input from the processing unit 96 on the image display unit 50 (step SC7). Thereby, the work name and the author are displayed on the image display unit 50, for example, as shown in FIG. These pieces of information are superimposed as a virtual image in the real field of view through the combiner 60 as shown in FIG. 19, and as a result, an image as shown in FIG. 20 is visually recognized by the user.

As described above, according to the information terminal device of the present embodiment, in the direct view mode, for example, as shown in FIG. In the case of the virtual vision mode, for example, as shown in FIG. 20, a simple image of the exhibit is displayed on the external image that the user is viewing via the combiner 60, that is, the exhibit is a virtual image. Are superimposed and displayed.
As described above, since the image displayed on the image display unit 50 is different depending on the mode, in the case of the direct viewing mode, for example, the user can provide useful information as long as it can be displayed on the image display unit 50, In other words, detailed information that cannot be obtained in the virtual vision mode is displayed, and in the virtual vision mode, the simple information is projected as a virtual image on the exhibit so that the user can easily understand the information on the exhibit. Can be provided.
In particular, in the virtual vision mode, information that interests each exhibit may be displayed to provide an opportunity to switch to the direct viewing mode. By displaying such information, it is possible to prompt the user to switch to the direct viewing mode and to provide more information to the user in the direct viewing mode. Thereby, the user can acquire various information about each exhibit, and can enjoy visiting the museum.

In the embodiment described above, the exhibit detection unit 91 for specifying the exhibit may be configured as follows.
In the first aspect, the exhibit detection unit 91 includes the imaging device 40 and the processing unit 96. For example, images of all exhibits and their identification information are registered in advance in the storage unit 95. Then, the processing unit 96 determines whether or not the image of the exhibit that is registered in the storage unit 95 exists in the landscape of the real field of view captured by the imaging device 40. This determination can be realized by, for example, image matching. When an image of an exhibit that is registered in advance in the storage unit 95 exists in the real field of view, the user visually recognizes it by acquiring identification information associated with the image. Identify exhibits.
In particular, the user holds the information terminal device 10 so that the exhibit is located in the center of the imaging device 40, and performs the above determination when a predetermined operation is performed in this state. By doing so, it becomes possible to reduce the processing load.

In the second mode, the exhibit detection unit 91 includes the GPS reception unit 81, the direction detection unit 82, and the processing unit 96 illustrated in FIG. For example, the identification information of each exhibit and the position information where the exhibit is arranged are registered in the storage unit 95 in association with each other.
Then, the processing unit 96 is based on the position information acquired by the GPS receiving unit 81, the direction information of the real field of view acquired by the direction detecting unit 82, and the position information of each exhibit registered in the storage unit. By detecting what exhibits are present in the real field of view, the exhibits that the user is viewing are identified.

In the present embodiment, when displaying information projected as a virtual image so as not to overlap a real image existing in the real field of view, the contour of the real image is recognized by image matching, and predetermined information outside the contour is displayed. Was displayed as a virtual image. Instead, for example, a predetermined frame is provided on the real field of view through the combiner 60, and when an exhibit is placed in this frame, the information on the exhibit is projected as a virtual image outside the frame. It is possible to make it the structure which makes it.
Here, whether or not the exhibit is stored in the frame can be determined based on, for example, an image acquired by the imaging device 40. Further, when an exhibit is stored in the frame, the user may detect that the exhibit is stored in the frame by operating the operation unit.
By adopting this method, it is possible to fix the position for displaying characters and the like even in the virtual vision mode, so that the predetermined information to be displayed as a virtual image can be displayed in a contour of the real image by simpler processing. It is possible to display on the outside.

As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the specific structure is not restricted to this embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.
First, in the above-described first to fourth embodiments, the current mode is determined based on the positional relationship between the image display unit 50 and the combiner 60. Instead of this, the following modes are also possible.
As a first other aspect, the information terminal device 10 is provided with a mode switching unit for switching modes in the operation unit 70, and the processing unit 86 or 96 acquires a signal from the mode switching unit, so that The mode may be determined.
As a second other aspect, an attitude detection unit that detects the attitude of the information terminal device 10 is provided, and the processing unit 86 or 96 determines the current mode based on a signal from the attitude detection unit. Also good. Specifically, as illustrated in FIGS. 1 and 2, the posture of the information terminal device is different between the direct viewing mode and the virtual viewing mode. Therefore, for example, a gyro sensor or the like is provided as the posture detection unit, and the gyro sensor detects how much the information terminal device 10 is inclined with respect to the horizontal plane. Then, the processing unit 86 or 96 may determine the current mode based on the detection result of the gyro sensor.

  Secondly, in the above-described embodiment, provision of information in route guidance, provision of information on exhibits in a museum, and the like have been described. However, information provided by an information terminal device is not limited to such information.

Thirdly, in the information terminal device according to the first or second embodiment described above, the information displayed in the direct view mode or the virtual vision mode does not need to be provided in the information terminal device 10, for example, as shown in FIG. As shown, it may be provided in a server 100 connected to the information terminal device 10 via an electric communication line.
For example, the server 100 includes a communication unit 101 that enables transmission / reception of information to / from the information terminal device 10, a processing unit 102, and an information storage unit 110.
The information storage unit 110 stores a first database 103 that stores information (hereinafter referred to as “direct viewing information”) displayed on the display control unit 50 (see FIG. 1 and the like) when the information terminal device 10 is in the direct viewing mode. And a second database 104 storing information (hereinafter referred to as “virtual vision information”) displayed on the display control unit 50 (see FIG. 1 and the like) when the information terminal device 10 is in the virtual vision mode. It is prepared for.
Thus, by providing the server 100 with the information storage unit 110 that stores direct-view information and virtual-vision information, it is possible to reduce the data capacity of the information terminal device 10 and the processing load.
In addition to the server 100 described above, the information storage unit 110 is an external storage device that is electrically connected directly via a UBS terminal or the like, for example, if the information terminal device 10 includes a USB terminal or the like. May be provided.

  Fourth, in the above-described embodiment, the processing unit 86 or 96 implements various functions by executing processing by software. However, the present invention is not limited to this configuration example, and the various functions are performed by an analog circuit or the like. May be realized.

It is an external appearance perspective view which shows the structural example of the information terminal device which concerns on the 1st Embodiment of this invention. It is an external appearance perspective view which shows the structural example of the information terminal device which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the effect | action of the image display part shown in FIG. 1, and a combiner. It is a block diagram which shows the electrical structural example of the information terminal device which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the effect | action of the information terminal device which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the information displayed on an image display part in the direct view mode of the information terminal device which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the information displayed on an image display part in the virtual vision mode of the information terminal device which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the real image which spreads in the real visual field via a combiner in the virtual vision mode of the information terminal device which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the information visually recognized via the combiner by the user in the virtual vision mode of the information terminal device which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the effect | action of the information terminal device which concerns on the 2nd Embodiment of this invention. It is a figure for demonstrating the information which the information terminal device which concerns on the 3rd Embodiment of this invention acquires. It is a figure for demonstrating the information which the information terminal device which concerns on the 3rd Embodiment of this invention acquires. It is a figure for demonstrating the information which the information terminal device which concerns on the 3rd Embodiment of this invention acquires. It is a figure for demonstrating the information which the information terminal device which concerns on the 3rd Embodiment of this invention acquires. It is a block diagram which shows the electrical structural example of the information terminal device which concerns on the 4th Embodiment of this invention. It is a flowchart which shows the effect | action of the information terminal device which concerns on the 4th Embodiment of this invention. It is a figure which shows an example of the information displayed on an image display part in the direct view mode of the information terminal device which concerns on the 4th Embodiment of this invention. It is a figure which shows an example of the information displayed on an image display part in virtual vision mode of the information terminal device which concerns on the 4th Embodiment of this invention. It is a figure which shows an example of the real image which spreads in the real visual field via a combiner in the virtual vision mode of the information terminal device which concerns on the 4th Embodiment of this invention. It is a figure which shows an example of the information visually recognized via the combiner by the user in the virtual vision mode of the information terminal device which concerns on the 4th Embodiment of this invention. It is a block diagram which shows the other structural example of the information terminal device of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Information terminal device 20 Case 30 Display apparatus 40 Imaging device 50 Image display part 60 Combiner 70 Operation part 81 GPS receiving part 82 Direction detection part 83 Communication part 84 Mode detection part 85, 95 Storage part 86, 96 Processing part 87 Image display Control unit 91 Exhibit detection unit

Claims (8)

An image display unit;
An image display control unit for displaying information on the image display unit;
A translucent optical element that transmits light from the real field of view of the user's line of sight, and displays the information displayed on the image display unit as an optical virtual image in a superimposed manner in the real field;
The current mode is a direct-view mode that directly provides information displayed on the image display unit, or a virtual-view mode that provides information displayed on the image display unit as a virtual image through the semi-transmissive optical element A mode detection unit for detecting
The information terminal device in which the image display control unit varies information to be displayed on the image display unit according to a detection result of the mode detection unit. The positional relationship between the image display unit and the transflective optical element differs depending on the mode,
The information terminal device according to claim 1, wherein the mode detection unit detects a current mode according to a positional relationship between the image display unit and the transflective optical element. An operation unit for switching the mode;
The information terminal device according to claim 1, wherein the mode detection unit detects the mode based on a signal from the operation unit. It has a posture detection unit that detects the posture of the main body,
The information terminal device according to claim 1, wherein the mode detection unit detects a current mode according to a detection result of the posture detection unit. In the virtual vision mode, comprising an information acquisition unit for acquiring information in a real field of view through the transflective optical element,
5. The information terminal according to claim 1, wherein the image display control unit causes the image display unit to display information related to the information in the real field of view acquired by the information acquisition unit. apparatus. The information acquisition unit
An imaging unit for imaging a landscape in the direction of the real field of view;
An object specifying unit for specifying an object existing in the real field of view based on an image acquired by the imaging unit;
In the virtual mode, the image display control unit displays the information on the object in the vicinity of the object specified by the object specifying unit so that the information on the object appears as a virtual image. The information terminal device according to claim 5, which is displayed on a section. An input section where the destination is entered;
A location information acquisition unit that acquires location information of the current location;
A direction detection unit for detecting the direction of the real field of view; a route acquisition unit for acquiring a route from the current location to the destination;
A traveling direction identifying unit that identifies the traveling direction to the destination based on the location information of the current location, the direction of the real field of view, and the route to the destination;
The information terminal device according to any one of claims 1 to 6, wherein in the virtual vision mode, the image display control unit displays information indicating the traveling direction on the image display unit. An image display unit, an image display control unit that displays information on the image display unit, and transmits light from a real field of view in a user's line of sight, and the information displayed on the image display unit is optically converted into a virtual image A transflective optical element that superimposes in a real field of view and a current mode is a direct view mode that directly provides information displayed on the image display unit, or the image display unit through the transflective optical element. An information terminal device having a mode detection unit for detecting whether it is a virtual vision mode for providing the displayed information as a virtual image;
An information storage unit for storing information associated with the direct viewing mode and information associated with the virtual viewing mode;
In the direct viewing mode, information associated with the direct viewing mode is displayed on the image display unit, and in the virtual viewing mode, the virtual viewing mode displayed on the image display unit via the semi-transmissive optical element. An information display system that superimposes and displays information associated with the optical field as an optical virtual image in the real field of view.

Images