JP2017004532A - Head-mounted display and information display - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a head-mounted display having excellent operability conforming to the intuition of a user, and an information display.SOLUTION: In an information display 100, a head-mounted display 1 comprises a receiving part, an image display element 14, and a display processing part. The receiving part receives an operation signal outputted from an input device 2, and including information regarding a relative position of a detection object in contact with an input operation face 21. The image display element 14 forms an image presented to a user. The display processing part displays an operation image in which an image is superimposed with an auxiliary image for indicating a position of the detection object, on the image display element 14 on the basis of the operation signal.SELECTED DRAWING: Figure 1

Description

  The present technology relates to a head mounted display and an information display device.

  2. Description of the Related Art A head mounted display (HMD) is known that is mounted on a user's head and can present an image to the individual user with a display or the like disposed in front of the eyes. As the HMD, a non-transmissive HMD that does not transmit external light and displays only a predetermined image to the user, and a transmissive HMD that displays a predetermined image superimposed on the user's visual field while allowing the user to visually recognize the external environment, for example. And classified. For example, Patent Document 1 describes a non-transparent HMD in which an operation input unit such as a button is disposed on an HMD main body.

  On the other hand, in recent years, as an input method to an information processing apparatus or the like, it is possible to present a large number of GUIs to a user by switching indication items (GUI: Graphical User Interface) instead of buttons having a large occupation area and a limit to diversification. The touch panel system is becoming mainstream. Therefore, Patent Document 2 describes a transmissive HMD in which an input operation surface of a touch panel is arranged in a user's visual field range and an input operation is performed.

JP 2008-70817 A JP 2010-145861 A

  However, in the transmissive HMD described in Patent Document 2, the input operation surface of the touch panel has to be used in the field of view via the HMD and is inconsistent with the input operation and the display on the image. There was a problem with operability.

  In view of the circumstances as described above, an object of the present technology is to provide a head mounted display and an information display device having excellent operability in accordance with the user's intuition.

In order to achieve the above object, a head mounted display according to an embodiment of the present technology includes a receiving unit, an image display element, and a display processing unit.
The reception unit receives an operation signal that is output from the input device and includes information on a relative position of a detection target that is in contact with the input operation surface.
The image display element forms an image presented to the user.
The display processing unit causes the image display element to display an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image based on the operation signal.

  By wearing the head-mounted display (HMD), the user can visually recognize information on the relative position of a detection target such as a finger that touches the input operation surface with an auxiliary image in the operation image presented by the HMD. it can. Accordingly, the user can perform an intuitive operation while viewing the operation image by the HMD without confirming the input operation surface.

In the head mounted display, the receiving unit may further receive an image signal output from the input device, and the display processing unit may display the image on the image display element based on the image signal. .
The HMD configured as described above can use not only a dedicated input device but also a mobile terminal equipped with a touch panel, for example, as an input device. Thereby, various applications such as a portable terminal can be used by the HMD.

The display processing unit may move the auxiliary image in the display area of the image based on the operation signal.
As a result, the user can confirm the temporal relative position of the detection target in contact with the input operation surface from the operation image, and the operability can be further improved.

The operation signal includes information regarding the contact pressure of the detection target with respect to the input operation surface, and the display processing unit displays a display form of the auxiliary image according to the contact pressure of the detection target with respect to the input operation surface. It may be changed.
Accordingly, the HMD can reflect, for example, a “push” operation on the input operation surface of the user on the operation image. Therefore, the user can perform various operations combining the touch on the input operation surface and “push”.

The head mounted display further includes a calculation unit that calculates the coordinate position of the detection target on the input operation surface based on the operation signal, and the display processing unit is configured to detect the detection calculated by the calculation unit. The operation image may be displayed on the image display element based on the target coordinate position.
The head mounted display can process a signal detected from an input device, and can be used even with an input device having a simple apparatus configuration.

In order to achieve the above object, an information display apparatus according to an embodiment of the present technology includes an input device and a head mounted display.
The input device includes an input operation surface and a sensor unit that detects contact of the detection target with the input operation surface and outputs information on the coordinate position of the detection target on the input operation surface.
The head mounted display includes an image display element that forms an image to be presented to a user, and an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image based on the output of the sensor unit. And a display processing unit to be formed.

  The information display device can present an image based on the input operation to the input device to the user by the head-mounted display, and the user can perform an intuitive operation without checking the input operation surface. It becomes.

The sensor unit may be configured to detect a contact pressure applied to the input operation surface by the detection target.
Furthermore, at this time, the head mounted display may change the display form of the auxiliary image according to the contact pressure of the detection target from the operation surface.
The input device can detect a contact pressure (pushing force) by a detection target with respect to the input operation surface, and can reflect the information on an operation image by the HMD. Thereby, the user can perform more various operations.

  The sensor unit may detect the movement of the detection target on the input operation surface, and the head mounted display may display the auxiliary image that moves in accordance with the detected movement of the detection target. .

The input device further includes a display element that displays an image on the input operation surface,
The head mounted display may display an image obtained by superimposing the auxiliary image on the image displayed on the input operation surface as the operation image.

In order to achieve the above object, an information display apparatus according to an embodiment of the present technology includes an input device and a head mounted display.
The input device includes an input operation surface and a sensor unit that detects contact of the detection target with the input operation surface.
The head-mounted display includes an image display element that forms an image presented to a user, a calculation unit that calculates a coordinate position of the detection target on the input operation surface based on an output of the sensor unit, and the calculation unit And a display processing unit that causes the image display element to form an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image based on the coordinate position of the detection target calculated in step (1).

  The information display apparatus can simplify the configuration of the input device. Thereby, weight reduction of an input device etc. can be implement | achieved and the information display apparatus with few feelings of fatigue can be provided even for a long time operation.

  As described above, according to the present technology, it is possible to provide a head mounted display and an information display device having excellent operability in accordance with the user's intuition.

It is a typical perspective view showing an information processor concerning a 1st embodiment of this art. It is a block diagram showing an internal configuration of an input device concerning a 1st embodiment of this art. 1 is a block diagram showing an internal configuration of a head mounted display (HMD) according to a first embodiment of the present technology. 6 is a flowchart in an operation example of the information processing apparatus according to the first embodiment of the present technology. It is a figure explaining the typical operation example of the information display apparatus which concerns on 1st Embodiment of this technique, (A) shows the input operation surface of the input device which the user is performing input operation, (B ) Shows an operation screen presented to the user via the HMD. 12 is a flowchart illustrating an operation example of the information display device according to the second embodiment of the present technology. It is a block diagram showing an internal configuration of an input device concerning a 3rd embodiment of this art. It is a block diagram which shows the internal structure of HMD which concerns on the 3rd Embodiment of this technique. It is a figure explaining the typical operation example of the information display apparatus which concerns on 3rd Embodiment of this technique, (A) shows the input operation surface of the input device which the user is performing input operation, (B ) Shows an operation screen presented to the user via the HMD. It is a block diagram showing an internal configuration of HMD concerning a 4th embodiment of this art. In a 5th embodiment of this art, it is a figure showing an example where an auxiliary picture on a picture of HMD becomes large according to contact pressure with a finger, and (A) is a user's finger lightly touching an input operation side. A mode is shown, (B) shows the operation image of HMD in that case. In a 5th embodiment of this art, it is a figure showing an example where an auxiliary picture on a picture of HMD becomes large according to contact pressure with a finger, and (A) is a user's finger pushed on an input operation surface. (B) shows an operation image of the HMD at that time. It is a figure which shows an example of the operation image which concerns on 5th Embodiment of this technique, and shows the example from which the color of the auxiliary image on the image of HMD changes according to the contact pressure by a finger | toe. It is a figure which shows an example of the operation image which concerns on 5th Embodiment of this technique, and shows the example from which the color of the auxiliary image on the image of HMD changes according to the contact pressure by a finger | toe. It is a figure which shows the modification of 2nd Embodiment of this technique, (A) shows the aspect which extended these distances, while the user pushed two fingers on the input operation surface, (B) is In this case, an operation image is enlarged. It is a figure which shows the modification of 2nd Embodiment of this technique, (A) shows the aspect which narrowed these distance, while the user pushed two fingers on the input operation surface, (B) In this case, the operation image is reduced.

  Hereinafter, embodiments according to the present technology will be described with reference to the drawings.

<First Embodiment>
[Information display device]
FIG. 1 is a schematic perspective view showing an information display device according to an embodiment of the present technology. The information display apparatus 100 according to the present embodiment includes a head mounted display (HMD) 1 and an input device 2.

  The HMD 1 is configured as a transmissive HMD in the present embodiment. The HMD 1 has a glasses-like shape as a whole, and is configured to allow the user to present an image based on information output from the input device 2 while wearing the head and visually recognizing the outside world. .

  The input device 2 is wirelessly communicably connected to the HMD 1, but may be configured to be communicable with the HMD 1 by a cable or the like. In this embodiment, the input device 2 is comprised with the portable terminal which has a touchscreen, for example, and is comprised so that the internet etc. can be connected. The input device 2 functions as a so-called remote controller of the HMD 1 and can be configured to allow input operations related to various settings of the HMD 1.

  Hereinafter, the input device 2 and the HMD 1 will be described.

[Input device]
The input device 2 includes, for example, a housing 2A that can be gripped by the user. The housing 2A is a substantially rectangular parallelepiped having a longitudinal direction in the x-axis direction, a short direction in the y-axis direction, and a thickness direction in the z-axis direction, and an input operation surface 21 is formed on one surface of the housing 2A. Yes. The input operation surface 21 belongs to a two-dimensional coordinate system having a coordinate axis on the x-axis and a y-axis orthogonal thereto, and has a long side parallel to the x-axis direction and a short side parallel to the y-axis direction. It has a rectangular shape perpendicular to the z-axis.

  The input device 2 has a function of detecting the xy coordinate position of a finger on the input operation surface 21 and a change thereof, for example, with a user's finger as a detection target. The input device 2 further has a function of detecting the amount of bending of the input operation surface 21 by the finger in the z-axis direction, that is, the contact pressure. Thereby, the contact pressure in the z-axis direction on the input operation surface 21 by the finger is acquired together with the movement direction, movement speed, movement amount, and the like of the finger on the input operation surface 21. The detection target is not limited to the user's finger, and may be a stylus or the like.

  In this embodiment, the input operation surface 21 is configured by a light-transmissive display cover or the like, and performs an input operation depending on a detection target and also has a function as a screen for displaying an image. For example, the input operation surface 21 is disposed so as to cover an opening 21A formed on the surface of the housing 2A, and the periphery thereof is fixed to the housing 2A. Further, the material, thickness, size, and the like of the input operation surface 21 are set so that the input operation surface 21 can be bent and deformed with respect to a predetermined contact pressure or more. As a material constituting the input operation surface 21, for example, an acrylic resin, a polycarbonate resin, a transparent plastic plate such as PET (polyethylene terephthalate), a glass plate, a ceramic plate, or the like is employed.

  FIG. 2 is a block diagram showing an internal configuration of the input device 2. The input device 2 includes a housing 2A, an input operation surface 21, a touch sensor 22, a pressure sensor 23, a display element 24, a control unit 25, a storage unit 26, a transceiver 27, and a communication unit 28. And a battery BT. The touch sensor 22 and the pressure sensor 23 constitute a “sensor unit” in the present embodiment.

  The touch sensor 22 has a panel shape having substantially the same shape and size as the input operation surface 21. The touch sensor 22 is disposed immediately below the input operation surface 21 and detects a detection target (finger) that contacts the input operation surface 21. The touch sensor 22 detects a coordinate position corresponding to the movement of the detection target on the input operation surface 21 on the xy plane, and outputs a detection signal corresponding to the coordinate position.

  In the present embodiment, the touch sensor 22 is a capacitive touch panel that can electrostatically detect a detection target that contacts the input operation surface 21. The capacitive touch panel may be a projected capacitive type (projection type) or a surface capacitive type (surface type). This type of touch sensor 22 is typically parallel to the x-axis direction and the first sensor 22x for x-position detection in which a plurality of first wirings parallel to the y-axis direction are arranged in the x-axis direction. The second wiring 22 has a plurality of position detection second sensors 22y arranged in the y-axis direction, and the first and second sensors 22x and 22y are arranged to face each other in the z-axis direction. The The touch sensor 22 is sequentially supplied with signal currents to the first and second wirings, for example, by a drive circuit included in the control unit 25 described later.

  In addition to the above, the touch sensor 22 is not particularly limited as long as it can detect the coordinate position of the detection target, and is a resistive film type, an infrared type, an ultrasonic type, a surface acoustic wave type, an acoustic wave matching type, an infrared ray type, and the like. Various types such as an image sensor are applicable.

  In the present embodiment, a pressure sensor that can electrostatically detect a contact pressure in the Z-axis direction of the input operation surface 21 by a detection target is used as the pressure sensor 23. The pressure-sensitive sensor 23 has, for example, one or more pairs of electrodes that are arranged between the input operation surface 21 and the housing 2 </ b> A and face each other in the z-axis direction. The contact pressure is detected by the change in capacitance. In addition, a sensor in which an elastic body is disposed between the pair of electrodes may be employed as the pressure sensor 23. In this case, an electrode generated by elastic deformation of the elastic body accompanying a pressing operation to the input operation surface 21. The contact pressure is detected by a change in capacitance between the two. Note that the pressure sensor 23 can be driven by, for example, a drive circuit included in a control unit 25 described later.

  The pressure sensor 23 is not limited to the above configuration. For example, a piezoelectric sensor using a piezoelectric element, a strain gauge, or the like can be used.

  The display element 24 is disposed immediately below the input operation surface 21 and the touch sensor 22. The display element 24 in the present embodiment is not particularly limited, and a liquid crystal display, an organic EL display, or the like can be used. As a result, it is possible to display images such as characters and pictures on the input operation surface 21.

  The control unit 25 is typically configured by a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). In the present embodiment, the control unit 25 includes a calculation unit 251 and a signal generation unit 252, and executes various functions according to programs stored in the storage unit 26. The calculation unit 251 performs a predetermined calculation process on the electrical signals output from the touch sensor 22 and the pressure sensor 23 and includes an operation signal including information on the relative position of the detection target that contacts the input operation surface 21. Is generated. The signal generation unit 252 generates an image signal for displaying an image on the display element 24 based on these calculation results. The control unit 25 has a drive circuit for driving the touch sensor 22, and the drive circuit is incorporated in the calculation unit 251 in the present embodiment.

  Specifically, the calculation unit 251 calculates the xy coordinate position and the contact pressure of the finger on the input operation surface 21 based on signals output from the touch sensor 22 and the pressure sensor 23. Further, based on the above calculation result, for example, the calculation unit 251 detects that the detection target is at a predetermined xy coordinate position, and further detects a contact pressure equal to or higher than a predetermined threshold at the coordinate position. A predetermined process assigned to the GUI corresponding to the coordinate position is executed. These processing results by the calculation unit 251 are transmitted to the signal generation unit 252.

  The signal generation unit 252 generates an image signal for forming an image to be displayed on the display element 24 based on the processing result transmitted from the calculation unit 251. At this time, the signal generation unit 252 generates a signal for displaying an auxiliary image such as a pointer or a region in which luminance or saturation has changed at a position corresponding to the xy coordinate position of the detection target of the image on the input operation surface 21. Is also possible. The signal generation unit 252 can also generate a signal that changes the display form of the auxiliary image according to the contact pressure of the detection target.

  The operation signal related to the xy coordinate position and the contact pressure calculated by the calculation unit 251 of the control unit 25 and the image signal generated by the signal generation unit 252 are configured to be transmitted to the HMD 1 via the transceiver 27. The Although not shown, the input device 2 is an A / D converter that converts a detection signal (analog signal) output from the touch sensor 22 or the pressure sensor 23 into a digital signal, or a D that converts a digital signal into an analog signal. Includes / A converter.

  The storage unit 26 includes a RAM (Random Access Memory), a ROM (Read Only Memory), and other semiconductor memories, and stores programs used for various operations by the control unit 25. For example, the ROM is configured by a non-volatile memory, and stores a program and a set value for causing the control unit 25 to perform arithmetic processing such as calculation of the xy coordinate position and the contact pressure. In addition, the storage unit 26 can store, for example, a xy coordinate position and a contact pressure to be detected, and a program for executing a function assigned corresponding to the xy coordinate position to be detected by a nonvolatile semiconductor memory. . Furthermore, these programs stored in advance in a semiconductor memory or the like may be loaded into the RAM and executed by the calculation unit 251 of the control unit 25.

  The transceiver 27 is configured to be able to transmit various control signals generated by the control unit 25 to the HMD 1 wirelessly, for example. The transceiver 27 is configured to be able to receive a predetermined signal transmitted from the HMD 2. On the other hand, the communication unit 28 is connected to a communication network such as the Internet. The communication unit 28 is used when, for example, a predetermined program such as an application is downloaded to the input device 2. Transmission / reception of information in the communication unit 28 may be performed by wire using a LAN cable or the like, or may be performed wirelessly such as high-speed data communication.

  The battery BT constitutes a power source for the input device 2 and supplies necessary power to each part inside the housing 2A. The battery BT may be a primary battery or a secondary battery. Battery BT may be formed of a solar cell. Furthermore, the input device 2 can also have an external switch (not shown) that controls activation of the input device 2 by pressing.

[HMD]
The HMD 1 is a transmissive HMD, and is configured to be able to present a predetermined image in the visual field region while allowing the user to visually recognize the outside world. The HMD 1 according to the present embodiment includes an image display device 11 and a support unit 16 and has a glasses-like shape as a whole. For example, by adopting a transmissive HMD as the HMD 1, it is possible to visually recognize the outside world even during operation of the input device 2, and safety during operation can be improved.

  The support unit 16 can be mounted on the user's head, and is configured such that an image display element 14 described later is supported in front of the user's eyes when mounted. The shape of the support portion 16 is not particularly limited, but in the present embodiment, the shape can be a glasses shape as a whole. The support portion 16 has, for example, a temple-like structure that can be worn on the left and right ears of the user. For example, a nose pad 161 can be attached near the nose bridge of the support portion 16. Further, the earphone 162 may be mounted on the support portion 16. For example, a metal, an alloy, a plastic, or the like can be adopted as the material constituting the support portion 16 and is not particularly limited.

  The image display device 11 includes a housing 11 </ b> A and is disposed at a predetermined position of the support unit 16, for example. The arrangement of the image display device 11 is not particularly limited. For example, the housing 11A is arranged outside the left and right eyes of the user so as not to be included in the user's visual field region.

  FIG. 3 is a block diagram showing an internal configuration of the image display device 11. In the present embodiment, the image display device 11 includes a housing 11A, a transceiver (reception unit) 12, a control unit 13, an image display element 14, and a storage unit 15. In the present embodiment, the control unit 13 constitutes a “display processing unit”.

  The transmitter / receiver 12 receives an operation signal that is output from the transmitter / receiver 27 of the input device 2 and includes information on the xy coordinate position of the detection target that contacts the input operation surface 21 and the contact pressure. Furthermore, it is also possible to receive an image signal of an image displayed on the input operation surface 21 of the input device 2. The signal transmission method is not particularly limited. For example, communication between devices such as “Wi Fi” (registered trademark), “ZigBee” (registered trademark), “Bluetooth” (registered trademark), communication via the Internet, or the like can be used as wireless communication. Also, a wired connection via USB (Universal Serial Bus), HDMI (registered trademark), or the like is possible.

  The control unit 13 is typically composed of a CPU or MPU, and executes various functions according to a program stored in the storage unit 15. In the present embodiment, the control unit 13 generates a control signal for causing the image display element 14 to form an image presented to the user based on the operation signal and the image signal received by the transceiver 12. Accordingly, for example, an operation image in which an auxiliary image indicating the position of the detection target is superimposed on an image similar to the image displayed on the input operation surface 21 can be displayed on the image display element 14. The shape of the auxiliary image is not particularly limited, and is composed of, for example, an annular (ring-shaped) pointer or a region where luminance and saturation are changed.

  In the present embodiment, the image display element 14 includes a liquid crystal display element (LCD) 141 and an optical system 142, and is configured to present an image formed by the LCD 141 to the user via the optical system 142. The

  The LCD 141 has a plurality of pixels arranged in a matrix. The LCD 141 modulates light incident from a light source (not shown) formed of an LED (light emitting diode) or the like for each pixel according to a control signal generated by the control unit 13, and forms light that forms an image presented to the user. Exit. The LCD 141 may be, for example, a single plate type that simultaneously emits image light corresponding to each color of R (red), G (green), and B (blue), or three plates that individually emit image light corresponding to each color. The method may be used.

  The optical system 142 is configured to deflect light emitted from the LCD 141 and guide the light to the eyes of the user. The optical system 142 can be composed of, for example, a light guide plate made of a transparent substrate capable of guiding light by total reflection, a reflective volume hologram diffraction grating capable of diffraction reflection, and the like. The light guide plate is formed in, for example, a rectangular or circular flat plate shape, and is disposed in front of the user's eyes like a lens of glasses. A hologram diffraction grating or the like is appropriately disposed in the light guide plate, and is manufactured so as to reflect the light emitted from the LCD 141 and guide it to the eyes of the user. In addition to the hologram diffraction grating, for example, a reflector or the like can be employed.

  The optical system 142 having such a configuration can present an operation image in the visual field area of the user by reflecting the light emitted from the LCD 141 at a predetermined position. In addition, since the light guide plate is formed of a transparent substrate, it is possible to present the operation image in a superimposed manner in the visual field region while allowing the user to visually recognize the outside world.

  The storage unit 15 includes a RAM, a ROM, and other semiconductor memories, and stores programs used for various calculations by the control unit 13.

  The speaker 17 converts an electrical sound signal transmitted from the input device 2 or generated by the control unit 13 or the like into physical vibration, and provides sound to the user via the earphone 162. The configuration of the speaker 17 is not particularly limited.

  In addition, the image display apparatus 11 is not restricted to the structure arrange | positioned corresponding to both eyes. For example, it is possible to adopt a configuration in which one piece is arranged corresponding to only one of the left and right eyes.

[Operation example of information display device]
Next, a basic operation example of the information display apparatus 100 will be described.

  FIG. 4 is a flowchart of an operation example of the HMD 1 (control unit 13) and the input device 2 (control unit 25). FIG. 5 is a diagram for explaining a typical operation example of the information display apparatus 100. FIG. 5A shows the input operation surface 21 of the input device 2 on which the user performs an input operation, and FIG. The operation screen shown to a user via HMD1 is shown. Here, an operation example of the information display apparatus 100 when the user wears the HMD 1 and performs a pushing operation at a predetermined position on the input operation surface 21 of the input device 2 is shown. Note that the X-axis direction and the Y-axis direction in the figure correspond to the x-axis direction and the y-axis direction of the input operation surface 21, respectively, and both indicate planar directions that are orthogonal to each other. The Z-axis direction corresponds to the z-axis direction of the input operation surface 21 and indicates directions orthogonal to the X-axis direction and the Y-axis direction, respectively.

  For example, an image v1 in which a large number of GUIs are displayed is displayed on the input operation surface 21 of the activated input device 2 (FIG. 5A). The image v1 is, for example, a menu selection screen for various settings of the HMD1, and each GUI corresponds to switching to the mute mode of the HMD1, volume adjustment, enlargement / reduction of the operation image, change of the display form of the pointer, and the like. . That is, the input device 2 is configured to be able to change the setting of the HMD 1 by selecting a specific GUI by the user.

  At this time, the user who is connected to the input device 2 and wears the activated HMD1 is also presented with the image V1 similar to the image v1 in the visual field region via the HMD1.

  The input device 2 determines contact of the user's finger (detection target) on the input operation surface 21 with the touch sensor 22 (step ST101). When contact is detected (YES in step ST101), the touch sensor 22 outputs a detection signal related to the xy coordinate position on the input operation surface 21 touched by the finger to the control unit 25.

  The computing unit 251 of the control unit 25 calculates the xy coordinate position of the finger on the input operation surface 21 based on the detection signal (step ST102). A signal related to the xy coordinate position calculated by the calculation unit 251 is output to the transceiver 27.

  The transceiver 27 transmits an operation signal related to the xy coordinate position to the transceiver 12 of the HMD 1.

  Based on the operation signal and the image signal received by the transceiver 12, the control unit 13 of the HMD 1 generates a signal for controlling the operation image V10 in which the auxiliary image (pointer P) indicating the position of the detection target is superimposed on the image V1. . The image display element 14 to which this control signal has been output presents the operation image V10 to the user (step ST103, FIG. 5B). Thus, the user can perform a desired operation by checking the movement of the pointer P on the operation image V10 presented by the HMD 1 without looking at the input operation surface 21 at hand, and the user's intuition. It is possible to perform an operation in accordance with. At this time, the image displayed on the input operation surface 21 may be only the image v1 or an image in which a pointer or the like is superimposed on the image v1.

  Further, when the user's finger moves while touching on the input operation surface 21 (see arrows in FIGS. 5A and 5B), information on the xy coordinate position that changes with time is acquired by the touch sensor 22. . This information is processed as an operation signal by the control unit 25 of the input device 2 and is output to the control unit 13 of the HMD 1 via the transceivers 27 and 12. Accordingly, the control unit 13 can output a control signal for moving the pointer P to the LCD 141 based on the operation signal, and the pointer P in the display area of the image V1 can be displayed according to the movement of the user's finger. It can be moved.

  The control unit 25 of the input device 2 sets a GUI closest to the calculated xy coordinate position (hereinafter referred to as a selection GUI) as a selection candidate (step ST104). Correspondingly, the GUI of the selection candidates of the images v1 and V10 displayed on the input operation surface 21 and the HMD 1 may change the display form such as the color of the frame. In addition, the user can check the GUI of the selection candidate by visually recognizing the image V10 displayed by the HMD1.

  The controller 25 determines the contact state between the finger and the input operation surface 21 over time (step ST105). Here, when the user moves the finger away from the input operation surface 21, the touch sensor 22 outputs a signal when the finger is not touching. The control unit 25 determines non-contact between the input operation surface 21 and the finger based on the output of the touch sensor 22 (YES in step ST105). In this case, the pointer P disappears from the operation image V10 of the HMD1, and the screen returns to the image V1.

  On the other hand, when the user does not move the finger away from the input operation surface 21, the touch sensor 22 outputs a signal when the finger is in contact. Based on the output of the touch sensor 22, the control unit 25 determines that the user's finger is in contact with the input operation surface 21 (NO in step ST105), and then inputs based on the output of the pressure sensor 21. The contact pressure of the finger on the operation surface 21 is detected (step ST106).

  The selection candidate GUI is selected (determined) by the user's pressing operation on the display position of the selection candidate GUI. Based on the output of the pressure-sensitive sensor 23, the control unit 25 calculates the contact pressure in the calculation unit 251. If the calculated contact pressure is equal to or greater than the predetermined threshold (YES in step ST106), the control unit 25 determines the selection candidate GUI as the selection GUI, and uses this selection GUI stored in the storage unit 26 as a selection GUI. Corresponding code information is acquired (step ST107).

  On the other hand, when the calculated contact pressure is less than the predetermined threshold (NO in step ST106), the control unit 25 determines that the selection candidate GUI has not been selected. In this case, the control part 25 returns to step ST105, and repeats the process mentioned above.

  Furthermore, the control unit 25 transmits the acquired code information to the HMD 1 via the transceiver 27 (step ST108). The control unit 13 of the HMD 1 executes processing corresponding to the selected GUI based on the code information received via the transceiver 12. This process is executed based on, for example, a program stored in the storage unit 15. For example, when the function corresponding to the selected GUI is “switching to the mute mode”, the control unit 13 executes the process based on the code information corresponding to the GUI to set the setting of the HMD 1 to the mute mode. It is possible to switch.

  If the code information acquired in step ST107 is, for example, volume adjustment, the control unit 13 can generate an image control signal based on the code information and output it to the image display element 14. As a result, the user wearing the HMD 1 is presented with a new operation image (not shown) on which, for example, a volume adjustment bar is superimposed. In this case, the image control signal generated by the control unit 13 can be output from the transceiver 12 to the input device 2. As a result, an image similar to the new operation image presented by the HMD 1 can be displayed on the input operation surface 21 of the input device 2.

  In the present embodiment, the input device 2 has a pressure sensor 23. Therefore, when the user presses a finger on the input operation surface 21, it is possible to cause the input device 1 and the HMD 1 to execute processing corresponding to a desired GUI. Thereby, for example, even when the input device 2 is inside a bag or a pocket, it is possible to perform an input operation from the outside while being placed in the bag, thereby improving convenience. In addition, various operations can be performed by combining the contact operation with the input operation surface 21 and the push-in operation.

  Further, by performing the above-described operation by combining the touch on the input operation surface 21 and the pressing, the GUI is not easily selected only by the touch, and it is possible to prevent malfunction.

  Furthermore, since the HMD 1 can be operated while viewing an image larger than the image displayed on the input operation surface 21 of the input device 2, the portable terminal can be easily used even for an elderly person who is difficult to see a small screen. It becomes possible to do.

  In the present embodiment, the image v <b> 1 is also displayed on the input operation surface 21 of the input device 2. Thereby, the input device 2 is not limited to a device dedicated to the HMD 1, and a portable terminal equipped with a touch panel display can be connected and used. Therefore, in this case, it is possible to enjoy the above-described effects of the HMD 1 while using abundant applications of the mobile terminal.

<Second Embodiment>
FIG. 6 is a flowchart illustrating an operation example of the information display apparatus according to the second embodiment of the present technology. In the present embodiment, the description of the same parts as those of the first embodiment will be omitted or simplified, and different parts from the first embodiment will be mainly described.

  Also in the present embodiment, the operation image presented to the user by the HMD 1 and the image displayed on the input operation surface 21 will be described as, for example, a menu selection image in which a large number of GUIs are arranged. The difference from the first embodiment is that, in steps ST207 to ST212, the control unit 25 moves the auxiliary image (pointer P) in the operation image presented to the user and the selected GUI to a desired position. It is possible to change the arrangement. That is, according to the present embodiment, a “drag and drop” operation is possible.

  Steps ST201 to ST206 in FIG. 6 correspond to steps ST101 to ST106 in FIG. First, the control unit 25 detects contact of the user's finger on the input operation surface 21 (YES in step ST201), and calculates the xy coordinate position where the finger touches (step ST202). Based on the xy coordinate position, the control unit 13 of the HMD 1 displays the pointer P on the image V1 (ST203).

  On the other hand, the control unit 25 of the input device 2 sets the GUI closest to the xy coordinate position as a selection candidate (step ST204). Control unit 25 determines the contact state of the finger based on the output from touch sensor 22 (NO in step ST205), and determines that the contact pressure in the z-axis direction output from pressure-sensitive sensor 23 is equal to or greater than the threshold ( In step ST206, it is determined whether or not a contact pressure equal to or higher than the threshold is output for a predetermined time or more (step ST207).

  Here, if it is equal to or shorter than the predetermined time (NO in step ST207), control unit 25 returns to step ST206 and repeats the above-described processing.

  On the other hand, when the contact pressure is detected for a certain time or longer (YES in step ST207), the control unit 25 determines that the selection candidate GUI is a draggable GUI (hereinafter referred to as draggable GUI) (step ST208).

  The control unit 25 calculates the xy coordinate position detected from the touch sensor 22 over time as the finger moves on the input operation surface 21 (step ST209). Further, this signal is output over time to the control unit 13 of the HMD 1 via the transceiver 27. The control unit 13 generates an image signal for moving the pointer P and the draggable GUI on the image V10 based on the change in the xy coordinate position (step ST210).

  During the movement, the control unit 25 calculates the contact pressure output from the pressure-sensitive sensor 23 over time, and continues to determine whether the contact pressure is greater than or equal to a threshold value (YES in step ST211). Here, when the contact pressure is less than the threshold value (NO in step ST211) and it is further determined that the finger is not in contact (YES in step ST212), the control unit 25 executes a process of “dropping” the draggable GUI. (Step ST213). The position where the draggable GUI is dropped is a position corresponding to the xy coordinate position when the control unit 25 determines that the contact pressure is less than the threshold (NO in step ST211).

  On the other hand, if the control part 25 does not determine non-contact (it is NO at step ST214), the control part 25 will return to step ST206 and will determine again whether the contact pressure output from the pressure sensor 23 is more than a threshold value. .

  As described above, the information display device 100 according to the present embodiment allows the user to move the GUI, the folder, and the like in the operation image by gazing only at the operation image of the HMD 1 without checking the input operation surface 21 at hand. be able to. That is, the “drag and drop” operation can be performed by the outputs of the touch sensor 22 and the pressure sensor 23, and the operability of the user can be further improved.

<Third Embodiment>
7 to 9 are diagrams illustrating a third embodiment of the present technology. FIG. 7 is a block diagram illustrating an internal configuration of the input device according to the present embodiment, and FIG. 8 is a block diagram illustrating an internal configuration of the image display apparatus according to the head mounted display of the present embodiment. FIG. 9 is a diagram for explaining a typical operation example of the information display device. FIG. 9A shows an input operation surface of an input device on which a user performs an input operation, and FIG. 9B shows an operation via an HMD. The operation screen presented to the user is shown. In the present embodiment, the description of the same parts as those of the first embodiment will be omitted or simplified, and different parts from the first embodiment will be mainly described.

  The input device 20 of the present embodiment does not have a display element and a control unit, and is different from the first embodiment described above in that no image is displayed on the input operation surface 210. That is, the detection signal of the xy coordinate position by the touch sensor 220 and the detection signal of the contact pressure by the pressure sensor 230 can be transmitted to the HMD 10 via the transceiver 270 as operation signals. Thus, as shown in FIG. 9A, no image is displayed on the input operation surface 210, and the input device 20 can be configured to have only a so-called touch panel. That is, the input device 20 according to the present embodiment can be a remote controller dedicated to the HMD 10.

  In this embodiment, the input operation surface 210 of the input device 20 is only subjected to an input operation based on a detection target, and thus can be made of a material that does not have optical transparency. Thereby, in addition to the materials listed in the first embodiment, for example, a synthetic resin that can be bent and deformed with respect to a predetermined contact pressure or more can be employed.

  The control unit 130 according to the image display device 110 of the HMD 10 includes a calculation unit 131 and a display processing unit 132 in the present embodiment. The calculation unit 131 calculates the xy coordinate position and the contact pressure from the detection signal of the xy coordinate position by the touch sensor 220 and the detection signal of the contact pressure by the pressure sensor 230, respectively. The display processing unit 132 generates an image signal for forming an image to be displayed on the image display element 140 based on the processing result input from the calculation unit 131. At this time, the display processing unit 132 can also generate a signal for displaying an auxiliary image such as a pointer at a position corresponding to the xy coordinate position of the image detection target on the input operation surface 210. As a result, as shown in FIG. 9B, the HMD 10 can display an operation image V30 similar to the operation image V10 according to the first embodiment to the user.

  Furthermore, the image presented to the user by the HMD 10 can be acquired via the communication unit 280 of the input device 20 from the Internet or the like. It is also possible to provide the communication unit 170 in the HMD 10 and generate an image based on information directly acquired by the HMD 10.

  In the present embodiment, the configuration of the input device 20 can be simplified. As a result, the input device 20 can be reduced in size and weight, and it is possible to provide the input device 20 that is less likely to get tired and easy to carry even with a long touch operation.

<Fourth Embodiment>
FIG. 10 is a block diagram illustrating an internal configuration of the input device according to the fourth embodiment of the present technology. In the present embodiment, parts similar to those in the configuration and operation of the first embodiment are denoted by the same reference numerals, description thereof will be omitted or simplified, and description will be focused on parts different from the first embodiment.

  The input device 200 according to the present embodiment is different from the first embodiment described above in that it does not have a pressure sensor. That is, in the present embodiment, the input device 200 can detect the coordinate position corresponding to the movement of the detection target on the input operation surface 21 on the xy plane and the contact pressure by the touch sensor 2200. Configured. The touch sensor 2200 constitutes a “sensor unit” in the present embodiment.

  As the touch sensor 2200, for example, as in the first embodiment, a capacitive touch panel capable of electrostatically detecting a detection target in contact with the input operation surface 21 can be used. The touch sensor 2200 detects a coordinate position corresponding to the movement of the detection target on the input operation surface 21 on the xy plane, and outputs a detection signal corresponding to the coordinate position to the control unit 2500. Furthermore, for example, when a pressing operation is performed at the coordinate position by the detection target, the touch sensor 2200 detects the capacitance according to the contact pressure of the detection target or the amount of change thereof. As a result, the touch sensor 2200 can output a detection signal corresponding to the contact pressure at the coordinate position.

  In addition to the above, for example, a resistive film type touch panel may be used. The resistive film type touch sensor 2200 typically has two transparent electrodes or the like disposed opposite to each other with a predetermined gap immediately below the input operation surface 21 and has a size corresponding to the contact pressure of the detection target. The voltage (contact resistance) is output. Thus, the control unit 2500 can calculate the contact pressure depending on the detection target. In addition to the resistive film type, various touch panel sensors such as an infrared type, an ultrasonic type, and a surface acoustic wave type can be applied as the touch sensor according to the present embodiment.

  The control unit 2500 is configured such that the calculation unit 2510 can calculate the xy coordinate position of the detection target and the contact pressure at the position based on the output from the touch sensor 2200.

  Hereinafter, an operation example of the input device 200 (control unit 2500) will be briefly described. Here, similarly to the operation example described in the first embodiment, an example in which a specific GUI on the image displayed on the input operation surface 21 and the HMD 1 is selected by the user will be described.

  First, the control unit 2500 acquires a detection signal of the xy coordinate position from the touch sensor 2200. Thereby, control unit 2500 determines the GUI or the like closest to the xy coordinate position as the selection candidate GUI or the like (see step ST104 in FIG. 4).

  Subsequently, the control unit 2500 determines whether the change amount is equal to or greater than a predetermined threshold using the change amount of the capacitance output from the touch sensor 2200 as a contact pressure detection signal (see step ST106 in FIG. 4). If it is equal to or greater than the threshold value, the processing corresponding to the selection GUI or the like is executed as in the first embodiment (see step ST107 in FIG. 4).

  On the other hand, the control unit 2500 is configured to determine whether the capacitance value is equal to or greater than a predetermined threshold, using the capacitance value output from the touch sensor 2200 or its change amount itself as a contact pressure detection signal. Is also possible.

  The information display apparatus having the input device 200 configured as described above can contribute to cost reduction and productivity improvement, and the input device 200 can be reduced in weight and size.

<Fifth Embodiment>
FIGS. 11-14 is a figure explaining 4th Embodiment of this technique. In the present embodiment, the same components as those in the first embodiment will be denoted by the same reference numerals, the description thereof will be omitted or simplified, and the description will focus on portions that are different from the first embodiment.

  This embodiment is different from the first embodiment in that the display form of the auxiliary image (pointer) P4 (P411 to 422) is changed according to the contact pressure of the detection target (finger) with respect to the input operation surface 21.

  11 and 12 show an example in which the annular pointer P411 on the image V4 increases in accordance with the contact pressure by the finger. For example, when the user's finger lightly touches the input operation surface 21 (FIG. 11A), the operation image V411 on which the pointer P411 is superimposed with a predetermined size is displayed (FIG. 11B). Further, from this state, when the user presses the finger on the input operation surface 212 (FIG. 12A), an operation image V412 in which a pointer P412 larger than the pointer P411 is superimposed on the image V4 is displayed ( FIG. 12 (B)).

  As an operation of the information display device 100, for example, when the control unit 25 determines that the contact pressure acquired by the pressure sensor 23 is equal to or higher than a predetermined threshold value, the control unit 25 outputs the information to the control unit 13 of the HMD 1. Thereby, the control part 13 produces | generates the control signal which displays the pointer of the magnitude | size according to contact pressure. As a result, pointers P411 and P412 having a size corresponding to the contact pressure are displayed on the operation images V411 and 412 at positions corresponding to the finger contact positions on the input operation surface 21.

  13 and 14 show an example in which the colors of the annular pointers P421 and P422 on the image V4 change according to the contact pressure by the finger. For example, when the user's finger touches lightly on the input operation surface 21, the pointer P421 becomes a black operation image V421 (FIG. 13), and when the user presses the finger on the input operation surface 212 from that state, An operation image V422 in which a gray pointer P422 is superimposed on the image V4 is displayed (FIG. 14).

  It is also possible to configure the transmittance of the annular pointer P4 on the image V4 to change according to the contact pressure by the finger.

  With the information display device 100 as described above, the user can visually confirm his / her push-in amount (contact pressure). As a result, the user can more accurately perform the operation combining the contact and the push-in by confirming the operation image by the HMD 1.

  The embodiment of the present technology has been described above, but the present technology is not limited to this, and various modifications can be made based on the technical idea of the present technology.

  For example, FIGS. 15 and 16 are diagrams illustrating a modification of the second embodiment. The information display apparatus 100 is configured such that the touch sensor 22 and the pressure sensor 23 of the input device 2 can detect two or more contact positions on the input operation surface 21, respectively. Furthermore, by moving these fingers while pressing them into the input operation surface 21, it is possible to perform a so-called “pinch zoom” operation for enlarging or reducing the images V21 and V22.

  For example, when the control unit 25 determines that the contact pressure acquired by the pressure-sensitive sensor 23 is equal to or higher than a predetermined threshold, “pinch zoom” is started. For example, when the user widens these distances while pressing two fingers on the input operation surface 21 as shown in FIG. 15A, the detected distance and its change are shown as shown in FIG. Accordingly, the image V21 can be enlarged. On the other hand, when the user narrows these distances while pressing two fingers on the input operation surface 211 as shown in FIG. 16A, the detected distance and its change are shown in FIG. 16B. Accordingly, the image V22 can be reduced.

  As described above, according to the modified example, it is possible to change the display area of the image without looking at the input operation surface 21. Thus, for example, when the image V21 is small and difficult to see, it is easy to enlarge the display area. Furthermore, when it is desired to watch the outside world, it is possible to make the outside world easier to see by reducing the image V22. Therefore, it is possible to provide the information display device 100 that is easier for the user to use even when worn for a long time.

  Further, by the same “pinch zoom” operation, not only enlargement / reduction of the display area but also enlargement / reduction of a part of the image (for example, a specific command) can be performed.

  For example, as a modification of the first embodiment, it is possible to form an operation image in which the luminance, saturation, and the like of the region corresponding to the position of the finger on the input operation surface 21 are changed. In this case, the “auxiliary image” can be an image of a region in which luminance, saturation, and the like are changed.

  A specific operation example of the information display apparatus 100 according to the modification will be described. For example, the control unit 25 calculates the xy coordinate position of the finger on the input operation surface 21 based on the detection signal from the touch sensor 22 and outputs the information to the control unit 13 of the HMD 1. Based on the information, the control unit 13 generates a control signal for changing the luminance and saturation of the corresponding XY coordinate position.

  As a result, the HMD 1 can present the user with operation images having different brightness, saturation, and the like in the area corresponding to the finger contact position and other areas. Therefore, the user can also confirm the contact position of a finger or the like on the input operation surface by visually recognizing the operation image displayed by the HMD 1 according to the above modification.

  For example, in the above embodiments, the input device is described as a portable terminal or the like that is held and used by the user, but the present invention is not limited to this. For example, it may be a wristwatch type worn on the user's wrist or a wristband type worn on the user's arm. Further, it may be in the form of being attached to or attached to clothes. As described above, in the present technology, various types of input devices can be used as long as the detection target can contact the input operation surface.

  Moreover, although the above embodiment demonstrated as a transmissive | pervious HMD, it is not restricted to this, It can also be set as a non-transmissive | pervious HMD. Also in this case, the user can perform an operation in an intuitive manner without looking at the input operation surface of the input device by performing the operation while visually recognizing the image presented by the HMD.

  In addition, an image sensor can be mounted on the HMD so that the outside can be imaged. As a result, for example, a captured image can be displayed on the HMD and the input device.

  In the embodiment described above, the shape of the pointer has been described as an annular example, but of course it is not limited to this. For example, any shape such as an arrow, a triangle, a quadrangle, and a circle can be employed.

In addition, this technique can also take the following structures.
(1) a receiving unit that receives an operation signal that is output from the input device and includes information related to a relative position of a detection target that contacts the input operation surface;
An image display element for forming an image presented to a user;
And a display processing unit that displays, on the image display element, an operation image obtained by superimposing an auxiliary image indicating the position of the detection target on the image based on the operation signal.
(2) The head mounted display according to (1) above,
The receiving unit further receives an image signal output from the input device,
The display processing unit displays the image on the image display element based on the image signal.
(3) The head mounted display according to (1) or (2) above,
The display processing unit moves the auxiliary image in the image display area based on the operation signal.
(4) The head mounted display according to any one of (1) to (3) above,
The operation signal includes information on the contact pressure of the detection target with respect to the input operation surface,
The display processing unit changes a display form of the auxiliary image according to a contact pressure of the detection target with respect to the input operation surface.
(5) The head mounted display according to (3) or (4) above,
Based on the operation signal, further comprising a calculation unit for calculating the coordinate position of the detection target on the input operation surface,
The display processing unit displays the operation image on the image display element based on the coordinate position of the detection target calculated by the calculation unit.
(6) An input device having an input operation surface and a sensor unit that detects a contact of the detection target on the input operation surface and outputs information on the coordinate position of the detection target on the input operation surface;
An image display element that forms an image to be presented to the user, and a display processing unit that forms, on the image display element, an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image based on the output of the sensor unit An information display device comprising: a head-mounted display comprising:
(7) The information display device according to (6) above,
The input device is
A display element for displaying an image on the input operation surface;
The head mounted display displays an image obtained by superimposing the auxiliary image on an image displayed on the input operation surface as the operation image.
(8) The information display device according to (6) or (7) above,
The information display device, wherein the sensor unit detects a contact pressure applied to the input operation surface by the detection target.
(9) The information display device according to (8) above,
The head mounted display changes the display mode of the auxiliary image according to the contact pressure of the detection target from the operation surface.
(10) The information display device according to any one of (6) to (9) above,
The sensor unit detects the movement of the detection target on the input operation surface,
The head-mounted display displays the auxiliary image that moves according to the detected movement of the detection target.
(11) An input device having an input operation surface and a sensor unit that detects contact of the detection target with the input operation surface;
An image display element that forms an image presented to the user, a calculation unit that calculates the coordinate position of the detection target on the input operation surface based on an output of the sensor unit, and the detection calculated by the calculation unit An information display device, comprising: a display processing unit configured to form, on the image display element, an operation image in which an auxiliary image indicating a position of a detection target is superimposed on the image based on a coordinate position of the target.

1 ... HMD (Head Mounted Display)
2,20 ... Input device 12,120 ... Transmitter / receiver (receiver)
13: Control unit (display processing unit)
131 ... Calculation unit 132 ... Display processing unit 14,140 ... Image display element 21,210 ... Input operation surface 22,220 ... Touch sensor (sensor unit)
23, 230 ... Pressure-sensitive sensor (sensor part)
24 ... display element 100 ... information display device P, P411, P412, P421, P422 ... pointer (auxiliary image)
V10, V30, V411, V412, V421, V422 ... Operation image

Claims (11)

A receiving unit that receives an operation signal that is output from the input device and includes information on a relative position of a detection target that touches the input operation surface;
An image display element for forming an image presented to a user;
And a display processing unit configured to display, on the image display element, an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image based on the operation signal. The head mounted display according to claim 1,
The receiving unit further receives an image signal output from the input device,
The display processing unit displays the image on the image display element based on the image signal. The head mounted display according to claim 1 or 2,
The display processing unit moves the auxiliary image in a display area of the image based on the operation signal. The head mounted display according to any one of claims 1 to 3,
The operation signal includes information on the contact pressure of the detection target with respect to the input operation surface,
The display processing unit changes a display form of the auxiliary image according to a contact pressure of the detection target with respect to the input operation surface. The head mounted display according to any one of claims 1 to 4,
Based on the operation signal, further comprising a calculation unit for calculating the coordinate position of the detection target on the input operation surface,
The display processing unit displays the operation image on the image display element based on the coordinate position of the detection target calculated by the calculation unit. An input device comprising: an input operation surface; and a sensor unit that detects a contact of the detection target on the input operation surface and outputs information on the coordinate position of the detection target on the input operation surface;
An image display element that forms an image presented to the user, and a display processing unit that forms an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image on the image display element based on the output of the sensor unit An information display device comprising: a head-mounted display comprising: The information display device according to claim 6,
The input device is
A display element for displaying an image on the input operation surface;
The head mounted display displays an image obtained by superimposing the auxiliary image on an image displayed on the input operation surface as the operation image. The information display device according to claim 6 or 7,
The sensor unit detects an contact pressure applied to the input operation surface by the detection target. The information display device according to claim 8, wherein
The head mounted display changes the display form of the auxiliary image according to the contact pressure of the detection target from the operation surface. An information display device according to any one of claims 6 to 9,
The sensor unit detects a movement of the detection target on the input operation surface;
The head mounted display displays the auxiliary image that moves in accordance with the detected movement of the detection target. An input device having an input operation surface and a sensor unit that detects contact of the detection target with the input operation surface;
An image display element that forms an image presented to a user, a calculation unit that calculates the coordinate position of the detection target on the input operation surface based on an output of the sensor unit, and the detection calculated by the calculation unit An information display device, comprising: a display processing unit configured to form, on the image display element, an operation image in which an auxiliary image indicating a position of a detection target is superimposed on the image based on a coordinate position of the target.

Images