WO2013099081A1 - ヘッドマウントディスプレイ及び情報表示装置 - Google Patents

ヘッドマウントディスプレイ及び情報表示装置 Download PDF

Info

Publication number
WO2013099081A1
WO2013099081A1 PCT/JP2012/007032 JP2012007032W WO2013099081A1 WO 2013099081 A1 WO2013099081 A1 WO 2013099081A1 JP 2012007032 W JP2012007032 W JP 2012007032W WO 2013099081 A1 WO2013099081 A1 WO 2013099081A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
operation surface
display
input operation
detection target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2012/007032
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
翼 塚原
正俊 上野
憲一 樺澤
栗屋　志伸
後藤　哲郎
尚子 菅野
大介 川上
中川　俊之
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Priority to US14/367,030 priority Critical patent/US9779700B2/en
Priority to EP12862497.0A priority patent/EP2799958A1/en
Priority to CN201280063109.9A priority patent/CN103999030A/zh
Publication of WO2013099081A1 publication Critical patent/WO2013099081A1/ja
Anticipated expiration legal-status Critical
Priority to US15/702,288 priority patent/US20180005607A1/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/37—Details of the operation on graphic patterns
    • G09G5/377—Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416—Control or interface arrangements specially adapted for digitisers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0485—Scrolling or panning
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
    • G06F2203/04101—2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04801—Cursor retrieval aid, i.e. visual aspect modification, blinking, colour changes, enlargement or other visual cues, for helping user do find the cursor in graphical user interfaces
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04808—Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
    • G09G3/003—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to produce spatial visual effects

Definitions

  • This technology relates to a head mounted display and an information display device.
  • Patent Document 2 describes a transmissive HMD in which an input operation surface of a touch panel is arranged in a user's visual field range and an input operation is performed.
  • an object of the present technology is to provide a head mounted display and an information display device having excellent operability in accordance with the user's intuition.
  • a head mounted display includes a receiving unit, an image display element, and a display processing unit.
  • the reception unit receives an operation signal that is output from the input device and includes information on a relative position of a detection target that is in contact with the input operation surface.
  • the image display element forms an image presented to the user.
  • the display processing unit causes the image display element to display an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image based on the operation signal.
  • the user By wearing the head-mounted display (HMD), the user can visually recognize information on the relative position of a detection target such as a finger that touches the input operation surface with an auxiliary image in the operation image presented by the HMD. it can. Accordingly, the user can perform an intuitive operation while viewing the operation image by the HMD without confirming the input operation surface.
  • a detection target such as a finger that touches the input operation surface
  • an auxiliary image in the operation image presented by the HMD it can. Accordingly, the user can perform an intuitive operation while viewing the operation image by the HMD without confirming the input operation surface.
  • the receiving unit may further receive an image signal output from the input device, and the display processing unit may display the image on the image display element based on the image signal.
  • the HMD configured as described above can use not only a dedicated input device but also a mobile terminal equipped with a touch panel, for example, as an input device. Thereby, various applications such as a portable terminal can be used by the HMD.
  • the display processing unit may move the auxiliary image in the display area of the image based on the operation signal.
  • the user can confirm the temporal relative position of the detection target in contact with the input operation surface from the operation image, and the operability can be further improved.
  • the operation signal includes information regarding the contact pressure of the detection target with respect to the input operation surface, and the display processing unit displays a display form of the auxiliary image according to the contact pressure of the detection target with respect to the input operation surface. It may be changed. Accordingly, the HMD can reflect, for example, a “push” operation on the input operation surface of the user on the operation image. Therefore, the user can perform various operations combining the touch on the input operation surface and “push”.
  • the head mounted display further includes a calculation unit that calculates the coordinate position of the detection target on the input operation surface based on the operation signal, and the display processing unit is configured to detect the detection calculated by the calculation unit.
  • the operation image may be displayed on the image display element based on the target coordinate position.
  • the head mounted display can process a signal detected from an input device, and can be used even with an input device having a simple apparatus configuration.
  • an information display apparatus includes an input device and a head mounted display.
  • the input device includes an input operation surface and a sensor unit that detects contact of the detection target with the input operation surface and outputs information on the coordinate position of the detection target on the input operation surface.
  • the head mounted display includes an image display element that forms an image to be presented to a user, and an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image based on the output of the sensor unit. And a display processing unit to be formed.
  • the information display device can present an image based on the input operation to the input device to the user by the head-mounted display, and the user can perform an intuitive operation without checking the input operation surface. It becomes.
  • the sensor unit may be configured to detect a contact pressure applied to the input operation surface by the detection target. Furthermore, at this time, the head mounted display may change the display form of the auxiliary image according to the contact pressure of the detection target from the operation surface.
  • the input device can detect a contact pressure (pushing force) by a detection target with respect to the input operation surface, and can reflect the information on an operation image by the HMD. Thereby, the user can perform more various operations.
  • the sensor unit may detect the movement of the detection target on the input operation surface, and the head mounted display may display the auxiliary image that moves in accordance with the detected movement of the detection target. .
  • the input device further includes a display element that displays an image on the input operation surface,
  • the head mounted display may display an image obtained by superimposing the auxiliary image on the image displayed on the input operation surface as the operation image.
  • an information display apparatus includes an input device and a head mounted display.
  • the input device includes an input operation surface and a sensor unit that detects contact of the detection target with the input operation surface.
  • the head-mounted display includes an image display element that forms an image presented to a user, a calculation unit that calculates a coordinate position of the detection target on the input operation surface based on an output of the sensor unit, and the calculation unit And a display processing unit that causes the image display element to form an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image based on the coordinate position of the detection target calculated in step (1).
  • the information display device can simplify the configuration of the input device. Thereby, weight reduction of an input device etc. can be implement
  • FIG. 1 is a block diagram showing an internal configuration of a head mounted display (HMD) according to a first embodiment of the present technology.
  • HMD head mounted display
  • 6 is a flowchart in an operation example of the information processing apparatus according to the first embodiment of the present technology. It is a figure explaining the typical operation example of the information display apparatus which concerns on 1st Embodiment of this technique, (A) shows the input operation surface of the input device which the user is performing input operation, (B ) Shows an operation screen presented to the user via the HMD.
  • FIG. 12 is a flowchart illustrating an operation example of the information display device according to the second embodiment of the present technology. It is a block diagram showing an internal configuration of an input device concerning a 3rd embodiment of this art. It is a block diagram which shows the internal structure of HMD which concerns on the 3rd Embodiment of this technique. It is a figure explaining the typical operation example of the information display apparatus which concerns on 3rd Embodiment of this technique, (A) shows the input operation surface of the input device which the user is performing input operation, (B ) Shows an operation screen presented to the user via the HMD. It is a block diagram showing an internal configuration of HMD concerning a 4th embodiment of this art.
  • a 5th embodiment of this art it is a figure showing an example where an auxiliary picture on a picture of HMD becomes large according to contact pressure with a finger, and (A) is a user's finger lightly touching an input operation side. A mode is shown, (B) shows the operation image of HMD in that case.
  • a 5th embodiment of this art it is a figure showing an example where an auxiliary picture on a picture of HMD becomes large according to contact pressure with a finger, and (A) is a user's finger pushed on an input operation surface. (B) shows an operation image of the HMD at that time.
  • FIG. 1 is a schematic perspective view showing an information display device according to an embodiment of the present technology.
  • the information display apparatus 100 according to the present embodiment includes a head mounted display (HMD) 1 and an input device 2.
  • HMD head mounted display
  • input device 2 input device 2.
  • the HMD 1 is configured as a transmissive HMD in this embodiment.
  • the HMD 1 has a glasses-like shape as a whole, and is configured to allow the user to present an image based on information output from the input device 2 while wearing the head and visually recognizing the outside world. .
  • the input device 2 is connected to be able to communicate with the HMD 1 wirelessly, but may be configured to be able to communicate with the HMD 1 by a cable or the like.
  • the input device 2 is comprised with the portable terminal which has a touchscreen, for example, and is comprised so that the internet etc. can be connected.
  • the input device 2 functions as a so-called remote controller of the HMD 1 and can be configured to allow input operations related to various settings of the HMD 1.
  • the input device 2 includes, for example, a housing 2A that can be gripped by the user.
  • the housing 2A is a substantially rectangular parallelepiped having a longitudinal direction in the x-axis direction, a short direction in the y-axis direction, and a thickness direction in the z-axis direction, and an input operation surface 21 is formed on one surface of the housing 2A.
  • the input operation surface 21 belongs to a two-dimensional coordinate system having a coordinate axis on the x-axis and a y-axis orthogonal thereto, and has a long side parallel to the x-axis direction and a short side parallel to the y-axis direction. It has a rectangular shape perpendicular to the z-axis.
  • the input device 2 has a function of detecting the xy coordinate position of the finger on the input operation surface 21 and a change thereof, for example, with the user's finger as a detection target.
  • the input device 2 further has a function of detecting the amount of bending of the input operation surface 21 by the finger in the z-axis direction, that is, the contact pressure. Thereby, the contact pressure in the z-axis direction on the input operation surface 21 by the finger is acquired together with the movement direction, movement speed, movement amount, and the like of the finger on the input operation surface 21.
  • the detection target is not limited to the user's finger, and may be a stylus or the like.
  • the input operation surface 21 is configured by a light-transmissive display cover or the like, and performs an input operation depending on a detection target and also has a function as a screen for displaying an image.
  • the input operation surface 21 is disposed so as to cover an opening 21A formed on the surface of the housing 2A, and the periphery thereof is fixed to the housing 2A.
  • the material, thickness, size, and the like of the input operation surface 21 are set so that the input operation surface 21 can be bent and deformed with respect to a predetermined contact pressure or more.
  • a material constituting the input operation surface 21 for example, an acrylic resin, a polycarbonate resin, a transparent plastic plate such as PET (polyethylene terephthalate), a glass plate, a ceramic plate, or the like is employed.
  • FIG. 2 is a block diagram showing the internal configuration of the input device 2.
  • the input device 2 includes a housing 2A, an input operation surface 21, a touch sensor 22, a pressure sensor 23, a display element 24, a control unit 25, a storage unit 26, a transceiver 27, and a communication unit 28. And a battery BT.
  • the touch sensor 22 and the pressure sensor 23 constitute a “sensor unit” in the present embodiment.
  • the touch sensor 22 has a panel shape having almost the same shape and size as the input operation surface 21.
  • the touch sensor 22 is disposed immediately below the input operation surface 21 and detects a detection target (finger) that contacts the input operation surface 21.
  • the touch sensor 22 detects a coordinate position corresponding to the movement of the detection target on the input operation surface 21 on the xy plane, and outputs a detection signal corresponding to the coordinate position.
  • the touch sensor 22 is a capacitive touch panel that can electrostatically detect a detection target that contacts the input operation surface 21.
  • the capacitive touch panel may be a projected capacitive type (projection type) or a surface capacitive type (surface type).
  • This type of touch sensor 22 is typically parallel to the x-axis direction and the first sensor 22x for x-position detection in which a plurality of first wirings parallel to the y-axis direction are arranged in the x-axis direction.
  • the second wiring 22 has a plurality of position detection second sensors 22y arranged in the y-axis direction, and the first and second sensors 22x and 22y are arranged to face each other in the z-axis direction.
  • the touch sensor 22 is sequentially supplied with signal currents to the first and second wirings, for example, by a drive circuit included in the control unit 25 described later.
  • the touch sensor 22 is not particularly limited as long as it can detect the coordinate position of the detection target, and is a resistive film type, an infrared type, an ultrasonic type, a surface acoustic wave type, an acoustic wave matching type, an infrared ray type, and the like.
  • Various types such as an image sensor are applicable.
  • a pressure-sensitive sensor capable of electrostatically detecting the contact pressure in the Z-axis direction of the input operation surface 21 by the detection target is used.
  • the pressure-sensitive sensor 23 has, for example, one or more pairs of electrodes that are arranged between the input operation surface 21 and the housing 2 ⁇ / b> A and face each other in the z-axis direction.
  • the contact pressure is detected by the change in capacitance.
  • a sensor in which an elastic body is disposed between the pair of electrodes may be employed as the pressure sensor 23. In this case, an electrode generated by elastic deformation of the elastic body accompanying a pressing operation to the input operation surface 21.
  • the contact pressure is detected by a change in capacitance between the two.
  • the pressure sensor 23 can be driven by, for example, a drive circuit included in a control unit 25 described later.
  • the pressure sensor 23 is not limited to the above configuration.
  • a piezoelectric sensor using a piezoelectric element, a strain gauge, or the like can be used.
  • the display element 24 is disposed immediately below the input operation surface 21 and the touch sensor 22.
  • the display element 24 in the present embodiment is not particularly limited, and a liquid crystal display, an organic EL display, or the like can be used. As a result, it is possible to display images such as characters and pictures on the input operation surface 21.
  • the control unit 25 is typically composed of a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit).
  • the control unit 25 includes a calculation unit 251 and a signal generation unit 252, and executes various functions according to programs stored in the storage unit 26.
  • the calculation unit 251 performs a predetermined calculation process on the electrical signals output from the touch sensor 22 and the pressure sensor 23 and includes an operation signal including information on the relative position of the detection target that contacts the input operation surface 21. Is generated.
  • the signal generation unit 252 generates an image signal for displaying an image on the display element 24 based on these calculation results.
  • the control unit 25 has a drive circuit for driving the touch sensor 22, and the drive circuit is incorporated in the calculation unit 251 in the present embodiment.
  • the calculation unit 251 calculates the xy coordinate position and the contact pressure of the finger on the input operation surface 21 based on signals output from the touch sensor 22 and the pressure sensor 23. Further, based on the above calculation result, for example, the calculation unit 251 detects that the detection target is at a predetermined xy coordinate position, and further detects a contact pressure equal to or higher than a predetermined threshold at the coordinate position. A predetermined process assigned to the GUI corresponding to the coordinate position is executed. These processing results by the calculation unit 251 are transmitted to the signal generation unit 252.
  • the signal generator 252 generates an image signal for forming an image to be displayed on the display element 24 based on the processing result transmitted from the calculator 251. At this time, the signal generation unit 252 generates a signal for displaying an auxiliary image such as a pointer or a region in which luminance or saturation has changed at a position corresponding to the xy coordinate position of the detection target of the image on the input operation surface 21. Is also possible. The signal generation unit 252 can also generate a signal that changes the display form of the auxiliary image according to the contact pressure of the detection target.
  • the operation signal related to the xy coordinate position and the contact pressure calculated by the calculation unit 251 of the control unit 25 and the image signal generated by the signal generation unit 252 are configured to be transmitted to the HMD 1 via the transceiver 27.
  • the input device 2 is an A / D converter that converts a detection signal (analog signal) output from the touch sensor 22 or the pressure sensor 23 into a digital signal, or a D that converts a digital signal into an analog signal. Includes / A converter.
  • the storage unit 26 includes a RAM (Random Access Memory), a ROM (Read Only Memory), and other semiconductor memories, and stores programs used for various operations by the control unit 25.
  • the ROM is configured by a non-volatile memory, and stores a program and a set value for causing the control unit 25 to perform arithmetic processing such as calculation of the xy coordinate position and the contact pressure.
  • the storage unit 26 can store, for example, a xy coordinate position and a contact pressure to be detected, and a program for executing a function assigned corresponding to the xy coordinate position to be detected by a nonvolatile semiconductor memory. .
  • these programs stored in advance in a semiconductor memory or the like may be loaded into the RAM and executed by the calculation unit 251 of the control unit 25.
  • the transceiver 27 is configured to be able to transmit various control signals generated by the control unit 25 to the HMD 1 wirelessly, for example.
  • the transceiver 27 is configured to be able to receive a predetermined signal transmitted from the HMD 2.
  • the communication unit 28 is connected to a communication network such as the Internet.
  • the communication unit 28 is used when, for example, a predetermined program such as an application is downloaded to the input device 2. Transmission / reception of information in the communication unit 28 may be performed by wire using a LAN cable or the like, or may be performed wirelessly such as high-speed data communication.
  • the battery BT constitutes a power source for the input device 2 and supplies necessary power to each part inside the housing 2A.
  • the battery BT may be a primary battery or a secondary battery. Battery BT may be formed of a solar cell.
  • the input device 2 can also have an external switch (not shown) that controls activation of the input device 2 by pressing.
  • the HMD 1 is a transmissive HMD, and is configured to be able to present a predetermined image in the visual field region while allowing the user to visually recognize the outside world.
  • the HMD 1 according to the present embodiment includes an image display device 11 and a support unit 16 and has a glasses-like shape as a whole.
  • a transmissive HMD as the HMD 1, it is possible to visually recognize the outside world even during operation of the input device 2, and safety during operation can be improved.
  • the support unit 16 can be attached to the user's head, and is configured such that an image display element 14 described later is supported in front of the user's eyes when attached.
  • the shape of the support portion 16 is not particularly limited, but in the present embodiment, the shape can be a glasses shape as a whole.
  • the support portion 16 has, for example, a temple-like structure that can be worn on the left and right ears of the user.
  • a nose pad 161 can be attached near the nose bridge of the support portion 16.
  • the earphone 162 may be mounted on the support portion 16.
  • a metal, an alloy, a plastic, or the like can be adopted as the material constituting the support portion 16 and is not particularly limited.
  • the image display device 11 includes a housing 11A and is disposed at a predetermined position of the support portion 16, for example.
  • the arrangement of the image display device 11 is not particularly limited.
  • the housing 11A is arranged outside the left and right eyes of the user so as not to be included in the user's visual field region.
  • FIG. 3 is a block diagram showing the internal configuration of the image display device 11.
  • the image display device 11 includes a housing 11A, a transceiver (reception unit) 12, a control unit 13, an image display element 14, and a storage unit 15.
  • the control unit 13 constitutes a “display processing unit”.
  • the transmitter / receiver 12 receives an operation signal that is output from the transmitter / receiver 27 of the input device 2 and includes information on the xy coordinate position of the detection target that contacts the input operation surface 21 and the contact pressure. Furthermore, it is also possible to receive an image signal of an image displayed on the input operation surface 21 of the input device 2.
  • the signal transmission method is not particularly limited. For example, inter-device communication such as “Wi Fi” (registered trademark), “ZigBee” (registered trademark), “Bluetooth” (registered trademark), communication via the Internet, or the like can be used as wireless communication. Also, a wired connection via USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) or the like is also possible.
  • the control unit 13 is typically composed of a CPU or MPU, and executes various functions according to a program stored in the storage unit 15.
  • the control unit 13 generates a control signal for causing the image display element 14 to form an image presented to the user based on the operation signal and the image signal received by the transceiver 12. Accordingly, for example, an operation image in which an auxiliary image indicating the position of the detection target is superimposed on an image similar to the image displayed on the input operation surface 21 can be displayed on the image display element 14.
  • the shape of the auxiliary image is not particularly limited, and is composed of, for example, an annular (ring-shaped) pointer or a region where luminance and saturation are changed.
  • the image display element 14 includes a liquid crystal display element (LCD) 141 and an optical system 142, and is configured to present an image formed by the LCD 141 to the user via the optical system 142.
  • LCD liquid crystal display element
  • the LCD 141 has a plurality of pixels arranged in a matrix.
  • the LCD 141 modulates light incident from a light source (not shown) formed of an LED (light emitting diode) or the like for each pixel according to a control signal generated by the control unit 13, and forms light that forms an image presented to the user.
  • the LCD 141 may be, for example, a single plate type that simultaneously emits image light corresponding to each color of R (red), G (green), and B (blue), or three plates that individually emit image light corresponding to each color. The method may be used.
  • the optical system 142 is configured to deflect light emitted from the LCD 141 and guide the light to the eyes of the user.
  • the optical system 142 can be composed of, for example, a light guide plate made of a transparent substrate capable of guiding light by total reflection, a reflective volume hologram diffraction grating capable of diffraction reflection, and the like.
  • the light guide plate is formed in, for example, a rectangular or circular flat plate shape, and is disposed in front of the user's eyes like a lens of glasses.
  • a hologram diffraction grating or the like is appropriately disposed in the light guide plate, and is manufactured so as to reflect the light emitted from the LCD 141 and guide it to the eyes of the user.
  • a reflector or the like can be employed.
  • the optical system 142 configured as described above can present an operation image in the visual field area of the user by reflecting the light emitted from the LCD 141 at a predetermined position.
  • the light guide plate is formed of a transparent substrate, it is possible to present the operation image in a superimposed manner in the visual field region while allowing the user to visually recognize the outside world.
  • the storage unit 15 includes a RAM, a ROM, and other semiconductor memories, and stores programs used for various calculations by the control unit 13.
  • the speaker 17 converts an electrical audio signal transmitted from the input device 2 or generated by the control unit 13 or the like into physical vibration, and provides audio to the user via the earphone 162.
  • the configuration of the speaker 17 is not particularly limited.
  • the image display apparatus 11 is not limited to the configuration in which two image display devices 11 are arranged corresponding to both eyes.
  • FIG. 4 is a flowchart in an operation example of the HMD 1 (control unit 13) and the input device 2 (control unit 25).
  • FIG. 5 is a diagram for explaining a typical operation example of the information display apparatus 100.
  • FIG. 5A shows the input operation surface 21 of the input device 2 on which the user performs an input operation
  • FIG. The operation screen shown to a user via HMD1 is shown.
  • an operation example of the information display apparatus 100 when the user wears the HMD 1 and performs a pushing operation at a predetermined position on the input operation surface 21 of the input device 2 is shown.
  • the X-axis direction and the Y-axis direction in the figure correspond to the x-axis direction and the y-axis direction of the input operation surface 21, respectively, and both indicate planar directions that are orthogonal to each other.
  • the Z-axis direction corresponds to the z-axis direction of the input operation surface 21 and indicates directions orthogonal to the X-axis direction and the Y-axis direction, respectively.
  • an image v1 on which a large number of GUIs are displayed is displayed on the input operation surface 21 of the activated input device 2 (FIG. 5A).
  • the image v1 is, for example, a menu selection screen for various settings of the HMD1, and each GUI corresponds to switching to the mute mode of the HMD1, volume adjustment, enlargement / reduction of the operation image, change of the display form of the pointer, and the like.
  • the input device 2 is configured to be able to change the setting of the HMD 1 by selecting a specific GUI by the user.
  • the user who is connected to the input device 2 and wears the activated HMD1 is also presented with the image V1 similar to the image v1 in the visual field region via the HMD1.
  • the input device 2 determines the touch of the user's finger (detection target) on the input operation surface 21 by the touch sensor 22 (step ST101).
  • the touch sensor 22 outputs a detection signal related to the xy coordinate position on the input operation surface 21 touched by the finger to the control unit 25.
  • the calculation unit 251 of the control unit 25 calculates the xy coordinate position of the finger on the input operation surface 21 based on the detection signal (step ST102). A signal related to the xy coordinate position calculated by the calculation unit 251 is output to the transceiver 27.
  • the transmitter / receiver 27 transmits an operation signal related to the xy coordinate position to the transmitter / receiver 12 of the HMD 1.
  • the control unit 13 of the HMD 1 Based on the operation signal and the image signal received by the transceiver 12, the control unit 13 of the HMD 1 generates a signal for controlling the operation image V10 in which the auxiliary image (pointer P) indicating the position of the detection target is superimposed on the image V1. .
  • the image display element 14 to which this control signal has been output presents the operation image V10 to the user (step ST103, FIG. 5B).
  • the user can perform a desired operation by checking the movement of the pointer P on the operation image V10 presented by the HMD 1 without looking at the input operation surface 21 at hand, and the user's intuition. It is possible to perform an operation in accordance with.
  • the image displayed on the input operation surface 21 may be only the image v1 or an image in which a pointer or the like is superimposed on the image v1.
  • control unit 13 can output a control signal for moving the pointer P to the LCD 141 based on the operation signal, and the pointer P in the display area of the image V1 can be displayed according to the movement of the user's finger. It can be moved.
  • control unit 13 can output a control signal for moving the pointer P to the LCD 141 based on the operation signal, and the pointer P in the display area of the image V1 can be displayed according to the movement of the user's finger. It can be moved.
  • the control unit 25 determines the contact state between the finger and the input operation surface 21 over time (step ST105).
  • the touch sensor 22 outputs a signal when the finger is not touching.
  • the control unit 25 determines non-contact between the input operation surface 21 and the finger based on the output of the touch sensor 22 (YES in step ST105). In this case, the pointer P disappears from the operation image V10 of the HMD1, and the screen returns to the image V1.
  • the touch sensor 22 outputs a signal when the finger is in contact. Based on the output of the touch sensor 22, the control unit 25 determines that the user's finger is in contact with the input operation surface 21 (NO in step ST105), and then inputs based on the output of the pressure sensor 21. The contact pressure of the finger on the operation surface 21 is detected (step ST106).
  • the selection candidate GUI is selected (determined) by the user's pressing operation on the display position of the selection candidate GUI.
  • the control unit 25 Based on the output of the pressure-sensitive sensor 23, the control unit 25 calculates the contact pressure in the calculation unit 251. If the calculated contact pressure is equal to or greater than the predetermined threshold (YES in step ST106), the control unit 25 determines the selection candidate GUI as the selection GUI, and uses this selection GUI stored in the storage unit 26 as a selection GUI. Corresponding code information is acquired (step ST107).
  • step ST106 when the calculated contact pressure is less than the predetermined threshold (NO in step ST106), the control unit 25 determines that the selection candidate GUI has not been selected. In this case, the control part 25 returns to step ST105, and repeats the process mentioned above.
  • control unit 25 transmits the acquired code information to the HMD 1 via the transceiver 27 (step ST108).
  • the control unit 13 of the HMD 1 executes processing corresponding to the selected GUI based on the code information received via the transceiver 12. This process is executed based on, for example, a program stored in the storage unit 15. For example, when the function corresponding to the selected GUI is “switching to the mute mode”, the control unit 13 executes the process based on the code information corresponding to the GUI to set the setting of the HMD 1 to the mute mode. It is possible to switch.
  • the control unit 13 can generate an image control signal based on the code information and output it to the image display element 14.
  • the user wearing the HMD 1 is presented with a new operation image (not shown) on which, for example, a volume adjustment bar is superimposed.
  • the image control signal generated by the control unit 13 can be output from the transceiver 12 to the input device 2.
  • an image similar to the new operation image presented by the HMD 1 can be displayed on the input operation surface 21 of the input device 2.
  • the input device 2 has a pressure sensor 23. Therefore, when the user presses a finger on the input operation surface 21, it is possible to cause the input device 1 and the HMD 1 to execute processing corresponding to a desired GUI. Thereby, for example, even when the input device 2 is inside a bag or a pocket, it is possible to perform an input operation from the outside while being placed in the bag, thereby improving convenience. In addition, various operations can be performed by combining the contact operation with the input operation surface 21 and the push-in operation.
  • the GUI is not easily selected only by the touch, and the malfunction can be prevented.
  • the portable terminal can be easily used even for an elderly person who is difficult to see a small screen. It becomes possible to do.
  • the image v1 is also displayed on the input operation surface 21 of the input device 2.
  • the input device 2 is not limited to a device dedicated to the HMD 1, and a portable terminal equipped with a touch panel display can be connected and used. Therefore, in this case, it is possible to enjoy the above-described effects of the HMD 1 while using abundant applications of the mobile terminal.
  • FIG. 6 is a flowchart illustrating an operation example of the information display apparatus according to the second embodiment of the present technology.
  • the description of the same parts as those of the first embodiment will be omitted or simplified, and different parts from the first embodiment will be mainly described.
  • the operation image presented to the user by the HMD 1 and the image displayed on the input operation surface 21 will be described as, for example, a menu selection image in which a large number of GUIs are arranged.
  • the control unit 25 moves the auxiliary image (pointer P) in the operation image presented to the user and the selected GUI to a desired position, and sets the GUI. It is possible to change the arrangement. That is, according to the present embodiment, a “drag and drop” operation is possible.
  • control unit 25 detects contact of the user's finger on the input operation surface 21 (YES in step ST201), and calculates the xy coordinate position where the finger touches (step ST202). Based on the xy coordinate position, the control unit 13 of the HMD 1 displays the pointer P on the image V1 (ST203).
  • control unit 25 of the input device 2 selects the GUI closest to the xy coordinate position as a selection candidate (step ST204).
  • Control unit 25 determines the contact state of the finger based on the output from touch sensor 22 (NO in step ST205), and determines that the contact pressure in the z-axis direction output from pressure-sensitive sensor 23 is equal to or greater than the threshold ( In step ST206, it is determined whether or not a contact pressure equal to or higher than the threshold is output for a predetermined time or more (step ST207).
  • control unit 25 if it is equal to or shorter than the predetermined time (NO in step ST207), the control unit 25 returns to step ST206 and repeats the above-described processing.
  • step ST207 when the contact pressure is detected for a predetermined time or longer (YES in step ST207), the control unit 25 determines that the selection candidate GUI is a draggable GUI (hereinafter referred to as draggable GUI) (step ST208).
  • draggable GUI a draggable GUI
  • the control unit 25 calculates the xy coordinate position detected from the touch sensor 22 over time as the finger moves on the input operation surface 21 (step ST209). Further, this signal is output over time to the control unit 13 of the HMD 1 via the transceiver 27. The control unit 13 generates an image signal for moving the pointer P and the draggable GUI on the image V10 based on the change in the xy coordinate position (step ST210).
  • the control unit 25 calculates the contact pressure output from the pressure-sensitive sensor 23 over time, and continues to determine whether the contact pressure is equal to or greater than the threshold (YES in step ST211).
  • the control unit 25 executes a process of “dropping” the draggable GUI. (Step ST213).
  • the position where the draggable GUI is dropped is a position corresponding to the xy coordinate position when the control unit 25 determines that the contact pressure is less than the threshold (NO in step ST211).
  • control part 25 determines non-contact (it is NO at step ST214)
  • the control part 25 will return to step ST206 and will determine again whether the contact pressure output from the pressure sensor 23 is more than a threshold value. .
  • the information display device 100 allows the user to move the GUI, the folder, and the like in the operation image by gazing only at the operation image of the HMD 1 without checking the input operation surface 21 at hand. be able to. That is, the “drag and drop” operation can be performed by the outputs of the touch sensor 22 and the pressure sensor 23, and the operability of the user can be further improved.
  • FIG. 7 to 9 are diagrams describing a third embodiment of the present technology.
  • FIG. 7 is a block diagram illustrating an internal configuration of the input device according to the present embodiment
  • FIG. 8 is a block diagram illustrating an internal configuration of the image display apparatus according to the head mounted display of the present embodiment.
  • FIG. 9 is a diagram for explaining a typical operation example of the information display device.
  • FIG. 9A shows an input operation surface of an input device on which a user performs an input operation
  • FIG. 9B shows an operation via an HMD. The operation screen presented to the user is shown.
  • the description of the same parts as those of the first embodiment will be omitted or simplified, and different parts from the first embodiment will be mainly described.
  • the input device 20 of this embodiment is different from the first embodiment described above in that it does not have a display element and a control unit, and an image is not displayed on the input operation surface 210. That is, the detection signal of the xy coordinate position by the touch sensor 220 and the detection signal of the contact pressure by the pressure sensor 230 can be transmitted to the HMD 10 via the transceiver 270 as operation signals. Thus, as shown in FIG. 9A, no image is displayed on the input operation surface 210, and the input device 20 can be configured to have only a so-called touch panel. That is, the input device 20 according to the present embodiment can be a remote controller dedicated to the HMD 10.
  • the input operation surface 210 of the input device 20 is only subjected to an input operation based on a detection target, and thus can be made of a material that does not have optical transparency.
  • a synthetic resin that can be bent and deformed with respect to a predetermined contact pressure or more can be employed.
  • the control unit 130 according to the image display device 110 of the HMD 10 includes a calculation unit 131 and a display processing unit 132 in the present embodiment.
  • the calculation unit 131 calculates the xy coordinate position and the contact pressure from the detection signal of the xy coordinate position by the touch sensor 220 and the detection signal of the contact pressure by the pressure sensor 230, respectively.
  • the display processing unit 132 generates an image signal for forming an image to be displayed on the image display element 140 based on the processing result input from the calculation unit 131.
  • the display processing unit 132 can also generate a signal for displaying an auxiliary image such as a pointer at a position corresponding to the xy coordinate position of the image detection target on the input operation surface 210.
  • the HMD 10 can display an operation image V30 similar to the operation image V10 according to the first embodiment to the user.
  • the image presented to the user by the HMD 10 can be acquired via the communication unit 280 of the input device 20 from the Internet or the like. It is also possible to provide the communication unit 170 in the HMD 10 and generate an image based on information directly acquired by the HMD 10.
  • the configuration of the input device 20 can be simplified. As a result, the input device 20 can be reduced in size and weight, and it is possible to provide the input device 20 that is less likely to get tired and easy to carry even with a long touch operation.
  • FIG. 10 is a block diagram illustrating an internal configuration of the input device according to the fourth embodiment of the present technology.
  • parts similar to those in the configuration and operation of the first embodiment are denoted by the same reference numerals, description thereof will be omitted or simplified, and description will be focused on parts different from the first embodiment.
  • the input device 200 is different from the first embodiment described above in that it does not have a pressure-sensitive sensor. That is, in the present embodiment, the input device 200 can detect the coordinate position corresponding to the movement of the detection target on the input operation surface 21 on the xy plane and the contact pressure by the touch sensor 2200. Configured.
  • the touch sensor 2200 constitutes a “sensor unit” in the present embodiment.
  • the touch sensor 2200 for example, a capacitive touch panel capable of electrostatically detecting a detection target in contact with the input operation surface 21 can be used as in the first embodiment.
  • the touch sensor 2200 detects a coordinate position corresponding to the movement of the detection target on the input operation surface 21 on the xy plane, and outputs a detection signal corresponding to the coordinate position to the control unit 2500.
  • the touch sensor 2200 detects the capacitance according to the contact pressure of the detection target or the amount of change thereof. As a result, the touch sensor 2200 can output a detection signal corresponding to the contact pressure at the coordinate position.
  • a resistive film type touch panel can be used.
  • the resistive film type touch sensor 2200 typically has two transparent electrodes or the like disposed opposite to each other with a predetermined gap immediately below the input operation surface 21 and has a size corresponding to the contact pressure of the detection target.
  • the voltage (contact resistance) is output.
  • the control unit 2500 can calculate the contact pressure depending on the detection target.
  • various touch panel sensors such as an infrared type, an ultrasonic type, and a surface acoustic wave type can be applied as the touch sensor according to the present embodiment.
  • the control unit 2500 is configured such that the calculation unit 2510 can calculate the xy coordinate position of the detection target and the contact pressure at the position based on the output from the touch sensor 2200.
  • control unit 2500 an operation example of the input device 200 (control unit 2500) will be briefly described.
  • an example in which a specific GUI on an image displayed on the input operation surface 21 and the HMD 1 is selected by the user will be described.
  • control unit 2500 acquires a detection signal of the xy coordinate position from the touch sensor 2200. Thereby, control unit 2500 determines the GUI or the like closest to the xy coordinate position as the selection candidate GUI or the like (see step ST104 in FIG. 4).
  • control unit 2500 determines whether the change amount is equal to or greater than a predetermined threshold using the change amount of the capacitance output from the touch sensor 2200 as a contact pressure detection signal (see step ST106 in FIG. 4). If it is equal to or greater than the threshold value, the processing corresponding to the selection GUI or the like is executed as in the first embodiment (see step ST107 in FIG. 4).
  • control unit 2500 is configured to determine whether the capacitance value is equal to or greater than a predetermined threshold, using the capacitance value output from the touch sensor 2200 or its change amount itself as a contact pressure detection signal. Is also possible.
  • the information display apparatus having the input device 200 configured as described above can contribute to cost reduction and productivity improvement, and the input device 200 can be reduced in weight and size.
  • ⁇ Fifth Embodiment> 11 to 14 are diagrams for explaining a fourth embodiment of the present technology.
  • the same components as those in the first embodiment will be denoted by the same reference numerals, the description thereof will be omitted or simplified, and the description will focus on portions that are different from the first embodiment.
  • This embodiment is different from the first embodiment in that the display form of the auxiliary image (pointer) P4 (P411 to 422) is changed according to the contact pressure of the detection target (finger) with respect to the input operation surface 21.
  • FIG. 11 and 12 show an example in which the annular pointer P411 on the image V4 increases in accordance with the contact pressure by the finger.
  • the operation image V411 on which the pointer P411 is superimposed with a predetermined size is displayed (FIG. 11B).
  • FIG. 12 (B) an operation image V412 in which a pointer P412 larger than the pointer P411 is superimposed on the image V4 is displayed ( FIG. 12 (B)).
  • the control unit 25 determines that the contact pressure acquired by the pressure sensor 23 is equal to or higher than a predetermined threshold value
  • the control unit 25 outputs the information to the control unit 13 of the HMD1.
  • the control part 13 produces
  • pointers P411 and P412 having a size corresponding to the contact pressure are displayed on the operation images V411 and 412 at positions corresponding to the finger contact positions on the input operation surface 21.
  • FIG. 13 and 14 show an example in which the colors of the annular pointers P421 and P422 on the image V4 change according to the contact pressure by the finger.
  • the pointer P421 becomes a black operation image V421 (FIG. 13)
  • An operation image V422 in which a gray pointer P422 is superimposed on the image V4 is displayed (FIG. 14).
  • the user can visually confirm his / her push-in amount (contact pressure). As a result, the user can more accurately perform the operation combining the contact and the push-in by confirming the operation image by the HMD 1.
  • FIGS. 15 and 16 are diagrams illustrating a modification of the second embodiment.
  • the information display apparatus 100 is configured such that the touch sensor 22 and the pressure sensor 23 of the input device 2 can detect two or more contact positions on the input operation surface 21, respectively. Furthermore, by moving these fingers while pressing them into the input operation surface 21, it is possible to perform a so-called “pinch zoom” operation for enlarging or reducing the images V21 and V22.
  • the control unit 25 determines that the contact pressure acquired by the pressure-sensitive sensor 23 is equal to or higher than a predetermined threshold. For example, when the user widens these distances while pressing two fingers on the input operation surface 21 as shown in FIG. 15A, the detected distance and its change are shown as shown in FIG. Accordingly, the image V21 can be enlarged. On the other hand, when the user narrows these distances while pressing two fingers on the input operation surface 211 as shown in FIG. 16A, the detected distance and its change are shown in FIG. 16B. Accordingly, the image V22 can be reduced.
  • the “auxiliary image” can be an image of a region in which luminance, saturation, and the like are changed.
  • control unit 25 calculates the xy coordinate position of the finger on the input operation surface 21 based on the detection signal from the touch sensor 22 and outputs the information to the control unit 13 of the HMD 1. Based on the information, the control unit 13 generates a control signal for changing the luminance and saturation of the corresponding XY coordinate position.
  • the HMD 1 can present a user with operation images having different brightness, saturation, and the like between the region corresponding to the contact position of the finger and other regions. Therefore, the user can also confirm the contact position of a finger or the like on the input operation surface by visually recognizing the operation image displayed by the HMD 1 according to the above modification.
  • the input device is described as a portable terminal that is held and used by the user, but is not limited thereto.
  • it may be a wristwatch type worn on the user's wrist or a wristband type worn on the user's arm. Further, it may be in the form of being attached to or attached to clothes.
  • various types of input devices can be used as long as the detection target can contact the input operation surface.
  • the transmissive HMD has been described.
  • the present invention is not limited to this, and a non-transmissive HMD may be used.
  • the user can perform an operation in an intuitive manner without looking at the input operation surface of the input device by performing the operation while visually recognizing the image presented by the HMD.
  • an image sensor can be mounted on the HMD so that the outside can be photographed.
  • a captured image can be displayed on the HMD and the input device.
  • the shape of the pointer has been described as an example of a ring, but of course it is not limited to this.
  • any shape such as an arrow, a triangle, a quadrangle, and a circle can be employed.
  • this technique can also take the following structures.
  • a receiving unit that receives an operation signal that is output from the input device and includes information related to a relative position of a detection target that contacts the input operation surface; An image display element for forming an image presented to a user; And a display processing unit that displays, on the image display element, an operation image obtained by superimposing an auxiliary image indicating the position of the detection target on the image based on the operation signal.
  • the receiving unit further receives an image signal output from the input device, The display processing unit displays the image on the image display element based on the image signal.
  • the display processing unit moves the auxiliary image in the image display area based on the operation signal.
  • the head mounted display according to any one of (1) to (3) above,
  • the operation signal includes information on the contact pressure of the detection target with respect to the input operation surface
  • the display processing unit changes a display form of the auxiliary image according to a contact pressure of the detection target with respect to the input operation surface.
  • the display processing unit Based on the operation signal, further comprising a calculation unit for calculating the coordinate position of the detection target on the input operation surface,
  • the display processing unit displays the operation image on the image display element based on the coordinate position of the detection target calculated by the calculation unit.
  • An input device having an input operation surface and a sensor unit that detects a contact of the detection target on the input operation surface and outputs information on the coordinate position of the detection target on the input operation surface;
  • An image display element that forms an image to be presented to the user, and a display processing unit that forms, on the image display element, an operation image in which an auxiliary image indicating the position of the detection target is superimposed on the image based on the output of the sensor unit
  • An information display device comprising: a head-mounted display comprising: (7) The information display device according to (6) above,
  • the input device is A display element for displaying an image on the input operation surface;
  • the head mounted display displays an image obtained by superimposing the auxiliary image on an image displayed on the input operation surface as the operation image.
  • the information display device (8) The information display device according to (6) or (7) above, The information display device, wherein the sensor unit detects a contact pressure applied to the input operation surface by the detection target. (9) The information display device according to (8) above, The head mounted display changes the display mode of the auxiliary image according to the contact pressure of the detection target from the operation surface. (10) The information display device according to any one of (6) to (9) above, The sensor unit detects the movement of the detection target on the input operation surface, The head-mounted display displays the auxiliary image that moves according to the detected movement of the detection target.
  • An input device having an input operation surface and a sensor unit that detects contact of the detection target with the input operation surface;
  • An image display element that forms an image presented to the user, a calculation unit that calculates the coordinate position of the detection target on the input operation surface based on an output of the sensor unit, and the detection calculated by the calculation unit
  • An information display device comprising: a display processing unit configured to form, on the image display element, an operation image in which an auxiliary image indicating a position of a detection target is superimposed on the image based on a coordinate position of the target.
  • HMD Head Mounted Display

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • Optics & Photonics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Position Input By Displaying (AREA)
PCT/JP2012/007032 2011-12-26 2012-11-02 ヘッドマウントディスプレイ及び情報表示装置 Ceased WO2013099081A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US14/367,030 US9779700B2 (en) 2011-12-26 2012-11-02 Head-mounted display and information display apparatus
EP12862497.0A EP2799958A1 (en) 2011-12-26 2012-11-02 Head-mounted display and information display device
CN201280063109.9A CN103999030A (zh) 2011-12-26 2012-11-02 头戴式显示器和信息显示装置
US15/702,288 US20180005607A1 (en) 2011-12-26 2017-09-12 Head-mounted display and information display apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2011-282869 2011-12-26
JP2011282869A JP5957875B2 (ja) 2011-12-26 2011-12-26 ヘッドマウントディスプレイ

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US14/367,030 A-371-Of-International US9779700B2 (en) 2011-12-26 2012-11-02 Head-mounted display and information display apparatus
US15/702,288 Continuation US20180005607A1 (en) 2011-12-26 2017-09-12 Head-mounted display and information display apparatus

Publications (1)

Publication Number Publication Date
WO2013099081A1 true WO2013099081A1 (ja) 2013-07-04

Family

ID=48696634

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2012/007032 Ceased WO2013099081A1 (ja) 2011-12-26 2012-11-02 ヘッドマウントディスプレイ及び情報表示装置

Country Status (5)

Country Link
US (2) US9779700B2 (https=)
EP (1) EP2799958A1 (https=)
JP (1) JP5957875B2 (https=)
CN (1) CN103999030A (https=)
WO (1) WO2013099081A1 (https=)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015014977A (ja) * 2013-07-08 2015-01-22 コニカミノルタ株式会社 画像処理システム、画像形成装置、表示装置、表示方法、および制御プログラム
WO2015105815A1 (en) * 2014-01-10 2015-07-16 Microsoft Technology Licensing, Llc Hover-sensitive control of secondary display
CN104793857A (zh) * 2014-01-17 2015-07-22 卡西欧计算机株式会社 电子装置以及系统、方法
JP2016224595A (ja) * 2015-05-28 2016-12-28 株式会社コロプラ システム、方法、およびプログラム
US10895672B2 (en) 2015-10-16 2021-01-19 Toppan Printing Co., Ltd. Optical element, optical element device and optical element control system

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5957875B2 (ja) * 2011-12-26 2016-07-27 ソニー株式会社 ヘッドマウントディスプレイ
US20160291687A1 (en) * 2013-05-21 2016-10-06 Sony Corporation Display control device, display control method, and recording medium
JP6492419B2 (ja) * 2014-05-23 2019-04-03 セイコーエプソン株式会社 頭部装着型表示装置、頭部装着型表示装置を制御する方法、コンピュータープログラム、画像表示システム、および、情報処理装置
JP5975183B2 (ja) * 2013-11-01 2016-08-23 株式会社村田製作所 表示装置
CN107357510B (zh) * 2013-11-21 2021-06-04 华为终端有限公司 触摸选择的视觉反馈方法和装置
EP3116616B1 (en) * 2014-03-14 2019-01-30 Sony Interactive Entertainment Inc. Gaming device with volumetric sensing
JP6484079B2 (ja) * 2014-03-24 2019-03-13 株式会社 ハイディープＨｉＤｅｅｐ Ｉｎｃ． 感性伝達方法及びそのための端末機
CN104063092B (zh) * 2014-06-16 2016-12-07 青岛歌尔声学科技有限公司 一种触摸屏控制方法及装置
DE102014009303B4 (de) 2014-06-26 2020-07-23 Audi Ag Verfahren zum Betreiben einer Virtual-Reality-Brille und System mit einer Virtual-Reality-Brille
KR102283546B1 (ko) 2014-10-16 2021-07-29 삼성전자주식회사 웨어러블 디바이스 및 웨어러블 디바이스에서의 어플리케이션 실행 방법
JP5996605B2 (ja) * 2014-10-21 2016-09-21 株式会社コロプラ ヘッドマウントディスプレイとコントローラとを連動させて画面操作するシステム、プログラム、及び方法
US20160196693A1 (en) * 2015-01-06 2016-07-07 Seiko Epson Corporation Display system, control method for display device, and computer program
WO2017113188A1 (zh) 2015-12-30 2017-07-06 深圳市柔宇科技有限公司 头戴式显示设备及其控制方法
US10095266B2 (en) 2016-01-28 2018-10-09 Colopl, Inc. System and method for interfacing between a display and a controller
CN105739879A (zh) * 2016-01-29 2016-07-06 广东欧珀移动通信有限公司 一种虚拟现实的应用方法及终端
CN105867635A (zh) * 2016-04-26 2016-08-17 乐视控股（北京）有限公司 压力触控方法及装置
JP6883394B2 (ja) * 2016-08-03 2021-06-09 株式会社コロプラ プログラム、コンピュータ及び方法
US10152851B2 (en) 2016-11-29 2018-12-11 Microsoft Technology Licensing, Llc Notification artifact display
CN106547378A (zh) * 2016-12-07 2017-03-29 歌尔科技有限公司 一种用于vr设备的触控装置和虚拟现实系统
CN106951171B (zh) 2017-03-14 2021-10-12 北京小米移动软件有限公司 虚拟现实头盔的控制方法和装置
US11054894B2 (en) 2017-05-05 2021-07-06 Microsoft Technology Licensing, Llc Integrated mixed-input system
JP6285604B2 (ja) * 2017-06-23 2018-02-28 晃輝 平山 入力装置
US11023109B2 (en) 2017-06-30 2021-06-01 Microsoft Techniogy Licensing, LLC Annotation using a multi-device mixed interactivity system
US10895966B2 (en) 2017-06-30 2021-01-19 Microsoft Technology Licensing, Llc Selection using a multi-device mixed interactivity system
JP2019105678A (ja) * 2017-12-11 2019-06-27 京セラドキュメントソリューションズ株式会社 表示装置、及び画像表示方法
JP2019164431A (ja) * 2018-03-19 2019-09-26 セイコーエプソン株式会社 頭部装着型表示装置、及び頭部装着型表示装置の制御方法
JP7238456B2 (ja) * 2019-02-21 2023-03-14 セイコーエプソン株式会社 表示システム、情報処理装置の制御プログラム、及び情報処理装置の制御方法
US11385789B1 (en) 2019-07-23 2022-07-12 Facebook Technologies, Llc Systems and methods for interacting with displayed items
US11106288B1 (en) * 2020-03-02 2021-08-31 John Walter Downey Electronic input system
JP7686663B2 (ja) * 2020-10-27 2025-06-02 富士フイルム株式会社 表示制御装置、表示制御方法、および表示制御プログラム
US12260081B1 (en) 2023-12-24 2025-03-25 John Walter Downey Non-standard keyboard input system
US20250251897A1 (en) * 2024-02-01 2025-08-07 Apple Inc. Remote viewing and control of an electronic device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06202776A (ja) * 1992-10-29 1994-07-22 Internatl Business Mach Corp <Ibm> 入力装置への圧力の表示方法及びシステム
JP2008070817A (ja) 2006-09-15 2008-03-27 Brother Ind Ltd ヘッドマウントディスプレイ
JP2009042967A (ja) * 2007-08-08 2009-02-26 Nikon Corp 情報入力表示システム、情報端末および表示装置
JP2010145861A (ja) 2008-12-19 2010-07-01 Brother Ind Ltd ヘッドマウントディスプレイ

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001202192A (ja) * 2000-01-18 2001-07-27 Sony Corp 情報処理装置及びその方法並びにプログラム格納媒体
JP2003279881A (ja) * 2002-03-27 2003-10-02 Hitachi Ltd 携帯情報装置
JP4115198B2 (ja) * 2002-08-02 2008-07-09 株式会社日立製作所 タッチパネルを備えた表示装置
US8760408B2 (en) * 2004-08-02 2014-06-24 Koninklijke Philips N.V. Touch screen with pressure-dependent visual feedback
US7683889B2 (en) * 2004-12-21 2010-03-23 Microsoft Corporation Pressure based selection
JP4797504B2 (ja) * 2005-08-09 2011-10-19 ソニー株式会社 入力装置及びこれを用いた表示装置
US20080273015A1 (en) * 2007-05-02 2008-11-06 GIGA BYTE Communications, Inc. Dual function touch screen module for portable device and opeating method therefor
JP2009104449A (ja) * 2007-10-24 2009-05-14 Sharp Corp カーソル制御装置、カーソル制御システムおよびカーソル制御装置の制御方法
CN101609363B (zh) * 2008-06-16 2011-02-09 林志雄 输入装置及输入方法
KR101569176B1 (ko) * 2008-10-30 2015-11-20 삼성전자주식회사 오브젝트 실행 방법 및 장치
US9891708B2 (en) * 2009-06-09 2018-02-13 Immersion Corporation Method and apparatus for generating haptic effects using actuators
WO2011019154A2 (en) * 2009-08-14 2011-02-17 Lg Electronics Inc. Remote control device and remote control method using the same
JP5428943B2 (ja) * 2010-03-02 2014-02-26 ブラザー工業株式会社 ヘッドマウントディスプレイ
JP5845585B2 (ja) * 2011-02-04 2016-01-20 セイコーエプソン株式会社 情報処理装置
US9152288B2 (en) * 2011-05-19 2015-10-06 Microsoft Technology Licensing, Llc Remote multi-touch
US20130083003A1 (en) * 2011-09-30 2013-04-04 Kathryn Stone Perez Personal audio/visual system
JP5957875B2 (ja) * 2011-12-26 2016-07-27 ソニー株式会社 ヘッドマウントディスプレイ

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06202776A (ja) * 1992-10-29 1994-07-22 Internatl Business Mach Corp <Ibm> 入力装置への圧力の表示方法及びシステム
JP2008070817A (ja) 2006-09-15 2008-03-27 Brother Ind Ltd ヘッドマウントディスプレイ
JP2009042967A (ja) * 2007-08-08 2009-02-26 Nikon Corp 情報入力表示システム、情報端末および表示装置
JP2010145861A (ja) 2008-12-19 2010-07-01 Brother Ind Ltd ヘッドマウントディスプレイ

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015014977A (ja) * 2013-07-08 2015-01-22 コニカミノルタ株式会社 画像処理システム、画像形成装置、表示装置、表示方法、および制御プログラム
WO2015105815A1 (en) * 2014-01-10 2015-07-16 Microsoft Technology Licensing, Llc Hover-sensitive control of secondary display
CN104793857A (zh) * 2014-01-17 2015-07-22 卡西欧计算机株式会社 电子装置以及系统、方法
JP2016224595A (ja) * 2015-05-28 2016-12-28 株式会社コロプラ システム、方法、およびプログラム
US10895672B2 (en) 2015-10-16 2021-01-19 Toppan Printing Co., Ltd. Optical element, optical element device and optical element control system

Also Published As

Publication number Publication date
EP2799958A1 (en) 2014-11-05
CN103999030A (zh) 2014-08-20
US9779700B2 (en) 2017-10-03
US20150002434A1 (en) 2015-01-01
US20180005607A1 (en) 2018-01-04
JP5957875B2 (ja) 2016-07-27
JP2013134532A (ja) 2013-07-08

Similar Documents

Publication Publication Date Title
JP5957875B2 (ja) ヘッドマウントディスプレイ
JP2013125247A (ja) ヘッドマウントディスプレイ及び情報表示装置
US9632318B2 (en) Head-mounted display including an operating element having a longitudinal direction in a direction of a first axis, display apparatus, and input apparatus
JP6206099B2 (ja) 画像表示システム、画像表示システムを制御する方法、および、頭部装着型表示装置
JP5884502B2 (ja) ヘッドマウントディスプレイ
CN106168848B (zh) 显示装置以及显示装置的控制方法
JP6064988B2 (ja) ヘッドマウントディスプレイ
JP6264871B2 (ja) 情報処理装置および情報処理装置の制御方法
US10884498B2 (en) Display device and method for controlling display device
JP2014142751A (ja) 表示装置、頭部装着型表示装置および表示装置の制御方法
JP2015133088A (ja) Ｇｕｉシステム、表示処理装置、入力処理装置及びプログラム
US20190095077A1 (en) Electronic apparatus
JP6036217B2 (ja) 表示装置、頭部装着型表示装置および表示装置の制御方法
JP6740613B2 (ja) 表示装置、表示装置の制御方法、及び、プログラム
JP6176368B2 (ja) ヘッドマウントディスプレイ及び情報表示装置
JP2018022426A (ja) 表示装置、頭部装着型表示装置、及び、表示装置の制御方法
JP6754329B2 (ja) 画像表示装置、ヘッドマウントディスプレイ、情報表示装置、表示処理方法及びプログラム
JP2014154074A (ja) 入力装置、頭部装着型表示装置および入力装置の制御方法
WO2016052061A1 (ja) ヘッドマウントディスプレイ
WO2016072271A1 (ja) 表示装置、表示装置の制御方法及びその制御プログラム
US20260016951A1 (en) Electronic device for supporting xr content, and input mode support method therefor
KR20240142245A (ko) Xr 컨텐츠를 지원하는 전자 장치 및 이의 입력 모드 지원 방법
KR20250023244A (ko) 가상 환경에서 객체 선택을 지원하는 방법 및 이를 지원하는 전자 장치
JP2018081415A (ja) 入力装置、入力方法、コンピュータープログラム
JPWO2017065050A1 (ja) 入力装置、電子機器、電子機器の入力方法及びその入力プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12862497

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2012862497

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 14367030

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE