WO2014035118A1 - Visiocasque et procédé de commande de dispositif numérique l'utilisant - Google Patents

Visiocasque et procédé de commande de dispositif numérique l'utilisant Download PDF

Info

Publication number
WO2014035118A1
WO2014035118A1 PCT/KR2013/007680 KR2013007680W WO2014035118A1 WO 2014035118 A1 WO2014035118 A1 WO 2014035118A1 KR 2013007680 W KR2013007680 W KR 2013007680W WO 2014035118 A1 WO2014035118 A1 WO 2014035118A1
Authority
WO
WIPO (PCT)
Prior art keywords
hmd
digital device
external digital
user
gesture input
Prior art date
Application number
PCT/KR2013/007680
Other languages
English (en)
Inventor
Eunhyung Cho
Original Assignee
Lg Electronics Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020120096132A external-priority patent/KR101958778B1/ko
Application filed by Lg Electronics Inc. filed Critical Lg Electronics Inc.
Publication of WO2014035118A1 publication Critical patent/WO2014035118A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/002—Specific input/output arrangements not covered by G06F3/01 - G06F3/16
    • G06F3/005—Input arrangements through a video camera
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Definitions

  • the present invention relates to a head mounted display and a method of controlling a digital device using the same. More particularly, the present invention relates to a method of receiving a gesture input of a user using a head mounted display and synthetically controlling a digital device using the received gesture input.
  • a Head Mounted Display refers to a variety of image display devices that are worn on the head of a user like glasses, to assist the user in viewing an image. With the tendency of a smaller weight and size of digital devices, various types of wearable computers have been developed, and the HMD has also been widely used.
  • the HMD may provide the user with various conveniences in combination with augmented reality techniques, N-screen techniques, and the like over a simple display function.
  • the HMD is available in association with various digital devices.
  • the present invention is directed to a head mounted display and a method of controlling a digital device using the same that substantially obviate one or more problems due to limitations and disadvantages of the related art.
  • One object of the present invention is to provide a method of synthetically controlling various digital devices using an HMD.
  • Another object of the present invention is to provide a method of controlling at least one of display objects of an HMD and an external digital device using a user gesture input detected via the HMD.
  • a further object of the present invention is to provide a method of grasping a user’s intention in that a user who performs a gesture input wishes to control display of any one digital device, and controlling a display object of the corresponding digital device to conform to the user’s intention.
  • a control method of a digital device using a Head Mounted Display includes detecting whether or not the HMD is worn by a user, detecting a positional state of an external digital device linked with the HMD, wherein the positional state includes a first state in which the external digital device is located in a preset view angle region of the HMD and a second state in which the external digital device is not located in the view angle region of the HMD, detecting a gesture input of the user, determining at least one digital device, display of which will be controlled, under application of the gesture input, based on whether or not the HMD is worn by the user and based on the detected positional state of the external digital device, wherein the at least one digital device includes at least one of the HMD and the external digital device, and controlling a display object of the determined digital device under application of the gesture input.
  • HMD Head Mounted Display
  • a Head Mounted Display includes a processor configured to control operation of the HMD, a display unit configured to output an image based on a command of the processor, a sensor unit configured to sense a peripheral environment of the HMD and transmit the sensed result to the processor, and a communication unit configured to perform data transmission/reception with an external digital device based on a command of the processor, wherein the processor is configured to detect whether or not the HMD is worn by a user, to detect a positional state of the external digital device linked with the HMD, wherein the positional state includes a first state in which the external digital device is located in a preset view angle region of the HMD and a second state in which the external digital device is not located in the view angle region of the HMD, to detect a gesture input of the user, to determine at least one digital device, display of which will be controlled, under application of the gesture input, based on whether or not the HMD is worn by the user and based on the detected positional
  • display of a plurality of digital devices may be controlled using an HMD.
  • the user can control a display object of an external digital device linked with the HMD as well as a display object of the HMD using the HMD.
  • a user’s intention in that the user wishes to control a display object of any one digital device among a plurality of external digital devices, and to provide a simplified and intuitive user interface to assist the user in controlling the display object of the corresponding digital device to conform to the user’s intention.
  • the present invention it is possible to determine whether or not to reverse a gesture input of a user and whether or not to correct the gesture input based on whether or not the HMD is worn by the user. Accordingly, according to an embodiment of the present invention, it is possible to determine an application direction of the gesture input on the basis of the user who performs the gesture input regardless of different states of the HMD.
  • FIG. 1 is a view illustrating an HMD, a user who wears the HMD, and a digital device linked with the HMD according to an embodiment of the present invention
  • FIG. 2 is a view illustrating a control method of a digital device according to a first embodiment of the present invention
  • FIG. 3 is a view illustrating a control method of a digital device according to a second embodiment of the present invention.
  • FIGs. 4 and 5 are views illustrating a control method of a digital device according to a third embodiment of the present invention.
  • FIGs. 6 to 8 are views illustrating a method of controlling a plurality of external digital devices using an HMD according to an embodiment of the present invention
  • FIGs. 9 and 10 are views illustrating a control method of a digital device according to a fourth embodiment of the present invention.
  • FIG. 11 is a block diagram illustrating an HMD according to an embodiment of the present invention.
  • FIG. 12 is a flowchart illustrating a control method of a digital device using an HMD according to an embodiment of the present invention.
  • FIG. 1 is a view illustrating an HMD 100, a user 10 who wears the HMD 100, and an external digital device 200 linked with the HMD 100 according to an embodiment of the present invention.
  • the HMD 10 may output various content, and may be operated in linkage with at least one external digital device 200.
  • the HMD 100 may perform separate pairing or communication connection for linkage with the external digital device 200. Pairing or communication connection may be performed by a user input through the external digital device 200 or the HMD 100.
  • the HMD 100 may provide a separate user interface for pairing or communication connection with the external digital device 200, and a user can perform a user input using the user interface so as to achieve pairing or communication connection between the HMD 100 and the external digital device 200.
  • the HMD 100 may display at least one object through a display unit equipped therein.
  • the HMD 100 may display internal content and a user interface.
  • the HMD 100 may display various content of the external digital device 200 linked with the HMD 100.
  • the external digital device 200 may output various kinds of content.
  • the external digital device 200 may output multimedia content, such as live broadcasts, movies, music, dramas, web-pages, games, and applications.
  • the external digital device 200 may include at least one display unit 220, and the content output from the external digital device 200 may be displayed on the display unit 220.
  • the external digital device 200 may further include a communication unit (not shown), and may perform data transmission/reception with the HMD 100 and other external devices through the communication unit.
  • the external digital device 200 includes a variety of digital devices including the display unit 220 and the communication unit.
  • the external digital device 200 includes a variety of digital devices, such as a Personal Computer (PC), Personal Digital Assistant (PDA), laptop computer, tablet PC, television, etc., that are capable of displaying an image and of performing data communication.
  • PC Personal Computer
  • PDA Personal Digital Assistant
  • the HMD 100 and the external digital device 200 are capable of performing data transmission/reception using various types of wireless or wired communication means.
  • available wireless communication means include Near Field Communication (NFC), ZigBee, infrared communication, Bluetooth, Wi-Fi communication, etc., but the present invention is not limited thereto.
  • communication between the HMD 100 and the external digital device 200 may be accomplished using any one of the above enumerated communication means, or may be accomplished using combinations thereof.
  • the HMD 100 of the present invention may include a view angle region 102 within a preset range.
  • the view angle region 102 is a preset region corresponding to a visual field of the user 10 who wears the HMD 100, and may include a region forward of the HMD 100 within a predetermined angular range.
  • the HMD 100 may detect whether or not the external digital device 200 linked with the HMD 100 is located in the view angle region 102 of the HMD 100. That is, the HMD 100 may detect a positional state of the external digital device 200 linked with the HMD 100.
  • the positional state may include a first state in which the external digital device 200 is located in the view angle region 102 of the HMD 100, and a second state in which the external digital device 200 is not located in the view angle region 102 of the HMD 100.
  • the HMD 100 may detect a gesture input 15 of the user 10.
  • the gesture input 15 includes an input in which the hand or the finger of the user 10 moves from a first position to a second position.
  • the gesture input 15 includes various user inputs to control content of the external digital device 200.
  • the HMD 100 of the present invention may further detect pointing direction information of the gesture input 15.
  • the pointing direction information of the gesture input 15 indicates which digital device the user wishes to control using the corresponding gesture input 15.
  • the pointing direction information of the gesture input 15 may be derived from the direction indicated by the finger included in the gesture input 15. For example, if the finger included in the gesture input 15, as illustrated in FIG.
  • the pointing direction information of the corresponding gesture input 15 may indicate the external digital device 200.
  • the finger included in the gesture input 15 points the HMD 100
  • the pointing direction information of the corresponding gesture input 15 may indicate the HMD 100.
  • a method of determining the pointing direction information of the gesture input 15 is not limited to the above description, and the method may be altered to various embodiments.
  • the pointing direction information may be differently determined according to a preset motion included in the gesture input 15 (for example, a motion of spreading a specific finger).
  • the HMD 100 may further detect whether or not the HMD 100 is worn by the user 10. That is, the HMD 100 may detect whether or not the user 10 wears the corresponding HMD 100 using the sensor unit equipped in the HMD 100. In the present invention, the HMD 100 may control a digital device corresponding to the gesture input 15 of the user 10 using the above described detected conditions. More specifically, the HMD 100 may determine a digital device, display of which will be controlled via the gesture input 15, based on whether or not the user 100 wears the HMD 100 and based on the positional state of the external digital device 200. In this case, the digital device, display of which will be controlled, may include at least one of the HMD 100 and the external digital device 200. A detailed embodiment thereof will be described with reference to FIGs. 2 to 10.
  • FIG. 2 is a view illustrating a control method of a digital device according to one embodiment of the present invention.
  • the HMD 100 is worn by the user, and an external digital device linked with the HMD 100 is located at the outside of the view angle region 102 of the HMD 100.
  • the HMD 100 of the present invention detects the gesture input 15 of the user 10.
  • the HMD 100 controls a display object 125 of the HMD 100 using the detected gesture input 15 of the user.
  • the HMD 100 may detect the gesture input 15 that moves from the left side L of the HMD 100 to the right side R of the HMD 100 within the view angle region 102.
  • the HMD 100 may control the display object 125 output on the display unit 120 of the HMD 100 under application of the detected gesture input 15. That is, the HMD 100 may apply an input for movement from the left side to the right side to the display object 125.
  • the display object 125 controlled by the gesture input 15 may be at least one of various content and a user interface displayed on the display unit 120 of the HMD 100.
  • the HMD 100 may determine whether or not the external digital device linked with the HMD 100 provides the same display object as the display object 125 of the HMD 100. That is, the HMD 100 determines whether or not the external digital device is in an N-screen state in which the external digital device outputs a display object equal to the display object 125 of the HMD 100. If the external digital device provides the same display object as the display object 125 of the HMD 100, the display object of the external digital device may be controlled along with the display object 125 of the HMD 100. That is, the HMD 100 controls the display object of the external digital device and the display object 125 of the HMD 100 together under application of the detected gesture input 15.
  • the display object of the external digital device controlled by the HMD 100 may be equal to the display object 125 output from the HMD 100.
  • the HMD 100 may control only the display object 125 of the HMD 100 under application of the gesture input 15.
  • FIG. 3 is a view illustrating a control method of a digital device according to another embodiment of the present invention.
  • the HMD 100 is worn by the user 10, and the external digital device 200 linked with the HMD 100 is located within the view angle region 102 of the HMD 100.
  • the HMD 100 of the present invention detects the gesture input 15 of the user 10.
  • the HMD 100 may determine whether or not the external digital device 200 linked with the HMD 100 provides the same display object as the display object 125 of the HMD 100. That is, the HMD 100 determines whether or not the external digital device 200 is in an N-screen state in which the external digital device 200 provides a display object equal to the display object 125 of the HMD 100.
  • a display object 225 of the external digital device 200 may be controlled along with the display object 125 of the HMD 100.
  • the HMD 100 controls the display object 225 of the external digital device 200 and the display object 125 of the HMD 100 together under application of the detected gesture input 15.
  • the display object 225 of the external digital device 200 controlled by the HMD 100 may be equal to the display object 125 output from the HMD 100.
  • the HMD 100 controls the display object 225 of the external digital device 200 and the display object 125 of the HMD 100 together using the detected gesture input 15 of the user.
  • the HMD 100 may detect the gesture input 15 that moves from the left side L of the HMD 100 to the right side R of the HMD 100 within the view angle region 102.
  • the HMD 100 may control the display object 125 output on the display unit 120 of the HMD 100 and the display object 225 output on the display unit 220 of the external digital device 200 under application of the detected gesture input 15. That is, the HMD 100 may apply an input for movement from the left side to the right side with respect to the display object 125 and the display object 225.
  • the display object 125 and the display object 225, which are controlled based on the gesture input 15, may be at least one of various content and user interfaces displayed respectively on the display unit 120 of the HMD 100 and the display unit 220 of the external digital device 200.
  • FIGs. 4 and 5 are views illustrating a control method of a digital device according to another embodiment of the present invention.
  • the HMD 100 is worn by the user 10, and the external digital device 200 linked with the HMD 100 is located within the view angle region 102 of the HMD 100.
  • the HMD 100 of the present invention detects the gesture input 15 of the user 10.
  • the HMD 100 may determine whether or not the external digital device 200 linked with the HMD 100 provides the same display object as the display object 125 of the HMD 100. If the external digital device 200 does not provide the same display object as the display object 125 of the HMD 100 as illustrated in the embodiment of FIGs. 4 and 5, the HMD 100 detects pointing direction information of the detected gesture input 15.
  • the pointing direction information indicates a target digital device of the gesture input 15.
  • the pointing direction information may be derived from the direction indicated by the finger included in the gesture input 15.
  • the HMD 100 may control a display object of a digital device that is pointed by the gesture input 15.
  • the HMD 100 may control the display object 225 of the external digital device 200 using the detected gesture input 15 of the user. For example, in FIG. 4, the HMD 100 may detect the gesture input 15 that moves from the left side L of the HMD to the right side R of the HMD 100 within the view angle region 102. In this case, the pointing direction information of the gesture input 15 indicates the external digital device 200. The HMD 100 may control the display object 225 output on the display unit 220 of the external digital device 200 under application of the detected gesture input 15.
  • the HMD 100 may apply an input for movement from the left side to the right side with respect to the display object 225 of the external digital device 200.
  • the HMD 100 may transmit the detected gesture input 15 to the external digital device 200.
  • the external digital device 200 may receive the gesture input 15 transmitted from the HMD 100, and control the display object 225 under application of the gesture input 15.
  • the HMD 100 may control the display object 125 of the HMD 100 using the detected gesture input 15 of the user. That is, in the embodiment of FIG. 5, if the HMD 100 detects the gesture input 15 that moves from the left side L of the HMD 100 to the right side R of the HMD 100 within the view angle region 102, the HMD 100 may apply an input for movement from the left side to the right side with respect to the display object 125 of the HMD 100.
  • the HMD 100 may determine a digital device to be controlled based on the pointing direction information of the detected gesture input 15. That is, the HMD 100 may control any one of the display objects 125 and 225 of the HMD 100 and the external digital device 200 based on the pointing direction information of the gesture input 15.
  • the external digital device 200 and the HMD 100 linked with each other may have different platforms. That is, the display object 125 of the HMD 100 may be controlled by the HMD 100, and the display object 225 of the external digital device 200 may be controlled by the external digital device 200.
  • the HMD 100 may transmit the detected gesture input 15 to the external digital device 200.
  • the HMD 100 of the present invention may serve as one of user input receiving devices of the external digital device 200.
  • FIGs. 6 to 8 are views illustrating a method of controlling a plurality of external digital devices 200a, 200b and 200c using the HMD 100 according to an embodiment of the present invention.
  • the HMD 100 is worn by the user 10.
  • the HMD 100 is linked with the plurality of external digital devices 200a, 200b and 200c, and the respective external digital devices 200a, 200b and 200c provide different display objects from the display object 125 of the HMD 100.
  • the HMD 100 may detect whether or not the respective external digital devices 200a, 200b and 200c are located within the view angle region 102, and control a display object of the corresponding external digital device located within the view angle region 102. That is, if any one external digital device linked with the HMD 100 is located within the view angle region 102 and the pointing direction information of the detected gesture input 15 indicates the corresponding external digital device located within the view angle region 102, the HMD 100 may control a display object of the corresponding external digital device using the detected gesture input 15.
  • the HMD 100 may determine to control display of the external digital device 200a. That is, the HMD 100 may control a display object 225a output on a display unit 220a of the external digital device 200a under application of the detected gesture input 15. According to the embodiment of the present invention, the HMD 100 may transmit the detected gesture input 15 to the external digital device 200a.
  • the external digital device 200a may receive the gesture input 15 transmitted from the HMD 100, and control the display object 225a using the gesture input 15.
  • the HMD 100 may determine to control display of the external digital device 200b.
  • the HMD 100 may determine to control display of the external digital device 200c.
  • a method of controlling display objects 225b and 225c of the respective external digital devices 200b and 200c using the HMD 100 is equal to the above description with reference to FIG. 6.
  • the display objects 225a, 225b and 225c of the plurality of external digital devices 200a, 200b and 200c may be controlled using the HMD 100. That is, the HMD 100 may detect whether or not the respective external digital devices 200a, 200b and 200c are located within the view angle region 102 of the HMD 100, and control the respective external digital devices 200a, 200b and 200c based on the detected results.
  • the user 10 can perform a control input with respect to the display object of the external digital device that the user 10 wishes to control by performing the gesture input 15 while looking at the corresponding external digital device.
  • FIGs. 9 and 10 are views illustrating a control method of a digital device according to another embodiment of the present invention.
  • the HMD 100 is not worn by the user 10.
  • the HMD 100 may detect the gesture input 15 of the user 10, and control the display object 225 of the external digital device 200 using the detected gesture input 15. That is, according to the embodiment of the present invention, if the HMD 100 is not worn by the user 10, the HMD 100 may determine to control display of the external digital device 200 using the detected gesture input 15. Accordingly, the external digital device 200 may utilize the HMD 100 of the present invention as a user input receiving device.
  • the HMD 100 may correct an application direction of the gesture input 15 with respect to the display object 225 of the external digital device 200 based on whether or not the HMD 10 is worn by the user. That is, if the HMD 10 of the present invention is not worn by the user, the HMD 100 may reverse left and right sides of a gesture input application direction with respect to the display object 225.
  • the gesture input application direction corresponds to a movement direction of the detected gesture input 15. For example, as illustrated in FIG. 9, if the gesture input 15 of the user 10 is performed within the view angle region 102 of the HMD 100, the HMD 100 detects the gesture input 15.
  • the gesture input 15 is an input for movement from the right side R to the left side L of the HMD 100.
  • the HMD 100 controls the display object 225 by reversing left and right sides of the detected gesture input 15. Thereby, the HMD 100 may apply an input for movement from the left side to the right side with respect to the display object 225 of the external digital device 200. This may assist the user in more conveniently and easily performing the gesture input 15 with respect to the external digital device 200 using the HMD 100.
  • the HMD 100 may transmit the detected gesture input 15 to the external digital device 200.
  • the HMD 100 may reverse left and right sides of the detected gesture input 15 prior to transmitting the gesture input 15 to the external digital device 200.
  • the external digital device 200 may receive the left-to-right reversed gesture input 15 transmitted from the HMD 100, and apply the received gesture input 15 to the display object 225.
  • the HMD 100 may transmit additional information on the state of the HMD 100 that detects the gesture input 15.
  • the additional information may include information on whether or not the HMD 100 that detects the gesture input 15 is worn by the user 10.
  • the external digital device 200 may reverse the left and right sides of the gesture input 15 transmitted from the HMD 100, and then apply the reversed gesture input 15 to the display object 225. In this way, the external digital device 200 may reverse the left and right sides of the gesture input 15 initially detected by the HMD 100, and then apply the resulting gesture input 15 to the display object 225, which may allow the display object 225 to be controlled to conform to the user’s intentions.
  • FIG. 10 illustrates another embodiment of a method of controlling a digital device when the HMD 100 is not worn by the user 10.
  • the HMD 100 may include a sensor for sensing an orientation angle of the HMD 100, such as, for example, a G-sensor or a gyroscope sensor.
  • the HMD 100 may sense a tilt angle of the HMD 100 on the basis of the ground, i.e. the orientation angle of the HMD 100.
  • the HMD 100 may sense the orientation angle of the HMD 100 on the basis of at least one axis of X-axis, Y-axis and Z-axis perpendicular to one another.
  • the HMD 100 of the present invention may correct the detected gesture input 15 of the user using the orientation angle of the HMD 100.
  • the HMD 100 may detect the gesture input 15 of the user in a state in which the HMD 100 is tilted by an angle ⁇ on the basis of the X-axis.
  • the HMD 100 may recognize a gesture input that moves from the right upper side of the HMD 100 to the left lower side of the HMD 100.
  • the HMD 100 of the present invention may correct the detected gesture input 15 using the orientation angle of the HMD 100, and apply the corrected gesture input 15 to the display object 225.
  • the HMD 100 may add or subtract the tilt angle ⁇ to or from the detected gesture input 15.
  • the HMD 100 may correct the detected gesture input 15 by reversing the left and right sides of the gesture input 15 and adding the orientation angle ⁇ of the HMD 100 to the gesture input 15.
  • the HMD 100 may apply an input for movement from the left side to the right side with respect to the display object 225.
  • the external digital device 200 may receive the gesture input 15 of the user under the same condition regardless of the orientation angle of the HMD 100.
  • the HMD 100 may transmit the detected gesture input 15 to the external digital device 200.
  • the HMD 100 may transmit the gesture input 15, which has been corrected using the orientation angle of the HMD 100, to the external digital device 200.
  • the external digital device 200 may receive the corrected gesture input 15 from the HMD 100, and apply the received gesture input 15 to the display object 225.
  • the HMD 100 may transmit additional information on the state of the HMD that detects the gesture input.
  • the additional information may include information on whether or not the HMD 100 that detects the gesture input 15 is worn by the user and information on the orientation angle of the HMD 100.
  • the external digital device 200 may correct the gesture input 15 transmitted from the HMD 100 using the additional information, and apply the corrected gesture input 15 to the display object 225.
  • FIG. 11 is a block diagram illustrating the HMD according to an embodiment of the present invention.
  • the HMD 100 of the present invention may include a processor 110, a display unit 120, an audio output unit 130, a sensor unit 140 and a communication unit 150.
  • the display unit 120 outputs an image on a display screen.
  • the display unit 120 may output various display objects including content executed by the processor 110 or a user interface based on a control command of the processor 110, for example.
  • the display unit 120 may display an image based on a control command of a digital device linked with the HMD 100 (for example, the external digital device 200 of the present invention).
  • the display unit 120 may display content executed by the digital device linked with the HMD 100.
  • the HMD 100 may receive data from the digital device through the communication unit 150, and output a display object based on the received data.
  • the display unit 120 of the HMD 100 and the display unit of the external digital device 200 may provide the same display object, or may provide different display objects. If the HMD 100 and the external digital device 200 are in an N-screen state in which they are linked with each other, the display unit 120 of the HMD 100 and the display unit of the external digital device 200 may provide the same display object. However, if the HMD 100 and the external digital device 200 are not in an N-screen state or have different platforms, the display unit 120 of the HMD 100 and the display unit of the external digital device 200 may provide different display objects.
  • the audio output unit 130 includes various audio output means, such as, for example, a speaker and earphone. Likewise, the audio output unit 130 may output content executed by the processor 110, or may output voice based on a control command of the processor 110. Additionally, according to the embodiment of the present invention, the audio output unit 130 may output voice based on a control command of the digital device linked with the HMD 100. For example, the audio output unit 130 may output voice data of content executed by the digital device linked with the HMD 100. In the present invention, the audio output unit 130 may be selectively provided on the HMD 100.
  • the sensor unit 140 may transmit a user input or an environment recognized by the HMD 100 to the processor 110 using a plurality of sensors equipped in the HMD 100.
  • the sensor unit 150 may include a plurality of sensing means.
  • the plurality of sensing means may include a gravity sensor, a geomagnetic sensor, a motion sensor, a kinetic sensor, a gyroscope sensor, an accelerometer, an infrared sensor, an inclination sensor, an illumination sensor, a height sensor, an olfactory sensor, a temperature sensor, a depth sensor, a pressure sensor, a bending sensor, an audio sensor, a video sensor, a Global Positioning System (GPS) sensor, a touch sensor, and a proximity sensor, for example.
  • GPS Global Positioning System
  • the sensor unit 150 is a generic term of the above enumerated various sensing means, and may sense various user inputs and user environments, and transmit the sensed results to the processor 110 so as to assist the processor 110 in performing corresponding operations.
  • the above enumerated sensors may be provided as individual elements included in the HMD 100, or may be combined to constitute at least one element included in the HMD 100.
  • the sensor unit 140 may include various kinds of sensors to detect whether or not the HMD 100 is worn by the user.
  • the HMD 100 may detect whether or not the HMD 100 is worn by the user using a proximity sensor.
  • the HMD 100 may include a sensor that detects whether or not a hinge, configured to connect a front portion and a leg portion of the HMD 100 to each other, is folded. As such, the HMD 100 may determine whether or not the HMD 100 is worn by the user based on the folded or unfolded state of the hinge.
  • the sensor unit 140 may include a sensor to detect a gesture input of the user.
  • the HMD 100 may include a kinetic sensor that detects a gesture input of the user within a preset angular range forward of the HMD 100.
  • the HMD may include a sensor to detect motion of a glove for the gesture input.
  • the HMD 100 may detect information on the position and direction of a glove in a 3-dimensional space, and information on the bending of the finger.
  • the user may perform a gesture input with the hand wearing the glove, and the HMD 100 may detect the gesture input of the user via motion of the glove.
  • the sensor unit 140 may include an image capturing sensor.
  • the image capturing sensor may detect an image within a preset view angle region of the HMD 100, and provide the detected image to the processor 110.
  • the processor 110 may detect whether the external digital device 200 linked with the HMD 100 is present within the view angle region of the HMD 100 based on the image detected via the image capturing sensor.
  • the image capturing sensor may be replaced by an infrared sensor according to the purpose of the present invention.
  • the communication unit 150 may perform data transmission/reception via communication with the external digital device 200 using a variety of protocols.
  • the communication unit 150 may be connected to a wired or wireless network for transmission/reception of digital data, such as content, for example.
  • the HMD 100 may perform paring and communication connection with the external digital device 200 using the communication unit 150, and may perform data transmission/reception with the connected external digital device 200.
  • the communication unit 150 may include at least one antenna.
  • the HMD 100 may detect a positional state of the external digital device 200 linked with the HMD 100 using the at least one antenna.
  • the HMD 100 may detect whether or not the external digital device 200 linked with the HMD 100 is located within the view angle region of the HMD 100 using, for example, a time difference or phase difference of signals transmitted/received through a plurality of antennas equipped in the HMD 100.
  • the processor 110 of the present invention may execute various content and applications, and process internal data of the digital device. Additionally, the processor 110 may control the above described respective units of the HMD 100, and control data transmission/reception between the units.
  • FIG. 11 is a block diagram of the HMD 100 according to the embodiment of the present invention, and separately displayed blocks represent logically distinguished elements of the device. Accordingly, the elements of the above described device may be mounted in a single chip or in a plurality of chips according to a device design.
  • FIG. 12 is a flowchart illustrating a control method of a digital device using the HMD according to an embodiment of the present invention. In the following description, respective operations of FIG. 12 may be controlled by the processor 110 of the HMD 100 illustrated in FIG. 11.
  • the HMD of the present invention may detect whether or not the HMD is worn by the user (S1210).
  • the HMD may detect whether or not the HMD is worn by the user using a variety of sensing means.
  • the HMD may include a proximity sensor to detect whether or not the HMD is worn by the user.
  • the HMD may include a sensor to detect whether or not the hinge, used to connect the front portion and the leg portion of the HMD to each other, is folded. As such, the HMD may determine whether or not the HMD is worn by the user based on the folded or unfolded state of the hinge.
  • the HMD of the present invention detects a positional state of an external digital device linked with the HMD (S1220).
  • the positional state includes a first state in which the external digital device is located in a preset view angle region of the HMD, and a second state in which the external digital device is not located in the view angle region of the HMD.
  • the view angle region is a preset region corresponding to a visual field of the user who wears the HMD, and may include a region forward of the HMD within a predetermined angular range.
  • the HMD may detect the positional state of the external digital device using at least one antenna.
  • the HMD may include a plurality of antennas, and perform data transmission/reception with the external digital device through the plurality of antennas.
  • the HMD may detect relative positions and directions between the HMD and the external digital device linked with the HMD using, for example, a time difference or phase difference of signals transmitted/received through the plurality of antennas equipped in the HMD.
  • the positional state of the external digital device may be accurately detected even in a situation in which a constant distance between the external digital device and the HMD is maintained and the direction (orientation angle) of the external digital device is changed.
  • the HMD may include a directional antenna, and detect the positional state of the external digital device using a signal transmitted/received through the directional antenna.
  • the HMD may detect the positional state of the external digital device using an image capturing sensor. That is, in a state in which the HMD is linked with the external digital device, the image capturing sensor may detect an image within a preset view angle region and provide the detected image to the processor of the HMD.
  • the processor may detect whether the external digital device linked with the HMD is present within the view angle region of the HMD based on the image detected by the image capturing sensor.
  • the external digital device linked with the HMD may output a preset optical pattern, and the HMD may detect the optical pattern output from the external digital device using the image capturing sensor.
  • the optical pattern may include at least one of a temporal pattern and a spatial pattern.
  • the optical pattern may be output through the display unit of the external digital device.
  • the HMD may detect the optical pattern output from the external digital device using the image capturing sensor.
  • the image capturing sensor cannot detect the optical pattern output from the external digital device.
  • the HMD of the present invention may detect the positional state of the external digital device based on whether or not the optical pattern of the external digital device within the view angle region is detected.
  • the positional state of the external digital device may be determined based on the eyes of the user who wears the HMD.
  • the HMD of the present invention may further include an image capturing sensor to detect positions of pupils of the user who wears the HMD, and may track the user’s eyes using the image capturing sensor.
  • the positional state of the external digital device of the present invention may be determined based on whether or not the external digital device is located within the visual field of the user who wears the HMD.
  • the visual field may include a predetermined range corresponding to the eyes of the user who wears the HMD.
  • the positional state of the external digital device may include a third state in which the external digital device is located in the visual field of the user who wears the HMD and a fourth state in which the external digital device is not located in the visual field of the user who wears the HMD.
  • the HMD when the positional state of the external digital device is changed, the HMD may provide appropriate feedback. For example, if the positional state of the external digital device linked with the HMD is changed from a first state to a second state, the HMD may provide vibration feedback. Likewise, if the positional state of the external digital device linked with the HMD is changed from a third state to a fourth state, the HMD may provide vibration feedback.
  • the HMD of the present invention detects a gesture input of the user (S1230).
  • the HMD of the present invention may include a variety of sensing means to detect the gesture input of the user, such as, for example, a kinetic sensor.
  • the HMD may include a sensor to detect motion of a glove for the gesture input.
  • the HMD may detect information on the position and direction of the glove in a 3-dimensional space, and information on the bending of the finger.
  • the user may perform a gesture input with the hand wearing the glove, and the HMD may detect the gesture input of the user via motion of the glove.
  • the HMD may detect the gesture input of the user within a preset angular range forward of the HMD.
  • the external digital device linked with the HMD may detect a gesture input of the user, and the HMD may receive the gesture input detected by the corresponding external digital device.
  • the HMD of the present invention may determine at least one digital device, display of which will be controlled, under application of the gesture input (S1240).
  • the at least one digital device includes at least one of the HMD and the external digital device of the present invention.
  • the HMD may determine a digital device, display of which will be controlled, based on the detected environment and preset conditions of the HMD. For example, the HMD may determine the digital device, display of which will be controlled, based on the result with regard to whether or not the HMD is worn by the user (S1210) and based on the detected positional state of the external digital device (S1220). Additionally, the HMD may further determine whether or not the HMD and the external digital device provide the same display object, and may determine the digital device, display of which will be controlled, based on the determined result.
  • the HMD of the present invention controls the display object of the digital device determined in Operation S1240 using the gesture input detected in Operation S1230 (S1250).
  • the HMD according to the embodiment of the present invention may control at least one of the display object of the HMD and the display object of the external digital device based on the result of Operation S1240.
  • the HMD may correct the detected gesture input, and control display object using the corrected gesture input.
  • the HMD may transmit the detected gesture input to the corresponding external digital device. Details of Operations S1240 and S1250 of the present invention are equal to the above description with reference to FIGs. 2 to 10.
  • the HMD described herein may be altered to or replaced by a variety of devices according to purposes thereof.
  • the HMD of the present invention may include various other devices that can be worn by the user and provide a display function, such as, for example, an Eye Mounted Display (EMD), eyeglass, eyepiece, eye-wear, and Head Worn Display (HWD), and are not limited to the terms used herein.
  • EMD Eye Mounted Display
  • HWD Head Worn Display
  • the present invention is totally or partially applicable to electronic devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Optics & Photonics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention porte sur un procédé de réception d'une entrée gestuelle d'un utilisateur à l'aide d'un visiocasque (HMD) et de commande synthétique d'un dispositif numérique à l'aide de l'entrée gestuelle reçue. Le procédé consiste à détecter si le HMD est porté ou non par un utilisateur, à détecter l'état positionnel d'un dispositif numérique externe lié au HMD, l'état positionnel comprenant un premier état dans lequel le dispositif numérique externe se trouve dans une région d'angle de vision prédéfinie du HMD et un second état dans lequel le dispositif numérique externe ne se trouve pas dans la région d'angle de vision, à détecter une entrée gestuelle de l'utilisateur, à déterminer au moins un dispositif numérique à commander sur la base du fait que le HMD est porté ou non par l'utilisateur et de l'état positionnel détecté, et à commander un objet d'affichage du dispositif numérique déterminé par application de l'entrée gestuelle.
PCT/KR2013/007680 2012-08-31 2013-08-27 Visiocasque et procédé de commande de dispositif numérique l'utilisant WO2014035118A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR10-2012-0096132 2012-08-31
KR1020120096132A KR101958778B1 (ko) 2012-08-31 2012-08-31 헤드 마운트 디스플레이 및 이를 이용한 디지털 디바이스 제어 방법
US13/662,033 2012-10-26
US13/662,033 US9223402B2 (en) 2012-08-31 2012-10-26 Head mounted display and method of controlling digital device using the same

Publications (1)

Publication Number Publication Date
WO2014035118A1 true WO2014035118A1 (fr) 2014-03-06

Family

ID=50183865

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2013/007680 WO2014035118A1 (fr) 2012-08-31 2013-08-27 Visiocasque et procédé de commande de dispositif numérique l'utilisant

Country Status (1)

Country Link
WO (1) WO2014035118A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3001148A1 (fr) * 2014-09-24 2016-03-30 Samsung Electronics Co., Ltd. Procédé d'acquisition de données de capteur et son dispositif électronique
US10884525B1 (en) 2019-04-23 2021-01-05 Lockheed Martin Corporation Interactive mixed masking system, method and computer program product for a simulator

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060197832A1 (en) * 2003-10-30 2006-09-07 Brother Kogyo Kabushiki Kaisha Apparatus and method for virtual retinal display capable of controlling presentation of images to viewer in response to viewer's motion
KR20110080915A (ko) * 2010-01-07 2011-07-13 엘지전자 주식회사 영상 서비스 제공 방법, 그리고 이를 수행하는 영상 시스템 및 이동 단말기
JP2011180867A (ja) * 2010-03-02 2011-09-15 Brother Industries Ltd ヘッドマウントディスプレイ
JP2012048659A (ja) * 2010-08-30 2012-03-08 Canon Inc 画像処理装置、画像処理方法
US8253685B2 (en) * 2008-12-19 2012-08-28 Brother Kogyo Kabushiki Kaisha Head mount display

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060197832A1 (en) * 2003-10-30 2006-09-07 Brother Kogyo Kabushiki Kaisha Apparatus and method for virtual retinal display capable of controlling presentation of images to viewer in response to viewer's motion
US8253685B2 (en) * 2008-12-19 2012-08-28 Brother Kogyo Kabushiki Kaisha Head mount display
KR20110080915A (ko) * 2010-01-07 2011-07-13 엘지전자 주식회사 영상 서비스 제공 방법, 그리고 이를 수행하는 영상 시스템 및 이동 단말기
JP2011180867A (ja) * 2010-03-02 2011-09-15 Brother Industries Ltd ヘッドマウントディスプレイ
JP2012048659A (ja) * 2010-08-30 2012-03-08 Canon Inc 画像処理装置、画像処理方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3001148A1 (fr) * 2014-09-24 2016-03-30 Samsung Electronics Co., Ltd. Procédé d'acquisition de données de capteur et son dispositif électronique
CN106574847A (zh) * 2014-09-24 2017-04-19 三星电子株式会社 用于获取传感器数据的方法及其电子设备
US10408616B2 (en) 2014-09-24 2019-09-10 Samsung Electronics Co., Ltd. Method for acquiring sensor data and electronic device thereof
US10884525B1 (en) 2019-04-23 2021-01-05 Lockheed Martin Corporation Interactive mixed masking system, method and computer program product for a simulator

Similar Documents

Publication Publication Date Title
US9223402B2 (en) Head mounted display and method of controlling digital device using the same
EP3469457B1 (fr) Extension modulaire d'une unité de commande inertielle pour six ddl(degrée de liberté) entrées de réalité mixte
CN108700941B (zh) 用于在虚拟现实环境中对准组件的方法和设备
WO2014069776A1 (fr) Dispositif d'affichage monté sur tête et procédé de sortie de signal audio l'utilisant
WO2019093846A1 (fr) Système et procédé pour suivi d'un dispositif distribué
WO2015046686A1 (fr) Dispositif d'affichage pouvant être porté et procédé permettant de commander une couche dans celui
WO2014109430A1 (fr) Dispositif d'affichage facial réalisant un étalonnage du regard, et procédé de commande associé
WO2016117758A1 (fr) Système d'exercice de rééducation de la main et méthode associée
WO2015122565A1 (fr) Système d'affichage permettant d'afficher une imagé de réalité augmentée et son procédé de commande
WO2017086708A1 (fr) Procédé et appareil d'affichage de contenu
WO2015030301A1 (fr) Montre intelligente et son procédé de commande
WO2012077922A2 (fr) Système d'affichage tridimensionnel (3d) répondant au mouvement d'un utilisateur, et interface utilisateur pour le système d'affichage 3d
WO2013162236A1 (fr) Appareil tactile virtuel d'affichage transparent sans pointeur
US20130002559A1 (en) Desktop computer user interface
EP3881167A1 (fr) Système et procédé pour dispositif d'entrée polyvalent pour environnements bidimensionnels et tridimensionnels
WO2019135550A1 (fr) Dispositif électronique de commande d'un affichage d'image sur la base d'une entrée de défilement et procédé associé
WO2016080557A1 (fr) Dispositif pouvant être porté et son procédé de commande
WO2013133624A1 (fr) Appareil d'interface utilisant une reconnaissance de mouvement, et procédé destiné à commander ce dernier
WO2014035118A1 (fr) Visiocasque et procédé de commande de dispositif numérique l'utilisant
WO2016208881A1 (fr) Interface utilisateur tridimensionnelle pour un visiocasque
US20240061798A1 (en) Debug access of eyewear having multiple socs
US20230262207A1 (en) Dual system on a chip eyewear having a mipi bridge
JP2021056371A (ja) 表示システム、表示方法及び表示プログラム
US20230117690A1 (en) Dual system on a chip eyewear
WO2014178477A1 (fr) Visiocasque et procédé de fourniture de contenus en l'utilisant

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13832014

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13832014

Country of ref document: EP

Kind code of ref document: A1