KR20170084443A - Method for controlling head mount display device - Google Patents

Method for controlling head mount display device Download PDF

Info

Publication number
KR20170084443A
KR20170084443A KR1020160003498A KR20160003498A KR20170084443A KR 20170084443 A KR20170084443 A KR 20170084443A KR 1020160003498 A KR1020160003498 A KR 1020160003498A KR 20160003498 A KR20160003498 A KR 20160003498A KR 20170084443 A KR20170084443 A KR 20170084443A
Authority
KR
South Korea
Prior art keywords
display device
head
gesture
mounted display
reference position
Prior art date
Application number
KR1020160003498A
Other languages
Korean (ko)
Inventor
김영윤
Original Assignee
엘지전자 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 엘지전자 주식회사 filed Critical 엘지전자 주식회사
Priority to KR1020160003498A priority Critical patent/KR20170084443A/en
Publication of KR20170084443A publication Critical patent/KR20170084443A/en

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems

Abstract

A method of controlling a head mounted display device, the method comprising: setting a position at a time when a device is activated to a reference position; Providing a reference screen on a display unit; Sensing a change in the position of the head-mounted display device; Adjusting a direction of an image to be output to the display unit in accordance with the positional change; Resetting the reference position; And controlling the display unit such that the reference screen is positioned at the reset reference position. The method of controlling a head-mounted display device may easily change the reference direction to improve convenience of use.

Description

[0001] The present invention relates to a method of controlling a head mount display device,

The present invention relates to a head-mounted display device control method relating to a method of changing a reference direction.

A head-mounted type display device has appeared, in which a variety of display forms come into view while attaching to the head to view an image. Such a device is called a head mount display (HMD), and the head mount display is positioned adjacent to the eye, so focus adjustment is necessary.

In particular, in recent years, a virtual reality head mount display has been introduced to provide a 3D stereoscopic image, not a 2D image, and a variety of images can be provided by turning a head 360 ° panoramic image as well as a vertical panoramic image according to a user's movement.

The head-mounted display device switches the screen according to the detected movement to provide a realistic image, and detects a change in direction and position relative to the reference direction to provide another image.

It is an object of the present invention to provide a control method of a head-mounted display device which easily changes a reference direction to switch to a screen desired by a user.

A method of controlling a head mounted display device, the method comprising: setting a position at a time when a device is activated to a reference position; Providing a reference screen on a display unit; Sensing a change in the position of the head-mounted display device; Adjusting a direction of an image to be output to the display unit in accordance with the positional change; Resetting the reference position; And controlling the display unit such that the reference screen is located at the reset reference position.

Wherein the resetting of the reference position comprises: sensing a first gesture of the user's hand using the camera of the head-mounted display device; And setting a position of a point of time when the first gesture is sensed as a reference position.

The first gesture is in a state in which the hand is in an unfolded state, and the reference position can be reset when the camera senses the palm of a hand.

The method may further include activating the camera prior to sensing the first gesture.

Wherein the resetting of the reference position comprises: sensing a second gesture of the user's hand using the camera of the head-mounted display device; Fixing a screen position at a point of time when the second gesture is sensed; Changing a position of the head-mounted display device; And setting a position of a point of time when the second gesture is not detected as a reference position.

The second gesture may be a single finger.

The step of resetting the reference position may include: confirming whether the user should change the reference position in the current direction when the position is fixed for a predetermined time or longer in a state where the reference position is out of a predetermined range; And change the reference position according to the result of the check.

Sensing a third gesture of the user's hand using the camera of the head-mounted display device; And capturing an image of the display unit at the time of sensing the third gesture.

According to at least one embodiment of the present invention, it is possible to easily change the reference direction of the head-mounted display device, thereby improving convenience of use.

1 is a block diagram of a head-mounted display device of the present invention.
2 is a perspective view showing the head-mounted display device of the present invention.
3 is a view for explaining a screen viewed through the head-mounted display device of the present invention.
Figs. 4 to 7 are views showing an embodiment of a step of adjusting the screen of the head-mounted display device of the present invention.
8 is a view showing another embodiment of the step of adjusting the screen of the head-mounted display device of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals are used to designate identical or similar elements, and redundant description thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

FIG. 1 is a block diagram of a head-mounted display device 100 of the present invention, and FIG. 2 is a perspective view of the head-mounted display device 100 of the present invention as viewed from the other side. The head mount display device 100 of the present invention includes a display unit 210, an acoustic output unit 165, a driver 260, a user input unit 123, an interface unit 160, a sensing unit 140, And a wireless communication unit 190.

The housing 110 is located on the front of the user's eyes, and the front surface may be either flat or curved but the back surface may include a curved surface bent to the shape of the user's face. The housing 110 may be mounted with various components and the housing 110 may be formed by injection molding of synthetic resin or may be formed of metal such as stainless steel (STS), aluminum (Al), titanium (Ti), or the like. Each case can use the same material or different materials.

And a hair fixing part 130 for fixing the housing 110 to the user's head. The head fixing unit 130 may be formed in various forms such as a band type, a spectacle type, and a helmet type.

The display unit 210 is a device for outputting image information from a video output unit on one side, and is provided on the left and right eyes of the head-mounted display device 100, respectively. The display unit 210 provides an image to be provided to the left or right eye and may provide the same image or may provide a 3D stereoscopic image when providing two images having different imaging angles to the left eye and the right eye, respectively.

The display unit 210 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) display, a 3D display, and an e-ink display.

As shown in FIG. 2, the display unit 210 may be built in. However, the smart phone 200 may be coupled to the housing 110 to output an image through the smart phone 200. At this time, the user input unit may receive a command through wireless communication to control the smart phone 200 or may be connected through the interface unit of the smart phone 200 to receive the command.

The display unit 210 may further include a lens positioned between the image output unit and a user's eye to adjust a focal length of an image output from the display unit 210. [

The driving unit 260 adjusts the position of the display unit 210 according to the pupil distance of the left and right eyes of the user and adjusts the position of the lens 250 positioned on the front surface of the display unit 210 according to the visual acuity of the user It plays a role. The driving unit 260 includes horizontal adjustment units 223, 265, and 267 for adjusting the distance between the left eye and the right eye and a focus adjustment unit 223 for adjusting the distance between the lens 250 and the display unit 210 according to the visual acuities of the left and right eyes, (227, 247).

The sound output unit 165 may include a bone conduction speaker to transmit sound directly to the user's skull or an ear jack to transmit sound to the user through the earphone.

The user input unit 123 is for receiving information from a user. When information is inputted through the user input unit 123, the control unit 180 can control the operation of the mobile terminal 100 to correspond to the input information .

The user input unit 123 may include a mechanical input means (or a mechanical key, for example, a button located on the front, rear or side of the head mounted display device 100, a dome switch, Wheel, jog switch, etc.) and touch-type input means. The user input unit 123 of the present embodiment may be provided in a form exposed on the side surface of the housing 110 using a button shape including a dome switch.

The interface unit 160 is a path through which the head-mounted display device 100 can be connected to an external device. For example, the interface unit 160 may receive a video signal or receive power from an external device. Since the apparatus is mounted on the face, if the battery is provided in the head mount display apparatus 100 itself, there is a problem that the weight increases, and external power can be supplied through the interface unit 160.

The sensing unit 140 may include a proximity sensor 143 (see FIG. 3) that senses whether or not the user wears the head-mounted display device 100. When the proximity sensor 143 senses wearing, the display unit 210 can be activated.

The sensing unit 140 may further include a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor An RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, and the like.

The wireless communication unit 190 may wirelessly communicate with an external terminal and receive image information from the external terminal. And is mainly used in a state where the distance from the external terminal is not too far. (Bluetooth), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity Direct, and Wireless USB (Wireless Universal Serial Bus) technology.

The control unit 180 is connected to the display unit 210, the wireless communication unit, the sensing unit 140, the interface unit 160, and the user input unit 123 to control the head mount display device 100. The control unit 180 controls the display unit 210 to output an image on the basis of the image data received through the wireless communication unit or the interface unit 160. Also, the controller 180 can activate the display unit 210 of the head-mounted display device 100 using the sensing result sensed by the sensing unit 140.

The display unit 210 can be controlled according to the user command signal input from the user input unit 123. [ The user input unit 123 may control the external terminal through the interface unit 160 and the wireless communication unit, if necessary.

The user input unit 123 can perform input through sensors as well as buttons. For example, a user command may be inputted through a touch sensor or a gesture may be sensed through an image sensor to receive a user command.

3 is a view for explaining a screen viewed through the head mount display device 100. The screen viewed using the head mount display device 100 may be graphically created as a virtual reality image, You can see the video. The conversion of the image changes according to the movement of the user as if the actual scene or object is being viewed.

When the head-mounted display device 100 is driven for the first time, the direction that the user looks at is set as the reference direction. The reference direction changes the screen in accordance with the change of the position and direction relative to the reference direction in a direction that becomes the reference point of the movement thereafter. A screen provided through the display unit has a reference screen and a reference screen is provided when the reference direction matches the direction of the head-mounted display device 100. [ For example, the image of the portion A in Fig. 3 can be the reference screen. However, the present invention is not limited thereto.

The position of the head mount display device 100 is detected through a gyro sensor, an acceleration sensor, or the like, and is set as a reference direction. Then, the head-mounted display device 100 senses the movement of the user, and when the head-mounted display device 100 rotates in the leftward direction, the left-side image B of the reference screen A is provided. And the image C on the right side of the reference screen A is provided.

However, when the user starts driving the head-mounted display device 100 in a state in which the user does not look at the front as shown in FIG. 4, the reference direction is turned to the side direction other than the front side of the user, Since the image on the left side is provided instead of the image, it is necessary to reset the reference direction because there is inconvenience that the state must be maintained.

It is necessary to reset the reference direction even when the posture of the user is changed later, such as when the user wants to use the head-mounted display device by lying down after starting.

5 and 6 are views showing an embodiment of a step of adjusting the screen of the head-mounted display apparatus 100 of the present invention. When the user starts driving the head-mounted display device 100 in the state shown in Fig. 4, the screen shown in Fig. 5 (a) (now a quasi-screen A) at the posture of Fig. When the line of sight is turned to the left with the front view (see Fig. 6), the left image B of the reference screen is provided as shown in Fig. 5 (b).

At this time, if the predetermined direction is longer than a predetermined range in the reference direction, the icon 21 for resetting the reference direction may be provided as shown in FIG. 5 (b).

In the present embodiment, it is possible to set using the hand gesture to reset the reference direction. 6, when the user stretches the palm of the hand and places the palm of the hand in the front direction, the camera of the head-mounted display device 100 detects the hand shape through the camera (if a smart phone is mounted, the camera of the smart phone can be used) can do.

When the hand shape is the first gesture 11 (in this embodiment, the palm is shaped), the reference direction is switched to the direction at the time when the first gesture is sensed, The screen is switched to the reference screen. That is, when the image is started in an uncomfortable posture or when the image is laid down, the posture can be reset to the reference direction and the image can be appreciated.

7 is a view showing another embodiment of the step of adjusting the screen of the head-mounted display apparatus 100 of the present invention. In a case where it is continued for a predetermined time or longer in a state deviating from the reference direction, instead of displaying the icon 21 on the screen in the embodiment of FIG. 5, a window 22 for asking whether to change the reference direction directly is provided .

If yes is selected, the reference direction is changed to the direction that the current head-mounted display device 100 faces, and the screen is changed as shown in FIG. 7 (b). If no is selected, the reference direction is maintained.

Next, Fig. 8 is a view showing another embodiment of the step of adjusting the screen of the head-mounted display device 100 of the present invention. If the user wishes to continue changing the pose while listening, he / she can use the pause function to stop the current screen and restart the pose after changing the pose.

8, when the second gesture 12 (in this embodiment, one finger is tilted) is sensed through the camera as shown in FIG. 8, the screen is stopped while the second gesture 12 is detected, If the second gesture 12 is no longer detected, the screen may be restarted again.

When stopping the screen, even when the screen is not facing the reference direction, the reference direction also changes when stopping and restarting after changing the direction, so that the image can be provided according to the new reference direction.

Or a third gesture (e.g., a gesture of gripping a fist) is detected, the user can capture the screen currently being viewed and store it in the memory. The above-described first to third gestures are not limited to the above-mentioned embodiments and can be changed.

According to at least one embodiment of the present invention, it is possible to easily change the reference direction of the head-mounted display device, thereby improving convenience of use.

The foregoing detailed description should not be construed in all aspects as limiting and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

100: head mount display device 110: housing
123: user input unit 130:
143: proximity sensor 160: interface unit
185: main board 190: wireless communication section
200: smartphone 210: display unit

Claims (8)

A method of controlling a head-mounted display device,
Setting a position at a time when the device is activated to a reference position;
Providing a reference screen on a display unit;
Sensing a change in the position of the head-mounted display device;
Adjusting a direction of an image to be output to the display unit in accordance with the positional change;
Resetting the reference position; And
And controlling the display unit to position the reference screen at the reset reference position. The method according to claim 1,
Wherein the step of resetting the reference position comprises:
Sensing a first gesture of the user's hand using the camera of the head-mounted display device; And
And setting a position of a point of time when the first gesture is detected as a reference position. 3. The method of claim 2,
Wherein the first gesture resets the reference position when the hand is in the unfolded state and the camera senses the palm of the hand. 3. The method of claim 2,
Before the step of sensing the first gesture,
Further comprising the step of activating the camera. The method according to claim 1,
Wherein the step of resetting the reference position comprises:
Sensing a second gesture of the user's hand using the camera of the head-mounted display device;
Fixing a screen position at a point of time when the second gesture is sensed;
Changing a position of the head-mounted display device; And
And setting a position of a point of time when the second gesture is not detected as a reference position. 6. The method of claim 5,
Wherein the second gesture is a state in which one finger is in a flat state. The method according to claim 1,
Wherein the step of resetting the reference position comprises:
Confirming whether the user should change the reference position in the current direction when the position is fixed for a predetermined time or more in a state where the existing reference position is out of a predetermined range; And
And the reference position is changed according to the result of the checking. The method according to claim 1,
Sensing a third gesture of the user's hand using the camera of the head-mounted display device; And
Further comprising the step of capturing an image of a display unit at a point of time when the third gesture is sensed.
KR1020160003498A 2016-01-12 2016-01-12 Method for controlling head mount display device KR20170084443A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020160003498A KR20170084443A (en) 2016-01-12 2016-01-12 Method for controlling head mount display device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020160003498A KR20170084443A (en) 2016-01-12 2016-01-12 Method for controlling head mount display device

Publications (1)

Publication Number Publication Date
KR20170084443A true KR20170084443A (en) 2017-07-20

Family

ID=59443347

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020160003498A KR20170084443A (en) 2016-01-12 2016-01-12 Method for controlling head mount display device

Country Status (1)

Country Link
KR (1) KR20170084443A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113574592A (en) * 2019-03-19 2021-10-29 佳能株式会社 Electronic device, control method for electronic device, program, and storage medium
US11687209B2 (en) 2019-02-27 2023-06-27 Beijing Sensetime Technology Development Co., Ltd. Display method and apparatus for displaying display effects
WO2023132539A1 (en) * 2022-01-07 2023-07-13 삼성전자 주식회사 Electronic device for providing augmented reality, and operation method therefor

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11687209B2 (en) 2019-02-27 2023-06-27 Beijing Sensetime Technology Development Co., Ltd. Display method and apparatus for displaying display effects
CN113574592A (en) * 2019-03-19 2021-10-29 佳能株式会社 Electronic device, control method for electronic device, program, and storage medium
WO2023132539A1 (en) * 2022-01-07 2023-07-13 삼성전자 주식회사 Electronic device for providing augmented reality, and operation method therefor

Similar Documents

Publication Publication Date Title
US10983593B2 (en) Wearable glasses and method of displaying image via the wearable glasses
EP3740811B1 (en) Head mounted display apparatus and operating method thereof
CN105700676B (en) Wearable glasses, control method thereof, and vehicle control system
KR101595957B1 (en) Mobile terminal and controlling system
US11199830B2 (en) Operating system for a machine of the food industry
WO2016063801A1 (en) Head mounted display, mobile information terminal, image processing device, display control program, and display control method
CN111630477A (en) Apparatus for providing augmented reality service and method of operating the same
KR20160006053A (en) Wearable glass-type device and control method of the wearable glass-type device
JP2018142857A (en) Head mounted display device, program, and control method of head mounted display device
KR20160056133A (en) Method for controlling display of image and apparatus implementing the same
KR20150146295A (en) Head mounted display and method for controlling the same
KR20150142516A (en) Glass type terminal and control method thereof
TW201604587A (en) Wearable glasses, displaying image method and non-transitory computer-readable storage medium
KR20180004112A (en) Eyeglass type terminal and control method thereof
KR20150131837A (en) Mobile terminal and control method for the mobile terminal
CN106067833B (en) Mobile terminal and control method thereof
KR20170084443A (en) Method for controlling head mount display device
KR20190067523A (en) Glass type terminal and operpation method of the same
US9298010B2 (en) Wearable optical display with audio functionality
JP2016126589A (en) Display system, program, and method of controlling display device
KR20190079263A (en) Head mounted display
JP2016090853A (en) Display device, control method of display device and program
KR20150084485A (en) Mobile terminal and controlling method thereof
KR101786515B1 (en) Head mounted display and method for controlling the same
KR101607379B1 (en) Mobile terminal and method for controlling the same