TW201606352A - Head mounted display device for displaying image and method thereof - Google Patents

Abstract

A head mounted display (HMD) device, a method by the HMD of displaying an image, a non-transitory computer readable recording medium, and a chipset are provided. The HMD device includes a color information obtaining unit configured to obtain color information regarding a lens arranged at a portion of the HMD device; and a control unit configured to output an image modified based on the color information via a display unit arranged at the HMD device. The method includes obtaining color information regarding a lens arranged at a portion of the HMD device; and outputting an image modified based on the color information via a display unit arranged at the HMD device.

Description

Head-mounted display device for displaying images and method thereof

This application claims the priority of the following patent applications: Korean Patent Application No. 10-2014-0096765, filed at the Korea Intellectual Property Office on July 29, 2014, and January 15, 2015 The Korean Patent Application No. 10-2015-0007447 filed by the Korean Intellectual Property Office, the entire contents of which are hereby incorporated by reference.

The present invention relates generally to a wearable display device and, more particularly, to a Head Mounted Display (HMD) device.

As the weight and size of digital devices have decreased, various wearable devices have been developed. Among wearable devices, a head mounted display (HMD) device refers to a variety of digital devices that are worn by a user on the head and provide multimedia content to the user. The HMD device can be embodied in any of a variety of forms to be worn on the head, such as glasses, helmets, and the like.

Since the HMD device is used after being attached to the user's body, the HMD device provides the user with images from various environments as the user moves. Therefore, it is necessary for the HMD device to include lenses having various colors or transmittances. For example, the HMD device can include a dark lens to block daylight when the daylight is strong, or a transparent color lens that is used when in an indoor position. Therefore, the user can properly view the surroundings thereof.

The HMD device can switch a lens disposed at its side to a lens having another color. Alternatively, if the lens disposed at the side of the HMD device is a smart window capable of changing at least one of its color and transmittance, the HMD device can control the color and transmittance of the lens. When the color or transmittance of the lens disposed at the side of the HMD device is modified, the HMD device can provide an image that may be more easily recognized by the user.

The present invention has been made to solve the above-mentioned problems and disadvantages, and at least to provide the advantages described below.

Accordingly, an aspect of the present invention provides an HMD device capable of providing content via a user interface that may be more clearly recognized by a user, and a method of displaying content via an HMD device.

Additional aspects will be set forth in part in the description which follows, and the additional aspects will be apparent from the description.

According to one aspect of the invention, an HMD is provided. The HMD includes: a color information obtaining unit configured to obtain color information about a lens disposed at a portion of the HMD device; and a control unit configured to pass through a display unit disposed at the HMD device An image modified based on the color information is output.

According to another aspect of the present invention, a method for displaying an image by an HMD device is provided. The method includes: obtaining color information about a lens disposed at a portion of the HMD device; and outputting an image modified based on the color information via a display unit disposed at the HMD device.

According to another aspect of the present invention, a non-transitory computer readable recording medium having a computer program recorded thereon for implementing a method for displaying an image by an HMD device is provided. The method includes: obtaining color information about a lens disposed at a portion of the HMD device; and outputting an image modified based on the color information via a display unit disposed at the HMD device.

According to another aspect of the present invention, a wafer set for displaying images is provided. The wafer set is configured to obtain color information about a lens disposed at a portion of the HMD device, and to output an image modified based on the color information via a display unit disposed at the HMD device.

The embodiments of the present invention will be described in detail with reference to the accompanying drawings, In this regard, the embodiments of the invention may have different forms and should not be construed as being limited to the descriptions set forth herein. Therefore, the embodiments of the present invention are merely described below to explain the aspects of the present invention. The term "and/or" as used herein includes any or all of one or more of the associated listed items. When preceded by a list of components, an expression such as "at least one of" is used to modify the entire list of elements and does not modify the individual elements of the list.

Throughout the present invention, it will be understood that when a part is referred to as "connected to" another portion, it can be "directly connected" to another portion or "electrically connected" to another portion via another element. In addition, it should be further understood that the term "comprises and/or "comprising" as used herein refers to the existence of the stated feature or component, but does not exclude the presence or addition of one or more other features or components.

Throughout the invention, a "lens" may refer to an article that is capable of transmitting light therethrough. The lens is not limited to objects having a surface for forming an optical image. For example, the lens can be an article having a flat surface. The lens can be formed from glass, crystal or plastic. Alternatively, the lens can be a smart window. The smart window can include an element, at least one of which can be altered based on the signal applied thereto. For example, an electrochromic glass, a Suspended Particle Device (SPD), or a Liquid Crystal (LC) can be used to form a smart window.

Throughout the present invention, an image displayed by an HMD device includes not only an image directly displayed on the lens but also an image recognized by the user as displayed on the lens.

In addition, throughout the present invention, "displaying an image on a lens" includes not only the condition in which the image is directly displayed on the lens, but also the condition in which the output image is displayed for the user to be displayed on the lens.

FIG. 1 illustrates an HMD device 100 in accordance with an embodiment of the present invention.

Referring to Figure 1, the HMD device 100 can be embodied in the form of glasses that can be secured to the user's face via the ears and nose of the user (or wearer). However, the configuration of the HMD device 100 is not limited thereto. For example, the HMD device 100 can be attached to a helmet or can be embodied in the form of a goggle.

The HMD device 100 according to an embodiment of the present invention may include a frame 110. The HMD device 100 in accordance with an embodiment of the present invention may include a bridge 140 to be secured to the nose of the user at the central portion 130. According to an embodiment of the invention, the bridge 140 can include a bridge arm 131 and a pad 141.

According to an embodiment of the invention, the detachable lens 120 can be attached to the frame 110. Multiple lenses 120 of different colors can be used. For example, a user may use an HMD device 100 having a clear lens 120 attached thereto therein, and may use an HMD device 100 having a lens 120 attached thereto in a dark outdoor environment. For example, the clip 121 disposed at the lens 120 can be attached to the bridge arm 131 of the HMD device 100.

Further, the HMD device 100 may include a display unit 111 for displaying an image. According to an embodiment of the present invention, the display unit 111 can be embodied in various forms. For example, display unit 111 can be configured to form an image on a user's retina or can be configured to display an image on a lens as shown in FIG.

FIG. 2 illustrates an HMD device 100 in accordance with an embodiment of the present invention.

Referring to Figure 2, the HMD device 100 can be embodied in the form of glasses that can be secured to the user's face via the ears and nose of the user (or wearer). However, the configuration of the HMD device 100 is not limited thereto. For example, the HMD device 100 can be attached to a helmet or can be embodied in the form of a goggle.

The HMD device 100 shown in FIG. 2 may include a lens frame 201, a right arm 202 and a left arm 203, a lens 204, an on-board computer 205, a camera 206, a user input unit 207, and a sensor 208. 209 and 210, a haptic module 211, a display 212, an optical output unit 213, a microphone 214, a right side speaker 215 and a left side speaker 216, and a power supply unit 217.

Some of the components included in the HMD device 100 may be disposed inside the HMD device 100, and the remaining components may be disposed outside the HMD device 100. For example, the sensors 208, 209, and 210 can be disposed inside the HMD device 100. The display 212 can be disposed outside of the HMD device 100. The components disposed inside the HMD device 100 and the components disposed outside the HMD device 100 are not limited thereto.

The components included in the HMD device 100 are not limited to those components shown in FIG. 2. For example, HMD device 100 may not include some of sensors 208, 209, and 210 and haptic module 211. For example, HMD device 100 may only include sensor 210 among sensors 208, 209, and 210, or may include more sensors at more locations inside or outside of HMD device 100 (with Figure 2 Compare these sensors as shown).

The lens frame 201 and the right side arm 202 and the left side arm 203 may be formed of a solid material such as plastic and/or metal, and may have a hollow structure to include wiring for interconnecting components included in the HMD device 100.

The lens frame 201 and the right arm 202 and the left arm 203 shown in Fig. 2 can be integrated with each other without a connecting member. Therefore, the lens frame 201 and the right arm 202 and the left arm 203 may be collectively referred to as a frame of the HMD device 100.

The structure of the lens frame 201 and the right arm 202 and the left arm 203 is not limited to the structure shown in FIG. 2. For example, the HMD device 100 can be configured to have a structure in which a connecting member is disposed between the lens frame 201 and the right arm 202 and the left arm 203. In other words, if the HMD device 100 is configured to have a structure in which the connecting member is disposed between the lens frame 201 and the right arm 202 and the connecting member is disposed between the lens frame 201 and the left arm 203, the right arm 202 and the left arm The 203 can be folded toward the lens frame 201.

If the HMD device 100 is configured to have a structure in which the connecting member is disposed at the center 201' of the lens frame 201, the connecting member is disposed between the lens frame 201 and the right arm 202, and the connecting member is disposed on the lens frame 201 and the left arm Between 203, and the connecting members are disposed at the respective centers 202' and 203' of the right arm 202 and the left arm 203, and the lens frame 201 and the right arm 202 and the left arm 203 can be folded inward via the respective connecting members. The volume of the HMD device 100 is reduced.

The connecting member may be assembled with a screw or may be attached via a hook and a groove. However, the invention is not limited thereto.

The right arm 202 and the left arm 203 can be formed to have a shape to be hung on the user's ear. The right arm 202 and the left arm 203 can be configured to extend to the back of the user's head. If the right arm 202 and the left arm 203 are configured to extend to the back of the user's head, a strip for securing the right arm 202 and the left arm 203 to the user's head can be added to the right arm 202. And the left arm 203. The strap may be an elastic strip for securing the right arm 202 and the left arm 203 to the user's head regardless of the size of the user's head.

The anti-slip member may be attached to the inner portion of the right arm 202 and the left arm 203 that contacts the portion of the user's head behind the ear cup to prevent the right arm 202 and the left arm 203 from sliding.

The lens frame 201 can be configured to attach and detach the lens 204 thereto. The lens 204 shown in Figure 2 is integrated with the bridge 204'. However, the invention is not limited thereto. For example, if the lens frame 201 is integrated with the nose bridge 204', the lens 204 can include a right side lens 404 and a left side lens 405, as shown in FIG.

Lens 204 may be formed from a transparent material that allows a user to view through it. Lens 204 may be formed from a material that transmits light reflected by display 212. The material used to form the lens 204 may comprise a plastic such as polycarbonate and glass, but the material is not limited thereto. Lens 204 can comprise at least one of: an anti-reflective and anti-glare coating, an anti-fog coating, and an ultraviolet radiation (UltraViolet; UV) light blocking coating.

The onboard computer 205 can be connected to the HMD device 100 by wires or wirelessly. The onboard computer 205 is located near a portion of the left arm 203 of the lens frame 201. However, the invention is not limited thereto. For example, the onboard computer 205 can be disposed at another portion of the HMD device 100, such as a portion of the right arm 202, another portion of the left arm 203, and the like.

The onboard computer 205 can receive data from the camera 206, the user input unit 207, and the sensors 208, 209, and 210, analyze the received data, and generate to be passed through the optical output unit 213, the display 212, and the right speaker 215 and the left speaker. At least one of the 216 provides information to the user of the HMD device 100. The information to be provided to the user may include at least one of images, text, video, and audio. However, the invention is not limited thereto.

The camera 206 may be disposed at an outer portion of the left arm 203 or may be disposed inside the left arm 203. Camera 206 can be attached to other locations of HMD device 100. For example, camera 206 can be disposed at a portion of right arm 202 or at a portion of lens frame 201 (eg, center). Camera 206 can be a small camera for a smart phone or webcam. The camera 206 can be disposed at a location for photographing at least a portion of the environment of the user of the HMD device 100.

The user input unit 207 can include at least one of the following: a touchpad that can be operated by a user's finger; and a button that can be pushed by the user. However, the invention is not limited thereto. The user input unit 207 can be disposed at a position of the left arm 203 or at other positions of the HMD device 100. For example, the user input unit 207 can be disposed at another portion of the left arm 203, a portion of the right arm 202, or a portion of the lens frame 201.

The user input unit 207 is available to the user for inputting commands. The user input unit 207 can include an "on/off" switch for turning on/off the HMD device 100.

The sensor 208 can be a biometric information detecting sensor for detecting biometric information of the user. The sensor 208 is configured to protrude from the lower right portion of the lens frame 201. However, the attachment of the sensor 208 is not limited thereto. For example, the sensor 208 can be disposed inside the lens frame 201 or can be disposed on the lens frame 201. The sensor 208 can be configured at any portion of the HMD device 100.

If the sensor 208 consists essentially of an image sensor based camera, the sensor 208 can detect changes in the user's eye congestion, dry eyes, eye blink rate, pupil position, and eye pupil. However, the invention is not limited thereto.

If the sensor 208 is at least one of an oxygen sensor, an air sensor, and a fine dust sensor, the sensor 208 can detect the user's dry eyes. However, the invention is not limited thereto.

The sensor 209 can be a surrounding environment sensor for detecting the surrounding environment of the HMD device 100. The sensor 209 is disposed at an inner central portion of the lens frame 201. However, the invention is not limited thereto. For example, the sensor 209 may be disposed at at least one of the inner portion of the lens frame 201 except the inner central portion of the lens frame 201, the outer portion of the lens frame 201, the right side arm 202, and the left side. The inner portion of the arm 203, and the outer portion of the right arm 202 and the left arm 203. However, the position and structure for attaching the sensor 209 are not limited thereto. The sensor 209 can be a sensor for detecting environmental conditions such as temperature near the HMD device 100, light conditions, humidity, wind speed, altitude, and atmospheric pressure. However, the detectable environmental conditions are not limited to this.

If the sensor 208 is configured to at least one of an oxygen sensor, an air sensor, and a fine dust sensor as described above, information about the surrounding environment of the HMD device 100 may be based on sensing The signal detected by the device 208. In this case, the HMD device 100 may not include the sensor 209.

The sensor 210 may be a sensor for detecting a state in which the HMD device 100 is worn. The sensor 210 is disposed at an inner portion of the right arm 202 that contacts the surface of the user's head behind the ear. However, the position of the sensor 210 is not limited thereto. For example, the sensor 210 may be disposed at an inner portion of the left arm 203 that contacts the surface of the user's head behind the ear or a power supply unit 217 that contacts the surface of the user's head behind the ear. internal.

The sensor 210 can be a pressure sensor. In this situation, the sensor 210 can detect its pressure and magnitude and transmit a signal regarding the pressure to the onboard computer 205. Therefore, based on the detected value received by the sensor 210, the onboard computer 205 can detect the time period when the user wears the HMD device 100 or the time period when the user executes the application while wearing the HMD device 100. .

The haptic module 211 is disposed at an inner portion near the end of the left arm 203 of the HMD device 100. However, the position of the haptic module 211 is not limited thereto. For example, the haptic module 211 can be disposed at an inner portion of the right arm 202 or at another location of the HMD device 100.

The haptic module 211 can be formed of a material that is capable of producing various tactile sensations that can be perceived by a user. For example, the haptic module 211 can be formed of a material capable of producing various tactile sensations, such as materials capable of producing vibrations, stimuli that contact the skin surface, stimuli based on jets or inhaled air, stimuli based on electrode-based contacts. The stimulus achieved by electrostatic force, the cold/heat sensation based on the heat absorbing device or the heat generating device, and the like. However, the invention is not limited thereto.

Display 212 is coupled to left arm 203 of HMD device 100 and is located at the upper left portion of lens 204. However, the position of the display 212 is not limited thereto. For example, display 212 can be coupled to right arm 202 to be located at the right center of lens 204 or can be coupled to left arm 203 to be located at the left center of lens 204. Display 212 can be formed from a translucent optical waveguide (eg, germanium). However, the inventive concept is not limited to this. Accordingly, the display 212 can reflect the light output by the optical output unit 213 and focus the image at the fovea of the retina of the user's eye of the HMD device 100.

The optical output unit 213 can be configured as a micro projector. However, the invention is not limited thereto.

The microphone 214 can receive a voice signal of the user of the HMD device 100 and other sounds. The microphone 214 can be configured to be selectively set to a close range mode or a remote mode. For example, if the microphone 214 is set to the close range mode, the microphone 214 can be used to receive sound regarding the biometric information of the user of the HMD device 100 (eg, information about the blink of the eye). If the microphone 214 is set to the remote mode, the microphone 214 can be used to receive the voice of the user of the HMD device 100. The close mode of the microphone 214 may be a mode for receiving sound generated inside the HMD device 100, and the remote mode of the microphone 214 may be a mode for receiving sound generated outside the HMD device 100. However, the operating range of the close mode and the remote mode can be set in different ways.

The right side speaker 215 and the left side speaker 216 can be configured as earphones to be worn on the ear of the user of the HMD device 100. The right side speaker 215 is located at an inner portion of the right arm 202 so as to be worn on the right ear of the user. The left side speaker 216 is located at an inner portion of the left side arm 203 so as to be worn on the left ear of the user.

The right side speaker 215 and the left side speaker 216 can be secured to the HMD device 100, as shown in FIG. However, the attachment of the right side speaker 215 and the left side speaker 216 is not limited thereto. For example, the right side speaker 215 and the left side speaker 216 can be detachably attached to the HMD device 100, and thus, the user of the HMD device 100 can selectively insert the right side speaker 215 and the left side speaker 216 into the user's ear. .

If the right speaker 215 and the left speaker 216 are detachable from the HMD device 100, the right speaker 215 and the left speaker 216 may be disposed at the inner portion of the right arm 202 and the left arm 203 that contact the user's head behind the ear. . Therefore, the user can detach the right side speaker 215 and the left side speaker 216 from the respective right side arm 202 and left side arm 203 and insert the right side speaker 215 and the left side speaker 216 into the right ear and the left ear of the user, respectively. In the case where the right side speaker 215 and the left side speaker 216 are detached from the respective right side arm 202 and the left side arm 203 and the right side speaker 215 and the left side speaker 216 are respectively inserted into the right ear and the left ear of the user as described above, the connecting line may be It is disposed between the right side speaker 215 and the left side speaker 216 and the HMD device 100.

When the right side speaker 215 and the left side speaker 216 are attached to the HMD device 100, the connection line between the right side speaker 215 and the left side speaker 216 and the HMD device 100 may be disposed in the HMD device 100. However, the invention is not limited thereto. The right side speaker 215 and the left side speaker 216 and the HMD device 100 can be wirelessly connected to each other based on short-range communication.

A connecting member based on an elastic member (for example, a spring) may be disposed between the right side speaker 215 and the left side speaker 216 and the HMD device 100. In this case, the elastic member-based connecting member can be deformed based on the position of the user's ear. For example, the connecting member may be deformed to enable the right side speaker 215 and the left side speaker 216 to be tilted in at least one of a leftward direction, a rightward direction, an upward direction, and a downward direction. If there is a resilient member-based connecting member between the right side speaker 215 and the left side speaker 216 and the HMD device 100, the right side speaker 215 and the left side speaker 216 and the HMD device 100 can be configured to communicate with each other via wires or wirelessly. According to an embodiment of the invention, the right side speaker 215 and the left side speaker 216 may be configured as a bone conduction type speaker.

The power supply unit 217 is disposed at an end portion of the right arm 202 of the HMD device 100. However, the position of the power supply unit 217 is not limited thereto. For example, the power supply unit 217 may be disposed at an end portion of the left arm 203 of the HMD device 100.

The power supply unit 217 can be detachably attached to the right arm 202 of the HMD device 100. The power supply unit 217 can include an external power connector that can be connected to an external power supply to charge the HMD device 100 after being connected to or detached from the right arm 202. The power supply unit 217 can be integrated with the right arm 202 of the HMD device 100. Even when the power supply unit 217 is integrated with the right arm 202 of the HMD device 100, the power supply unit 217 may also include an external power connector that is connectable to the external power supply device to charge the HMD device 100.

FIG. 3 illustrates an HMD device 300 in accordance with an embodiment of the present invention. The HMD device 300 shown in Figure 3 includes a monocular lens 303 that can be embodied in the form of a glass-type display device as shown in Figure 2 or in the form of a monocular display device.

Referring to FIG. 3, an HMD device 300 in accordance with an embodiment of the present invention is a monolith-like device that is secured to a user's face via one of the user's ears and one side of the nose. The HMD device 300 can be attached to a helmet structure.

The HMD device 300 can include a lens frame 301, a side arm 302, a monocular lens 303, an onboard computer 304, a camera 305, a user input unit 306, sensors 307, 308, and 309, a haptic module 310, a display 311, and an optical output. The unit 312, the microphone 313, the speaker 314, and the power supply unit 315.

The components included in the HMD device 300 are not limited to those components shown in FIG. For example, HMD device 300 may not include some of sensors 307, 308, and 309 and haptic module 310. For example, HMD device 300 may only include sensor 309 among sensors 307, 308, and 309, or may include more sensors at locations internal or external to HMD device 300 (as shown in FIG. 3) Comparison of their sensors). The position at which the sensors 307, 308, and 309 are attached to the HMD device 300 can vary.

The lens frame 301 can be configured to be attachable to and detachable from the HMD device 300, as described above with reference to FIG. Further, as described above with reference to FIG. 2, the HMD device 300 may include a connecting member between the lens frame 301 and the side arm 302. Further, as described above with reference to FIG. 2, the HMD device 300 can include a connecting member at the center of the side arm 302.

The monocular lens 303, the onboard computer 304, the camera 305, the user input unit 306, the sensors 307, 308, and 309, the haptic module 310, the display 311, the optical output unit 312, and the microphone 313 are included in the HMD device 300. The speaker 314 and the power supply unit 315 can be in accordance with the lens 204, the onboard computer 205, the camera 206, the user input unit 207, the sensors 208, 209 and 210, the haptic module 211, the display 212, The optical output unit 213, the microphone 214, the speaker 216, and the power supply unit 217 (when located in the left arm 203) are positioned and configured in a similar manner.

The HMD device 100 in accordance with an embodiment of the present invention may be configured similar to the HMD device 400 shown in FIG. The HMD device 400 shown in Figure 4 is configured as a binocular display device. The HMD device 400 can be attached to a helmet structure or can have a similar eyeglass structure.

FIG. 4 illustrates an HMD device 400 in accordance with an embodiment of the present invention.

Referring to FIG. 4, the HMD device 400 can include a lens frame 401, a right arm 402 and a left arm 403, a right lens 404 and a left lens 405, an onboard computer 406, a camera 407, a user input unit 408, sensors 409, 410, and 411, haptic module 412, right side display 413 and left side display 414, microphone 415, right side speaker 416 and left side speaker 417, and power supply unit 418.

The lens frame 401 included in the HMD device 400 shown in Fig. 4 is integrated with the bridge 401'. The lens frame 401 can have any of a variety of shapes including a rectangular shape, an elliptical shape, and an elliptical shape. However, the shape of the lens frame 401 is not limited thereto. The lens frame 401 may be formed of a material similar to that constituting the lens frame 201 as shown in FIG. 2.

Right side lens 404 and left side lens 405 may comprise any material suitable for displaying material (including images or graphics projected by an optical output unit), the optical output unit being a projector as set forth above with reference to FIG. The right side lens 404 and the left side lens 405 may include materials that are capable of overlapping data (including images or graphics projected by the optical output unit according to the surrounding environment that the user can view through the right side lens 404 and the left side lens 405). The right side lens 404 and the left side lens 405 may be formed of a plastic material or glass, similar to the lens 204 described above with reference to FIG. However, materials for forming the right side lens 404 and the left side lens 405 are not limited thereto.

Although the right arm 402 and the left arm 403, the onboard computer 406, the camera 407, the user input unit 408, the sensors 409, 410, and 411, the haptic module 412, the microphone 415, and the right speaker 416 are shown in FIG. Some of the left side speaker 417 and the power supply unit 418 are disposed on the right side arm 202 and the left side arm 203, the onboard computer 205, the camera 206, the user input unit 207, the sensors 208, 209 and shown in FIG. 210, the haptic module 211, the microphone 214, the right speaker 215 and the left speaker 216, and the power supply unit 217 are at different positions, but the functions and configurations of the components set forth above in FIG. 4 may be similar to those in FIG. The functions and configurations of the components shown. However, the invention is not limited thereto.

The right side display 413 and the left side display 414 shown in FIG. 4 can display data including images or graphics projected by the optical output unit, and the optical output unit can be attached to the inner surfaces of the respective right arm 402 and left side arm 403.

The HMD device 400 can be transformed to operate the right lens 404 and the left lens 405 as display units. In this case, the right side lens 404 and the left side lens 405 can be configured as a transparent display unit or a translucent display unit. If the right lens 404 and the left lens 405 are configured as translucent display units, the right lens 404 and the left lens 405 may include at least one optical waveguide (eg, germanium), an electroluminescent display unit, or a liquid crystal display (Liquid Crystal Display) ;LCD) unit. However, the invention is not limited thereto.

FIG. 5 is a block diagram of an HMD device 500 in accordance with an embodiment of the present invention.

Referring to FIG. 5, an HMD device 500 according to an embodiment of the present invention may include an input/output unit 510, a memory 520, a sensor unit 525, a battery 530, a power management unit 535, a communication unit 540, and a touch sensor. 550, camera 560, display unit 570, lens 580, and control unit 590.

The input/output unit 510 includes means for receiving input from a user, providing information to a user, receiving data externally, and outputting data externally. The input/output unit 510 can include at least one of a speaker 511, at least one microphone 512, a button, a connector, and a keypad, or a combination thereof. However, the invention is not limited thereto.

The speaker 511 can output sound corresponding to various materials from the HMD device 500 under the control of the control unit 590. Further, the speaker 511 can output a sound corresponding to a function performed by the HMD device 500. The speaker 511 may be disposed at a suitable location of the HMD device 500 or the plurality of speakers 511 may be disposed at a suitable location of the HMD device 500. According to an embodiment of the invention, the speaker 511 can be configured as an earphone as shown in FIG.

The microphone 512 can receive a voice or sound outside the HMD device 500, generate an electrical signal based on the received voice or the received sound, and output the generated electrical signal to the control unit 590. The microphone 512 can be configured at a suitable location of the HMD device 500 or a plurality of microphones 512 can be disposed at a suitable location of the HMD device 500. Throughout the present invention, signals may also be referred to as data and data may also be referred to as data signals.

The sensor unit 525 can include at least one sensor for detecting the state of the HMD device 500 or the state of the surrounding environment. For example, the sensor unit 525 can include at least one of the following sensors: a proximity sensor for detecting a user approaching the HMD device 500, for detecting motion of the HMD device 500 (eg, Motion/direction sensor for rotation, acceleration, deceleration, vibration, etc., illumination sensor for detecting ambient illumination, photosensitive sensor for detecting color or spectrum of light, or a combination thereof. In addition, the motion/direction sensor may include a gravity sensor, a geomagnetic sensor, a gyroscope sensor, an impact sensor, a Global Positioning System (GPS) module, a compass, and an acceleration sensor. At least one of them. For example, the GPS module can receive radio signals from a plurality of GPS satellites orbiting the earth and can calculate the location of the HMD device 500 by using radio signals from the GPS satellites to the arrival time of the HMD device 500.

The power management unit 535 can supply power to the HMD device 500 under the control of the control unit 590. Power management unit 535 can be coupled to one or more batteries 530. Further, the power management unit 535 can supply power to the HMD device 500, which is input from an external power source via a cable.

The communication unit 540 can be a wired communication unit, a wireless communication unit, or a wired/wireless communication unit and can wirelessly transmit data from the control unit 590 to an external device via a wire or via an external communication network or over the air, via wires or via external communication. The network wirelessly receives data from the electronic device and transmits the received data to the control unit 590. According to an embodiment of the present invention, the communication unit 540 may include at least one of a mobile communication module, a local area network (LAN) module, and a short-range communication module.

The mobile communication module can perform communication with the external device via the mobile communication network by using at least one antenna under the control of the control unit 590. Mobile communication module can transmit wireless signals for voice calls, video calls, Short Message Service (SMS) or Multimedia Message Service (MMS) to mobile phones, smart phones, tablet PCs (Personal Computer; PC) or other communication device with a network address (such as an Internet Protocol (IP) address or phone number) or from a mobile phone, smart phone, tablet PC (Personal Computer) ; PC) or other communication device with a network address (such as an Internet Protocol (IP) address or phone number) for receiving voice calls, video calls, short message services (Short Message Service; SMS) ) or wireless signal of Multimedia Message Service (MMS).

The wireless LAN module can be connected to the Internet via a wireless access point AP around an access point (AP) under the control of the control unit 590. The wireless LAN module supports the IEEE 802.11x wireless LAN standard of the Institute of Electrical and Electronics Engineers (IEEE).

The short-range communication module can perform wireless short-range communication with the external communication device under the control of the control unit 590. The communication method may be a short-range Bluetooth ^TM, Infrared Data Association (Infrared Data Association; IrDA) standard, wireless fidelity (Wireless Fidelity; Wi-Fi) in direct communication, near field communication (Near Field Communication; NFC), or combinations thereof.

The touch sensor 550 can transmit a signal corresponding to the at least one touch input to the control unit 590. The user can touch the touch sensor 550 by using a body part (for example, a finger) or another touch input unit, and the touch sensor 550 can receive the touch by detecting the user's contact. Input. Further, the touch sensor 550 can receive an input (eg, a drag input) based on a continuous touch at the touch location. Touch input information can include touch coordinates and/or contact status. The contact state may be a state in which the body part contacts the touch sensor 550, a state in which the body part is separated from the touch sensor 550, or a body part is in contact with the touch sensor 550 when the body part contacts the touch sensor 550 The drag state of sliding on the touch sensor 550. The control unit 590 can determine user input information based on the touch input, for example, selecting or moving a menu item or item, handwriting input, and the like. The control unit 590 can perform functions corresponding to touch input (eg, establish a telephone call, photograph, select a menu, write/read a message, transfer a material, etc.).

However, in the present invention, the touch input is not limited to the input based on the contact between the touch sensor 550 and the touch input device. The touch input can include a contactless input (eg, the touch sensor 550 and the touch input device are separated from each other). This contactless input can also be referred to as a hovering input.

The touch sensor 550 can be embodied as a resistive type, a capacitive type, an infrared type, an acoustic wave type, an electromagnetic resonance (EMR) type, or a combination thereof.

Camera 560 can include a lens system and an image sensor and can further include a flash memory. The camera 560 can convert the input (or picked up) light into an electrical image signal via a lens system and output the converted image signal to the control unit 590. The user can obtain a moving picture or a still image by using the camera 560. In addition, camera 560 can also be used to receive input based on the user's motion or gesture.

The lens system can form an image of the object by concentrating light received externally. The lens system includes at least one or more lenses, wherein each of the lenses can be a convex lens, an aspheric lens, or the like. The lens system is symmetrical about an optical axis extending from the center of the lens system, wherein the optical axis can be defined as the central axis of the lens system. The image sensor can convert an optical image formed based on external light input through the lens system into an electrical image signal. The image sensor can include a plurality of pixel cells configured in an M x N matrix shape. Each of the pixel units may include a photodiode, and a plurality of transistors. The pixel unit accumulates the charge generated by the light input, and the voltage based on the accumulated charge may indicate the intensity of the brightness of the light input. In the case of processing a still image or an image constituting a moving picture, the image signal output by the image sensor is basically composed of a set of voltages (ie, pixel values) output by the pixel unit, and an image signal may indicate a Frame (ie, still image). In addition, the frame may include M x N pixels. The image sensor can be a Charge-Coupled Device (CCD) image sensor or a Complementary Metal-Oxide Semiconductor (CMOS) image sensor.

Based on the control signals received from control unit 590, the image sensor can operate all pixels of the image sensor or only pixels in the area of interest. The image data output by the pixels can be output to the control unit 590.

The control unit 590 can process images input from the camera 560, images stored in the memory 520, or images formed by the control unit 590 frame by frame using the data stored in the memory 520. The control unit 590 can externally output a frame of the image via the display unit 570 or store the frame of the image in the memory 520. In the present invention, the image output by the display unit 570 may be visual content including a moving picture or a still image or a Graphical User Interface (GUI).

Display unit 570 can be embodied in any of a variety of forms in accordance with embodiments of the present invention. For example, the display unit 570 can display an image by forming an image on the user's retina according to various optical system principles. Alternatively, display unit 570 can include a transparent display unit and display an image on the transparent display unit.

Lens 580 in accordance with an embodiment of the present invention can be configured to be attachable to and detachable from HMD device 500, as shown in FIG. In other words, the lens 580 may not be included in the HMD device 500, and a structure that may only be used to mount the lens 580 is included in the HMD device 500. The user can change the color of the lens 580 by switching the lens 580 of the HMD device 500 to a lens having a different color. Alternatively, the lens 580 according to an embodiment of the present invention may be a smart window that can modify color and transmittance under the control of the control unit 590.

FIG. 6 is a diagram of a method for displaying an image by the HMD device 600, in accordance with an embodiment of the present invention. The HMD device 600 according to an embodiment of the present invention may include a display unit 570 that includes an optical system. The display unit 570 according to an embodiment of the present invention can display an image by forming an image on the eye 610 of the user 1 by radiating light through the lens 580.

FIG. 7 is a diagram of a portion of an HMD device 500 in accordance with an embodiment of the present invention.

Referring to FIG. 7, the HMD device 500 can include a frame 720 for attaching the lens 580. According to an embodiment of the present invention, the frame 720 may have a structure in which the frame 720 is connected via a hinge 722. The opening 711 may be disposed on an inner surface of the frame 720 that is a path for externally outputting the projected light 710 output from the projector included in the display unit included in the frame 720 from the frame 720. In this case, a glass member or a transparent plastic member for preventing dust from being introduced into the frame 720 may be disposed at the opening 711.

Further, at least one speaker may be disposed on an inner surface of the frame 720.

The projector outputs the projected light 710 for use in forming a virtual image. The lens 580 focuses and reflects the projected light 710 output by the projector, and the focused and reflected projected light 730 can form a virtual image on the retina of the user's eye 610. In this case, focusing can refer to the collimation of light. Focusing can include the reduction of light as a convergence of light spots or a beam spot of light. The reflected projected light 730 can converge onto the aperture of the lens or eye 610. The HMD device 500 can use multiple projectors to form virtual images on the user's two eyes, respectively.

Figure 8 is a block diagram of a display unit in accordance with an embodiment of the present invention. In detail, FIG. 8 is a block diagram showing the structure of a projector in a state where the display unit includes a projector.

Referring to FIG. 8, if the display unit 570 includes a projector, the display unit 570 includes a light source 810, an illumination optical system 820 for illuminating the display device 840 by using light output by the light source 810, for reflection transmission through the illumination optical system. A mirror 830 of light of 820, a display device 840 for forming a virtual image by reflecting light reflected by the mirror surface 830 pixel by pixel, and a projection optical system 850 for externally projecting light reflected by the display device 840.

Illumination optics 820 can be configured along a first optical axis 571 that is parallel to the x-axis. Illumination optics 820 can include at least one collimating lens, at least one filter, at least one equalizing lens, a focusing lens, or a combination thereof.

Optical elements (eg, lenses, cymbals, filters, etc.) of illumination optics 820 can be aligned along first optical axis 571. In general, the optical axis can refer to an axis that satisfies the following situation: when the optical system is rotated about it, along the axis, no optical changes occur in the optical system. When the optical elements are aligned along the optical axis, the center of curvature of the optical elements that make up the optical system is on the optical axis, or the center of symmetry of the optical elements is on the optical axis.

Light source 810 can output light traveling along first optical axis 571. For example, at least one Light Emitting Diode (LED) that outputs white light, primary color light (ie, blue, red, and yellow) or a combination of primary colors (eg, purple, green, orange, etc.) may be used. Used as the light source 810.

Illumination optics 820 can collimate, filter, and/or focus the light input from light source 810 and output the processed light to mirror 830.

The mirror 830 can reflect the light input via the illumination optical system 820 toward the display device 840. The mirror 830 can have a structure in which a highly reflective dielectric layer or a highly reflective metal layer is deposited on the substrate.

The display device 840 can display an image pixel by pixel based on the material input by the control unit 590. Display device 840 can include pixel elements corresponding to a predetermined resolution. The display device 840 displays an image by "turning on" and "turning off" the pixel elements. For example, a Digital Micro-mirror Device (DMD) including a micromirror configured in an M×N matrix shape can be used as the display device 840. Each of the micromirrors is rotatable according to the drive signal to a position corresponding to the "on" state and a position corresponding to the "off" state. Each of the micromirrors can reflect the incident light at an angle displayed outside the display device 840 in the "on" state, and can reflect the incident light at an angle that is not displayed outside the display device 840 in the "off" state.

Projection optics 850 can be configured along a second optical axis 572 that is parallel to the y-axis. Projection optics 850 can include relay lens 860 and projection lens 870. Relay lens 860 and projection lens 870 can be aligned on second optical axis 572.

The relay lens 860 can align light reflected by the mirror 830 toward the display device 840 in consideration of light overflow. In other words, the relay lens 860 can allow light reflected by the mirror surface 830 to be incident on a region larger than a region occupied by the pixel elements of the display device 840.

In addition, relay lens 860 can receive light reflected by display device 840, reduce the beam spot size of the light, and output the processed light. Since the light reflected by the display device 840 has a large beam spot size, light may not be transmitted to the projection lens 870 due to light loss. The relay lens 860 focuses the light reflected by the display device 840 and reduces its beam spot size, and thus, more light can be transmitted to the projection lens 870.

Projection lens 870 can receive light transmitted through relay lens 860, collimate or focus the received light, and project the processed light externally.

9 is a diagram of a lens capable of changing color or transmittance, in accordance with an embodiment of the present invention.

Referring to Figure 9, a lens 580 in accordance with an embodiment of the present invention can change its color or transmittance. The lens 580 according to an embodiment of the present invention may include a glass 581 capable of changing its transmittance (that is, modifying the color or transmittance of the glass 581 based on a signal applied to the glass 581), and a holographic optical element serving as a concave mirror (Holographic) Optical Element; HOE) 582.

The lens 580 can transmit the ambient light 911 that is externally input through it and can reflect and focus the light 921 output by the display unit 570.

The transmitted ambient light 911 and the light 921 reflected by the lens 580 are input to the user's eye 610, and the ambient image 910 based on the transmitted ambient light 912 and the displayed image 920 (based on the reflected light) are overlapped and formed. The retina 613 of the eye 610. In other words, the user sees an image formed when the surrounding environment image 910 and the displayed image 920 overlap each other. The user can recognize that the displayed image 920 is floating on the transparent layer on the ambient image 910 as if it were displayed. The displayed image 920 may include a static GUI, the static GUI includes a menu and a graphic corresponding to a phone call, contact information, a message, a main menu, or the displayed image 920 may include a content image, and the content image includes a still image. Or move the picture. Although the area in which the displayed image 920 is displayed is shown as a non-transparent area in FIG. 9, the area in which the displayed image 920 is displayed may be transparent to allow the user to view the surrounding environment in the area in which the displayed image 920 is displayed. Image 910. In other words, the transparency of the area in which the displayed image 920 is displayed may vary according to embodiments of the present invention.

The light 921 output by the display unit 570 may be parallel light having a wavelength λ (that is, collimated light). Light 921 can be incident on HOE 582 to form an angle θ with the normal to HOE 582. HOE 582 can be a wavelength selective element. HOE 582 can reflect and focus light having a wavelength λ (i.e., light 921 according to the present invention) and can transmit light having a wavelength different from wavelength λ (i.e., ambient light 911 according to the present invention) through it Without gathering the light.

The HOE 582 can reflect and focus the input light 921. Additionally, reflected light 922 from HOE 582 can converge at a location corresponding to eye 610 that is spaced a distance from HOE 582 (ie, eye gap). The reflected light 922 can converge at a location corresponding to the aperture 611 or the lens 612 of the eye 610. In this case, the concentrated light 922 may have a convergence angle or a viewing angle φ. The crystal 612 controls the focus of the light incident on the eye 610, and since the transmitted ambient light 912 converges at a location corresponding to the pupil 611 or the crystal 612, the transmitted ambient light 912 can be projected onto the retina 613 instead of Converged by the crystal 612 and an image 920 is formed.

Although the light 921 incident to the HOE 582 is described above as parallel light, the present invention is not limited thereto. If the reflected light 922 from the HOE 582 converges at a position corresponding to the aperture 611 or the crystal 612, the light 921 may not be parallel light but divergent or concentrated.

If the user wearing the HMD device deflects his or her eyes to view the actual object in its surroundings, the image (virtual object) displayed by the HMD device may not be clearly formed on the user's retina. Under this circumstance, the user cannot clearly view the actual object and the virtual object at the same time, and therefore, the augmented reality cannot be implemented. Furthermore, the inconsistency in focus between the actual object and the displayed image can increase user fatigue (due to inconsistencies between eye convergence and monocular focus adjustment).

An HMD device according to an embodiment of the present invention can condense light for forming a projection of a virtual image at a position of a pupil, a crystal, or near a pupil, a crystal, thereby reducing a focus change of light based on projection of the crystal. If the projected light can be directly projected onto the retina regardless of the action of the crystal, the image projected onto the retina can be recognized by the user as a clear image regardless of the focus adjustment and aberration of the crystal.

The HOE 582 can have a focal length that corresponds to the distance between the aperture 611 or the crystal 612 and the HOE 582.

Figure 10 is a diagram of a glass 581 included in a lens of an HMD device in accordance with an embodiment of the present invention. The transmittance or color of the glass 581 according to an embodiment of the present invention may be modified based on a signal or voltage applied thereto by the control unit 590.

An electrochromic glass, a suspended particle device (SPD) or a liquid crystal (LC) can be used as the glass 581. Alternatively, if necessary, a photochromic glass or a thermochromic glass can be used as the glass 581, and the transmittance of the photochromic glass or the thermochromic glass is passively modified based on the wavelength or temperature change of the light, rather than via a signal. Apply active control.

Various methods can be used to make the glass 581. For example, the glass 581 can be fabricated by applying a material having a controlled transmittance to the glass or attaching a film having a controlled transmittance to the glass.

The case where electrochromic glass is used as the glass 581 will be described below with reference to FIG.

The glass 581 includes: a first insulating substrate and second insulating substrates 1010 and 1015; a first conductive electrode 1020 deposited on a top surface of the first insulating substrate 1010; and a second conductive electrode 1025 deposited on a bottom of the second insulating substrate 1015 On the surface; the insulating spacer 1030 separates the first insulating substrate 1010 from the second insulating substrate 1015 and seals a space therebetween; and the electrochromic layer 1040 and the electrolyte 1050, the electrolyte fills the first insulating substrate 1010 and the second A space between the insulating substrates 1015.

The first insulating substrate 1010 and the second insulating substrate 1015 may be formed of transparent glass or transparent plastic. For example, the transparent plastic may comprise at least one of polyacrylate, polyvinyl ether phthalate, polyethylene naphthalate, polycarbonate, polyarylate, polyether oxime Imine, polyether oxime and polyimine.

The first conductive electrode 1020 may be formed of a transparent conductor. For example, the first conductive electrode 1020 may contain an inorganic conductive material such as Indium Tin Oxide (ITO), Fluorine-doped Tin Oxide (FTO), or antimony-doped antimony-doped Tin Oxide; ATO), or an organic conductive material such as polyacetylene or polythiophene.

The second conductive electrode 1025 may be formed of a transparent or non-transparent conductive material. For example, the second electrode 1025 can contain ITO, FTO, a metal similar to aluminum (Al), ATO, or a combination thereof.

An electrochromic layer 1040 containing an electrochromic material can be disposed on the first conductive electrode 1020. The electrochromic layer 1040 may be disposed on the upper first electrode 1020 in the form of a film, but is not limited thereto.

The first insulating substrate 1010 and the second insulating substrate 1015 may be fixed to the insulating spacer 1030. The space between the first insulating substrate 1010 and the second insulating substrate 1015 may be filled with the electrolyte 1050. The electrolyte 1050 can comprise an oxidizing/reducing material that reacts with the electrochromic material. The electrolyte 1050 can be a liquid electrolyte or a solid electrolyte. With a lithium salt (such as lithium hydroxide (LiOH) or lithium perchlorate LiClO _4), potassium salt (such as potassium hydroxide (of KOH)) or a salt (such as, sodium hydroxide (of NaOH)) was formed Can be used as a liquid electrolyte. Meanwhile, for example, (poly(2-acrylamido-2-methylpropanesulfonic acid) or poly(ethylene) oxide (PEO)) can be used as the solid electrolyte.

The material constituting the electrochromic layer 1040 (i.e., the electrochromic material) may contain a metal organic composite which is a combination of a metal and an organic compound (including a functional group capable of coordinating with a metal). The metal may comprise a light metal, a transition metal, a lanthanide metal, an alkali metal, or a combination thereof. For example, the metal may comprise beryllium (Be), barium (Ba), copper (Cu), zinc (Zn), cerium (Ce), magnesium (Mg), aluminum (Al), titanium (Ti), or a combination thereof . Further, for example, the functional group may include a carboxyl group, a pyridyl group, an imidazolyl group, or a combination thereof. Further, for example, the organic compound may contain a viologen derivative, an anthracene derivative, or a combination thereof.

11A through 11C are diagrams of a method for displaying an image by an HMD device, in accordance with an embodiment of the present invention.

Referring to Figure 11A, display unit 570 can be disposed between lens 580 and the user's eyes. Display unit 570 can include a transparent display that is capable of transmitting light therethrough.

The ambient light 1110-1 can be transmitted through the lens 580 and the display unit 570 and incident on the user's eyes, and the light 1120-1 generated by the display unit 570 can be directly incident on the user's eyes without being transmitted through Lens 580.

Alternatively, referring to FIG. 11B, lens 580 can be disposed between display unit 570 and the user's eyes. In this case, the ambient light 1110-2 can be transmitted through the lens 580 and the display unit 570 and incident on the user's eyes. Light 1120-2 produced by display unit 570 can be transmitted through lens 580 and incident on the user's eyes. In this case, since both the ambient light 1110-2 and the light 1120-2 generated by the display unit 570 are transmitted through the lens 580 and incident on the user's eyes, the HMD device can be considered by the user. The image is modified by the color produced by display unit 570 and transmitted through light 1120-2 of lens 580.

Alternatively, referring to FIG. 11C, the display unit 570 and the lens 580 may be integrated with each other as the display unit/lens 570-1. In this case, the color or transmittance of the display unit/lens 570-1 can be controlled and the display unit/lens 570-1 can be configured as a smart window capable of displaying images. Ambient light 1110-3 can be transmitted through display unit/lens 570-1 and incident on the user's eyes. Further, the light 1120-3 generated by the display unit/lens 570-1 can be directly incident on the eyes of the user.

12 is a flow chart of a method for an HMD device to display an image, in accordance with an embodiment of the present invention.

Referring to Fig. 12, in step S1210, the HMD device can obtain color information about the lens. The color information about the lens may refer to identification information for the HMD device to recognize the color of the lens. For example, the color information about the lens can include a Red-Green-Blue (RGB) value that indicates the color of the lens. Color information may vary in accordance with embodiments of the present invention. For example, the color information can be information indicating the color of the lens. Alternatively, according to an embodiment of the invention, the color information may indicate the color of the color identified as the background around the image displayed by the HMD device. In this case, the color of the light transmitted through the lens is detected by the photosensitive sensor disposed at the HMD device, and thus, the HMD device can obtain color information indicating the color recognized as the background color. In this case, the color information indicating the color recognized as the background color may also be referred to as the color information about the image display area.

The HMD device can use various methods to obtain color information about the lens. For example, if the HMD device is interchangeable for the lens, the HMD device can obtain the color of the replacement lens when the lens is interchanged. Further, according to an embodiment of the present invention, if the lens is a smart window whose transmittance or color can be modified under the control of the control unit of the HMD device, the HMD device can control the color of the lens, and thus, the current lens can be obtained. color.

Next, in step S1220, the HMD device may modify the image based on the color information obtained in step S1210 and display the modified image via the display unit. For example, when the HMD device displays an image containing text and the obtained color information about the lens indicates yellow, the HMD device can modify the color of the text to blue (blue is a complementary color of yellow), and via the display unit Press the blue to display the modified text.

If the lens is attachable to and detachable from the HMD device or the HMD device is capable of controlling the color of the lens, the HMD device may obtain color information about the lens in step S1210. However, if the lens is integrated with the HMD device and the color of the lens cannot be changed, the color information about the lens can be preset. In other words, if the fixed color lens is integrated with the HMD device, the HMD device can correct the color of the image based on the preset color information about the lens. For example, if the blue lens is fixed to the HMD device, the HMD device can change the color of the image to red (red to blue complementary color) without obtaining color information about the lens and displaying the modified image.

Figure 13 is a block diagram for modifying and displaying an image in accordance with an embodiment of the present invention.

Referring to FIG. 13, an HMD device in accordance with an embodiment of the present invention may include a color correction module 1330. The color correction module 1330 can display the modified image 1340. The modified image 1340 is generated by modifying the color of the original image 1320 based on the color information 1310 about the lens and the original image 1320. In this case, the color correction module 1330 according to an embodiment of the present invention may use the control unit 590. For example, color correction module 1330 can include a processor loaded with a program for generating modified image 1340.

Color correction module 1330 can modify original image 1320 using various methods in accordance with embodiments of the present invention. For example, if the original image 1320 contains text, the color correction module 1330 can modify the original image 1320 to display the text in a complementary color of the color of the lens.

However, the modification of the image by the color correction module 1330 is not limited to modification of the color of the original image 1320. For example, the color correction module 1330 can increase or decrease the size of an object (eg, a GUI object like the illustration) contained in the original image 1320. As another example, color correction module 1330 can change the shape of an object (eg, a GUI article like the illustration) contained in original image 1320.

If the lens is integrated with the HMD device and the HMD device is unable to change the color of the lens, the color information 1310 may not be separately input to the color correction module 1330, but the color information may be preset. The color correction module 1330 can modify the image based on the preset color information. In other words, if the lens with fixed color is integrated with the HMD device, the color correction module 1330 can modify the image based on the preset color information about the lens. For example, if the blue lens is fixed to the HMD device, the color correction module 1330 can change the color of the image to red (red to blue complementary color) without obtaining color information about the lens, and displaying the modified image.

The display unit 570 can display the modified image 1340 output by the color correction module 1330.

The HMD device can obtain color information about the lens using various methods in accordance with embodiments of the present invention. For example, the HMD device can obtain color information about the lens by using a metal wire included in the lens, a code printed on the lens, a photosensitive sensor, or an image picked up by a camera. In this case, obtaining color information about the lens may include two situations in which the color information about the lens is input by the user, or wherein the HMD device is capable of recognizing the lens when the HMD device controls the color or transmittance of the lens. Color information.

14 is a flow chart of a method for obtaining color information about a lens, in accordance with an embodiment of the present invention.

Referring to Fig. 14, in step S1410, the HMD device can obtain identification information about the lens. In this case, the identification information about the lens refers to information for identifying the lens by the HMD device. For example, the identification information about the lens may be a code assigned based on the type of the lens. The method for obtaining identification information about the lens for the HMD device may vary according to embodiments of the present invention. For example, if the metal line is disposed at the lens, the identification information about the lens can be obtained based on the position of the metal line. As another example, if a code (eg, a barcode or a Quick Response (QR) code) is printed on the surface of the lens, the HMD device can obtain identification information about the lens based on the code printed on the lens.

In this case, the obtained identification information about the lens may be information based on the color distribution of the lens. For example, the identification information about the black lens may be "01" and the identification information about the transparent (eg, colorless) lens may be "00". In this case, the HMD device can obtain color information about the lens corresponding to the obtained identification information.

Next, the HMD device can modify the image to be displayed based on the obtained color information. The HMD device can modify the image using various methods in accordance with embodiments of the present invention. For example, the HMD device can modify the image such that the text contained in the image is displayed in a complementary color of the color of the lens.

However, the modification of the image by the HMD device is not limited to the modification of the color of the image. For example, an HMD device can increase or decrease the size of an object (eg, a GUI item like the illustration) contained in an image. By way of further example, an HMD device can change the shape of an object (eg, a GUI article like the illustration) contained in an image.

Next, in step S1440, the HMD device can display the modified image.

15 is a diagram of an HMD device 1500 for using a wire identification lens in accordance with an embodiment of the present invention.

Referring to FIG. 15, an HMD device 1500 in accordance with an embodiment of the present invention can include a display unit 1511 and an electrode 1532 disposed at a portion of the frame 1510. Lens 1520 can be attached to and detached from HMD device 1500 in accordance with embodiments of the present invention. In this case, the detachable lens 1520 can include a metal wire 1531. In this case, the electrode 1532 and the metal line 1531 may be disposed at positions where the electrode 1532 and the metal line 1531 may contact each other when the lens 1520 is attached to the frame 1510. When the lens 1520 is attached to the frame 1510, the HMD device 1500 can detect the position of the metal line 1531 by using the electrode 1532. The HMD device 1500 can obtain identification information about the lens 1520. According to an embodiment of the invention, the magnet 1540 can be disposed at a portion of the frame 1510 such that the lens 1520 can be secured to the frame 1510.

16 is a diagram of a portion of an HMD device 1500 including metal lines, electrodes, and lenses, in accordance with an embodiment of the present invention.

Referring to Figure 16, the detachable lens 1520 can include a metal wire 1531. In this case, the metal line 1531 may be disposed at a certain position based on the type of the lens 1520. When the lens 1520 is attached to the frame 1510, the metal line 1531 can contact the electrode 1532.

When the metal line 1531 contacts the electrode 1532, the HMD device 1500 can detect the position where the metal line 1531 is disposed. The HMD device 1500 can obtain identification information about the lens 1520 corresponding to the position where the metal wire 1531 is disposed.

17 is a diagram of a method of identifying a lens by using a code printed on a lens, in accordance with an embodiment of the present invention.

Referring to FIG. 17, the HMD device 1700 can obtain identification information about the lens 1720 based on the code 1721 included in the lens 1720. Although FIG. 17 shows that the code 1721 is printed on the front surface of the lens 1720, the position of the code 1721 can vary depending on the embodiment of the present invention.

Code 1721 may refer to a symbol that is provided for optical reading by HMD device 1700, in accordance with an embodiment of the present invention. For example, the code 1721 can be a QR code or a barcode. In this case, the HMD device 1700 can pick up an image in which the position of the code 1721 is arranged by using the camera 1710 and obtain identification information about the lens 1720 based on the picked up image.

18 is a flow chart of a method of obtaining color information about a lens by using a photosensitive sensor, in accordance with an embodiment of the present invention.

Referring to FIG. 18, an HMD device according to an embodiment of the present invention may include a photosensitive sensor. A photosensitive sensor is a sensor that can detect light by responding to radiant energy of similar light. In step S1810, the HMD device can detect the light transmitted through the lens and analyze the detected light via the photosensitive sensor.

Next, in step S1820, the HMD device may determine color information about the lens based on the result of the analysis. For example, the HMD device can analyze the spectrum of light detected by the photosensitive sensor. If the percentage of blue light in the detected light is high, the HMD device can obtain color information corresponding to blue as color information about the lens.

Once the color information about the lens is obtained, the HMD device can modify the image based on the obtained color information in step S1830 and display the modified image in step S1840.

19A and 19B are diagrams showing a method of obtaining color information about a lens by using a photosensitive sensor according to an embodiment of the present invention.

Referring to FIGS. 19A and 19B, an HMD device according to an embodiment of the present invention may include a photosensitive sensor 1930 at a position that can receive light emitted from a light source 1910 disposed outside the lens 1920 of the HMD device. For example, referring to Figure 19B, photosensitive sensor 1930 can be disposed at a portion of the frame of the HMD device. As a result of analyzing the light transmitted through the lens 1920, the HMD device according to an embodiment of the present invention can obtain color information about the lens 1920.

20 is a flow chart of a method of obtaining color information about a lens based on the size of a user's pupil, in accordance with another embodiment of the present invention.

Referring to Fig. 20, in step S2010, the HMD device can pick up an image by using a camera. In this case, the picked up image may contain an image of the user's eyes.

The HMD device can perform image recognition with respect to the picked up image. As a result of the image recognition, the HMD device can obtain a value regarding the size of the pupil included in the image. In step S2020, a change in the size of the pupil included in the image may be determined based on the obtained value regarding the size of the pupil.

Next, in step S2030, the HMD device may determine color information about the lens based on the change in the pupil. If the transmittance of the lens is high, a large amount of light can be incident on the eyes of the user, and therefore, the size of the pupil of the user may be reduced. On the contrary, if the transmittance of the lens is low, a small amount of light can be incident on the eyes of the user, and thus the size of the pupil of the user may increase. Based on the mechanism, the HMD device can obtain color information about the lens based on the size of the user's pupil. In other words, if the size of the user's pupil is large, the HMD device can determine that the lens has a bright color. Conversely, if the size of the user's pupil is small, the HMD device can determine that the lens has a dark color.

Once the color information about the lens is obtained, the HMD device can modify the image based on the obtained color information in step S2040 and display the modified image in step S2050.

21 is a diagram of a method of obtaining color information based on the size of a user's pupil, in accordance with an embodiment of the present invention.

Referring to FIG. 21, the HMD device 2100 can pick up an image of the user's eye 2120 by using a camera 2110 disposed at a portion of the HMD device 2100. Although FIG. 21 shows that the lens is disposed between the camera 2110 and the user, the position of the camera 2110 can vary depending on the embodiment of the present invention. The HMD device 2100 can detect the size of the pupil from the image of the eye 2120. The HMD device 2100 can obtain color information about the detected size of the pupil.

22 is a diagram of a method of obtaining color information about a lens based on a user's settings, in accordance with an embodiment of the present invention.

According to an embodiment of the invention, the user can set color information about the lens 2220. Referring to FIG. 22, HMD device 2200 can display a user interface 2210 for setting color information about lens 2220. For example, user interface 2210 can include a list of color information.

The HMD device 2200 can receive an input from the user for selecting a color information from among the color information about the lens 2220 via the input/output unit. For example, referring to FIG. 22, if brown 2211 is selected via user interface 2210, the HMD device can obtain RGB values corresponding to brown as color information about lens 2220.

23 is a flow chart of a method for displaying images for an HMD device in accordance with an embodiment of the present invention. In addition, FIG. 24 is a diagram of a method for displaying an image by an HMD device according to an embodiment of the present invention.

According to an embodiment of the invention, the HMD device may be a passive device that is incapable of processing images. In this case, the HMD device can modify the image by using an active device capable of communicating with the HMD device.

Referring to Figures 23 and 24, in step S2310, the HMD device 2300 can obtain identification information about the lens. Next, in step 2320, the HMD device 2300 may transmit the obtained identification information about the lens to the control terminal 2400, which is the active device.

Next, in step 2330, the control terminal 2400 may modify or generate an image to be displayed by the HMD device 2300 based on the identification information received from the HMD device 2300.

Next, in step 2340, the control terminal 2400 can transmit the modified or generated image to the HMD device 2300.

Next, in step 2350, the HMD device 2300 can display the modified or generated image.

25 is a flow chart of a method for recommending a lens for an HMD device in accordance with an embodiment of the present invention.

Referring to Fig. 25, in step S2510, the HMD device can obtain information about the external environment. The information about the external environment may be information indicating conditions near the HMD device detected by the at least one sensor. For example, information about the external environment may include information about the brightness near the HMD device, GPS coordinates about the location of the HMD device, or information about the sound detected by the HMD device.

Next, in step S2520, the HMD device may determine color information corresponding to the information about the external environment obtained in step S2510. According to an embodiment of the present invention, the color information about the external environment may be original information about the external environment, such as information about the brightness near the HMD device. For example, the HMD device can pick up an image of the external environment of the HMD device by using the camera 560. The HMD device can obtain information about the external environment based on the picked up image. In this case, step S2520 can be omitted. Further, according to an embodiment of the present invention, the color information about the external environment may be a color code selected based on color information contained in a color database (DataBase; DB) included in or outside the HMD device based on: Information about the brightness near the HMD device, GPS coordinates about the location of the HMD device, or information about the sound detected by the HMD device. In step S2530, the HMD device may recommend a lens corresponding to the determined color information about the external environment. For example, the expression "HMD device recommendable lens" instructs the HMD device to display a message indicating that the user has changed the lens. For example, if the brightness obtained in step S2510 is low, the HMD device can recommend a colorless lens. In contrast, if the brightness obtained in step S2510 is high, the HMD device may recommend a color lens such as a brown lens, a blue lens, a black lens, or the like.

FIG. 27 is a diagram of a method for recommending a lens for the HMD device 2700.

Referring to FIG. 27, the HMD device 2700 can display a message 2710 for recommending a lens based on information selection regarding the external environment. In this situation, HMD device 2700 can display message 2710 to cause the user to recognize that message 2710 is displayed on lens 2720.

According to an embodiment of the present invention, if the HMD device 2700 can control the color or transmittance of the lens 2720, the step S2530 can be replaced with the following step: wherein the HMD device 2700 controls the lens 2720 based on the color information about the external environment determined in step S2520. Color or transmittance.

26 is a flow chart of a method for recommending a lens for an HMD device in accordance with an embodiment of the present invention. In addition, FIGS. 28 to 31 are diagrams of a method for recommending a lens for an HMD device according to an embodiment of the present invention.

In step S2610, the HMD device selects the content to be displayed via the display unit. For example, the HMD device can display a list of content and select content from a list of content based on user input received via an input/output unit of the HMD device. In this case, the content list can include an executable application or multimedia content containing pictures or moving pictures.

For example, referring to FIG. 28, HMD device 2700 can display a list of contents 2810 on lens 2720. The HMD device 2700 can select a content from among the content contained in the content list 2810. As shown in FIG. 28, HMD device 2700 can select web browser 2811 from content list 2810.

Next, in step S2620 in Fig. 26, the HMD device 2700 can recommend a lens corresponding to the content selected in step S2610. For example, referring to FIG. 29, if web browser 2811 is selected, HMD device 2700 can display a message 2920 on lens 2820 for recommending a lens having a color suitable for use with web browser 2811. Additionally, HMD device 2700 can display a web browser that executes screen image 2910.

Next, in step S2630 of Fig. 26, it may be determined whether or not the recommended lens is attached to the HMD device. In step S2630, the HMD device can determine whether the recommended lens is attached to the HMD device by using various methods. For example, the HMD device can obtain identification information about the lens and determine whether the obtained identification information is identical to the identification information of the recommended lens. As another example, the HMD device can obtain color information about the lens and determine whether the obtained color information is the same as the color information of the recommended lens. As shown in FIG. 30, if the lens 2820 is replaced with a lens 2821 having a different color, the HMD device can determine whether the recommended lens is attached to the HMD device. If the recommended lens is not attached to the HMD device, the HMD device may perform step S2620 of FIG. 26 again.

In accordance with an embodiment of the present invention, if the HMD device is capable of controlling the color or transmittance of the lens 2820, step S2620 and step S2630 may be replaced with the following steps: controlling the lens 2820 to change the color or transmittance of the lens 2820 to display selected content. Recommended color or transmittance.

Next, in step S2640, the HMD device can display the selected content on the lens. The content displayed in step S2640 may be an image modified based on the color information about the lens modified in step S2620 and step S2630. As shown in FIG. 31, the HMD device 2700 can display a web browser for displaying a screen image 2911 having a modified color on the lens 2821 in a modified color.

32 is a flow chart of a method of an HMD device capable of changing the color or transmittance of a lens displaying an image, in accordance with an embodiment of the present invention.

In step S3110, the HMD device according to an embodiment of the present invention can control the color or transmittance of the lens. For example, an HMD device can measure brightness and control the transmittance of a lens by using a brightness sensor. For example, the brightness sensor can include at least one of: an illumination sensor for detecting ambient illumination, a photosensitive sensor for detecting color or spectrum of light, or a combination thereof. However, the invention is not limited thereto. As another example, the HMD device can measure the brightness of the surrounding environment by using a camera and measure the brightness based on the picked up image. Alternatively, the brightness in the vicinity of the HMD device can be measured by using information indicating the position of the HMD device. If the measured brightness is low, the transmittance of the lens can be increased. Conversely, if the measured brightness is higher, the transmittance of the lens can be reduced to protect the user's eyes.

When the lens is formed from an electrochromic glass, a liquid crystal (LC) panel, or a suspended particle device (SPD), the HMD device can control the color or transmittance of the lens. For example, the lens can comprise an electrochromic glass comprising an electrochromic material (EC) thin film transparent electrode, ITO glass, and an electrolyte. If the lens is formed of electrochromic glass, the HMD device can control the color or transmittance of the lens by controlling the magnitude of the voltage applied to the electrochromic glass.

Next, in step S3120, the HMD device may modify the image based on the color or transmittance of the lens. In step S3130, the HMD device can display the modified image. For example, if the HMD device controls the lens to change the color of the lens to blue, the HMD device can display the text in red and the red in blue as a complementary color.

33 is a chart of modified colors that can be used to modify an image, in accordance with an embodiment of the present invention.

An HMD device according to an embodiment of the present invention can modify an image by using a complementary color chart. For example, if the color information obtained in step S1210 of FIG. 12 is FFFF00 (for example, yellow), the HMD device can display the text in blue, and the blue is a complementary color of yellow. However, the invention is not limited thereto, and the method of modifying the image may vary according to an embodiment of the invention.

In this case, the color information may vary according to an embodiment of the present invention. For example, according to an embodiment of the invention, the color information may be information indicating the color of the lens. Alternatively, according to an embodiment of the invention, the color information may indicate the color of the color identified as the background around the image displayed by the HMD device. In this case, the color of the light transmitted through the lens is detected by the photosensitive sensor disposed at the HMD device, and thus, the HMD device can obtain color information indicating the color recognized as the background color. According to an embodiment of the present invention, the color information may be information determined based on information about the external environment.

34 and 35 are diagrams of an HMD device that displays a corrected image, in accordance with an embodiment of the present invention.

Referring to Fig. 34, when a user wearing the HMD device 3300 views the indoor landscape, an image 3310 is displayed. The HMD device 3300 can measure the brightness of the circumference. If the HMD device 3300 is in an indoor position and the measured brightness is low, the HMD device 3300 can increase the transmittance of the lens 3320 or can control the lens 3320 to be colorless. The user of the HMD device 3300 can recognize that the image 3310 is displayed on the lens 3320.

Referring to Fig. 35, when a user wearing the HMD device 3300 views the external landscape 3401, an image 3311 is displayed. The HMD device 3300 can control the color or transmittance of the lens 3321 as the measured brightness increases. When the color or transmittance of the lens 3321 is modified, the HMD device 3300 can modify the image 3311 based on the color information corresponding to the modified color or transmittance of the lens 3321. As shown in FIG. 35, HMD device 3300 can display image 3311 with a modified color or modified graphical size.

36 and 37 are diagrams of an HMD device that displays a modified image, in accordance with an embodiment of the present invention.

Referring to Figure 36, the HMD device 3500 can display an image 3510 on the lens 3520 for providing augmented reality functionality. For example, an image 3510 for providing augmented reality functionality may contain information about the actual object that the user is viewing. As shown in FIG. 37, if the color or transmittance of lens 3520 is modified, HMD device 3500 can display modified image 3511. The modified image 3511 can be not only an image with a modified color, but also an image containing the modified object, as shown in FIG.

Figure 38 is a block diagram of an HMD device in accordance with an embodiment of the present invention.

The HMD device may include a color information obtaining unit 3710, a control unit 3720, and a display unit 3730.

The color information obtaining unit 3710 according to an embodiment of the present invention can obtain color information about a lens. The color information obtaining unit 3710 can be configured in various manners according to an embodiment of the present invention. For example, as shown in FIG. 15, color information obtaining unit 3710 can include an electrode 1532. Alternatively, the color information obtaining unit 3710 may include an identification unit that recognizes the code by using the camera 1710, as shown in FIG. 17, or may obtain color information about the lens by using the photosensitive sensor 1930, such as Shown in Figure 19. Alternatively, if the control unit 3720 of FIG. 38 controls the color or transmittance of the lens, the color information obtaining unit 3710 may be included in the control unit 3720.

Further, if the lens is attachable to and detachable from the HMD device, the color information obtaining unit 3710 may determine whether the lens is attached to the HMD device. When it is recognized that the lens is attached to the HMD device, the color information obtaining unit 3710 can obtain color information about the attached lens.

Control unit 3720 can process information and control components of the HMD device. The control unit 3720 may modify the image based on the color information obtained by the color information obtaining unit 3710. The control unit 3720 can control the display unit 3730 to display the modified image.

According to an embodiment of the invention, the control unit 3720 can control at least one of the color and transmittance of the lens. In this case, the lens may comprise an electrochromic glass, SPD or LC. For example, the lens can comprise glass having the structure as shown in FIG. If the control unit 3720 of FIG. 38 controls the color or transmittance of the lens, the control unit 3720 can obtain color information about the lens corresponding to the modified color or transmittance. Further, the control unit 3720 can modify the image based on the obtained color information about the lens.

According to an embodiment of the present invention, the control unit 3720 can select the content that can be output via the display unit 3730. For example, the HMD device can display a list of content and select content from a list of content based on user input received via an input/output unit of the HMD device. In this case, the content list can include an executable application or multimedia content containing pictures or moving pictures.

Further, the HMD device according to an embodiment of the present invention may further include a brightness sensor for obtaining information on external brightness. The control unit 3720 may further modify the image in consideration of information about the external brightness obtained via the brightness sensor. For example, when the external brightness is increased, the control unit 3720 can increase the brightness of the image.

The HMD device according to an embodiment of the present invention may further modify the image in consideration of the color of the background (eg, the background of the image) that the user is viewing.

FIG. 39 is a flow diagram of an HMD device 4000 for displaying images in accordance with an embodiment of the present invention. In addition, FIGS. 40-43 are diagrams of an HMD device 4000 displaying a corrected image 4125, in accordance with an embodiment of the present invention.

First, in step S3910, the HMD device 4000 can pick up the background image 4020. The HMD device 4000 can include a camera 4010 for picking up images in the viewing direction of the user 1. The HMD device 4000 can pick up the background image 4020 by using the camera 4010.

Referring to FIG. 40, HMD device 4000 can include a camera 4010 at a portion of HMD device 4000. If the user 1 wearing the HMD device 4000 is viewing the pedestrian crossing, the camera 4010 can obtain a background image 4020 that captures the pedestrian crossing.

Next, in step S3920, the HMD device 4000 may determine the image display region 4120 in the background image 4020 regarding the position of the image 4110 to be displayed by the HMD device 4000. When the HMD device 4000 displays the image 4110, the image display area 4120 may be part of the background image 4020, which is recognized by the user as the area in which the image 4110 is displayed. According to an embodiment of the present invention, the image display area 4120 can be determined based on the position of the display image 4110.

Referring to FIG. 41, if the image 4110 is displayed on the upper left portion of the HMD device 4000, the HMD device 4000 can determine that the image display region 4120 is located at the upper left portion of the background image 4020. Referring to FIG. 42, the user 1 can recognize that the image 4110 is displayed on the image display area 4120 in the background image 4020. In this case, if the color of the background in the image display area 4120 is similar to the color of the image 4110, the user 1 may have difficulty recognizing the image 4110. Therefore, it is necessary to modify the color of the image 4110.

When the image display area 4120 is determined in step S3920 in FIG. 39, the HMD device 4000 can determine the color information about the image display area 4120 in step S3930. In this case, step S3930 may vary according to an embodiment of the present invention. For example, the HMD device 4000 can determine that the average color of the pixels included in the image display area 4120 is the color information about the image display area 4120. As another example, the HMD device 4000 can extract color information from feature points (such as a center point) in the image display area 4120.

Moreover, in accordance with an embodiment of the present invention, the HMD device 4000 can determine color information in consideration of both the color of the lens and the color of the image display area 4120. If the user 1 views the object through the lens, the user 1 can recognize the object in a color different from the actual color of the object. Therefore, in step S3930, the HMD device 4000 can determine that the color of the object recognized by the user 1 through the lens is the color information about the image display area 4120.

Next, in step S3940, the HMD device 4000 may modify the image based on the color information regarding the image display region 4120 determined in step S3930. In step S3950, the HMD device 4000 may display the modified image via the display unit.

For example, the HMD device 4000 can determine the complementary color of the color information about the image display area 4120 and modify the color of the image 4110 to the determined complementary color. Referring to FIG. 43, the HMD device 4000 can display the modified image 4125 such that the modified image 4125 can be easily distinguished from the background image 4020.

In step S3940, it is not necessary to modify the entire image. For example, you can modify only the color of the border of the image or add a separate background around the image.

Figure 44 is a flow diagram of a method for an HMD device to control the color or transmittance of a lens, in accordance with an embodiment of the present invention.

First, in step S4410, the HMD device can pick up an image by using a camera included in the HMD device. In this case, the camera can be aimed from outside the HMD device. In other words, the camera can pick up the image by receiving light that is not transmitted through the lens.

Next, in step S4420, the HMD device can determine the color or transmittance of the lens. In this case, the lens may include a smart window whose transmittance or color can be modified under the control of the control unit of the HMD device. For example, the HMD device can determine the external brightness based on the picked up image. In other words, the HMD device can analyze the brightness of the picked up image and determine the brightness outside the HMD device based on the result of the analysis. If the external brightness is high, the HMD device can change the color of the lens to a dark color, such as brown. Conversely, if the external brightness is low, the HMD device can control the lens to be colorless or determine the color or transmittance of the lens for a lens having high transmittance.

Next, in step S4430, the HMD device may control the color or transmittance of the lens such that the lens has the color or transmittance determined in step S4420.

45 is a diagram of a method for an HMD device to pick up an image for controlling color or transmittance, in accordance with an embodiment of the present invention.

According to an embodiment of the invention, the HMD device may comprise a plurality of cameras. Referring to Figure 45, the HMD device can include a first camera 560-1 and a second camera 560-2. The first camera 560-1 can be configured to receive light 4520 that is transmitted outside of the HMD device but that does not pass through the lens 580. The second camera 560-2 can be configured to receive light 4510 transmitted through the lens 580.

The HMD device can determine the color or transmittance of the lens 580 based on the light 4520 received via the first camera 560-1 and the light 4510 received via the second camera 560-2. For example, the HMD device can compare the light 4520 received via the first camera 560-1 with the light 4510 received via the second camera 560-2 (eg, luminance difference or chromatic aberration) and determine the color of the lens 580 based on the result of the comparison or Transmittance. Alternatively, the HMD device may select one of the light 4520 received via the first camera 560-1 and the light 4510 received via the second camera 560-2 and determine the color or transmittance of the lens 580 based on the brightness or color of the selected light. . According to an embodiment of the invention, the method of determining the color or transmittance of the lens can vary.

Figure 46 is a flow diagram of a method for an HMD device to control the color or transmittance of a lens, in accordance with an embodiment of the present invention.

First, in step S4610, the HMD device can pick up an image via the first camera 560-1. Further, in step S4615, the HMD device can pick up an image via the second camera 560-2.

Next, in step S4620, the HMD device can analyze the images picked up in steps S4610 and S4615. For example, the HMD device can compare the image picked up via the first camera 560-1 with the image picked up via the second camera 560-2. Alternatively, the HMD device may select one of the images picked up via the first camera 560-1 and the second camera 560-2. However, the invention is not limited thereto.

Next, in step S4630, the HMD device can determine the color or transmittance of the lens. When determining the color or transmittance of the lens, the HMD device may control the color or transmittance of the lens based on the determined color or transmittance in step S4640.

As described above, according to an embodiment of the present invention, a user can easily recognize a content or a user interface.

Embodiments of the invention may also be embodied as non-transitory computer readable recording media, including instructions executable by a computer, such as a program module. The non-transitory computer readable recording medium can be any available media that can be accessed by a computer and includes volatile media (eg, Random Access Memory (RAM)) and non-volatile media (eg, read only) Both memory (Read Only Memory; ROM) and both removable media and non-removable media. Furthermore, the non-transitory computer readable recording medium can include both computer storage media and communication media. Computer storage media includes computer readable instructions, data structures, program modules, or volatile, non-volatile, removable, and non-removable media embodied by any technology used to store information. Communication media typically includes computer readable instructions, data structures, program modules, other materials for modulating data signals or other transport mechanisms, and any information delivery media. For example, the non-transitory computer readable recording medium may be a ROM, a RAM, a flash memory, a compact disc (CD), a digital video disc (DVD), a magnetic disc, or a magnetic tape.

It is to be understood that the embodiments of the invention described herein are to be considered in a Descriptions of features or aspects within each embodiment of the invention should generally be considered as other similar features or aspects that may be used in other embodiments.

Although one or more embodiments of the present invention have been described with reference to the drawings, it will be understood by those skilled in the art that the spirit and scope of the invention as defined by the appended claims and their equivalents In this case, various changes can be made to the form and details.

1‧‧‧Users
100‧‧‧HMD device
110‧‧‧Frame
111‧‧‧Display unit
120‧‧‧Removable lens
121‧‧‧ Clips
130‧‧‧ central part
131‧‧‧Bridge arm
140‧‧‧Nose beam
141‧‧‧ cushion
201‧‧‧ lens frame
201'‧‧‧ Center
202‧‧‧ right arm
202'‧‧‧ Center
203‧‧‧left arm
203'‧‧‧ Center
204‧‧‧ lens
204'‧‧‧Nose beam
205‧‧‧ onboard computer
206‧‧‧ camera
207‧‧‧User input unit
208‧‧‧ sensor
209‧‧‧ sensor
210‧‧‧ Sensor
211‧‧‧Touch Module
212‧‧‧ display
213‧‧‧Optical output unit
214‧‧‧ microphone
215‧‧‧right speaker
216‧‧‧left speaker
217‧‧‧Power supply unit
300‧‧‧HMD device
301‧‧‧ lens frame
302‧‧‧ side arm
303‧‧‧ monocular lens
304‧‧‧Airborne computer
305‧‧‧ camera
306‧‧‧User input unit
307‧‧‧ sensor
308‧‧‧ sensor
309‧‧‧ sensor
310‧‧‧Touch Module
311‧‧‧ display
312‧‧‧Optical output unit
313‧‧‧Microphone
314‧‧‧Speaker
315‧‧‧Power supply unit
400‧‧‧HMD device
401‧‧‧ lens frame
401'‧‧‧Nose beam
402‧‧‧ right arm
403‧‧‧left arm
404‧‧‧right lens
405‧‧‧left lens
406‧‧‧ onboard computer
407‧‧‧ camera
408‧‧‧User input unit
409‧‧‧ sensor
410‧‧‧ sensor
411‧‧‧ sensor
412‧‧‧Touch Module
413‧‧‧right display
414‧‧‧left display
415‧‧‧ microphone
416‧‧‧right speaker
417‧‧‧left speaker
418‧‧‧Power supply unit
500‧‧‧HMD device
510‧‧‧Input/Output Unit
511‧‧‧Speaker
512‧‧‧ microphone
520‧‧‧ memory
525‧‧‧Sensor unit
530‧‧‧Battery
535‧‧‧Power Management Unit
540‧‧‧Communication unit
550‧‧‧Touch sensor
560‧‧‧ camera
560-1‧‧‧First camera
560-2‧‧‧Second camera
570‧‧‧Display unit
570-1‧‧‧Display unit/lens
571‧‧‧First optical axis
572‧‧‧second optical axis
580‧‧‧ lens
581‧‧‧ glass
582‧‧‧Whole Image Optical Element (HOE)
590‧‧‧Control unit
600‧‧‧HMD device
610‧‧ eyes
611‧‧‧Light
612‧‧‧Cell crystal
613‧‧‧ retina
710‧‧‧Projected light
711‧‧‧ openings
720‧‧‧Frame
722‧‧‧ Hinges
730‧‧‧reflected projected light
810‧‧‧Light source
820‧‧‧Lighting optical system
830‧‧ ‧ mirror
840‧‧‧ display device
850‧‧‧Projection optical system
860‧‧‧Relay lens
870‧‧‧Projection lens
910‧‧‧Environmental imagery
911‧‧‧ Ambient light
912‧‧‧Transmitted ambient light
920‧‧‧Image/displayed image
921‧‧‧Light
922‧‧‧Reflected light/converged light
1010‧‧‧First insulating substrate
1015‧‧‧Second insulating substrate
1020‧‧‧First conductive electrode
1025‧‧‧Second conductive electrode
1030‧‧‧Insulation spacer
1040‧‧‧Electrochromic layer
1050‧‧‧ Electrolytes
1110-1‧‧‧Light
1110-2‧‧‧Light
1110-3‧‧‧Light
1120-1‧‧‧Light
1120-2‧‧‧Light
1120-3‧‧‧Light
1310‧‧‧Color Information
1320‧‧‧ original image
1330‧‧‧Color Correction Module
1340‧‧‧Modified images
1500‧‧‧HMD device
1510‧‧‧Frame
1511‧‧‧Display unit
1520‧‧ lens
1531‧‧‧Metal wire
1532‧‧‧electrode
1540‧‧‧ magnet
1700‧‧‧HMD device
1710‧‧‧ camera
1720‧‧ lens
1721‧‧‧ yards
1910‧‧‧Light source
1920‧‧ lens
1930‧‧‧Photosensitive sensor
2100‧‧‧HMD device
2110‧‧‧ camera
2120‧‧‧ eyes
2200‧‧‧HMD device
2210‧‧‧User interface
2211‧‧‧Brown
2220‧‧‧ lens
2300‧‧‧HMD device
2400‧‧‧Control terminal
2700‧‧‧HMD device
2710‧‧‧Information
2720‧‧‧ lens
2810‧‧‧Content list
2811‧‧‧Web browser
2820‧‧ lens
2821‧‧‧ lens
2910‧‧‧Screen image
2911‧‧‧Screen image
2920‧‧‧Message
3300‧‧‧HMD device
Image of 3310‧‧‧
3311‧‧ images
3320‧‧‧ lens
3321‧‧‧ lens
3401‧‧‧External landscape
3500‧‧‧HMD device
3510‧‧‧ images
3511‧‧‧Modified image
3520‧‧ lens
3710‧‧‧Color Information Acquisition Unit
3720‧‧‧Control unit
3730‧‧‧Display unit
4000‧‧‧HMD device
4010‧‧‧ camera
4020‧‧‧Background image
4110‧‧‧Image
4120‧‧‧Image display area
4125‧‧‧Corrected images
4510‧‧‧Light
4520‧‧‧Light angle ‧‧‧ angle θ‧‧‧ angle
S1210‧‧‧Steps
S1220‧‧‧Steps
S1410‧‧‧Steps
S1420‧‧‧Steps
S1430‧‧‧Steps
S1440‧‧‧Steps
S1810‧‧‧Steps
S1820‧‧‧Steps
S1830‧‧‧Steps
S1840‧‧‧Steps
S2010‧‧‧Steps
S2020‧‧‧ steps
S2030‧‧‧Steps
S2040‧‧‧Steps
S2050‧‧‧Steps
S2310‧‧‧Steps
S2320‧‧‧Steps
S2330‧‧‧Steps
S2340‧‧‧Steps
S2350‧‧‧Steps
S2510‧‧‧Steps
S2520‧‧‧Steps
S2530‧‧‧Steps
S2610‧‧‧Steps
S2620‧‧‧Steps
S2630‧‧‧Steps
S2640‧‧‧Steps
S3110‧‧‧Steps
S3120‧‧‧Steps
S3130‧‧‧Steps
S3910‧‧‧Steps
S3920‧‧‧Steps
S3930‧‧‧Steps
S3940‧‧‧Steps
S3950‧‧‧Steps
S4410‧‧‧Steps
S4420‧‧‧Steps
S4430‧‧‧Steps

The foregoing and other aspects, features and advantages of the present invention will become more apparent from the description of the accompanying drawings in which: Figure 1 illustrates a head mounted display (HMD) device in accordance with an embodiment of the present invention. Figure 2 illustrates an HMD device in accordance with an embodiment of the present invention. Figure 3 illustrates an HMD device in accordance with an embodiment of the present invention. Figure 4 illustrates an HMD device in accordance with an embodiment of the present invention. Figure 5 is a block diagram of an HMD device in accordance with an embodiment of the present invention. 6 is a diagram of a method for displaying an image by an HMD device in accordance with an embodiment of the present invention. Figure 7 is a diagram of a portion of an HMD device in accordance with an embodiment of the present invention. Figure 8 is a block diagram of a display unit in accordance with an embodiment of the present invention. Figure 9 is a diagram of a lens in accordance with an embodiment of the present invention. Figure 10 is a diagram of glass contained in a lens of an HMD device, in accordance with an embodiment of the present invention. 11A through 11C are diagrams of a method for displaying an image by an HMD device, in accordance with an embodiment of the present invention. 12 is a flow chart of a method for an HMD device to display an image, in accordance with an embodiment of the present invention. Figure 13 is a block diagram for modifying and displaying an image in accordance with an embodiment of the present invention. 14 is a flow chart of a method of obtaining color information about a lens, in accordance with an embodiment of the present invention. 15 is a diagram of an HMD device for using a wire identification lens in accordance with an embodiment of the present invention. 16 is a diagram of a portion of an HMD device including metal wires, electrodes, and lenses, in accordance with an embodiment of the present invention. 17 is a diagram of a method of identifying a lens by using a code printed on a lens, in accordance with an embodiment of the present invention. 18 is a flow chart of a method of obtaining color information about a lens by using a photosensitive sensor, in accordance with an embodiment of the present invention. 19A and 19B are diagrams of a method of obtaining color information about a lens by using a photosensitive sensor, in accordance with an embodiment of the present invention. 20 is a flow chart of a method of obtaining color information about a lens based on the size of a user's pupil, in accordance with an embodiment of the present invention. 21 is a diagram of a method of obtaining color information based on the size of a user's pupil, in accordance with an embodiment of the present invention. 22 is a diagram of a method of obtaining color information about a lens based on a user's settings, in accordance with an embodiment of the present invention. 23 is a flow chart of a method for displaying images for an HMD device in accordance with an embodiment of the present invention. 24 is a diagram of a method for displaying an image by an HMD device in accordance with an embodiment of the present invention. 25 is a flow chart of a method for recommending a lens for an HMD device in accordance with an embodiment of the present invention. 26 is a flow chart of a method for recommending a lens for an HMD device in accordance with an embodiment of the present invention. 27 to 31 are diagrams of a method for recommending a lens for an HMD device according to an embodiment of the present invention. 32 is a flow chart of a method for an HMD device to change the color or transmittance of a lens that displays an image, in accordance with an embodiment of the present invention. 33 is a chart of modified colors that can be used to modify an image, in accordance with an embodiment of the present invention. 34 and 35 are diagrams of an HMD device that displays a corrected image, in accordance with an embodiment of the present invention. 36 and 37 are diagrams of an HMD device that displays a modified image in accordance with another embodiment of the present invention. Figure 38 is a block diagram of an HMD device in accordance with an embodiment of the present invention. 39 is a flow chart of a method for displaying an image by an HMD device in accordance with an embodiment of the present invention. 40 to 43 are diagrams of an HMD device that displays a corrected image, in accordance with another embodiment of the present invention. 44 is a flow chart of a method for an HMD device to control the color or transmittance of a lens, in accordance with an embodiment of the present invention. 45 is a diagram of a method of controlling color or transmittance of an HMD device for picking up images, in accordance with an embodiment of the present invention. Figure 46 is a flow diagram of a method for an HMD device to control the color or transmittance of a lens, in accordance with an embodiment of the present invention.

3710‧‧‧Color Information Acquisition Unit

3720‧‧‧Control unit

3730‧‧‧Display unit

Claims (15)

A head mounted display device comprising: a color information obtaining unit configured to obtain color information about a lens disposed at a portion of the head mounted display device; and a control unit configured to be configured via the head The display unit at the display device outputs an image modified based on the color information. The head mounted display device of claim 1, wherein the lens is configured to modify at least one of color or transmittance thereof under control of the control unit, and the color information obtaining unit Further configuration is made to obtain color information corresponding to the modified color or transmittance of the lens. The head mounted display device of claim 1, wherein the lens is attachable to and detachable from the head mounted display device, and the lens attachment is detected In the case of the head mounted display device, the control unit is further configured to obtain the color information of the lens. The head-mounted display device of claim 3, wherein the color information obtaining unit further comprises an identification unit configured to recognize a code on the lens and obtain corresponding to the identification The color information of the code identified by the unit. The head mounted display device of claim 3, wherein the control unit is further configured to select a content to be output via the display unit, determine a lens corresponding to the selected content, and The display unit outputs a recommendation message for recommending the determined lens. The head mounted display device of claim 3, further comprising a brightness sensor configured to obtain information about external brightness, wherein the control unit is further configured to determine A lens corresponding to the information about the external brightness obtained via the brightness sensor and outputting a recommendation message for recommending the determined lens via the display unit. The head mounted display device of claim 1, wherein the color information obtaining unit comprises a photosensitive sensor and is further configured to obtain information about the lens based on information obtained via the photosensitive sensor The color information. The head mounted display device of claim 1, further comprising a brightness sensor configured to obtain information about external brightness, wherein the image output via the display unit is further An image modified in consideration of the information about the external brightness. A method for displaying an image by a head mounted display device, the method comprising: obtaining color information about a lens disposed at a portion of the head mounted display device; and via a display unit disposed at the head mounted display device An image modified based on the color information is output. The method for displaying an image for a head mounted display device according to claim 9, further comprising modifying at least one of a color or a transmittance of the lens under control of a control unit, wherein In the case of color information, color information corresponding to the modified color or transmittance of the lens is obtained. The method for displaying an image for a head mounted display device according to claim 9, wherein the lens is attachable to the head mounted display device and detachable from the head mounted display device, and is recognized Where the lens is attached to the head mounted display device, the color information is obtained. The method for displaying an image for a head mounted display device according to claim 11, wherein obtaining the color information comprises: identifying a code on the lens; and obtaining color information corresponding to the code. The method for displaying an image for a head mounted display device according to claim 11, further comprising: selecting content to be output via the head mounted display device; and recommending a lens corresponding to the selected content, wherein When the modified image is displayed, an image containing the selected content modified based on color information about the recommended lens is output. The method for displaying an image for a head mounted display device according to claim 11, further comprising: obtaining information about external brightness; and recommending a lens corresponding to the information about the external brightness. A non-transitory computer readable recording medium having a computer program recorded thereon for implementing a method for displaying images by a head mounted display device, the method comprising: obtaining information about the configuration on the headset Color information of a lens at a portion of the display device; and outputting an image modified based on the color information via a display unit disposed at the head mounted display device.