KR20160014507A - Method for displayin image by head mounted display device and head maounted display device thereof - Google Patents

Abstract

Provided are a head mounted display device capable of providing a user interface or a content which can be recognized by a user more clearly, and a display method of the head mounted display device. The head mounted display device includes: a color information acquisition part for acquiring color information as to a lens at one side of the head mounted display device; and a control part which outputs an image corrected on the basis of the color information through a display part comprised in the head mounted display device.

Description

METHOD FOR DISPLAY IMAGE BY HEAD MOUNTED DISPLAY DEVICE AND HEAD MAOUNTED DISPLAY DEVICE THEREOF FIELD OF THE INVENTION [

The present invention relates to a wearable display device, and more particularly to a head mounted display device.

[0003] As the weight and size of digital devices have been reduced, various wearable devices have been developed. A head-mounted display (HMD), which is a type of wearable device, refers to various digital devices that can be worn on a user's head to receive multimedia contents and the like. The head mount display can be implemented in various forms for wearing on the head, such as glasses or helmets.

Since the HMD is used by being worn on the user's body, as the user moves, the image is provided to the user in various environments. Therefore, the HMD display needs to have a lens having various colors, transmissivity, and the like. For example, in a sunny outdoor environment, a dark color lens is provided to block sunlight and a transparent color lens is provided in the room, so that the user can easily see the surroundings.

The HMD can change the color of the lens by replacing the lens provided on one side of the HMD with a lens having a different color. Alternatively, if the lens provided on one side of the HMD is a smart window that can change at least one of color and transmittance, the HMD can control the color and transmittance of the lens. As the color, transmissivity, and the like of the lens on one side of the HMD are changed, it is necessary to provide an image that can be easily recognized by the user.

An embodiment of the present invention provides a head-mounted display device and a method of displaying the head-mounted display device that can provide contents or a user interface more clearly recognizable by a user.

According to another aspect of the present invention, there is provided a head-mounted display device including a color-information acquiring unit for acquiring color information for a lens on one side of a head-mounted display, and a color- And a controller for outputting the data through a display unit provided in the device.

According to another embodiment of the present invention, the lens changes at least one of hue and transmittance according to the control of the control unit, and the color information obtaining unit obtains color information corresponding to the hue or transmittance of the changed lens .

Further, according to still another embodiment, the lens is detachable to the head mount display, and the control unit can acquire color information when the lens coupled to the head mount display is recognized.

According to another embodiment of the present invention, the color information obtaining unit further includes a recognizing unit that recognizes a code included in the lens, and can acquire color information corresponding to the recognized code through the recognizing unit.

According to another embodiment of the present invention, the control unit may select the outputable content through the display unit, determine a lens corresponding to the selected content, and output a recommendation message recommending the determined lens through the display unit.

Further, the head-mounted display device according to still another embodiment may further include an illuminance sensor that obtains information on the external illuminance, and the control unit determines a lens corresponding to the information on the external illuminance obtained through the illuminance sensor And a recommendation message recommending the determined lens through the display unit can be output.

According to still another embodiment, the color information obtaining unit may include a photosensor and acquire color information about the lens based on the information obtained through the photosensor.

Further, the head-mounted display device according to another embodiment further includes an illuminance sensor that obtains information on the external illuminance, and the image output through the display unit is a modified image . ≪ / RTI >

As a technical means for solving the above-mentioned technical problem, a method for displaying an image by a head mounted display device includes the steps of acquiring color information for a lens on one side of the head mount display, And outputting through a display unit included in the mount display device.

Further, the display method according to some other embodiments may further include changing at least one of a color and a transmittance of the lens, and the step of acquiring the color information may include acquiring color information corresponding to the color or transmittance of the changed lens . ≪ / RTI >

Further, according to still another embodiment, the lens is detachable to the head mount display, and the display method performs the step of acquiring color information when the lens coupled to the head mount display is recognized can do.

Further, according to still another embodiment, acquiring the color information may include recognizing a code provided in the lens and acquiring color information corresponding to the code.

Further, a display method according to still another embodiment includes selecting content outputable through a head-mounted display and recommending a lens corresponding to the selected content, wherein the step of displaying the modified image is a recommended And outputting an image obtained by modifying the selected content based on the color information of the lens.

Further, the display method according to still another embodiment may further include acquiring information on external illuminance and recommending a lens corresponding to information on the external illuminance.

According to still another embodiment, the head mount display device includes a photosensor, and the step of acquiring color information may acquire color information based on information obtained through the photosensor.

According to still another aspect of the present invention, there is provided a display method, further comprising the step of acquiring information on an external illuminance, and the image output through the display unit is a modified image considering further information on external illuminance can do.

As a technical means for solving the above-mentioned technical problems, a computer-readable recording medium is characterized in that a program for executing the above-described method in a computer is recorded.

1 is an external view of an HMD device according to some embodiments.
2 is an external view of an HMD device according to some other embodiments.
3 is an external view of an HMD device according to still another embodiment.
4 is an external view of an HMD device according to still another embodiment.
5 is a block diagram showing a circuit structure of an HMD device according to some embodiments.
6 is a conceptual diagram showing a method of displaying an image by an HMD device according to some embodiments.
7 is an outline view showing a structure of an HMD device according to some other embodiments.
8 is a conceptual diagram showing a structure of a display unit according to some embodiments.
9 is a conceptual diagram for explaining the configuration and function of the lens according to some embodiments.
10 is a conceptual diagram showing a configuration of a glass included in a lens of an HMD device according to some embodiments.
11 is a conceptual diagram illustrating a method of displaying an image by an HMD device according to some other embodiments.
12 is a flow chart illustrating a process by which an HMD device displays an image in accordance with some embodiments.
13 is a conceptual diagram showing a structure in which an image is corrected and displayed according to some embodiments.
14 is a flow chart illustrating a process for obtaining color information in accordance with some embodiments.
15 is an outline view showing the structure of an HMD device for identifying a lens using a metal line according to some embodiments.
16 is a view showing a part of an HMD device and a lens including a structure of a metal wire and an electrode according to some embodiments.
17 is a conceptual diagram illustrating a method of identifying a lens using a code displayed on the lens in accordance with some embodiments.
18 is a flow chart illustrating a process for obtaining color information using a photosensor in accordance with some other embodiments.
19 is a conceptual diagram showing a method of acquiring color information using a photosensor according to some other embodiments.
20 is a flow chart illustrating a process for obtaining color information based on a size of a user ' s pupil in accordance with some further embodiments.
21 is a conceptual diagram illustrating a method for acquiring color information based on the size of a pupil of a user in accordance with some further embodiments.
22 is a conceptual diagram showing a method of acquiring color information based on a setting of a user according to still another embodiment.
23 is a flowchart illustrating a process by which an HMD device displays an image according to still another embodiment.
24 is a conceptual diagram showing a method of displaying an image by an HMD device according to still another embodiment.
25 is a flowchart illustrating a process in which an HMD device recommends a lens according to some embodiments.
Figure 26 is a flow chart illustrating a process by which an HMD device recommends a lens in accordance with some other embodiments.
Figures 27-31 are illustrations showing how an HMD device recommends a lens according to some embodiments.
32 is a flowchart illustrating a process by which an HMD device capable of changing the color or transmittance of a lens displays an image according to some embodiments.
33 is an exemplary diagram showing a complementary table that can be used for correcting an image according to some embodiments.
Figs. 34 and 35 are conceptual diagrams showing an example in which an HMD device displays a corrected image according to some embodiments.
36 and 37 are conceptual diagrams illustrating an example in which an HMD device displays a corrected image according to some other embodiments.
38 is a block diagram briefly showing a structure of an HMD device according to some embodiments.
39 is a flow chart illustrating a process by which an HMD device displays an image in accordance with some other embodiments.
40 to 43 are conceptual diagrams illustrating an example in which an HMD device displays a corrected image according to some other embodiments.
Figure 44 is a flow chart illustrating a process by which an HMD device controls the color or transmittance of a lens in accordance with some embodiments.
45 is a conceptual diagram showing a method by which an HMD device photographs an image to control color or transmittance according to some other embodiments.
46 is a flow chart illustrating a process by which an HMD device controls the color or transmittance of a lens in accordance with some other embodiments.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

Throughout the specification, the term "lens" may comprise an object capable of transmitting light. The term "lens" is not limited to an object whose surface forms an optical image. For example, "lens" in this specification may include the case where the surface is planar. The "lens" may be made of glass, quartz or plastic. Alternatively, the "lens" may be a smart window. The smart window may include a device in which at least one of the hue and transmittance is varied according to an application signal. For example, an electrochromic glass, a suspended particle device (SPD), or a liquid crystal (LC) may be used to form a smart window.

Throughout the specification, the image displayed by the HMD device includes not only the image directly displayed on the lens, but also the image in which the user is perceived as displaying the image on the lens.

In the entire specification, "displaying an image on a lens" means not only when an image is directly displayed on a lens, but also when an image is outputted to a user of the HMD device so that the image is recognized as being displayed on the lens .

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is an external view of an HMD device according to some embodiments.

Referring to FIG. 1, the HMD device may be configured in the form of glasses that are secured to the user's face through the ears and nose of the user (or wearer). However, the configuration of the HMD device 100 is not limited thereto. For example, the HMD device 100 may be attached to a helmet or implemented in the form of goggles.

The HMD device 100 according to some embodiments may include a frame 110. The HMD device 100 according to some embodiments may include a nosepiece 420 configured to be seated in the wearer's nose at the central portion 130. According to some embodiments, the nose pads 420 may include a bridge arm 131 and a pad 141.

According to some embodiments, a detachable lens 120 may be attached to the frame 110. Here, the lens 120 may have various colors. For example, the user may use the HMD device 100 in a state where a lens 120 having a transparent color is attached to the interior of the HMD device 100 while the lens 120 having a dark color is attached to the exterior of the HMD device 100, Can be used. For example, a clip 121 provided on the lens 120 may be coupled to the bridge arm 131 of the HMD device 100.

In addition, the HMD device 100 may include a display unit 111 for displaying an image. The display unit 111 may be implemented in various forms according to the embodiment. For example, the display unit 111 may be configured to form an image on the wearer's retina using an optical system as shown in FIG. 1, or may display an image on a lens.

2 is an external view of an HMD device according to some other embodiments.

Referring to FIG. 2, the HMD device 100 may be configured in the form of glasses that are secured to the user's face through the ears and nose of the user (or wearer). However, the configuration of the HMD device 100 is not limited thereto. For example, the HMD device 100 may be attached to a helmet structure or modified into a Goggles form.

The HMD device 100 shown in Figure 2 includes a lens frame 201, side-arms 202 and 203, a lens 204, an on-board computer 205, a camera A user input unit 207, sensors 208, 209 and 210, a haptic module 211, a display 212, an optical output unit 213, a microphone 214, speakers 215 and 216, And a supply unit 217.

Some components included in the HMD device 100 may be embedded in the HMD device 100, and some other components may be mounted outside the HMD device 100. For example, the sensors 208, 209, 210 may be embedded in the HMD device 100. The display 212 may be mounted to the exterior of the HMD device 100. The components incorporated in the HMD device 100 and the components mounted on the outside of the HMD device 100 are not limited to those described above.

The components included in the HMD device 100 are not limited to those shown in FIG. For example, the HMD device 100 may not include some of the sensors 208, 209, 210, and the haptic module 211. For example, the HMD device 100 may include only the sensor 210 of the sensors 208, 209, 210, or may house the sensors at more locations than shown in FIG.

The lens frame 201 and the side legs 202 and 203 may be constructed of a solid such as plastic or / and metal or may be hollowed to include a material such as wiring that may connect the components contained in the HMD device 100 to each other. Can be composed of solid plastic such as hollow plastic or / and metal.

The lens frame 201 and the side legs 202 and 203 shown in Fig. 2 may be integrally formed without a connecting portion (not shown). Thus, the lens frame 201 and the side legs 202, 203 can be referred to as the frame of the HMD device 100.

The structure of the lens frame 201 and the side legs 202, 203 is not limited to that shown in Fig. For example, the HMD device 100 may be configured with a coupling member (not shown) between the lens frame 201 and the side legs 202, 203. That is, the HMD device 100 has a connecting member (not shown) between the lens frame 201 and the right side legs 202 and a connecting member (not shown) between the lens frame 201 and the left side legs 203 The side legs 202 and 203 can be folded in the direction of the lens frame 201. In this case,

The HMD device 100 has a connecting member (not shown) at the center 201 'of the lens frame 201 and a connecting member (not shown) between the lens frame 201 and the right side leg 202 (Not shown) between the lens frame 201 and the left side leg 203 and a connecting member (not shown) is attached to each of the centers 202 'and 203' of the side legs 202 and 203, The lens frame 201 and the side legs 202 and 203 can be folded inward to minimize the size of the HMD device 100 based on each connecting member.

The above-described connecting members, which are not shown, may be configured in the form of a screw assembly, or may be configured in such a manner that a groove is interposed in the groove, but the present invention is not limited thereto.

The side legs 202, 203 can be configured to extend to the user's ear. The side legs 202, 203 can be configured to extend around the back of the user's head. A band (not shown) capable of securing the side legs 202, 203 to the user's head is provided on the side legs 202, 203 when the side legs 202, 203 are configured to extend around the back of the user's head. (202, 203). The band (not shown) may be configured in the form of a de-energizing band so that the side legs 202, 203 are fixed to the user's head, regardless of the size of the user's head.

A preventive member (not shown) can be attached to prevent the side legs 202, 203 from slipping or falling down to the inside position of the side legs 202, 203 contacting the back of the user's wheels.

The lens frame 201 can be configured so that the lens 204 can be detached. The lens 204 shown in FIG. 2 is constructed integrally with, but not limited to, a nosepiece 204 '. For example, in the case where the lens frame 201 is integral with the nose piece, the lens 204 may be composed of a right lens 404 and a left lens 405 as shown in Fig. 3 .

The lens 204 can be made of a transparent material that allows the user to see the real world or the physical world. The lens 204 may be made of a material that can pass light that is reflected through the display 212. The material that can be used as the lens 204 may include, but is not limited to, for example, plastic such as polycarbonate, or glass. The lens 104 may include at least one of a light reflection and anti-glare coating, an anti-fog coating, and an ultraviolet blocking coating.

The embedded computer 205 may be connected to the HMD device 100 either wired or wirelessly. The embedded computer 205 is located in a part of the left side leg 203 adjacent to the lens frame 201, but is not limited thereto. For example, the embedded computer 205 may be located in a portion of the right side leg 202 or in another portion of the HMD device 100, such as another portion of the left side leg 203.

The embedded computer 205 receives data from, for example, the camera 206, the user input 207, the sensors 208, 209 and 210, analyzes the received data, May generate information to be communicated to the user of the HMD device 100 through at least one of the display 212 and the speakers 215, The information to be conveyed to the user may include, but is not limited to, at least one of image, text, video, and audio.

The camera 206 may be located outside the left side leg 203 or may be embedded in the left side leg 203. The camera 206 may be mounted at another location of the HMD device 100. For example, the camera 206 may be located in a portion of the right side leg 202 or in a portion (e.g., a center) of the lens frame 201. The camera 206 may be a small camera such as a camera or a webcam used in a smart phone. The camera 206 may be mounted at a location capable of capturing at least a portion of the real world that is perceived by the user of the HMD device 100. [

The user input unit 207 may include at least one of a touch pad that can be operated by the user's finger and a button that can be operated by the user's pushing operation. The user input 207 may be located at a portion of the left side leg 203, but may be located at another portion of the HMD device 100. For example, the user input 107 may be located on another portion of the left side leg 203, a portion of the right side leg 202, and a portion of the lens frame 201.

The user input 207 can be used by the user to input a command. The user input unit 207 may include an on / off switch for turning on / off the power of the HMD device 100.

The sensor 208 may be a biometric information sensor capable of sensing biometric information of a user of the HMD device 100. The sensor 208 is provided in a protruding form at the lower right end of the lens frame 201, but the installation form of the sensor 208 is not limited thereto. For example, the sensor 208 may be embedded in the lens frame 201 or mounted on the lens frame 201. The sensor 208 may be located at any other portion of the HMD device 100.

In the case where the sensor 208 is constituted by an image sensor-based camera, the sensor 208 measures the degree of eye congestion of the user, the degree of dryness of the user's eyes, the number of times the user's eyes flicker, the position of the user's eyes, A change in the pupil, and the like.

In the case where the sensor 208 comprises at least one of an oxygen sensor, an air sensor, and a fine dust measurement sensor, the sensor 208 may detect the dryness of the user's eye, but is not limited thereto.

The sensor 209 may be an ambient environment sensing sensor capable of sensing the surrounding environment of the HMD device 100. The sensor 209 is located inside the center of the lens frame 201, but is not limited thereto. For example, the sensor 209 may be configured such that an inner portion of the lens frame 201, an outer portion of the lens frame 201, an inner portion of the side legs 202 and 203, And at least one of the outer portions of the side legs 202 and 203, but the installation position and the installation structure of the sensor 209 are not limited to the above-described ones. The sensor 209 may be a sensor capable of detecting the ambient environment such as temperature, light, humidity, wind, altitude, air pressure, etc. of the HMD device 100, but the detectable environment is not limited thereto.

When the sensor 208 is configured with at least one of an oxygen sensor, an air sensor, and a fine dust measurement sensor as described above, the ambient environment information of the HMD device 100 is based on the signal sensed through the sensor 208 . In this case, the HMD device 100 may not include the sensor 209.

The sensor 210 may be a sensor capable of sensing the wearing state of the HMD device 100. [ The sensor 110 is positioned inside the right side leg 202 contacting the back of the user's ear, but the position of the sensor 210 is not limited thereto. For example, the sensor 210 may be located inside the left side leg 203 contacting the back of the user's ear, or inside the power supply 217 contacting the back of the user's ear.

The sensor 210 may comprise, for example, a pressure sensor. In the case where the sensor 210 is constituted by a pressure sensor, the sensor 210 can transmit a signal to the built-in computer 205, which detects whether or not the pressure is applied and the magnitude of the pressure. Accordingly, the built-in computer 205 detects the wearing time of the HMD device 100 or wears the HMD device 100 based on the sensing value received from the sensor 210, Can be detected.

The haptic module 211 is located at a position adjacent to an end inside the left side leg 203 of the HMD device 100 but the mounting position of the haptic module 211 is not limited thereto. For example, the haptic module 211 may be located within a portion of the right side leg 202 or at another point of the HMD device 100.

The haptic module 211 may be formed of a material capable of generating various tactile effects that the user can feel. For example, the haptic module 211 may include a substance capable of generating a vibration effect, a substance capable of generating a stimulating effect on a contact skin surface, a substance capable of generating a stimulation effect based on injection or inhalation of air, electrode, it is possible to generate a variety of tactile effects such as a substance capable of generating a stimulation effect through contact with the electrode, a substance capable of generating a stimulation effect using electrostatic force, and a substance capable of generating a cold sensing effect using a device capable of endothermic or exothermic But the material used for the haptic module 211 is not limited thereto.

The display 212 is connected to the left side leg 203 of the HMD device 100 and is located at the upper left of the lens 204 but the position of the display 212 is not suggested. For example, the display 212 may be connected to the right side leg 202 to be located in the right center of the lens 204 or to the left side leg 203 to be located in the left center of the lens 204. The display 212 may comprise, but is not limited to, a translucent optical waveguide (e.g., a prism). The display 212 may reflect the light output from the optical output 213 and focus the image on the fovea of the retina of the user's eye of the HMD device 100. [

The optical output device 213 can be, but is not limited to, a mini projector, for example.

The microphone 214 may receive the voice of the user of the HMD device 100, and other sounds. The microphone 214 may be configured to selectively set the near mode and the remote mode. For example, when the microphone 214 is set to the close-up mode, the microphone 214 receives a guitar sound such as biometric information of the user of the HMD device 100 (for example, information based on blinking of eyes) For example. When the microphone 214 is set to the remote mode, the microphone 214 can be used to receive the voice of the user of the HMD device 100. The near mode of the microphone 214 is a mode capable of receiving sound generated inside the HMD device 100 and the remote mode of the microphone 214 is a mode capable of receiving sounds generated outside the HMD device 100 Lt; / RTI > However, the above-described operation range of the near mode and the far mode can be set differently.

The speakers 215 and 216 may be configured in the form of an earphone that can be mounted on the user's ear of the HMD device 100. [ The speaker 215 is located inside the right side leg 202 so as to be mounted on the right ear of the user of the HMD device 100. The speaker 216 may be located inside the left side leg 203 for mounting on the left ear of the user of the HMD device 100.

The speakers 215 and 216 may be fixedly mounted to the HMD device 100 as shown in FIG. 2, but the mounting form of the speakers 215 and 216 is not limited thereto. For example, the speakers 215 and 216 may be configured to be detachable from the HMD device 100 such that a user of the HMD device 100 may optionally insert the speakers 215 and 216 into the ear of the user .

When the speakers 215 and 216 are detachable to the HMD device 100, the speakers 215 and 216 may be located inside the respective side legs 202 and 203 contacting the back of the user's ear. Accordingly, the user can remove the speakers 215 and 216 from the respective side legs 202 and 203, and then insert them into the user's ear. There may be a connection line (not shown) between the speakers 215 and 216 and the HMD device 100 when the speakers 215 and 216 are pulled out from the respective side legs 202 and 203 and inserted into the user's ear .

A connection line (not shown) between the speakers 215 and 216 and the HMD device 100 may be embedded in the HMD device 100 when the speakers 215 and 216 are mounted on the HMD device 100, It does not. The speakers 215 and 216 and the HMD device 100 may be wirelessly connected based on short-range wireless communication.

There may be a coupling member (not shown) based on an elastic material (e.g., a spring) between the speakers 215 and 216 and the HMD device 100. [ In this case, the connecting member based on the elastic material can be changed in shape depending on the position of the user's ear. For example, the connection member can be changed in shape so as to tilt the speakers 215 and 216 in at least one of the left direction, the right direction, the upward direction, and the downward direction. The speaker 215 and 216 and the HMD device 210 can be configured to be able to communicate in a wired or wireless manner when there is a connection member based on elastic material between the speakers 215 and 216 and the HMD device 100 have. According to some embodiments, the speakers 215 and 216 may be comprised of bone conduction speakers.

The power supply unit 217 is located at the end of the right side leg 102 of the HMD device 100 but the position of the power supply unit 217 is not limited thereto. For example, the power supply 217 may be located at the end of the left side leg 103 of the HMD device 100.

The power supply unit 217 may be configured to be detachable to the right side leg 202 of the HMD device 100. [ The power supply 217 may include an external power connector that may be connected to an external power supply (not shown) and then charged after being connected to the right side leg 202 or separated from the right side leg 202 have. The power supply unit 217 may be integrated with the right side leg 202 of the HMD device 100. The power supply unit 217 may include an external power supply connector that can be connected to and charged with an external power supply unit (not shown).

3 is an external view of an HMD device according to still another embodiment. 3 is an HMD device 300 having an eye lens 303, and may be configured as a single display type or a monopod display type as shown in FIG.

Referring to FIG. 3, the HMD device 300 according to another embodiment of the present invention is configured as a one-eyed eyeglass that is fixed to a user's face through one ear and one side of the user. The HMD device 300 may be attached to a helmet structure.

The HMD device 300 includes a lens frame 301, a side-arm 302, a lens 303, an on-board computer 304, a camera 305, a user input 306, A haptic module 310, a display 311, an optical output unit 312, a microphone 313, a speaker 314, and a power supply unit 315. The sensors 317, 308,

The components included in the HMD device 300 are not limited to those shown in FIG. For example, the HMD device 300 may not include some of the sensors 307, 308, 309, and the haptic module 310. For example, the HMD device 300 may include only the sensor 309 of the sensors 307, 308, 309, or the sensors may be attached to more positions than shown in FIG. The positions of the sensors 307, 308, and 309 attached to the HMD device 300 may be changed.

The lens frame 301 can be detachably attached to the lens 303 as described with reference to FIG. In addition, the HMD device 300 may have a connecting member between the lens frame 301 and the side legs 302 as described with reference to FIG. In addition, the HMD device 300 may have a connecting member at the center of the side legs 302 as described in FIG.

The lens 303, the on-board computer 204, the camera 305, the user input 306, the sensors 307, 308 and 309 included in the HMD device 300, the haptic module 310 The display 311, the optical output 312, the microphone 313, the speaker 314 and the power supply 315 are connected to the lens 204, the on-board computer 205 A camera 206, a user input 207, sensors 208, 209 and 210, a haptic module 211, a display 212, an optical output 213, a microphone 214, speakers 215 and 216 And power supply 217, and may be similarly configured.

The HMD device 100 used in the preferred embodiments may be configured as the HMD device 400 shown in FIG. The HMD device 400 shown in FIG. 4 is configured in the form of a biometric display. The HMD device 400 may be attached to a helmet structure or deformed into a goggle shape.

4 is an external view of an HMD device according to still another embodiment. The HMD device 400 shown in FIG. 4 may be configured as a biometric display. The HMD device 400 may be attached to a helmet structure or deformed into a goggle shape.

The HMD device 400 shown in Figure 4 includes a lens frame 401, side-arms 402 and 403, lenses 404 and 405, an on-board computer 406, A camera 407, a user input 408, sensors 409, 410 and 411, a haptic module 412, displays 413 and 414, a microphone 415, speakers 416 and 417, (418).

The lens frame 401 included in the HMD device 400 shown in FIG. 4 is configured integrally with the nose pad 401 '. The lens frame 401 may have various shapes such as a rectangular shape, an elliptical shape, and a circular shape, but the structure of the lens frame 401 is not limited thereto. The lens frame 401 may be made of a material similar to the lens frame 201 mentioned in Fig.

Lenses 404 and 405 may be constructed of any material suitable for displaying information, such as images or graphics, projected through an unillustrated light output device (the projector referred to in FIG. 2). Lenses 404 and 405 are used to provide information such as images or graphics that are projected through a real world that a user of HMD device 400 can view through lenses 404 and 405 and an optical printer (not shown) Or the like. The material constituting the lenses 404 and 405 may be composed of plastic or glass, but not limited thereto, such as the material constituting the lens 204 mentioned in Fig.

Side-arms 402, 403, an on-board computer 406, a camera 407, a user input 408, sensors 409, 410, 411 ), The haptic module 412, the microphone 415, the loudspeakers 416 and 417 and the power supply 418 are connected to the side-arms 202 and 203 shown in Fig. a keyboard 206, a user input 207, sensors 208, 209 and 210, a haptic module 211, a microphone 214, speakers 215 and 216, But the functions and configurations may be similar, but are not limited thereto.

The displays 413 and 414 shown in Fig. 4 are used to display information such as images or graphics projected by the above-described optical printers (not shown) which can be mounted on the inner surface of each of the side legs 402 and 403 Can be displayed.

The HMD device 400 shown in FIG. 4 may be modified to operate the lenses 404, 405 as a display. In this case, the lenses 404 and 405 may be composed of a transparent display or a translucent display. When the lenses 404 and 405 are configured as a translucent display, the lenses 404 and 405 may include at least one optical waveguide (e.g., a prism), an electroluminescent display, or a liquid crystal display ), And the like, but is not limited thereto.

5 is a block diagram showing a circuit structure of an HMD device according to some embodiments.

The HMD device 500 according to some embodiments includes an input / output unit 510, a memory 520, a sensor unit 525, a battery 530, a power management unit 535, a communication unit 540, a touch sensor 550 A camera 560, a display unit 570, a lens 580, and a control unit 590. [0041]

The input / output unit 510 includes means for receiving a user's input, informing a user of the information, receiving data from the outside, or outputting data to the outside. The input / output unit 510 may include at least one speaker 511, at least one microphone 512, at least one button, a connector, a keypad, or a combination thereof. However, the embodiment of the present invention is not limited thereto.

The speaker 511 can output sound corresponding to various data to the outside of the HMD device 500 under the control of the control unit 590. [ Also, the speaker 511 can output sound corresponding to the function performed by the HMD device 500. [ The speakers 511 may be disposed at one or more positions in the HMD device 500 at appropriate positions or positions. According to some embodiments, the speaker 511 may be implemented in the form of an earphone, as shown in FIG.

The microphone 512 receives a voice or sound from the outside of the HMD device 100 and generates an electric signal based on the received voice or sound, To the control unit 590. The microphones 512 may be placed in one or more positions at appropriate positions or positions of the HMD device 100. Throughout this specification, a signal can be represented as data, and the data can also be represented as a data signal.

The sensor unit 525 may include at least one sensor for detecting the state of the HMD device 500 or the state of the surrounding environment. For example, the sensor unit 225 may include a proximity sensor for detecting whether or not the user accesses the HMD device 500, a proximity sensor for detecting the operation of the HMD device 500 (e.g., rotation, acceleration, deceleration, A light / darkness sensor for detecting a light / darkness, a light / darkness sensor for detecting an ambient illuminance, a light / darkness sensor for detecting a color or spectrum of light, or a combination thereof. Also, the motion / orientation sensor may include at least one of an acceleration sensor, a gravity sensor, a geomagnetic sensor, a gyro sensor, a shock sensor, a GPS, a compass sensor, and an acceleration sensor. For example, the GPS module receives radio waves from a plurality of GPS satellites on a global orbit and uses the time of arrival from the GPS satellite to the HMD device 500 to determine the location of the HMD device 500 Can be calculated.

The power management unit 535 can supply power to the HMD device 500 under the control of the control unit 590. [ The power management unit 535 may be connected to one or more batteries 530. In addition, the power management unit 535 can supply the HMD device 500 with power input from an external power source through the wired cable.

The communication unit 540 may be a wired or wireless communication unit or a wired or wireless communication unit. The communication unit 540 may transmit data from the control unit 590 to the external device via an external communication network or the air in a wired or wireless manner, Or wirelessly, and may transmit the received data to the control unit 590. According to an embodiment, the communication unit 540 may include at least one of a mobile communication module, a wireless LAN module, and a local area communication module.

The mobile communication module can perform communication with an external device through the mobile communication network using at least one antenna under the control of the controller 590. [ The mobile communication module transmits or receives wireless signals for voice calls, video calls, short messages (SMS), or multimedia messages (MMS) with cellular phones, smart phones, tablet PCs or other communication devices having network addresses or telephone numbers such as IP .

The wireless LAN module may be connected to the Internet through a wireless AP in the vicinity of a wireless access point under the control of the controller 590. [ The wireless LAN module can support the IEEE 802.11x standard of the Institute of Electrical and Electronics Engineers (IEEE).

The short-range communication module can perform short-range communication with an external communication device wirelessly under the control of the controller 590. Short-range communication method, for example, be a Bluetooth ^TM (bluetooth ^TM), infrared data association (IrDA, Infrared Data Association), Wi-Fi Direct (Wi-Fi direct) communication, (Near Field Communication) NFC, or a combination thereof.

The touch sensor 550 may transmit a signal corresponding to at least one touch input to the controller 590. The user can touch the touch sensor 550 through a part of the body (e.g., finger) or other touch input means, and the touch sensor 550 can receive the touch input by sensing the touch of this user . In addition, the touch sensor 550 can receive an input (e.g., a drag input) in accordance with the continuous movement of the touched position. The touch input information may include touch coordinates and / or a touch state. The contact state may be a mouse-down state in which a body or the like is in contact with the touch sensor 550, a mouse-up state in which the body or the like is separated from the touch sensor 550, or a drag state in which the touch sensor 550 is in contact with the touch sensor 550 have. The control unit 590 can grasp the user input information such as menu item or item selection or movement, handwriting input based on the touch input. The control unit 590 may perform functions corresponding to the touch input (e.g., telephone connection, camera shooting, menu selection, message creation / viewing, data transmission, and the like).

However, in this specification, the touch input is not limited to the input using touch of the touch sensor 550 and the touch input means. The touch input may include an input using non-contact (e.g., a state in which the touch sensor 550 and the touch input means are spaced from each other). This non-contact based input may also be referred to as a hovering input.

The touch sensor 550 may be a resistive type, a capacitive type, an infrared type, an acoustic wave type, an electromagnetic resonance (EMR), or a combination thereof. have.

The camera 560 includes a lens system and an image sensor, and may further include a flash lamp. The camera 560 may convert the light inputted (or photographed) through the lens system into an electrical image signal, and output the converted image signal to the control unit 590. The user can shoot a moving image or a still image using the camera 560. [ The camera 560 may also be used to receive input via the user's motion or gesture.

The lens system can form an image of a subject by converging light input from the outside. The lens system includes at least one lens, and each lens may be a convex lens, an aspherical lens, or the like. The lens system has symmetry with respect to the optical axis passing through its center, and the optical axis can be defined as the central axis of the lens system. The image sensor can convert the optical image formed based on the external light input through the lens system into an electrical image signal. The image sensor may have a plurality of pixel units arranged in an M X N matrix structure. The pixel unit may include a photodiode and a plurality of transistors. The pixel unit accumulates the charges generated by the input light, and the voltage due to the accumulated charges can indicate the illuminance of the input light. In the case of processing an image constituting a still image or a moving image, an image signal outputted from the image sensor is composed of a set of voltages (i.e., pixel values) output from the pixel units, (I.e., a still image). Also, the frame can be composed of M X N pixels. A charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, or the like can be used as the image sensor.

The image sensor can operate only the pixels of the ROI among all the pixels of the image sensor or all of the pixels according to the control signal received from the controller 590. [ The image data output from the pixels may be output to the control unit 590.

The control unit 590 processes the image formed by the control unit 590 in units of frames using an image input from the camera 560, an image stored in the memory 520, or data stored in the memory 520 . The control unit 590 may output the frame of the image to the outside through the display unit 570 or store it in the memory 520. In this specification, the image output through the display unit 570 may refer to visual content such as a moving image or a still image, or a GUI (Graphic User Interface).

The display unit 570 may be implemented in various forms according to embodiments. For example, the display unit 570 may display an image by causing an image of an image to be formed on a user's retina using an optical system. Alternatively, the display portion 570 includes a transparent display, and may display an image on the transparent display.

The lens 580 according to some embodiments may be provided in a detachable structure from the HMD device 500 as shown in FIG. That is, the lens 580 may not be included in the HMD device 500, and the HMD device 580 may include a structure for attaching the lens 580 in place of the lens 580. The user can change the color of the lens 580 of the HMD device 500 by replacing the lens 580. [ Alternatively, the lens 580 according to some other embodiments may be a smart window in which the transmittance or color can be changed under the control of the control unit 590.

6 is a conceptual diagram illustrating a method of displaying an image by the HMD device 600 according to some embodiments. The HMD device 600 according to some embodiments may include a display portion 570 including an optical system. The display unit 570 according to some embodiments may display an image by transmitting the lens 580 and causing the retina of the user's eye 610 to form a phase.

7 is an external view illustrating a portion of an HMD device according to some other embodiments.

The HMD device 500 may include a frame 720 to which the lens 580 may be secured. According to some embodiments, frame 720 may have a structure in which two frames are hingedly connected to hinge 722. An opening 711 through which the projection light 710 output from the projector included in the display portion disposed in the frame 720 is output to the outside of the frame 720 may be disposed on the inner side of the frame 720 have. Here, the opening 711 may be provided with glass or transparent plastic for blocking dust from the outside from flowing into the frame 720.

Also, at least one speaker 211 may be disposed on the inner surface of the frame 720.

The projector outputs the projection light 710 for forming a virtual image, and the projection light 710 output from the projector is continuously reflected by the lens 580 and the focused and reflected projection light 730 is reflected by the user A virtual image can be formed on the retina of the eye 610. Here, focusing can mean collecting light. Focusing may include convergence to gather light at a point, or reduction of beam spot of light. The reflected projected light 730 can converge to the lens of the eye 610 or the pupil. The HMD device may use a plurality of projectors in order to form a virtual image in each eye of the user.

8 is a conceptual diagram showing a structure of a display unit according to some embodiments. More specifically, FIG. 8 is a conceptual diagram showing the structure of the projector when the display unit includes the projector.

When the display unit 570 includes a projector, the display unit 570 includes a light source 810, a roughing optical system 820 for roughening the display element 840 through light output from the light source 810, A display 840 that reflects the light reflected by the mirror 830 on a pixel-by-pixel basis to form a virtual image, and a display 840 that reflects light reflected from the display 840. [ And a projection optical system 850 for projecting the projection optical system 850 to the outside.

The illumination optical system 820 can be disposed with respect to the first optical axis 571 parallel to the X axis. The illumination optics 820 may include at least one collimating lens, at least one filter, at least one equalization lens, a condensing lens, or a combination thereof.

The optical elements (lens, prism, or filter, etc.) of the illumination optical system 820 can be aligned with respect to the first optical axis 571. [ Normally, the optical axis may mean an axis in which no optical change occurs in the optical system even if the optical system is rotated around the optical axis. Alignment on the optical axis means that the center of curvature of the optical element constituting the optical system is located on the optical axis, or that the symmetry point (i.e., the center of symmetry) of the optical element or the center point is located on the optical axis.

The light source 810 may output light traveling along the first optical axis 571. For example, at least one LED that outputs a combination of white light, primary light (e.g., blue light or green light), or primary light may be used as the light source 810.

The illumination optical system 820 may collimate, filter and / or focus the light input from the light source 810, and output the processed light to the mirror 830.

The mirror 830 can reflect the input light transmitted through the illumination optical system 820 to the display element 840 side. The mirror 830 may have a structure in which a highly reflective dielectric layer or a metal layer is deposited on the substrate.

The display element 840 can display an image on a pixel-by-pixel basis based on the data input from the control unit 590. [ The display element 840 may have pixel elements corresponding to a predetermined resolution. The display element 840 displays an image through on / off driving of the pixel elements. For example, a DMD (Digital Micro-Mirror Device) including micromirrors arranged in a matrix structure of MxN (for example, 1280x720, 854x480, etc.) may be used as the display element 840. Each micromirror can be rotated to a position corresponding to the ON state and a position corresponding to the OFF state according to the driving signal. Each micromirror can reflect incident light at an angle that can be displayed outside when it is on, and can reflect light incident at an angle that is not displayed outside when it is off.

The projection optical system 850 can be disposed with respect to the second optical axis 572 parallel to the Y axis. The projection optical system 850 may include a relay lens 860 and a projection lens 870. The relay lens 860 and the projection lens 870 may be aligned with the second optical axis 572. [

The relay lens 860 may allow light reflected from the mirror 830 to be matched to the display element 840 in consideration of overfill. That is, the relay lens 860 can allow light reflected from the mirror 830 to be incident on an area larger than the area occupied by the pixel elements of the display element 840.

Further, the relay lens 860 receives the light reflected from the display element 840, and can reduce the beam area of the light and output it. The light reflected from the display element 840 has a large beam spot size, so that light loss that can not be transmitted to the projection lens 870 may occur. The relay lens 860 condenses the light reflected from the display element 840 to reduce a beam area, thereby allowing a larger amount of light to be transmitted to the projection lens 870.

The projection lens 870 receives the light passing through the relay lens 860, and can collimate or focus the received light and project it to the outside.

9 is a conceptual diagram for explaining the configuration and function of a lens capable of changing the hue or transmittance according to some embodiments.

The lens 580 according to some embodiments may change color or transmittance. The lens according to some embodiments may include a glass 581 capable of controlling the transmittance, i. E. Changing the hue or transmittance according to the applied signal, and a holographic optical element (HOE, 582) have.

The lens 580 transmits the ambient light 911 input from the outside and reflects and focuses the light 921 outputted from the display unit 570.

The light 921 reflected by the transmitted ambient light 912 and the lens 580 is input to the user's eye 610 and the retina 613 of the eye 610 is illuminated by the ambient light 912 The image 910 of the displayed image 920 and the displayed image 920 of the displayed image may be superimposed. That is, the user sees an image in which the image 910 of the surrounding environment and the displayed image 920 are superimposed. The image 920 displayed to the user may be perceived as floating in a transparent layer on the image 910 of the ambient environment. The displayed image 920 may include a GUI including a menu or an icon such as a phone, a contact, a message, a main menu, etc., or a content image including a still image or a moving image. In FIG. 9, the area where the displayed image 920 is displayed is opaque, but it may be displayed so that the user can see the image 910 of the surrounding environment in the area where the image 920 is displayed. That is, the transparency in the area where the image 920 is displayed may be changed according to the embodiment.

The light 921 output from the display unit 570 may be a parallel light having a wavelength? (I.e., collimated light). The light 921 may be incident on the holographic optical element 582 to form an angle? With the normal of the holographic optical element 582. The holographic optical element 582 may be an element having wavelength selectivity. The holographic optical element 582 reflects and focuses the light 921 in this example and the light of a different wavelength from the wavelength lambda (i.e., the ambient light 911 in this example) Can be transmitted as it is without convergence.

The holographic optical element 582 may reflect and focus the input light 921. [ The holographic optical element 582 also includes a plurality of optical elements 612 that are arranged such that the light 922 reflected from the holographic optical element 582 is transmitted from the holographic optical element 582 to the eye 610 at a certain distance, As shown in FIG. Preferably, the reflected light 922 can be converged to the position of the pupil 611 of the eye 610 or the lens 612. Here, the light 922 to be converged may have a converging angle or a viewing angle?. The lens 612 functions to adjust the focus of the light incident on the eye 610 and the transmitted ambient light 912 is converged at the position of the pupil 611 or the lens 612, The image 920 can be formed by being projected onto the retina 613 without convergence.

Although the light 921 incident on the holographic optical element 582 is illustrated as a parallel tube, it is not limited thereto. The light 921 may be divergent light or convergent light when the light 922 reflected by the holographic optical element 582 converges to the position of the pupil 611 or the lens 612. [

When the user wearing the HMD device focuses the eyes to see the actual object in the surrounding environment, there is a problem that the image (virtual object or virtual object) displayed by the HMD device is not formed clearly in the user's retina . In this case, since the user can not see the real object and the virtual object at the same time, it is difficult to obtain the effect of the augmented reality. In addition, the problem of the focus mismatch between the actual object and the displayed image can increase the fatigue of the user due to the mismatch between the binocular vision convergence and the monocular focus control.

The HMD device according to some embodiments can minimize the amount of focus change of the projection light by the lens by converging the projection light forming the virtual image to the position of the pupil or the lens or the position close thereto. When the projection light converges to the position of the pupil or lens, the projection light can be projected directly onto the retina regardless of the action of the lens. The image projected on the retina can be perceived by the user as a sharp image regardless of the aberration caused by the focus control of the lens and the lens.

The holographic optical element 582 may have a focal length corresponding to the distance between the pupil 611 and the lens 612 and the holographic optical element 582. [

10 is a conceptual diagram showing the configuration of the glass 581 included in the lens of the HMD device according to some embodiments. The glass 581 according to some embodiments can be changed in transmittance or color by a signal or voltage applied by the control unit 590. [

An electrochromic glass, an SPD (Suspended Particle Device), an LC (Liquid Crystal), or the like can be used as the glass 581. Alternatively, a photochromic glass or a thermochromic glass may be used as the glass 851 in which the transmittance is changed according to the wavelength or temperature of light, rather than the active control according to the signal application.

The glass 581 can be manufactured in various ways. For example, the glass 581 can be manufactured by applying a material capable of adjusting the transmittance to glass, or attaching a thin film (transmittance controllable thin film) to the glass.

In the following description of Fig. 10, the case where the electrochromic glass is used as the glass 581 will be described.

The glass 581 includes first and second insulating substrates 1010 and 1015 and a conductive first electrode 1020 laminated on the upper surface of the first substrate 1010. The first and second substrates 1010 and 1015 An insulating spacer 1030 which separates the first and second substrates 1010 and 1015 from each other and seals a space therebetween, a first and a second substrate 1015, The electrochromic layer 1040 and the electrolyte 1050 filled in the space between the electrodes 1030 and 1030).

Each substrate 1010, 1015 can be made of clear glass or plastic. For example, the plastic may comprise at least one of polyacrylate, polyethylene ether phthalate, polyether linaphthalate, polycarbonate, polyarylate, polyetherimide, polyether sulfone and polyimide.

The first electrode 1020 may be made of a transparent conductor. For example, the first electrode 1020 may include an inorganic conductive material such as indium tin oxide (ITO), fluorine tin oxide (FTO), or antimony doped tin oxide (ATO) Or an organic conductive material such as polyacetylene or polythiophene.

The second electrode 1025 may be made of a transparent or opaque conductive material. For example, the second electrode 1025 may comprise a metal such as indium tin oxide (ITO), fluorine containing tin oxide (FRO), Al, antimony doped tin oxide (ATO) have.

An electrochromic layer 1040 including an electrochromic material may be disposed on the first electrode 1020. The electrochromic layer 1040 may be disposed on the first electrode 1020 in the form of a film. But are not limited to this.

The first substrate 1010 and the second substrate 1015 may be fixed to the spacer 1030. An electrolyte 1050 may be filled between the first substrate 1010 and the second substrate 1015. The electrolyte (1050) can combine the redox materials that react with the electrochromic material. The electrolyte 1050 may be a liquid electrolyte or a solid polymer electrolyte. As the liquid electrolyte, for example, a solution in which a lithium salt such as LiOH or LiClO ₄ , a potassium salt such as KOH and a sodium salt such as NaOH are dissolved in a solvent may be used. As the solid electrolyte, for example, poly (2-acrylamino-2-methylpropane sulfonic acid), polyethylene oxide, or the like may be used. However, the present invention is not limited thereto.

The material forming the electrochromic layer 1040, that is, the electrochromic material, may include a metal-organic composite having a metal and an organic compound having a functional group capable of forming a coordination with the metal. The metal may include light metals, transition metals, lanthanide metals, alkali metals or combinations thereof. For example, the metal may be selected from the group consisting of Beryllium (Be), Ba, Cu, Zn, Ce, Mg, Al, Ti, . Also, for example, the functional group may include a carboxyl group, a pyridine group, an imidazole group, or a combination thereof. Also, for example, the organic compound may include a viologen derivative, an anthraquinone derivative, or a combination thereof.

11 is a conceptual diagram illustrating a method of displaying an image by an HMD device according to some other embodiments.

According to some other embodiments, the display portion 570 may be arranged to be positioned between the lens 580 and the user's eyes, as shown in Fig. 11 (a). The display unit 570 may be configured using a transparent display through which light can be transmitted.

The light 1110-1 generated by the surrounding environment may be incident on the user's eye through the lens 580 and the display unit 570 and may be incident on the light 1120-1 generated by the display unit 570, Can be incident on the user's eyes directly without passing through the lens 580. [

Alternatively, the lens 580 may be disposed between the display portion 570 and the lens 580 as shown in FIG. 11 (b). In this case, the light 1110-2 generated by the surrounding environment can be transmitted through the lens 580 and the display unit 570 and incident on the user's eyes. The light 1120-2 generated by the display unit 570 can be transmitted through the lens 580 and incident on the user's eye. In this case, the light 1110-2 generated by the surrounding environment and the light 1120-2 generated by the display unit 570 both pass through the lens 580 and are incident on the user's eye. In this case, the HMD The device can correct the image in consideration of the color recognized by the user by the light generated by the display unit 570 through the lens 580. [

Alternatively, the display portion 570 and the lens 580 may be a display portion / lens 570-1 integrally formed as shown in FIG. 11 (c). In this case, the color or transparency of the display unit / lens 570-1 can be controlled and can be configured as a smart window in which an image can be displayed. Light 1110-3 generated by the surrounding environment can be incident on the user's eye through the display unit / lens 570-1. In addition, the light 1120-3 generated by the display unit / lens 570-1 can be directly incident on the user's eyes.

12 is a flow chart illustrating a process by which an HMD device displays an image in accordance with some embodiments.

The HMD device can acquire color information about the lens (S1210). The color information of the lens may mean identification information for the HMD device to identify the color of the lens. For example, the color information of the lens may be an RGB value representing the color of the lens. The color information may be modified according to the embodiment. For example, according to some embodiments, the color information may be information indicating the color of the lens itself. Or according to some other embodiments, the color information may indicate to the user the color recognized as the background color around the image displayed by the HMD device. In this case, by detecting the color of the light transmitted through the lens using the photosensor provided in the HMD device, the HMD device can acquire the color information indicating the color recognized as the background color. In this case, the color information indicating the color recognized as the background color may be referred to as color information of the image display area.

The manner in which the HMD device acquires color information for the lens can be implemented using various methods. For example, when the lens is replaceable, the HMD device can acquire color information of the replaced lens when the lens is replaced. In another example, if the lens is a smart window that can change the transmittance or hue according to the control of the HMD device, the HMD device can control the hue of the lens so that the current hue of the lens can be known.

Thereafter, the HMD device modifies the image based on the color information obtained in step S1210, and displays the modified image through the display unit (S1220). For example, when the HMD device tries to display an image containing text, and the color information of the obtained lens indicates yellow, the HMD device converts the text into a blue complementary color of indigo And the modified text can be displayed on the display unit.

If the lens is detachable to the HMD device or the HMD device is capable of controlling the color of the lens, the HMD device may acquire the color information of the lens through step S1210. However, when the lens is integrally formed with the HMD device and the color of the lens can not be changed, the color information of the lens may be preset. That is, when the fixed color lens is integrated with the HMD device, the HMD device can correct the color of the image based on the color information of the predetermined lens. For example, when the blue lens is fixed to the HMD device, the HMD device can correct the color of the image to red, which is a complementary color of blue, and display the corrected image without acquiring color information for the lens separately.

13 is a conceptual diagram showing a structure in which an image is corrected and displayed according to some embodiments.

The HMD device according to some embodiments may include a color correction module 1330. The color correction module 1330 may output the correction image 1330 in which the color is changed from the original image 1320 based on the color information 1310 of the lens and the original image 1320. [ Here, the color correction module 1330 according to some embodiments may be implemented using the control unit 590. [ For example, the color correction module 1330 may include a processor loaded with a program for generating the corrected image 133. [

The color correction module 1330 may modify the original image 1320 in various manners according to an embodiment. For example, if the original image 1320 includes text, the color correction module 1330 may modify the original image 1320 so that the text is displayed in a complementary color of the color of the lens.

However, the present invention is not limited to the modification of only the color of the original image 1320 of the color correction module 1330. For example, the color correction module 1330 may enlarge or reduce the size of an object (e.g., a GUI object such as an icon) included in the original image 1320. [ As another example, the color correction module 1330 may change the shape of an object (e.g., a GUI object, such as an icon) included in the image 1320.

If the lens is constructed integrally with the HMD device and the color of the lens can not be changed then the color information 1310 is not separately input to the color correction module 1330 and may be pre-set in the color correction module 1330 have. The color correction module 1330 can correct the image based on the preset color information. That is, when the fixed color lens is integrally formed with the HMD device, the color correction module 1330 can correct the color of the image based on the color information of the predetermined lens. For example, when the blue lens is fixed to the HMD device, the color correction module 1330 does not separately acquire color information for the lens but modifies the color of the image to red, which is a complementary color of blue, can do.

The display unit 570 may display the corrected image 1330 output from the color correction module 1330. [

The HMD device may obtain color information for the lens in various ways, depending on the embodiment. For example, the HMD device can acquire color information about the lens using a metal wire provided on the lens, a cord displayed on the lens, a photosensor, or an image photographed by the camera. Here, acquiring the color information about the lens may include the case where the color information of the lens is inputted from the user or the HMD device can recognize the color information of the lens as the HMD device controls the color or transmittance of the lens have.

14 is a flow chart illustrating a process for obtaining color information for a lens in accordance with some embodiments.

The HMD device can acquire the identification information of the lens (S1410). Here, the identification information of the lens means information for the HMD device to identify the lens. For example, the identification information of the lens may be a code assigned according to the type of the lens. The manner in which the HMD device acquires the identification information of the lens can be variously implemented according to the embodiment. For example, when the lens is provided with the metal line, the identification information of the lens can be obtained by using the position where the metal line is disposed. As another example, when a code (e.g., a bar code or a Quick Response Code) is displayed on the surface of the lens, the HMD device can obtain the identification information of the lens from the code displayed on the lens.

Here, the identification information of the obtained lens may be information given according to the color of the lens. For example, the identification information may be " 01 "for a black lens and the identification information may be" 00 "for a transparent (colorless) lens. In this case, the HMD device can acquire color information on the lens corresponding to the obtained identification information (S1420).

Thereafter, the HMD device may modify the image to be displayed based on the obtained color information (S1430). The HMD device may modify the image in various ways, depending on the embodiment. For example, the HMD device can modify the image so that the text contained in the image is displayed as a complementary color of the lens color.

However, the embodiment of the present invention is not limited to the HMD device modifying the color of an image. For example, an HMD device can enlarge or reduce the size of an object (e.g., a GUI object such as an icon) included in an image. As another example, an HMD device may change the shape of an object included in an image (e.g., a GUI object such as an icon).

Thereafter, the HMD device can display the modified image (S1440).

15 is an outline view showing the structure of an HMD device for identifying a lens using a metal line according to some embodiments.

The HMD device 1500 according to some embodiments may include a display portion 1511 and an electrode 1532 located on one side of the frame 1510. [ The HMD device 1500 according to some embodiments can detach and attach the lens 1520. [ Here, the detachable lens 1520 may include metal lines 1531. Here, the electrodes 1532 and the metal wires 1531 can be disposed at positions where the lenses 1520 can contact each other when the lenses 1520 are attached to the frame 1510. When the lens 1520 is attached to the frame 1510, the HMD device 1500 can sense the position of the metal line 1531 using the electrode 1532. [ The HMD device 1500 may obtain the identification information of the lens 1520 based on the position of the sensed electrode 1531. [ According to some embodiments, the lens 1520 can be secured to the frame 1510 by having a magnet 1540 on one side of the frame 1510. [

16 is a view showing a part of an HMD device and a lens including a structure of a metal wire and an electrode according to some embodiments.

According to some embodiments, the detachable lens 1520 may include a metal wire 1531. Here, the metal wire 1531 may be disposed at a unique position assigned according to the type of the lens 1520. When the lens 1520 is attached to the frame 1510, the metal wire 1531 can contact the electrode 1532. [

When the metal wire 1531 contacts the electrode 1532, the HMD device 1500 can recognize the position where the metal wire 1531 is disposed. The HMD device 1500 can obtain the identification information of the lens 1520 corresponding to the position where the metal line 1531 is disposed.

17 is a conceptual diagram illustrating a method of identifying a lens using a code displayed on the lens in accordance with some embodiments.

According to some embodiments, the HMD device 1700 can obtain the identification information of the lens 1720 using the code 1721 provided in the lens 1720. [ 17, the code 1721 is shown on the front of the lens 1720, but the position of the code 1721 can be changed according to the embodiment.

According to some embodiments, the code 1721 may refer to a mark or the like marked by the HMD device 1700 to be optically readable. For example, the code 1721 may be a QR code, a bar code, or the like. In this case, the HMD device 1700 can use the camera 1710 to capture the position at which the code 1721 is displayed, and obtain the identification information of the lens 1720 from the captured image.

18 is a flow chart illustrating a process for acquiring color information of a lens using a photosensor in accordance with some other embodiments.

The HMD device according to some embodiments may include a photosensor. The photosensitive sensor means a sensor capable of detecting light by responding to radiation energy such as light. The HMD device can detect and analyze light transmitted through the lens using a photosensor (S1810).

Thereafter, the HMD device may determine color information for the lens based on the analysis result (S1820). For example, an HMD device can analyze the spectrum of light detected using a photosensor. When the ratio of the blue light to the detected light is high, the HMD device can obtain the color information corresponding to the blue color with respect to the lens.

Once the color information for the lens is obtained, the HMD device can modify the image based on the obtained color information (S1830) and display the modified image (S1840).

19 is a conceptual diagram showing a method of acquiring color information for a lens using a photosensor according to some other embodiments.

The HMD device according to some embodiments may include a photosensor 1930 in a position capable of receiving light generated from a light source 1910 located opposite the user relative to the lens 1920. For example, referring to FIG. 19, a photosensor 1930 may be disposed on one side of the frame of the HMD device. The HMD device according to some embodiments can obtain color information about the lens as a result of analyzing the light transmitted through the lens 1920.

20 is a flow chart illustrating a process for obtaining color information for a lens based on a size of a user ' s pupil in accordance with some further embodiments.

According to some embodiments, the HMD device may capture an image using a camera (S2010). Here, the photographed image may include an image of the user's eyeball.

The HMD device can perform image recognition on the photographed image. The HMD device can acquire the size value of the pupil included in the image as a result of performing image recognition. Based on the acquired pupil size value, the pupil size change included in the image can be determined (S2020).

The HMD device may then determine color information for the lens based on the pupil size change (S2030). When the transmittance of the lens is high, the size of the user's pupil can be reduced as much light is incident on the user's eyes. On the contrary, when the transmittance of the lens is low, the amount of the user's pupil can be enlarged because less light is incident on the user's eyes. Based on this principle, the HMD device can obtain color information about the lens based on the size of the user's pupil. That is, when the size of the user's pupil is large, the HMD device can determine that the color of the lens is bright. Conversely, when the size of the user's pupil is small, the HMD device can determine that the color of the lens is dark.

Once the color information for the lens is obtained, the HMD device can modify the image based on the obtained color information (S20400) and display the modified image (S2050).

21 is a conceptual diagram illustrating a method for acquiring color information based on the size of a pupil of a user in accordance with some further embodiments.

Referring to FIG. 21, the HMD device 2100 may photograph a user's eye 2120 using a camera 2110 provided at one side of the HMD device 2100. In FIG. 21, it is shown that the lens is disposed between the camera and the user, but the position of the camera can be variously changed according to the embodiment. The HMD device 2100 can detect the size of the pupil from the image of the user's eyeball 2120. The HMD device 2100 may obtain color information of the lens based on the detected pupil size.

22 is a conceptual diagram showing a method of acquiring color information for a lens based on a setting of a user according to another embodiment.

According to some embodiments, the user can manually set the color information of the lens 2220. [ Referring to FIG. 22, the HMD device 2200 may display a user interface 2210 for setting color information of the lens 2220. The user interface 2210 may include, for example, a list of color information.

The HMD device may receive input from the user to select any of the color information 2220 of the lens through the input / output portion. For example, referring to FIG. 22, when brown 2211 is selected through the user interface 2210, the HMD device can acquire RGB values corresponding to brown with color information on the lens.

23 is a flowchart illustrating a process by which an HMD device displays an image according to still another embodiment. 24 is a conceptual diagram showing a method of displaying an image by the HMD device according to still another embodiment.

 According to some embodiments, the HMD device may be a passive device that is incapable of performing image processing. In this case, the HMD device can modify the image using an active device capable of communicating with the HMD device.

Referring to FIGS. 23 and 24, the HMD device 2300 can acquire lens identification information (S2310). Then, the HMD device 2300 can transmit the obtained lens identification information to the control terminal 2400, which is the active device.

Thereafter, the control terminal 2400 may correct or generate an image to be displayed on the HMD device 2300 based on the identification information received from the HMD device 2300 (S2330).

25 is a flowchart illustrating a process in which an HMD device recommends a lens according to some embodiments.

First, the HMD device can acquire information related to the external environment (S2510). The information related to the external environment may be information indicating a state around the HMD device detected through at least one sensor. For example, the information related to the external environment may include information about illuminance around the HMD device, GPS (Global Positioning System) coordinates for the position of the HMD device, or information about sounds sensed by the HMD device.

Thereafter, the HMD device may determine the color information of the external environment corresponding to the information related to the external environment obtained in step S2510 (S2520). According to some embodiments, the color information of the external environment may be information pertinent to the external environment, such as information about the illuminance around the HMD device. For example, the HMD device can photograph the outside of the HMD device using the camera 560. [ The HMD device can acquire information related to the external environment from the photographed image. In this case, step S2520 may be omitted. Alternatively, according to some other embodiments, the color information of the external environment may be based on information about illuminance around the HMD device, GPS (Global Positioning System) coordinate values for the location of the HMD device, or information about sounds sensed in the HMD device And may be a color code selected from the color information included in the color DB (not shown) provided inside or outside the HMD device. The HMD device may recommend a lens corresponding to the determined color information of the external environment (S2530). Quot; recommending a lens "may mean, for example, that the HMD device displays a message to replace the lens. For example, if the illuminance obtained in step S2510 is a low value, the HMD device may recommend a colorless lens. Conversely, when the illuminance obtained in step S2510 is high, the HMD device can recommend a colored lens such as brown, blue, black, and the like.

FIG. 27 is an exemplary diagram showing a method by which the HMD device 2700 recommends a lens. Referring to FIG. 27, the HMD device 2700 may display a message 2710 recommending a selected lens based on information related to the external environment. Here, the HMD device 2700 can display the message 2710 so that the user can recognize that the message 2710 is displayed on the lens 2720.

If the HMD device is capable of controlling the color or transmittance of the lens 2720, the step S2530 may be based on the color information of the external environment determined in the step S2520, As shown in FIG.

Figure 26 is a flow chart illustrating a process by which an HMD device recommends a lens in accordance with some other embodiments. Figures 28-31 are illustrations showing how an HMD device recommends a lens according to some embodiments.

The HMD device can select the content that can be output through the display unit (S2610). For example, the HMD device may display the content list and may select the content from the content list according to the user input received through the input / output unit of the HMD device. Here, the content list may include executable applications, multimedia content such as photographs or moving pictures, and the like.

For example, referring to FIG. 28, an HMD device 2700 may display a content list 2810 on a lens 2720. The HMD device 2700 can select any one of the contents included in the contents list 2810. [ As shown in Fig. 28, the HMD device 2700 can select the web browser 2811 from the content list 2810. [

Thereafter, the HMD device may recommend a lens corresponding to the content selected in step S2610 (S2620). 29, the HMD device 2700 displays a message 2920 recommending a lens having an appropriate hue in using the web browser 2811, when the web browser 2811 is selected, on the lens 2820. For example, Can be displayed. Further, the HMD device 2700 can display a web browser execution screen 2910. [

Thereafter, it may be determined whether the recommended lens is coupled to the HMD device (S2630). In step S2630, the HMD device can determine whether a recommended lens is coupled to the HMD device in various ways. For example, the HMD device may obtain the identification information of the lens and determine whether the obtained identification information matches the identification information of the recommended lens. As another example, the HMD device can acquire the color information of the lens and determine whether the color information of the obtained lens and the color information of the recommended lens coincide with each other. As shown in FIG. 30, when the lens 2820 is replaced with a lens 2821 having a different color, the HMD device can determine whether or not the recommended lens is coupled to the HMD device. If the recommended lens is not coupled to the HMD device, the HMD device can perform step S2620 again.

If the HMD device is capable of controlling the color or transmittance of the lens 2820, steps S2620 and S2630 may be performed by the HMD device 2820 in accordance with the selected content depending on the transmittance and color of the lens 2820, And controlling the lens 2820 so as to change the transmittance to the hue.

Thereafter, the HMD device may display the selected content on the lens (S2640). The content displayed in step S2640 may be a modified image based on the color information of the changed lens in steps S2620 and S2630. As shown in Fig. 31, the HMD device 2700 can display a web browser execution screen 2911 in which the color is modified on the lens 2821 whose color has been changed.

32 is a flowchart illustrating a process by which an HMD device capable of changing the color or transmittance of a lens displays an image according to some embodiments.

The HMD device according to some embodiments may control the color or transmittance of the lens (S3110). For example, the HMD device can measure the illuminance using an illuminance sensor and control the transmittance of the lens based on the illuminance measured. However, this is not limited. As another example, the HMD device can photograph the surrounding environment using a camera, and measure the illuminance based on the photographed image. Alternatively, the illuminance around the HMD device may be measured using information indicating the position of the HMD device. If the measured illuminance is low, the transmittance of the lens can be increased. Conversely, if the measured illuminance is high, the transmittance of the lens can be reduced to protect the user's eyes.

The lens is composed of an electrochromic glass, an LC (Liquid Crystal) panel, or an SPD (Suspended Particle Device), so that the HMD device can control the color or transmissivity of the lens. For example, a lens can be made of an electrochromic material EC (Electrochromic Material) thin film transparent electrode (OCA), an Induim-Tin Oxide (ITO), and an electrochromic glass composed of an electrolyte filled between two ITO glasses / . When the lens is an electrochromic glass, the HMD device can control the color or transmissivity of the lens by adjusting the magnitude of the voltage applied to the electrochromic glass.

The HMD device may then modify the image based on the color or transmittance of the lens (S3120). The HMD device can display the modified image in step S3120. For example, if the HMD device controls the lens so that the color of the lens is blue, the HMD device can display the text in red, which is a complementary color to blue.

33 is an exemplary diagram illustrating a complementary color table that can be used to modify an image in accordance with some embodiments.

The HMD device according to some embodiments may modify an image using a complementary color table. For example, if the color information obtained in step S1210 of FIG. 12 is FFFF00 (yellow), the HMD device can display text with a blue color that is a complementary color of yellow. However, the present invention is not limited thereto, and the method of modifying the image may be changed according to the embodiment.

Here, the color information may be modified according to the embodiment. For example, according to some embodiments, the color information may be information indicating the color of the lens itself. Or according to some other embodiments, the color information may indicate to the user the color recognized as the background color around the image displayed by the HMD device. In this case, by detecting the color of the light transmitted through the lens using the photosensor provided in the HMD device, the HMD device can acquire the color information indicating the color recognized as the background color. According to still another embodiment, the color information may be information determined based on the external environment information.

Figs. 34 and 35 are conceptual diagrams showing an example in which an HMD device displays a corrected image according to some embodiments.

FIG. 34 is a view showing an example of displaying an image 3310 when a user wearing the HMD device 3300 sees the indoor scenery 3301. FIG. The HMD device 3300 can measure the illuminance. The HMD device 3300 may increase the transmittance of the lens 3320 or prevent the color from appearing on the lens 3320 when the illuminance measured by the HMD device 3300 is low. The user of the HMD device 3300 can recognize that the image 3310 is displayed on the lens 3320. [

35 is a diagram illustrating an example of displaying an image 3311 when a user wearing the HMD device 3300 views the outdoor scenery 3401. FIG. As the measured illuminance increases, the HMD device 3300 can control the transmittance of the lens 3321 and the hue of the lens 3321. When the transmittance or hue of the lens 3321 is changed, the HMD device 3300 can correct the image 3310 based on the color information corresponding to the transmittance or color of the changed lens 3321. [ As shown in FIG. 35, the HMD device 3300 can display an image 3311 in which the color, the icon size, and the like are modified.

36 and 37 are conceptual diagrams illustrating an example in which an HMD device displays a corrected image according to some other embodiments.

As shown in FIG. 36, the HMD device 3500 may display an image 3510 on the lens 3520 to provide an augmented reality function. The image 3510 for providing an augmented reality function may be, for example, related information about an actual object that the user is looking at. 37, when the color or transmittance of the lens 3520 is changed, the HMD device 3500 can display the modified image 3511. [ The modified image 3511 may be an image in which the shape of the object included in the image is changed as shown in FIG. 37 as well as the color.

38 is a block diagram briefly showing a structure of an HMD device according to some embodiments.

The HMD device may include a color information obtaining unit 3710, a control unit 3720, and a display unit 3730.

The color information obtaining unit 3710 according to some embodiments may obtain the color information of the lens. The color information obtaining unit 3710 may be implemented variously according to the embodiment. For example, the color information obtaining unit 4710 may include an electrode 1532 as shown in FIG. Or color information acquisition unit 4710 may include a recognition unit for recognizing a code using a camera 1710 or the like as shown in Fig. 17, or may include a recognition unit for recognizing a code by using a photosensor 1930, as shown in Fig. Color information may be obtained. Alternatively, when the control unit 3720 controls the transmittance or color information of the lens, the color information obtaining unit 3710 may be included in the control unit 3720.

Further, when the lens is detachable to the HMD device, the color information obtaining unit 3710 can recognize whether or not the lens is coupled to the HMD device. When the lens coupled to the HMD device is recognized, the color information acquiring unit 3710 can acquire color information on the combined lens.

The control unit 3720 can process information and control each part of the HMD device. The control unit 3720 can modify the image based on the color information obtained by the color information obtaining unit 3710. [ The control unit 3720 may control the display unit 3730 to display the modified image.

According to some embodiments, the controller 3720 may control at least one of the hue or transmittance of the lens. In this case, the lens may include an electrochromic glass, a suspended particle device (SPD), or a liquid crystal (LC). For example, the lens may comprise a glass having a structure as shown in Fig. When the controller 3720 controls the hue or transmittance of the lens, the controller 3720 can acquire color information of the lens corresponding to the color or control rate of the changed lens. Further, the control unit 3720 can correct the image based on the obtained color information of the lens.

According to some other embodiments, the control unit 3720 can select the content that can be output through the display unit 3730. [ For example, the HMD device may display the content list and may select the content from the content list according to the user input received through the input / output unit of the HMD device. Here, the content list may include executable applications, multimedia content such as photographs or moving pictures, and the like.

In addition, the HMD device according to some other embodiments may further include an illuminance sensor (not shown) for obtaining information on the external illuminance. The controller 3720 can modify the image by further considering the information about the external illuminance obtained using the illuminance sensor. For example, the controller 3720 can increase the brightness of the image as the external illuminance increases.

The HMD device according to some embodiments may modify the image by further considering the color of the scenery (background of the image) that the user is looking at. 39 is a flow chart illustrating an exemplary process by which an HMD device 4000 displays an image, in accordance with some embodiments. 40 to 43 are conceptual diagrams illustrating an example in which the HMD device 4000 displays the corrected image 4125 according to some other embodiments.

First, the HMD device 4000 can take a background image 4020 (S3810). The HMD device 4000 according to the present embodiment may include a camera 4010 capable of capturing an image in the direction of the user's eyes. The HMD device 4000 can shoot a background image using the camera 4010. [

Referring to FIG. 40, the HMD device 4000 may include a camera 4010 on one side. When the user 1 wearing the HMD device 4000 is looking at the crosswalk, the camera 4010 can acquire the background image 4020 from which the crosswalk is photographed.

The HMD device 4000 may determine the image display area 4120 related to the position of the image 4110 to be displayed by the HMD device 4000 within the captured background image 4020 at operation S3920. When the HMD device 4000 displays the image 4110, the image display area 4120 displays an area including a position where the user 1 recognizes that the image 4110 is displayed on the background image 4020 It can mean. According to some embodiments, the image display area 4120 may be determined according to the position at which the image 4110 is displayed.

41, when the image 4110 is displayed on the upper left of the HMD device 4000, the HMD device can determine that the image display area 4120 is positioned at the upper left of the background image 4020. Referring to FIG. 42, the user 1 can recognize that the image 4110 is displayed on the image display area 4120 in the background image 4020. If the background color in the image display area 4120 is similar to the color of the image 4110, the image 4110 is hardly recognized by the user. Therefore, it is necessary to correct the color of the image 4110. [

If the image display area 4120 is determined in step S3920, the HMD device 4000 can determine the color information of the image display area 4120 in step S3930. Here, the step S3930 may be variously implemented according to the embodiment. For example, the HMD device 4000 can determine the average hue of the pixels included in the image display area 4120 as the color information of the image display area 4120. As another example, the HMD device 4000 may extract color information from a feature point such as a center point in the image display area 4120. [

Further, according to some other embodiments, the HMD device 4000 can determine the color information by considering the color of the lens and the color of the image display area 4120 together. When the user 1 views the object through the lens, the user 1 can recognize the object in a color different from the color of the actual object. Accordingly, the HMD device 4000 can determine the hue of the object recognized by the user 1 through the lens as color information of the image display area 4120 in step S3930.

Thereafter, the HMD device 4000 can modify the image based on the color information of the image display area 4120 determined in step S3930 (S3940). The HMD device 4000 can display the modified image through the display unit (S3940).

For example, the HMD device 4000 may determine a complementary color to the color information of the display area 4120, and may modify the color of the image 4110 to a determined complementary color. Referring to Figure 43, the HMD device 4000 may display the modified image 4125 so that the image 4124 may be easily distinguished from the background 4020.

In step S3940, not all of the images need to be modified. The colors of the edges of the image may change, or a background may be added to the surroundings of the image.

Figure 44 is a flow chart illustrating a process by which an HMD device controls the color or transmittance of a lens in accordance with some embodiments.

First, in step S4410, the HMD device can photograph an image with a camera provided in the HMD device (S4410). Here, the camera may be disposed toward the outside of the HMD device. That is, the camera can take an image by receiving light not transmitted through the lens.

Then, in step S4420, the HMD device can determine the hue or transmittance of the lens based on the photographed image. Here, the lens may include a smart window capable of changing the hue or transmittance according to the control of the HMD device. For example, the HMD device can determine the external illuminance based on the photographed image. That is, the HMD device can analyze the brightness of the photographed image and determine the illuminance outside the HMD device according to the analysis result. The HMD device can determine the color of the lens to be a dark color such as brown when the external illuminance is high. Conversely, the HMD device can determine the hue or transmittance of the lens so that the lens has no hue or the transmittance is high when the external illuminance is low.

Thereafter, in step S4430, the HMD device can control the hue or transmittance of the lens so that the lens has the hue or transmittance determined in step S4420.

45 is a conceptual diagram showing a method by which an HMD device photographs an image to control color or transmittance according to some other embodiments.

According to some other embodiments, the HMD device may comprise a plurality of cameras. Referring to FIG. 45, the HMD device may include a first camera 560-1 and a second camera 560-2. The first camera 560-1 may be arranged to receive light 4520 that has not transmitted the lens 580 outside the HMD device. The second camera 560-2 may be arranged to receive the light 4510 transmitted through the lens 580. [

The HMD device can determine the color or transmittance of the lens based on the light 4520 received through the first camera 560-1 and the light 2510 received through the second camera 560-2. For example, the HMD device may compare the light 4520 received via the first camera 560-1 with the light 4510 received via the second camera 560-2 (e.g., Difference in color or difference in color), and the color or transmittance of the lens can be determined based on the comparison result. Alternatively, the HMD device may select either the light 4520 received through the first camera 560-1 or the light 4510 received through the second camera 560-2, and the light intensity or color of the selected light The color or transmittance of the lens can be determined. The method of determining the hue or transmittance of the lens according to the embodiment can be variously changed.

46 is a flow chart illustrating a process by which an HMD device controls the color or transmittance of a lens in accordance with some other embodiments.

First, the HMD device may take an image at the first camera 560-1 at step S4610. Further, the HMD device may take an image at the second camera 560-2 at step S4615.

Thereafter, in step S4620, the HMD device may analyze the photographed image in steps S4610 and S4615. For example, the HMD device can compare an image photographed through the first camera 560-1 with an image photographed through the second camera 560-2. Alternatively, the HMD device may select one of the image captured through the first camera 560-1 and the image captured through the second camera 560-2. However, the present invention is not limited thereto.

Thereafter, in step S4630, the HMD device can determine the hue or transmittance of the lens based on the analysis result. If the color or transmittance of the lens is determined, the HMD device can control the hue or transmittance of the lens according to the determined hue or transmittance in step S4640.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media such as RAM, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. A communication medium typically includes computer readable instructions, data structures, program modules, or other data of a modulated data signal, or other transport mechanism, and includes any information delivery media. For example, computer storage media may be embodied in ROM, RAM, flash memory, CD, DVD, magnetic disk, magnetic tape, or the like.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

Claims (17)

A head-mounted display device comprising:
A color information obtaining unit for obtaining color information of a lens on one side of the head mount display; And
And a control unit for outputting the corrected image based on the color information through a display unit provided in the head-mounted display device.
The method according to claim 1,
Wherein the lens changes at least one of a hue and a transmittance according to a control of the control unit,
Wherein the color information obtaining unit obtains color information corresponding to the hue or transmittance of the changed lens.
The method according to claim 1,
Characterized in that the lens is attachable to and detachable from the head mount display,
Wherein the control unit obtains the color information when the lens coupled to the head-mounted display is recognized.
The method of claim 3,
Wherein the color information obtaining unit comprises:
Further comprising a recognition unit for recognizing a code provided on the lens,
And acquires color information corresponding to the recognized code through the recognition section.
The method of claim 3,
Wherein the controller selects a content that can be output through the display unit, determines a lens corresponding to the selected content, and outputs a recommendation message recommending the determined lens through the display unit.
The method of claim 3,
Wherein the head mount display device comprises:
Further comprising an illuminance sensor for obtaining information on an external illuminance,
Wherein the control unit determines a lens corresponding to information on the external illuminance obtained through the illuminance sensor and outputs a recommendation message recommending the determined lens through the display unit.
The method according to claim 1,
Wherein the color information obtaining unit includes a photosensor, and obtains the color information for the lens based on information obtained through the photosensor.
The method according to claim 1,
Wherein the head mount display device comprises:
Further comprising an illuminance sensor for obtaining information on an external illuminance,
Wherein the image output through the display unit includes:
Wherein the image is a modified image further considering information on the external illuminance.
A method of displaying an image on a head mounted display device,
Obtaining color information for a lens on one side of the head mount display; And
And outputting a corrected image based on the color information through a display unit provided in the head mounted display device.
10. The method of claim 9,
In the display method,
Further comprising changing at least one of a color and a transmittance of the lens,
Wherein the obtaining of the color information comprises:
And acquires color information corresponding to the color or transmittance of the modified lens.
10. The method of claim 9,
Characterized in that the lens is attachable to and detachable from the head mount display,
In the display method,
And when the lens coupled to the head mount display is recognized, performing the step of obtaining the color information.
12. The method of claim 11,
Wherein the obtaining of the color information comprises:
Recognizing a code included in the lens; And
And obtaining color information corresponding to the code.
12. The method of claim 11,
In the display method,
Selecting content that is outputable through the head mount display; And
And recommending a lens corresponding to the selected content,
Wherein the step of displaying the modified image displays an image obtained by modifying the selected content based on the color information of the recommended lens.
12. The method of claim 11,
In the display method,
Obtaining information on external illuminance; And
Further comprising recommending a lens corresponding to information about the external illuminance.
10. The method of claim 9,
Wherein the head mount display device comprises a photosensor,
Wherein the obtaining of the color information comprises:
And acquiring the color information based on the information obtained through the photosensor.
10. The method of claim 9,
The display method may further include acquiring information on an external illuminance,
Wherein the image output through the display unit includes:
Wherein the image is a modified image considering further information on the external illuminance.
A computer-readable recording medium storing a program for causing a computer to execute the method of claim 9.

Images