KR102360557B1 - See-through head mount display with real-world image garbling function - Google Patents

Abstract

Disclosed are a real-world image distortion method for a see-through head mounted display and a see-through head mounted display having a real world image distortion function. A real-world image distortion method for a see-through head mounted display includes: capturing a real-world image through a see-through camera; converting the captured real-world image into a video signal; changing the sense of reality of the real world image by distorting the real world image converted into a video signal; and receiving the distorted real-world image, and driving the see-through head mounted display to display the received distorted real-world image. According to the present invention, a real-world image distortion method for a see-through head mounted display that is differentiated from the existing augmented reality technique of matching virtual information to a real world image and is differentiated from correction techniques for improving the quality of a see-through image is provided , by embedding a reality change unit in the see-through HMD, it is possible to cause effects such as psychological treatment and psychological stability of the user.

Description

See-through head mount display with real-world image garbling function

The present invention relates to a video see-through head mounted display (HMD) that distorts and outputs a see-through image in real time for the purpose of changing the sense of reality. A real-world image distortion method for a see-through HMD having a built-in realism change unit comprising

A head mounted display (HMD) is a wearable display device that is mounted on the head, and is variously called a VR headset, VR glasses, smart glasses, smart eyewear, smart goggles, VR goggles, and the like. All of these devices are common in that they are wearable display devices that are mounted on the user's face, and their names are subdivided according to wearing methods, components, shapes, and functions, but all of these devices can be represented by the term HMD.

The advantages of see-through HMDs are that, firstly, they are one of the devices capable of implementing augmented reality. Augmented reality is a technology that provides spatial matching between the real world and virtual objects or digital information, and provides a visual experience in which the real world and the virtual world are connected.

Second, information that cannot be seen with the naked eye, for example, infrared rays, ultraviolet rays, ultrasonic waves, electromagnetic waves, etc., can be provided in real time from the user's point of view using a see-through HMD equipped with a special camera. In addition, if you use the zoom-in function of the camera, you can see a distant object magnified several times to several tens of times. If the HMD is equipped with a fisheye lens, it can be used as a means of transcending visual ability, such as having a wide field of view.

Third, the see-through HMD wearable display devices can store the front view viewed by the user as it is. The video information stored in this way can be shared with others through the network, and the user can view it again in the future. Through the see-through HMD, users can share a visual experience that transcends the limitations of time and space.

Lastly, the see-through HMD also provides a function to protect the user's eyes from dangers that can damage the eyes, such as sunlight, laser, dirt, and harmful substances.

The Korean patent application (the title of the invention is "see-through head-mounted display") published as Publication No. 1020140178710 on December 11, 2014 is an optical combiner, a virtual image display that projects a virtual image to the optical combiner an element, and a shade cover disposed on the front surface of the optical synthesizer, wherein the shade cover contains an electrochromic material as a first transparent electrode, an electrolyte layer disposed on the first transparent electrode, and an electrochromic layer disposed on the electrolyte layer Disclosed is a see-through HMD comprising an electrochromic layer, and a second transparent electrode disposed on the electrochromic layer. According to the prior art, by employing embodiments of the shade cover, excellent visibility of the virtual image can be secured without amplifying the luminance of the virtual image even when the illuminance of the real environment increases.

However, the see-through HMD according to the prior art has the following disadvantages. That is, the see-through HMD shows the real world by changing it into a video image through a digital display, so the user feels unfamiliar with the visual feeling of the real world. This is because the digitally converted video signal has a sense of heterogeneity no matter how hard you try to express it the same as the real field of view. This heterogeneity of perspective is a characteristic and fundamental problem of the see-through HMD. In general, such heterogeneity is regarded as a disadvantage because it deteriorates visual abilities such as visual acuity, viewing angle, and color discrimination. Therefore, we are trying to overcome this through the development of various technologies such as increasing the display resolution, increasing the viewing angle, improving the performance of the see-through camera, and improving the picture quality so that there is no distortion of the real field of view.

However, this heterogeneity makes the user feel the visual confusion of the real world as if it is a virtual world, so that psychological burdens such as tension and fear of reality can be expected to be lowered.

Therefore, there is an urgent need for a technology capable of providing the visual confusion effect that occurs in the see-through HMD in various forms according to the situation.

Korean Patent Application Publication No. 1020140178710 (published on December 11, 2014, titled "See-through head mounted display")

An object of the present invention is to provide a real-world image distortion method for a see-through head mounted display that is differentiated from the existing augmented reality technique of matching virtual information to a real world image and is differentiated from correction techniques for improving the quality of a see-through image will do

Another object of the present invention is to provide a see-through head mounted display having a real world image distortion function, which can cause effects such as psychological treatment and psychological stability of a user by embedding a reality change unit in the see-through HMD.

One aspect of the present invention for achieving the above objects is a see-through head mounted display having a real-world image distortion function, a see-through camera for capturing a real-world image, and a digital conversion unit for converting the captured real-world image into a video signal and a reality change unit that distorts the real-world image converted into a video signal to change the sense of reality of the real-world image, and the see-through head mounted display to receive the distorted real-world image and display the received distorted real-world image and a display driving unit driving It may relate to a see-through head mounted display having a real-world image distortion function, including changing the real-world image and the sound collected together with the real-world image.

The see-through camera captures the real world image according to a MIPI CSI standard, and the digital converter converts the MIPI CSI signal to NTSC, PAL, RGB, YCbCr, HDMI, MPEG, AVI, VOB, H.264. can be converted to a format selected from

The reality change unit may include a distortion control unit that determines a distortion to be applied to the real world image, and an image distortion unit that distorts the real world image based on the determined distortion.

The distortion control unit determines the distortion based on a command input from the user, and determines the psychological state of the user based on a sensor signal obtained from a sensor for observing the user's physical state, and the determined psychological state performing at least one of determining the distortion based on can be configured to

The sensor for observing the user's body condition includes a sensor for detecting a physiological signal of the user including at least one of skin color, pulse rate, blood pressure, electrocardiogram, skin electrocardiogram, electromyogram, and perspiration, and detecting the user's behavior The observing sensor may include at least one of a position sensor, a GPS sensor, a speed sensor, and an acceleration sensor.

The image distortion unit, changing the pixel value of the real world image, cartoon rendering giving a cartoon-like feel by applying a 3D image technique to the real world image, changing the real world image contrast, binarizing the real world image performing at least one of black-and-white processing, changing the expression of a specific object included in the real-world image, and adjusting the contrast of the real-world image, the frequency of the sound collected together with the real-world image; It may be configured to change at least one of amplitude and pitch together.

The display driver may be configured to drive the see-through head mounted display by converting the received distorted real-world image into a MIPI DSI format compatible with the see-through head mounted display.

According to the present invention, there is provided a real-world image distortion method for a see-through head mounted display that is differentiated from the existing augmented reality technique of matching virtual information to a real world image and is differentiated from correction techniques for improving the quality of a see-through image. .

In addition, according to the present invention, there is provided a see-through head mounted display having a real-world image distortion function, which can cause effects such as psychological treatment and psychological stability of a user by embedding a reality change unit in the see-through HMD.

1 is a view for explaining an operating environment of an independent see-through HMD without a connection line with an external device.
2 is a flowchart schematically illustrating a real-world image distortion method for a see-through head mounted display according to an aspect of the present invention.
3 is a block diagram schematically illustrating a process of converting a MIPI CSI format to a MIPI DSI format.
4 is a block diagram schematically illustrating a see-through head mounted display having a real-world image distortion function according to another aspect of the present invention.
FIG. 5 is a block diagram schematically illustrating a configuration of an acoustic distortion module that may be embedded in the see-through HMD of FIG. 4 .
6 is a block diagram schematically illustrating a distortion control unit of FIG. 5 .
7 is a diagram illustrating results in which the real world is distorted according to the present invention.
8 is a diagram illustrating an operating environment of a stand-alone see-through HMD according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] Reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. In addition, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all scope equivalents as those claimed. Like reference numerals in the drawings refer to the same or similar functions throughout the various aspects. In addition, in order to clearly explain the present invention, parts irrelevant to the description are omitted, and the same reference numerals in the drawings indicate the same members.

1 is a view for explaining an operating environment of an independent see-through HMD without a connection line with an external device.

The see-through HMD includes a see-through HMD connected to an external device and a stand-alone see-through HMD.

An external device-linked see-through HMD is an HMD that performs image processing in an external device such as an external computer. The see-through camera converts the front real-world image into a video signal and outputs it to the digital converter. Such an image signal is generally a signal conforming to the MIPI (Mobile Industry Processor Interface) CSI (Camera Serial Interface) standard. The digital converter converts the MIPI CSI standard signal received from the see-through camera into a standard video signal. In this specification, standard video signal refers to all video signal standards except for MIPI CSI standards such as NTSC, PAL, or digital video signals such as HDMI, compression standards MPEG, and H.264 that are connected to common display devices such as CRTs and LCD monitors. covers The converted signal is transmitted to an external device.

The reason why a digital converter is needed is that, in general, a computer can perform image processing only when there is information converted into a standard video signal. An external device-linked HMD processes these standard video signals in an external computer device, and in the case of a stand-alone HMD, a built-in computer processes it.

The image information that has undergone the image processing process in the external device is sent back to the HMD. The transmission path can be both wired and wireless. Then, the transmitted image information is converted into a MIPI DSI (Display Serial Interface) standard signal by the display driver. The converted signal is shown to the user through an HMD, which is usually an LCD or OLED.

In the present specification, image processing refers to processes such as real-world-virtual-world coordinate matching and image rendering for generating an augmented reality image. For example, mixing a real-world see-through image with a specific object in virtual reality is included in image processing.

On the other hand, in stand-alone see-through, which operates independently, a built-in computer performs the image modulation process to achieve the desired function. After that, the modulated image is converted into a MIPI DSI signal through the display driver, which is shown to the user through the HMD. The difference between the external device-linked see-through HMD and the stand-alone see-through HMD is that a computer device that receives a standard video signal and processes the image through predetermined information processing is built into the HMD. In this case, since the HMD can operate independently without an external computer device, there is no need for a wired/wireless connection, and therefore there is no restriction on the user's motion or movement.

2 is a flowchart schematically illustrating a real-world image distortion method S200 for a see-through head mounted display according to an aspect of the present invention.

Referring to FIG. 2 , first, a real world image is captured according to MIPI CSI through a see-through camera (S210). The MIPI CSI format will be described later in detail with reference to FIG. 3 . Therefore, repetitive description is omitted for the sake of simplification of the specification.

When the real world image is captured, the captured real world image is converted into a video signal (S220). In this case, the converted video signal may be converted into various formats known in the art, such as NTSC, PAL, RGB, YCbCr, HDMI, MPEG, AVI, VOB, H.264, and the like. When a real world image is converted into a video signal, it can be processed by various image processing devices.

And, in the real world image distortion method S200 for the see-through head mounted display, the real world image converted into a video signal is distorted to change the sense of reality of the real world image.

To this end, first, distortion to be applied to the real world image is determined (S230). As used herein, garbling means changing the real-world image displayed to the user by applying various processing to the real-world image.

The following techniques can be used to determine the distortion. It should be noted that these are merely exemplified for convenience of understanding and do not limit the present invention.

1. Determine the distortion based on the use of the command input from the user. In this case, there is an advantage in that the user can directly determine the distortion suitable for the user.

2. An operator other than the user may determine the distortion through the remote control unit. The operator can be a doctor or a psychologist who heals the patient. In this case, the treatment effect can be maximized by distorting the real-world image according to the patient's condition.

3. The user's psychological state may be determined based on a sensor signal obtained from a sensor for observing the user's physical state, and distortion may be determined based on the determined psychological state. The sensor for observing the user's body state may be a camera capable of detecting skin color, pulse rate, blood pressure, electrocardiogram, skin electrocardiogram, electromyography, perspiration, and the like, and various biometric sensors. In this case, the user's physical state may be directly determined, and the real-world image may be distorted to fit the determined physical state.

4. The user's behavior may be determined based on a sensor signal obtained from a sensor observing the user's behavior, and distortion may be determined based on the determined behavior. The sensor for observing the user's behavior may be a location sensor, a GPS sensor, a speed sensor, an acceleration sensor, and the like. In this case, it is possible to detect the user's movement, hyperactivity, or underactivity, and appropriately distort the real-world image according to each behavioral state.

When the distortion is determined, the real-world image is distorted based on the determined distortion (S250). Distortions applicable to real-world images in the present invention include the following. It should be noted that these are merely exemplified for convenience of understanding and do not limit the present invention.

1. You can change the pixel value of the real world image.

For example, it may be undesirable for violent scenes to be exposed to people with an unstable psychological state. Accordingly, it is possible to expose to the user by changing the pixel values of the portions sensed as fatigue in the real world image.

2. By applying 3D image technique to real-world images, it can give a cartoon-like feel. This technique is called cartoon rendering, and it can produce a dreamy effect, and it has the effect of lowering the tension in reality because it makes reality feel like a space in a cartoon.

3. You can change the real world image contrast. In general, the higher the contrast, the sharper the real-world image provided, and the more sensitive people are. Accordingly, by adjusting the contrast based on the user's psychological state, it is possible to induce the user to relieve tension felt in reality.

4. If necessary, the real world image can be binarized and processed in black and white. In this case, grayscale processing may be performed by adjusting the black-and-white processing step. In the case of extreme tension, providing the real-world image in black and white may help to stabilize the user's emotions.

5. It is also possible to change the expression of a specific object included in the real world image. For example, for a user who is driving, traffic indicating facilities such as a center line or a traffic light may have very important meaning in themselves. In this case, it is possible to draw the user's attention by highlighting or magnifying the traffic indication facilities in the real world image.

6. You can also adjust the contrast of the real world image. For a user looking at a ski resort or a white sandy beach, eye damage can be prevented by lowering the brightness of the real world image, and for a user in a dark environment, visibility can be improved by increasing the brightness.

7. It is also possible to change at least one of the frequency, amplitude, and pitch of the sound collected together with the real-world image. For users in the library, concentration can be improved by canceling background noise, and for users exposed to high-noise environments such as baseball fields, hearing can be protected by reducing the amplitude of the sound provided. A technique for distorting the sound will be described later in detail with reference to FIG. 5 . Therefore, repetitive description is omitted for the sake of simplification of the specification.

When the real-world image is distorted, the distorted real-world image is converted into a MIPI DSI format compatible with the see-through head mounted display (S260). This conversion is to change a video signal processed by the image processing device to a signal that can be processed by the see-through HMD.

When the format is converted, the see-through head mounted display is driven to display a distorted real-world image (S270). Accordingly, the user views the scene in which the real world image is converted.

3 is a block diagram schematically illustrating a process of converting a MIPI CSI format to a MIPI DSI format.

As mobile devices such as smart phones have become widespread, the MIPI format has been introduced in order to satisfy the limitations of low current consumption and high speed. The MIPI format refers to the interface protocol required for connection between parts within a mobile device, and was initially used between the camera sensor and the mobile processor, AP, but is now being extended and applied to mobile display devices, storage devices, and Wi-Fi modules.

If you look at the MIPI format, each part is connected based on the central AP, and there are CSI and DSI formats for connecting these parts. Signal transmission in MIPI is achieved through a differential pairing method with a built-in differential amplifier called LVDS (Low-Voltage Differential Signaling). D-PHY and M-PHY exist as a physical layer for implementing a differential pairing scheme in charge of signal transmission. Among them, D-PHY is used for cameras or displays, and M-PHY is used for low-power, high-speed communication.

In FIG. 3 , the signal captured by the camera 310 is transmitted to the display device 390 via the SoC 350 . The signal captured by the camera 310 is transmitted to the SoC 350 through the D-PHY through the CSI-2 transmitter. The transmitted signal is received by the CSI-2 receiver and transmitted to the image processor. And, it is provided to the DSI host controller through the graphic controller. This video signal is also passed through the D-PHY to the DSI device controller. As can be seen from FIG. 3 , the signal obtained by the camera 310 is provided to the SoC 350 through the CSI interface, and the SoC 350 sends a signal to the display device 390 through the DSI interface.

4 is a block diagram schematically illustrating a see-through head mounted display having a real-world image distortion function according to another aspect of the present invention.

Referring to FIG. 4 , the see-through HMD 400 according to the present invention includes a see-through camera 410 , a digital conversion unit 420 , a reality change unit 450 , a display driver 480 , and a head mounted display 490 . ) is included. Also, the reality change unit 450 includes a distortion control unit 440 and an image distortion unit 460 , and the display driver 480 includes a format conversion unit 470 .

The see-through camera 410 captures the real world image and provides it to the digital converter 420 . Then, the digital conversion unit 420 converts the captured real world image into a video signal and provides it to the reality change unit 450 .

The reality change unit 450 serves to change the reality of the real world image by distorting the real world image converted into the video signal. To this end, the distortion control unit 440 determines the distortion to be applied to the real world image. The process of determining the distortion is the same as described above using FIG. 2 . Therefore, repetitive description is omitted for the sake of simplification of the specification.

When the distortion to be applied is determined, the image distortion unit 460 distorts the real world image based on the determined distortion. Since the applied distortion is the same as described above using FIG. 2 , a repetitive description is omitted for the sake of simplicity of the specification.

When the real-world image is distorted, the format converter 470 receives the distorted real-world image and converts the received real-world image into a MIPI DSI format compatible with the see-through head mounted display. Then, the display driver 480 receives the distorted real-world image and drives the head mounted display 490 to display the received distorted real-world image.

Therefore, since the HMD 400 according to the present invention controls the image distortion by setting the distortion control unit 440 separately, it provides image distortion that meets the user's needs in real time to change the sense of reality, thereby providing various psychological and psychological benefits. .

FIG. 5 is a block diagram schematically illustrating a configuration of an acoustic distortion module 500 that may be embedded in the see-through HMD of FIG. 4 .

The sound distortion module 500 includes an HMD microphone 510 , a sound encoder 520 , a reality change unit 550 , a sound decoder 570 , and an HMD earphone 590 . Also, the reality change unit 550 includes a distortion control unit 540 and an image distortion unit 560 .

The HMD microphone 510 collects real-world sound and provides it to the sound encoder 520 . Then, the sound encoder 520 converts the collected sound into an acoustic signal and provides it to the reality change unit 550 .

The reality change unit 550 serves to change the sense of reality by distorting the real world sound converted into the sound signal. To this end, the distortion control unit 540 determines the distortion to be applied to the real world sound. The process of determining the distortion is the same as described above using FIG. 2 . Therefore, repetitive description is omitted for the sake of simplification of the specification.

When the distortion to be applied is determined, the sound distortion unit 560 distorts the real world sound based on the determined distortion. The distorted sound may have a pitch, a frequency, an amplitude, and the like. In addition, examples of acoustic distortion may include removing or amplifying a specific frequency, adding a spatial effect such as echo or reverb to give a sense of space, or inducing attention by adding a specific acoustic component. However, this is not intended to limit the present invention, and all sound processing techniques known in the art may be applied to the present invention. The acoustic distortion may be provided simultaneously with the image distortion, or the two may be provided separately.

When the real-world sound is distorted, the sound decoder 570 receives the distorted real-world sound and decodes the received real-world sound to be reproduced in the HMD earphone 590 .

Therefore, since the acoustic distortion module 500 according to the present invention controls the acoustic distortion by setting the distortion control unit 540 separately, it provides acoustic distortion that meets the user's needs in real time to change the sense of reality, thereby providing various psychological and psychological benefits. to provide.

Although the acoustic distortion module 500 of FIG. 5 is illustrated as a separate module, it may be embedded in the see-through HMD 400 of FIG. 4 .

6 is a block diagram schematically illustrating a distortion control unit of FIG. 5 .

The distortion control unit 540 includes a direct manipulation unit 610 , a remote manipulation unit 630 , a user psychological state determining unit 660 , and a user behavioral state determining unit 680 . In addition, the direct manipulation unit 610 , the remote manipulation unit 630 , the user psychological state determining unit 660 , and the user behavioral state determining unit 680 include an HMD built-in button 620 , a wireless communication unit 640 , and a user psychological signal sensor. 670 , and a user behavior signal sensor 690 , respectively.

The direct manipulation unit 610 and the remote manipulation unit 630 receive a distortion control signal through the HMD built-in button 620 and the wireless communication unit 640, respectively. These signals may be provided by the user himself or directly by a third party other than the user.

The direct manipulation unit 610 transmits signals such as on/off of image distortion and switching of distortion rules to the image distortion unit (sound distortion unit).

The user's mental state determination unit 660 diagnoses the psychological state in response to a signal input from the sensor unit. For example, when the pulse rate surges, it is diagnosed as a tension or stress situation, and a tension state value is calculated. Then, the user's psychological state determination unit 660 determines the image (sound) distortion method corresponding to the calculated tension state, and transmits it to the image distortion unit (sound distortion unit).

The psychological signal sensor 670 collects various biological signals of the user in order to generate a signal for the user's psychological state. As used herein, the biosignal refers to an electrical signal that is measured in our body and holds energy or information about the human body. The human body generates electricity as a single conductor, and the electrical value changes according to the human metabolism and physical condition. In particular, since the transmission of information is made by voltage fluctuations on the surface of nerve or muscle cells, monitoring this can provide various information. Information is also transmitted through body fluids to the body surface, ie, the skin.

Examples of the sensor may include an electrocardiograph (ECG), an electroencephalograph (EEG), an electro neurography (ENG), an electromyograph (EMG), an electroretinograph (ERG), an electro gastrography (EGG), an electrooculogram (EOG) sensor, and the like. However, this is not intended to limit the present invention, and it should be noted that various sensors known in the art may be applied to the present invention. For example, a camera for detecting skin color, a pulse wave sensor, a blood pressure sensor, a perspiration sensor, and the like may also be applied to the present invention.

The behavioral state detected by the user behavioral state determination unit 680 includes all behaviors that can be determined to be related to a psychological state, such as long-time stopping, sitting, standing up, running, severe head movement, and screaming. The user behavioral state determination unit 680 diagnoses a psychological state corresponding to a signal input from the sensor unit. For example, when the user is stationary without movement for a long time, it is diagnosed as a state of concentration and a value of the state of concentration is calculated. Then, the user behavior state determination unit 680 determines an image (sound) distortion method corresponding to the calculated attention and concentration state value, and transmits it to the image distortion unit (sound distortion unit). For example, in order to lower the user's attention in the state of concentration, the image distortion unit performs binarization of the image, and lowers the binarization threshold value to darken the image as a whole. In this case, an operation such as restoring the see-through image to its original state when the user stops excessive attention and looks around may be an embodiment of the present invention.

As the user behavior signal sensor 690, a location sensor, a GPS sensor, a speed sensor, an acceleration sensor, etc. may be utilized. For example, if the movement acceleration of the head on which the HMD is worn is large, it may be determined that the user is in an anxious state.

The remote manipulator 630 is to remotely provide image (sound) distortion determined to be most suitable for the user at the present time while an outsider observing the user observes the user's behavior. The outsider transmits an image (sound) distortion change signal to the wireless communication unit 640 built in the HMD using a remote control device while observing the user. The change signal received by the wireless communication unit is transmitted to the remote control unit 630, and the remote control unit transmits the image (sound) distortion rule corresponding to the signal to the image distortion unit (sound distortion unit) so that the distortion rule is changed. . For example, the therapist who determines that the user is over-excited and needs to be calmed presses a button corresponding to the sedation inducing function of the remote control device in his possession. At this time, the remote control unit 630 transmits a wireless signal corresponding thereto to the wireless communication unit 640 built in the HMD. The remote control unit 630 transmits a command for operating cartoon render distortion corresponding to the received signal and distortion to change the phase of sound to the image distortion unit (sound distortion unit), and the image distortion unit (sound distortion unit) immediately distorts Change the rules and display (sound output). As a result, the user experiences changed visual (auditory) information. The remote control unit can also be controlled by the user himself.

7 is a diagram illustrating results in which the real world is distorted according to the present invention.

Distortion rule 1 is a typical simple binarization rule. Binarization is an image conversion rule that replaces all pixel values with 0 or 1 and outputs black in case of 0 and white in case of 1. At this time, if the threshold value is lowered, most of the pixel values become 0 and darken. Image distortion also includes adjustment of these parameter values. Distortion rule 2 is cartoon rendering. Cartoon renderer is a technique that unifies adjacent pixel values of similar colors into one matched value to produce a coloring effect. Also, the result value varies depending on various parameter values. Distortion rules 3 and 4 are examples of images output by complex image distortion rules. One or more of these distortion rules may be used in combination. In addition, these distortion rules are only for convenience of description, and do not limit the present invention. Therefore, repetitive description is omitted for the sake of simplification of the specification.

The distortion of the real-world image is selected and provided based on the user's psychological state, so that the user's psychological stability can be promoted.

8 is a diagram illustrating an operating environment of an independent acoustic distortion module according to the present invention.

As illustrated in FIG. 8 , the real-world image is processed in black and white by the reality change unit and provided to the HMD. Accordingly, the user's tension state can be relieved.

According to the present invention, it can be used to induce the attention of an autistic patient by emphasizing or removing a specific object in the see-through image according to a predetermined distortion rule. In addition, the phobia may be alleviated by distorting the see-through image in a direction to lower the sense of reality according to a predetermined distortion rule. Furthermore, by providing an image distorted by a predetermined distortion rule, it is possible to prevent panic disorder or reduce a psychological burden during seizures, and to prevent depression and manic depression or to reduce symptoms by providing an image distorted by a predetermined distortion rule can alleviate In addition, according to the present invention, it is possible to prevent symptoms such as delusions, hallucinations, and hallucinations by providing an image distorted by a predetermined distortion rule, or to reduce a psychological burden during seizures, and to provide an image distorted by a predetermined distortion rule Therefore, it can be used for gradual desensitization treatment to relieve symptoms of fear.

Although the present invention has been described with reference to the embodiment shown in the drawings, which is only exemplary, those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom.

In addition, the method according to the present invention can be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium may include any type of recording device in which data readable by a computer system is stored. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. include In addition, the computer-readable recording medium may store computer-readable codes that can be executed in a distributed manner by a network-connected distributed computer system.

In terms of terms used herein, the singular expression should be understood to include the plural expression unless the context clearly dictates otherwise, and terms such as "comprises" refer to the described feature, number, step, operation, element. , parts or combinations thereof are to be understood, but not to exclude the possibility of the presence or addition of one or more other features or numbers, step operation components, parts or combinations thereof. In addition, terms such as "... unit", "... group", "module", and "block" described in the specification mean a unit that processes at least one function or operation, which is hardware, software, or hardware. and a combination of software.

Accordingly, this embodiment and the drawings attached to this specification only clearly show a part of the technical idea included in the present invention, and within the scope of the technical idea included in the specification and drawings of the present invention, those skilled in the art can easily It will be apparent that all inferred modified examples and specific embodiments are included in the scope of the present invention.

The present invention can be applied to a see-through HMD.

400: see-through HMD 410: see-through camera
420: digital conversion unit 440: distortion control unit
450: reality change unit 460: image distortion unit
470: format conversion unit 480: display driving unit
490 : head mounted display

Claims (7)

As a see-through head mounted display with real world image distortion function,
A see-through camera that captures real-world images,
A digital conversion unit that converts the captured real-world image into a video signal;
a sense of reality change unit that distorts the real world image converted into a video signal to change the sense of reality of the real world image;
and a display driver configured to receive a distorted real-world image and drive the see-through head mounted display to display the received distorted real-world image,
Distorting the real-world image is
changing the real-world image and the sound collected together with the real-world image according to the user's location;
A see-through head mounted display having a real-world image distortion function, comprising changing the real-world image and the sound collected together with the real-world image based on the user's physical and psychological state. The method of claim 1,
The see-through camera captures the real-world image according to the MIPI CSI standard,
The digital conversion unit converts the MIPI CSI signal into a format selected from the group consisting of NTSC, PAL, RGB, YCbCr, HDMI, MPEG, AVI, VOB, H.264, a see-through head mounted display having a real world image distortion function . The method of claim 1,
The reality change unit,
a distortion control unit that determines distortion to be applied to the real world image;
A see-through head mounted display having a real-world image distortion function, comprising an image distortion unit configured to distort the real-world image based on the determined distortion. 4. The method of claim 3,
The distortion control unit,
determining the distortion based on the command received from the user;
determining the psychological state of the user based on a sensor signal obtained from a sensor for observing the user's physical state, and determining the distortion based on the determined psychological state;
having a real-world image distortion function, configured to perform at least one of determining the user's behavior based on a sensor signal obtained from a sensor observing the user's behavior, and determining the distortion based on the determined behavior See-through head mounted display. 5. The method of claim 4,
The sensor for observing the user's physical condition includes a sensor for detecting a physiological signal of the user including at least one of skin color, pulse rate, blood pressure, electrocardiogram, skin electrocardiogram, electromyogram, and sweating,
The sensor for observing the user's behavior includes at least one of a position sensor, a GPS sensor, a speed sensor, and an acceleration sensor, a see-through head mounted display having a real-world image distortion function. 4. The method of claim 3,
The image distortion unit,
changing the pixel value of the real-world image;
Cartoon rendering that gives a cartoon-like feel by applying a 3D image technique to the real-world image;
altering the real-world image contrast;
Binarizing the real-world image to black-and-white processing;
changing the representation of a specific object included in the real-world image, and
performing at least one of adjusting the contrast of the real-world image;
A see-through head mounted display having a real-world image distortion function, configured to change at least one of a frequency, an amplitude, and a pitch of the sound collected together with the real-world image. 3. The method of claim 2,
The display driving unit is configured to drive the see-through head mounted display by converting the received distorted real-world image into a MIPI DSI format compatible with the see-through head mounted display. display.

Images