KR20220033223A - Eye-based personalized head-mounted display device and control method thereof - Google Patents

Abstract

The present invention relates to a method for controlling an eye-based personalized head-mounted display device, which includes the following steps of: outputting a measurement screen for customization; partitioning a first area in the measurement screen and a second area lower in image quality than the first area; reducing the first area; measuring a first perception time point at which a user perceives a boundary between the first area and the second area as the first area is reduced and setting the boundary at the first perception time point as the user's viewing angle; and setting the area inside the viewing angle as the user's center vision area, setting the area outside the viewing angle as the user's peripheral vision area, and rendering the peripheral vision area to be relatively lower in image quality than the center vision area.

Description

TECHNICAL FIELD [0002] Eye-based personalized head-mounted display device and control method thereof

The present invention relates to an eye-based personalized head-mounted display device and a control method therefor, and more particularly, after calculating the optimal viewing angle and image quality for each user who uses the head-mounted display device, based on this The present invention relates to a head-mounted display device for performing rendering and a method for controlling the same.

A head mounted display device refers to a display device worn on a user's head, and the head mounted display device is mainly used as a display device for realization of virtual reality or augmented reality. It can also be combined with 3D display technology.

The head-mounted display device is characterized by providing a user with a sense of immersion that cannot be compared with other display devices due to its characteristics, and has the advantage that there are few spatial restrictions compared to the size of the display shown to the user. In addition, the head mounted display apparatus recognizes user movement such as head tracking, and can adjust a displayed image according to the recognition, thereby providing a very intuitive input method.

However, since the head mounted display device has to be worn by the user, inconvenience occurs due to the weight of the head mounted display device, and motion sickness may occur due to the large screen displayed in front of the eyes.

In addition, in the case of a head mounted display device, due to the nature of the large screen of the image provided to the user, it requires a lot of resources for the graphics card to maintain the high quality of the image, so there is a disadvantage in that it cannot provide a high-quality image compared to other display devices. do.

Accordingly, in order to solve this problem, in recent years, a forward rendering technique in consideration of a user's viewing angle has been applied to a head mounted display apparatus.

However, since the structure of the retina is different for each person, the range of such a viewing angle has a different range for each individual. It is a situation in which an optimized image cannot be provided, and since the image is provided based on the uniformly applied viewing angle without considering the viewing angle for each user, there is also a problem in that the effect of reducing motion sickness described above is halved.

Republic of Korea Patent Publication No. 10-2017-0013653

Accordingly, an eye-based personalized head-mounted display device and a method for controlling the same according to an embodiment are inventions proposed to improve the above-described problems, and the eye-based personalized head-mounted display device capable of providing an optimal image in consideration of individual characteristics is provided. An object of the present invention is to provide a display device and a method for controlling the same.

Specifically, an object is to provide a more optimized image to the user by measuring the user's viewing angle before outputting the image to the user, and then providing an image rendered based on the measured viewing angle.

The control method of the eyeball-based personalized head mounted display device according to the present embodiment comprises the steps of: (a) outputting a measurement screen for customizing, (b) a first area within the measurement screen, and more than the first area A step of partitioning a second area having a low image quality, a step of reducing the first area, a first recognition time point at which a user recognizes a boundary between the first area and the second area as the first area is reduced measuring , and setting the boundary at the first recognition time point as the user's viewing angle, setting the area inside the viewing angle as the central vision area of the user, and setting the area outside the viewing angle as the peripheral vision area of the user and rendering the image quality of the peripheral vision region with a relatively lower image quality than the image quality of the central vision region.

The image quality of the peripheral vision region may be degraded in stages as the distance from the viewing angle to the peripheral vision region increases.

The method for controlling the eye-based personalized head-mounted display device further includes the step of repeating steps (c) and (d) a plurality of times, and the setting of the viewing angle includes: It may include the step of setting the average.

The method for controlling the eyeball-based personalized head-mounted display device further includes repeating steps (c) and (d) a plurality of times, and setting the viewing angle includes: It can be set as the mean plus or minus the standard deviation.

The setting of the viewing angles may include setting values obtained by subtracting and adding a standard deviation to the average of the data repeated a plurality of times as the first viewing angles and the second viewing angles, respectively.

The image quality of the central viewing area may be gradually deteriorated as the first viewing angle approaches the second viewing angle.

The method of controlling the eye-based personalized head-mounted display apparatus may further include tracking the movement of the user's body or the user's eye before the step (b).

The method further includes repeating steps (b) to (f) a plurality of times by varying the image quality of the first region, wherein the image quality of the peripheral vision region is different from each of the different viewing angles measured for each image quality. It may be gradually decreased as the distance from the viewing angle to the peripheral vision region is increased based on .

The control method of the eyeball-based personalized head mounted display device according to the present embodiment comprises the steps of (a) outputting a measurement screen for customizing, (b) a first area within the measurement screen, and lower than the first area partitioning a second region having image quality, (c) reducing the first region, and (d) reducing the first region so that the user recognizes the boundary between the first region and the second region measuring a first recognition time point, and setting the boundary at the first recognition time point as the user's viewing angle, (e) setting the inner area of the viewing angle as the central viewing area of the user, and setting the boundary outside the viewing angle setting a region as the peripheral vision region of the user, and setting the image quality of the central vision region and the peripheral vision region to be the same, (f) reducing the image quality of the peripheral vision region, (g) the As the image quality of the peripheral vision region deteriorates, a second perception time point at which the user recognizes the boundary between the central vision area and the peripheral vision area is measured, and the image quality value of the peripheral vision area at the second perception time point and (h) rendering an image of the peripheral vision region based on the measured image quality value of the peripheral vision region.

The method for controlling the eyeball-based personalized head-mounted display device further includes repeating steps (f) and (g) a plurality of times, wherein the image quality value of the peripheral vision region is the value of the data repeated a plurality of times. can be set to average.

The method for controlling the eyeball-based personalized head-mounted display device further includes repeating steps (f) and (g) a plurality of times, wherein the image quality value of the peripheral vision region is the value of the data repeated a plurality of times. It can be set as a value by adding or subtracting the standard deviation from the mean.

The control method of the eyeball-based personalized head-mounted display apparatus according to the present embodiment includes (a) outputting a screen for customizing, (b) a third area and a fourth area having the same image quality in the screen In this partitioning step, (c) the image quality of the fourth region is lowered, (d) as the image quality of the fourth region is lowered, the user recognizes the boundary between the third region and the fourth region. measuring the second recognition time point, measuring the image quality value of the fourth area at the second recognition time point, and (e) measuring the image quality value of the fourth area measured by the step (d) to the surroundings After setting the image quality value of the viewing region, the method may include performing rendering on the peripheral vision region.

The method for controlling the eyeball-based personalized head-mounted display device further includes repeating steps (c) and (d) a plurality of times, wherein the image quality value of the peripheral vision region is the data repeated a plurality of times. can be set as the average of

The method for controlling the eyeball-based personalized head-mounted display device further includes repeating steps (c) and (d) a plurality of times, wherein the image quality value of the peripheral vision region is the data repeated a plurality of times. It can be set as a value by adding or subtracting the standard deviation from the mean of .

The eye-based personalized head-mounted display device according to the present embodiment includes a display, a region control unit dividing a first region in the display, and a second region having lower image quality than the first region, and the size of the first region is A measurement unit measuring the size of the first area based on a user input as the size is reduced and the first area measured by the measurement unit are set as a central vision area, and an outer area of the central vision area is set as a peripheral vision area After setting the image quality value of the peripheral vision region to a value relatively lower than the image quality value of the central vision region, a controller configured to perform rendering on the images of the central vision region and the peripheral vision region. may include

The eye-based personalized head-mounted display device further includes a tracking module for tracking the movement of the user's eyes and the movement of the user's head, and the area control unit determines the forward direction in which the user's eyes are looking, obtained from the tracking module. The first region and the second region may be determined by using the first region and the second region as centers.

The eye-based personalized head-mounted display device according to various embodiments of the present invention measures an individual's viewing angle without uniformly determining the viewing angle in performing foveit rendering, and then performs a foveated image based on the measured viewing angle. It has the advantage of being able to provide customized images optimized for individuals.

In addition, there is an advantage in that virtual motion sickness that occurs when using a head mounted display device can be reduced by providing customized images for each individual. There is an advantage of reducing the resources of the graphics card being used.

1 is a conceptual diagram for controlling an eyeball-based personalized head-mounted display device according to an embodiment of the present invention.
2 is a block diagram illustrating the configuration of an eyeball-based personalized head mounted display device according to various embodiments of the present disclosure.
3 is a flowchart illustrating a method of controlling an eyeball-based personalized head mounted display apparatus according to various embodiments of the present disclosure.
4 is a block diagram illustrating another configuration of an eye-based personalized head-mounted display apparatus according to various embodiments of the present disclosure.
5 is a flowchart of a control method for setting a viewing angle for each user according to various embodiments of the present invention.
6 is an exemplary view for explaining a viewing angle setting process according to various embodiments of the present disclosure.
7 is a flowchart of a control method for setting image quality for each user according to various embodiments of the present invention.
8 is an exemplary view for explaining an image quality setting process according to various embodiments of the present disclosure.
9 is a view for explaining a process of customizing a viewing angle and image quality according to various embodiments of the present disclosure;

Hereinafter, embodiments according to the present invention will be described with reference to the accompanying drawings. In adding reference numerals to the components of each drawing, it should be noted that the same components are given the same reference numerals as much as possible even though they are indicated on different drawings. In addition, in describing the embodiment of the present invention, if it is determined that a detailed description of a related known configuration or function interferes with the understanding of the embodiment of the present invention, the detailed description thereof will be omitted. In addition, although embodiments of the present invention will be described below, the technical spirit of the present invention is not limited thereto and may be modified by those skilled in the art and variously implemented.

In addition, the terms used herein are used to describe the embodiments, and are not intended to limit and/or limit the disclosed invention. The singular expression includes the plural expression unless the context clearly dictates otherwise.

In this specification, terms such as "comprises", "comprises" or "have" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, but one It does not preclude in advance the possibility of the presence or addition of other features or numbers, steps, operations, components, parts, or combinations thereof, or other features.

In addition, throughout the specification, when a certain part is "connected" with another part, it is not only "directly connected" but also "indirectly connected" with another element interposed therebetween. Including, terms including an ordinal number, such as "first", "second", etc. used herein may be used to describe various elements, but the elements are not limited by the terms.

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily implement them. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description will be omitted. In addition, for convenience of description, the eye-based personalized head mounted display device will be described as a head mounted display device.

1 is a conceptual diagram for controlling an eyeball-based personalized head-mounted display device according to an embodiment of the present invention.

Referring to FIG. 1 , the head mounted display apparatus 100 may output an image 200 , and the output image 200 may be provided to a user wearing the head mounted display apparatus 100 .

The viewing angle means the maximum side angle at which a normal screen can be viewed in the screen display device, and the head mounted display device 100 may adjust the field of view of the output image 200 . Here, the viewing angle may be expressed as an angle of view.

Specifically, the head mounted display apparatus 100 may adjust the viewing angle of the displayed image 200 by using a restrictor 250 for adjusting the viewing angle of the output image 200 .

In one embodiment, the head mounted display apparatus 100 applies the restrictor 250 to the output image 200 to output the image 200 whose viewing angle is adjusted, or the restrictor ( 250) together, the image 200 with the viewing angle adjusted may be output. As another embodiment, the head mounted display apparatus 100 may adjust the viewing angle of the output image 200 by using the restrictor 250 having a physical configuration.

In addition, the head mounted display apparatus 100 may output the center of the restrictor 250 to be located at the center of the output image 200 , and to position the center of the restrictor 250 at the center of the user's gaze. You can also print

The head mounted display apparatus 100 may track a movement of a user wearing the head mounted display apparatus 100 , for example, a head movement and/or an eye movement, and adjust a viewing angle based on the tracked movement. In this way, the head mounted display apparatus 100 may reduce the user's sense of motion sickness by adjusting the viewing angle according to the user's movement.

The basic viewing angle of the human eye is 60 degrees in the nose direction, 90 degrees outward, 60 degrees upward, and 60 degrees downward from the normal of the left eye (the right eye in relation to the other person) from the frontal direction. 180 degrees horizontally and 120 degrees vertically, respectively. However, since the structure of the retina is different for each person, the range of such a viewing angle has a different range for each individual. However, in the case of the prior art, in setting the viewing angle, since an average viewing angle of a known person is set as a reference, an image optimized for an individual viewing angle cannot be provided. In particular, since the head mounted display device provides an image based on a uniformly applied viewing angle without considering the viewing angle for each user wearing the head mounted display device even though the range of images recognizable by the user is wide, the image is provided as described above. There was a problem that the effect of reducing motion sickness did not occur efficiently.

In addition, although rendering can be performed more efficiently when rendering is performed in consideration of the user's viewing angle, in the case of the prior art, since rendering is performed based on a uniform viewing angle, there is a problem that the resource load of the graphics card is excessive existed. Therefore, the head mounted display apparatus 100 according to an embodiment is an invention devised to solve this problem. Before outputting an image, an individual's viewing angle is measured, and then rendering is performed based on the measured viewing angle. The purpose of providing an individually optimized image exists. Hereinafter, the present invention will be described in detail with reference to the drawings.

2 is a block diagram illustrating a configuration of a head mounted display apparatus according to various embodiments of the present disclosure.

Referring to FIG. 2 , the head mounted display apparatus 100 may include a display 110 , a memory 120 , a controller 130 , and a tracking module 140 .

The display 110 may output an image processed by the head mounted display apparatus 100 or an image transmitted to the head mounted display apparatus 100 .

For example, the display 110 may output a virtual reality (VR), augmented reality (AR), 3D image, etc., and the display 110 may output an image in which the viewing angle of the output image is adjusted.

The memory 120 may store data supporting various functions of the head mounted display apparatus 100 . For example, the memory 120 may store various application programs driven in the head mounted display apparatus 100 , data for control, and commands.

The controller 130 may control overall control of the head mounted display apparatus 100 .

For example, the controller 130 may control the head mounted display apparatus 100 to output an image, and may control the tracking module 140 to track the user's movement. Also, the controller 130 may calculate a viewing angle of an image to be output based on the tracked user movement, and may output an image to which the calculated viewing angle is applied to the display 110 .

Accordingly, the controller 130 may include at least one processor. In addition, the control of the controller 130 may be performed by the processor executing the execution words stored in the memory.

The tracking module 140 may include an eye tracking module 142 and a head tracking module 144 , and the tracking module 140 may detect and measure the movement of a user wearing the head mounted display device 100 . there is.

The eye tracking module 142 may track the eye movement of a user wearing the head mounted display apparatus 100 . Accordingly, the eye tracking module 142 may track not only the user's eye movement but also the eye movement.

The head tracking module 144 may track the head movement of the user wearing the head mounted display apparatus 100 . For example, the head tracking module 144 may sense each of a pitch, a yaw, and a roll based on the center of the head of the user wearing the head mounted display apparatus 100 .

The tracking module 140 may include various sensors to track the above-described user's movement. For example, the tracking module 140 is a proximity sensor (proximity sensor), an illumination sensor (illumination sensor), a touch sensor (touch sensor), an acceleration sensor (acceleration sensor), a magnetic sensor (magnetic sensor), a gravity sensor (G- sensor), a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor), an ultrasonic sensor, an optical sensor (eg, a camera) , may include at least one of a microphone (microphone).

The head mounted display apparatus 100 may include various configurations in addition to the aforementioned configuration. For example, the head mounted display apparatus 100 may include a communication module for wired/wireless communication, an input/output port for inputting/outputting data, an input module for receiving a user input, and the like. Another configuration of the head mounted display apparatus 100 will be described later.

A method of controlling the head mounted display apparatus 100 will be described with reference to FIG. 3 . 3 is a flowchart illustrating a method of controlling a head mounted display apparatus according to various embodiments of the present disclosure.

Referring to FIG. 3 , the head mounted display apparatus 100 may track the movement of at least one of the eyes and the head of the user wearing the head mounted display apparatus 100 ( S10 ).

For example, the tracking module 140 of the head mounted display apparatus 100 may sense the movement of at least one of the eyes and the head of the user wearing the head mounted display apparatus 100 .

In one embodiment, the eye tracking module 142 may track the movement of the user's eyes and gaze, and the head tracking module 144 may track the movement of the user's head.

In another embodiment, each of the eye tracking module 142 and the head tracking module 144 may track the user's eye movement, gaze movement, and head movement, respectively.

Accordingly, the head mounted display apparatus 100 may not only track the movement of the user based on the sensed movement of at least one of the user's eyes and the head, but also determine the position the user is looking at in the output image. there is.

When the user's eye or head movement is tracked, the head mounted display apparatus 100 may calculate a viewing angle of an image to be output based on the tracked movement ( S20 ).

Specifically, the controller 130 of the head mounted display apparatus 100 may calculate a field of view of an image to be output to the display 110 based on the user movement tracked by the tracking module 140 .

For example, the controller 130 may calculate the viewing angle of the image to be output to adjust the viewing angle in proportion to the tracked movement. Specifically, the controller 130 may calculate the viewing angle of the image to be output to adjust the viewing angle of the image to be output in proportion to the tracked movement of at least one of the user's eyes and head.

When the viewing angle of the image to be output is calculated, the head mounted display apparatus 100 may output the image based on the calculated viewing angle. (S30)

4 is a block diagram illustrating another configuration of a head mounted display apparatus according to various embodiments of the present disclosure.

4 is a block diagram illustrating another configuration of the head mounted display apparatus 100 according to various embodiments of the present disclosure. The configuration of FIG. 4 may be added to the configuration of the head mounted display apparatus 100 of FIG. 2 .

Referring to FIG. 4 , the head mounted display apparatus may include a control unit 310 , a region control unit 320 , an image quality control unit 330 , a measurement unit 340 , and a user manipulation unit 350 .

The controller 310 controls overall control of the head-mounted display apparatus. The controller 310 processes or drives signals, data, information, etc. input or output through the components of the above-described head-mounted display apparatus, thereby processing or driving the head-mounted display apparatus. It may provide or process appropriate information or functions for the operation of the device.

The region control unit 320 may divide the first region and the second region in the output image.

Specifically, the area control unit 320 may divide the first area and the second area based on the user's viewing angle, and set the first area and the second area based on the information received from the above-described tracking module 140 . can be adjusted The region control unit 320 may define a center of the region that the user gazes at as the first region, and may define a region outside the first region as the second region. Accordingly, the first region may be referred to as a central vision region, and the second region may be referred to as a peripheral vision region.

In the process of measuring the user's viewing angle, the area adjusting unit 320 may reduce or expand the size of the first area. For example, when the region within the image is referred to as a virtual circle, an area disposed inside the virtual circle is referred to as a first area, and an area disposed outside the virtual circle is referred to as a second area, the area adjusting unit 320 ) is a method of reducing or expanding the diameter of the virtual circle, and the size of the first region and the size of the second region can be varied.

In the process of defining the customized image quality for each user, the region adjusting unit 320 may define the first region and the second region based on the tracking module 140 or the viewing angle defined for each user. When the tracking module 140 in the head mounted display is present, the point the user's eyes look at can be designated as the center point of the first area and the second area, but when the tracking module 140 in the head mounted display does not exist , a central portion of the output image may be defined as a first area, and an outside of the first area may be defined as a second area.

The quality control unit 330 may adjust the image quality of the output image. Adjusting the image quality means adjusting the quality of the resolution or picture quality of an output image.

An image output through the image quality control unit 330 may have a plurality of regions having different image qualities. For example, the image quality adjusting unit 330 may output an image having a high image quality for the image output from the first region, and output an image with a relatively low image quality for the image output from the second region. there is. The image quality control unit 330 may adjust the image to be sequentially outputted from the first region to the second region. The image quality control unit 330 may render an image of the highest quality that can be output by the head mounted display device in the first area.

As in the above-described method, the first area belonging to the user's viewing angle is divided into a first area and a second area, and rendering is performed with the highest quality, and the second area corresponding to the user's peripheral vision area is rendered with relatively low quality. In this case, there is no need to perform high-quality rendering in an area that cannot be accurately recognized by the user, thereby reducing the resource load of the graphics card without compromising the user's immersion.

The measurement unit 340 may measure the viewing angle and image quality provided in the process of customizing an image for each user.

Specifically, the measurement unit 340 may measure the viewing angle and image quality value at a point in time recognized by the user among the viewing angles and image qualities that are changed through the area control unit 320 and the image quality control unit 330 . Here, the time point recognized by the user may be determined by the user's input through the user manipulation unit 350 . The viewing angle and image quality values for each user measured by the measurement unit 340 may be stored in the memory 120 , and the control unit 310 may use a plurality of data of a single user input to the memory 120 to obtain an average and standard deviation. can be calculated.

The user manipulation unit 350 is for inputting the recognition timing of the first region and the second region through the driving of the region adjusting unit 320 or the image quality adjusting unit 330 , and may be manipulated by the user. The user manipulation unit 350 may be disposed on the outer surface of the head mounted display device. As an example, the user manipulation unit 350 may include a button (not shown) pressed by the user.

Hereinafter, driving of the head mounted display apparatus 100 customized for each user will be described.

When the user first wears the head mounted display apparatus 100 , the user may go through a customizing step for outputting an image optimized for him/her.

Specifically, in the customizing step, the customizing screen may be output to the front that the user stares at. In addition, the memory 120 stores data related to different viewing angles and image quality for each user, so that the user can control an image to be output with a data value customized by the user.

5 is a flowchart of a control method for setting a viewing angle for each user according to various embodiments of the present invention, and FIG. 6 is an exemplary view for explaining a viewing angle setting process according to various embodiments of the present invention.

Among the regions partitioned based on the test screen shown in the left of FIG. 6 , the first region may be defined as an area disposed inside the circle, and the second area may be defined as an area disposed outside the circle. The yellow circle is a virtual circle for explaining the control process of the head mounted display apparatus 100 and may not be displayed on the actual customizing screen.

Referring to FIGS. 5 and 6 , a screen for customization may be output according to the user's wearing of the head mounted display apparatus 100 . The customized screen may be divided into a first area and a second area based on the area the user gazes at. An image having a relatively high image quality may be output to the first region, and an image having a relatively low image quality may be output to the second region.

Initially, a first region having the maximum size in the image may be formed. For example, as shown in the test screen of FIG. 6 , the yellow circle may have a diameter value of 10, which is the largest diameter. Next, the yellow circle of the test screen may be reduced so that the diameter value of the first area decreases from the periphery of the user's gaze toward the center through the area control unit 320 ( S110 ). Accordingly, the second region having a relatively low quality value increases, and the first region having a relatively high quality value decreases.

As described above, since the virtual circle is not displayed on the customizing screen, the second region of relatively low quality increases as the first region is reduced, and the boundary between the first region and the second region having different image quality. The first area may be reduced until the user recognizes .

When the first region continues to be reduced by the region adjusting unit 320 , the user may recognize the boundary between the first region and the second region. In this case, the user may input a recognition time according to the division of the first area and the second area through the manipulation of the user manipulation unit 350 . The measurement unit 340 may measure the size of the first area at the time of user recognition based on the user's input (S120).

For example, in the test screen of FIG. 6 , in the case of the first test, the user recognizes the distinction between the first region and the second region when the size of the first region has a diameter of 5.5 as an example, and the second test In the case of , it is exemplified that the user recognizes the distinction between the first area and the second area when the size of the first area has a diameter of 5.4.

The above-described control process may be performed a plurality of times. For example, the user recognition process related to the division of the first area and the second area may be performed five or more times. Accordingly, the memory 120 may store a data value related to the size of the first area at the time of user recognition for each process. In FIG. 6 , the recognition time is input by the user when the first region has a diameter of 5.4 in the second test process, and in the Nth test process, which is the final test, the recognition time is input by the user when the first region has a diameter of 5.3. As an example, a recognition time point is input.

Next, the controller 310 may derive the average and standard deviation of the measured data through a plurality of data values related to the size of the first region ( S130 ).

For example, the average value and standard deviation of the size of the first region may be derived from the plurality of data related to the size of the first region input in the above-described process. In FIG. 6 , the size of the first region at the point in time recognized by the user has an average value of 5.44 and a standard deviation of 0.11 as an example.

If the viewing angle of a specific individual is measured based on one test, the user's viewing angle may not be accurately measured due to the characteristics of objects existing in the image. However, as in the present invention, when the viewing angle is measured through an average value through a plurality of tests, there is an advantage in that the viewing angle can be measured with a relatively small error compared to the actual viewing angle. Although FIG. 6 illustrates that the test is performed n times based on one image, the embodiment of the present invention is not limited thereto, and the test n times may be performed using different images.

A viewing angle for each user may be defined by the process of obtaining the above-described average value. That is, the viewing angle of a specific user may be defined by an average value calculated from data through a plurality of viewing angle measurement processes. Accordingly, an image based on the viewing angle of a specific user may be rendered ( S140 ). That is, the central vision region may be set as the size of the defined first region, and the peripheral vision region having a relatively low image quality may be set and rendered outside the central vision region.

Referring to FIG. 6 , since a specific user has a viewing angle corresponding to an area in which the size of the first area is 5.4, the head mounted display apparatus 100 has a diameter of 5.4 with respect to the center of the area viewed by the specific user. The first area may be set, and the other areas may be set as the second area. In addition, based on the set characteristic user's viewing angle, the head mounted display apparatus 100 outputs an image of relatively high image quality to the first region and foveated to output an image of relatively low image quality to the second region. Rendering (Foved Rendering) may be implemented. When foveated rendering is performed as in the present invention, since foveated rendering is performed according to the user's actual viewing angle rather than a uniform viewing angle, it is possible to provide a more immersive image to the user, and motion sickness can also be reduced. In addition, since rendering is performed at a relatively low quality for an unnecessary part that is difficult for a user to recognize, there is also an effect of reducing the resource load of the graphic card.

Meanwhile, the size of the first area for each specific user may be defined as a value obtained by adding or subtracting a standard deviation from an average value of a plurality of measured data. For example, when the user desires that the first region of high image quality in the image be formed to be relatively large, the head mounted display apparatus 100 sets the size of the first region as a value obtained by adding a standard deviation to an average value of a plurality of data. can form. That is, like the diameter of the orange circle R2 in the rendering screen of FIG. 6 , the size of the first region may be determined by setting the diameter of the circle to a value obtained by adding the standard deviation to the average (5.55=5.44+0.11).

As another example, if it is desired that the first area be formed relatively small in consideration of the capacity of the graphic card or battery, the head mounted display apparatus 100 may display a circle like the diameter of the yellow circle R1 in the rendering screen of FIG. 6 . The size of the first region can be determined by setting the diameter of the ?

The image quality in the second region may be set to be lower as the distance from the first region increases. Accordingly, the image output from the head mounted display apparatus 100 may have a gradation effect of image quality that decreases toward the outside of the user-centered viewing area. If the image quality is sharply lowered from a specific boundary line, the immersion of the image may be reduced due to the difference from the boundary line. However, as in the present invention, if the image quality is gradually lowered from the outside of the central vision region to the peripheral vision region, the sense of disparity due to the rapid quality deterioration between regions disappears, thereby increasing the user's immersion feeling.

Next, a process of customizing image quality for each user will be described. The image quality customization process may be performed after the viewing angle customization process. However, the present invention is not limited thereto, and the image quality customizing process may be performed before the viewing angle customizing process.

7 is a flowchart of a control method for setting image quality for each user according to various embodiments of the present invention, and FIG. 8 is an exemplary view for explaining an image quality setting process according to various embodiments of the present invention.

7 and 8 , the custom screen may be divided into a third area and a fourth area according to a value set in the above-described viewing angle customizing process. Here, the third area and the fourth area may be areas partitioned by the viewing angle set in the above-described viewing angle customizing process. That is, the area within the user's viewing angle may be a third area, and a fourth area may be disposed outside the third area.

An image may be output on the initial image quality customizing screen so that the third region and the fourth region have the same image quality. The initial image quality customization screen of FIG. 8 shows that the image quality values of the third and fourth regions are set to the highest values (I=0), but the image quality values are not always output as the highest values. The test can be started from various values according to

Next, the image quality of the fourth region may be gradually lowered through the image quality adjusting unit 330 ( S210 ). That is, in the fourth region compared to the third region, a screen having a lower image value may be gradually output.

According to the deterioration of the image quality of the fourth region, the user may recognize the boundary between the third region and the fourth region. In this case, the user may input a recognition time according to the division of the third region and the fourth region through the manipulation of the user manipulation unit 350 .

The measurement unit 340 may measure the image quality value of the fourth region based on the usage field input at the time of user recognition ( S220 ). For example, in FIG. 8 , in the first test, which is the user's first test, the distinction between the third region and the fourth region is recognized when the image quality of the fourth region has a value of 30 as an example, and in the second test, the second test An example of recognizing the distinction between the third area and the fourth area when the image quality of area 4 has a value of 31 is exemplified.

The above-described control process may be performed a plurality of times. For example, the user recognition process according to the difference in image quality between the third region and the fourth region may be performed five or more times. Accordingly, the memory 120 may store data regarding the image quality value of the second area at the time of user recognition for each process. In FIG. 8 , in the second test process, the recognition time by the user is input when the image quality of the fourth region has a value of 31, and in the final n-th test process, the recognition time by the user is inputted to the user at the time when the image quality has a value of 29 An example in which a recognition time point is input by

Next, the controller 310 may derive the average and standard deviation of the measured data through a plurality of data related to the image quality value of the fourth region ( S230 ). For example, the average value and standard deviation of the image quality of the fourth region may be derived through a plurality of data regarding the image quality value of the fourth region input in the above-described process. In FIG. 8 , it is exemplified that the image quality value of the fourth region at a point in time recognized by the user has an average value of 30. As shown in FIG.

In addition, the image quality value of the fourth region for each user may be defined by the above-described process. That is, the image quality value of the fourth region based on the user's viewing angle may be set by the average value calculated from data through a plurality of measurement processes. Accordingly, an image based on the viewing angle and image quality of a specific user may be rendered ( S240 ).

If the image quality value of the fourth region is measured based on the result value of one test, the recognition range of the user may not be accurately measured due to the characteristics of objects existing in the image. However, as in the present invention, if the recognition range for image quality is measured through an average value through a plurality of tests, there is an advantage in that a more accurate recognition range can be measured.

Meanwhile, in the process of setting the image quality value, similar to the viewing angle setting process, the image quality value of the fourth region may be set by adding or subtracting the standard deviation from the average value of the image quality described above in consideration of various environments. am.

For example, if the user requires overall high image quality of the image, the image quality value of the fourth region may be determined by adding the standard deviation to the average value of a plurality of data. As another example, the graphic card Alternatively, if high image quality is not required for an image in consideration of the battery capacity, the image quality value of the fourth region may be determined by subtracting a standard deviation from an average value of a plurality of data.

Also, in the case of the present invention, as described above, the image quality customizing process may be performed after the viewing angle customizing process. Accordingly, it is said that the image quality customizing process is performed after the viewing angle customizing process, and the above-described third region may be referred to as a central vision region and the fourth region may be referred to as a peripheral vision region.

9 is a view for explaining a process of customizing a viewing angle and image quality according to various embodiments of the present disclosure;

Referring to FIG. 9 , the head mounted display apparatus 100 may implement foveated rendering by repeating the step of customizing the viewing angle for each specific user and the step of customizing the image quality.

Specifically, in carrying out the test as shown in the upper part of FIG. 9, after measuring the viewing angle at which the user can recognize the change of the boundary for each specific image value, forward rendering is performed based on each of the measured viewing angles. can be performed. Therefore, as shown in the lower part of Fig. 9, the image quality value is set to 10 within the viewing angle of 50 degrees, the image quality value is set to 20 when the viewing angle is between 50 and 60 degrees, and the image quality value is set to 20 when the viewing angle is between 60 and 70 degrees. By setting the quality value to 30, four-bit rendering can be performed. In the case of performing forward rendering as in the present invention, even if the forward rendering is performed step by step starting from the boundary line, the rendering is performed in consideration of individual viewing angles, thereby providing an image more optimized for individuals.

Up to now, the components and control method of the head mounted display apparatus 100 according to the embodiment of the present invention have been studied through the drawings.

The head mounted display apparatus 100 according to various embodiments of the present invention measures an individual's viewing angle without uniformly setting the viewing angle, and then performs a foveated image based on the measured viewing angle. There are advantages to being offered.

In addition, there is an advantage in that virtual motion sickness that occurs when using a head mounted display device can be reduced by providing customized images for each individual. There is an advantage of reducing the resources of the graphics card being used.

On the other hand, components, units, modules, components, etc. described as "~" described in this specification may be implemented together or individually as interoperable logic devices. Depictions of different features of modules, units, etc. are intended to emphasize different functional embodiments, and do not necessarily imply that they must be realized by separate hardware or software components. Rather, functionality associated with one or more modules or units may be performed by separate hardware or software components or integrated within common or separate hardware or software components.

Additionally, the logic flows and structural block diagrams described in this patent document describe corresponding acts and/or specific methods supported by corresponding functions and steps supported by the disclosed structural means, and corresponding It can also be used to build software structures and algorithms and their equivalents.

The present description sets forth the best mode of the invention, and provides examples to illustrate the invention, and to enable any person skilled in the art to make or use the invention. The specification thus prepared does not limit the present invention to the specific terms presented.

Although the above has been described with reference to the preferred embodiment of the present invention, those skilled in the art or those having ordinary knowledge in the technical field will not depart from the spirit and technical scope of the present invention described in the claims to be described later. It will be understood that various modifications and variations of the present invention can be made without departing from the scope of the present invention. Accordingly, the technical scope of the present invention should be defined by the claims rather than being limited to the contents described in the detailed description of the specification.

100: head mounted display device
110: display
130: memory
150: controller
170: tracking module
310: control unit
320: area control unit
330: quality control unit
340: measurement unit
350: user control panel

Claims (15)

(a) outputting a measurement screen for customizing;
(b) partitioning a first area in the measurement screen and a second area having a lower image quality than the first area;
(c) reducing the first region;
(d) as the first area is reduced, a first recognition time point at which the user recognizes the boundary between the first area and the second area is measured, and the boundary at the first recognition time point is used as the user's viewing angle setting up; and
(e) setting the region inside the viewing angle as the central vision region of the user and setting the region outside the viewing angle as the peripheral vision region of the user, wherein the image quality of the peripheral vision region is relatively higher than the image quality of the central vision region Rendering with low image quality; Control method of an eye-based personalized head-mounted display device comprising a. According to claim 1,
and the image quality of the peripheral vision region is progressively degraded as the distance from the viewing angle to the peripheral vision region increases.
According to claim 1,
Further comprising the step of repeating the steps (c) and (d) a plurality of times,
The setting of the viewing angle includes setting an average of the data repeated a plurality of times. According to claim 1,
Further comprising the step of repeating the steps (c) and (d) a plurality of times,
The setting of the viewing angle includes setting a value obtained by subtracting or adding a standard deviation to the average of the data repeated a plurality of times. 5. The method of claim 4,
The step of setting the viewing angle is
setting a value obtained by subtracting and adding a standard deviation to the average of the data repeated a plurality of times as a first viewing angle and a second viewing angle, respectively;
The image quality of the central vision region is gradually decreased as the first viewing angle approaches the second viewing angle. According to claim 1,
Before the step (b), the step of tracking the user's body movement or user's eye; According to claim 1,
The method further comprising repeating steps (b) to (f) a plurality of times with different image quality of the first region,
The image quality of the peripheral vision region is progressively lowered as the distance from the viewing angle to the peripheral vision region is increased based on different viewing angles measured for each image quality. (a) outputting a measurement screen for customizing;
(b) partitioning a first area in the measurement screen and a second area having a lower image quality than the first area;
(c) reducing the first region;
(d) as the first area is reduced, a first recognition time point at which the user recognizes the boundary between the first area and the second area is measured, and the boundary at the first recognition time point is used as the user's viewing angle setting up;
(e) setting the inner region of the viewing angle as the central vision region of the user, setting the outer region of the viewing angle as the peripheral vision region of the user, and setting the image quality of the central vision region and the peripheral vision region to be the same to do;
(f) degrading the image quality of the peripheral vision region;
(g) as the image quality of the peripheral vision region deteriorates, a second perception time point at which the user recognizes a boundary between the central vision area and the peripheral vision area is measured, and the peripheral vision area at the second perception time point measuring the image quality value of ; and
(h) rendering an image of the peripheral vision region based on the measured image quality value of the peripheral vision region; 9. The method of claim 8,
Further comprising the step of repeating the steps (f) and (g) a plurality of times,
The image quality value of the peripheral vision region is set as an average of the data repeated a plurality of times. 9. The method of claim 8,
Further comprising the step of repeating the steps (f) and (g) a plurality of times,
The image quality value of the peripheral vision region is set as a value obtained by adding or subtracting a standard deviation to an average of the data repeated a plurality of times. (a) outputting a screen for customizing;
(b) partitioning a third area and a fourth area having the same image quality in the screen;
(c) degrading the image quality of the fourth region;
(d) as the image quality of the fourth region is deteriorated, a second recognition time point is measured at which the user recognizes the boundary between the third area and the fourth area, and the fourth area at the second recognition time point is measured measuring an image quality value; and
(e) setting the image quality value of the fourth region measured in step (d) as the image quality value of the peripheral vision region, and then performing rendering on the peripheral vision region; A method of controlling a personalized head mounted display device. 12. The method of claim 11,
Further comprising the step of repeating the steps (c) and (d) a plurality of times,
The image quality value of the peripheral vision region is set as an average of the data repeated a plurality of times. 12. The method of claim 11,
Further comprising the step of repeating the steps (c) and (d) a plurality of times,
The image quality value of the peripheral vision region is set as a value obtained by adding or subtracting a standard deviation to an average of the data repeated a plurality of times. display;
a region adjusting unit dividing a first region in the display and a second region having a lower image quality than that of the first region;
a measuring unit measuring the size of the first region based on a user input as the size of the first region is reduced; and
After setting the first area measured by the measuring unit as the central vision area, and setting the outer area of the central vision area as the peripheral vision area, the quality value of the image of the peripheral vision area is the image quality of the central vision area and a controller configured to render the images of the central vision region and the peripheral vision region by setting it to a value relatively lower than the value. 15. The method of claim 14,
Further comprising; a tracking module for tracking the movement of the user's eyes and the movement of the user's head;
The area control unit,
An eye-based personalized head-mounted display device for determining the first area and the second area by using the forward direction that the user's eyes are looking at obtained from the tracking module as the center of the first area and the second area.

Images