KR102055481B1 - Method and apparatus for quantitative evaluation assessment of vr content perceptual quality using deep running analysis of vr sickness factors - Google Patents

Abstract

VR content sensation quality evaluation method according to the present invention, based on deep learning, analyzing at least one fatigue causing factor from haptic VR content; And based on deep learning, quantitatively predicting a viewer's haptic quality index for the haptic VR content from the analyzed at least one fatigue causing factor.

Description

Apparatus and method for quantitative evaluation of VR content sensation quality using deep learning analysis of VR fatigue-induced factors {METHOD AND APPARATUS FOR QUANTITATIVE EVALUATION ASSESSMENT OF VR CONTENT PERCEPTUAL QUALITY USING DEEP RUNNING ANALYSIS OF VR SICKNESS FACTORS}

The present invention is an invention for evaluating fatigue felt by a user watching a virtual reality content. Specifically, the cause of fatigue caused by a specific virtual reality content is analyzed through deep learning, and the fatigue degree is quantitatively evaluated. Related to the invention.

Recently, with the development of displays such as 3D display and head mount display (HMD), interest in virtual reality (VR) contents has increased, and the VR-related market is expanding to various fields such as games, broadcasting, and entertainment.

VR content refers to content that immerses viewers as if they are actually experiencing the video they are watching. In order to provide a realistic immersion to viewers, VR content photographed and produced generally has characteristics such as stereo format, 360 degree image information, and fast and many motions.

However, due to these characteristics, when watching high-immersion VR content, about 70 to 80% of viewers are known to have severe fatigue or cyber sickness. And when watching VR content, most viewers who are tired or sick are known to experience this phenomenon even after watching.

Therefore, it is very urgent to develop a technology for evaluating the haptic quality of VR content to prevent such side effects and to watch and produce VR content safely.

An embodiment of the present invention provides a VR content sensation quality evaluation method and apparatus for predicting haptic quality index of VR content to prevent the severe fatigue or cyber sickness that may occur to the viewer after watching the VR content in advance do.

The VR content sensation quality evaluation method according to an embodiment of the present invention includes analyzing at least one fatigue causing factor from haptic VR content based on deep learning; And based on deep learning, quantitatively predicting a viewer's haptic quality index for the haptic VR content from the analyzed at least one fatigue causing factor.

Analyzing the fatigue inducing factor may include obtaining reference information;

Configuring the haptic VR content by reflecting the reference jumbo on the original VR content; And analyzing the at least one fatigue causing factor from the haptic VR content.

The analyzing of the fatigue inducing factor may include analyzing a spatial factor and a temporal factor causing fatigue to the user from the haptic VR content based on deep learning.

Further, the step of analyzing the spatial and temporal factors may be performed based on a convolutional neural network (CNN) or an auto encoder, and analyzes the spatial factors, and may be a recurrent neural network (RNN) or long short (LSTM). Term Memory) may be included to analyze the temporal factor.

Analyzing the fatigue inducing factor is a predictive quality index that quantitatively predicts the fatigue degree for the haptic VR content based on deep learning and provided evaluation quality index provided by the user subjectively assessing the fatigue degree for the haptic VR content Learning using a cost function of the liver.

The analyzing of the fatigue inducing factor may include learning using a cost function between biometric information of a user viewing the haptic VR content and measured biometric information measured for the haptic VR content based on deep learning. have.

The quantitatively predicting the haptic quality index may further include quantitatively predicting the viewer's haptic quality index for the haptic VR content from the biometric information of the user viewing the haptic VR content.

VR content experience quality evaluation apparatus according to an embodiment of the present invention, based on deep learning, the analysis unit for analyzing at least one fatigue causing factor from the haptic VR content; And a prediction unit quantitatively predicting the viewer's haptic quality index for the haptic VR content from the analyzed at least one fatigue-inducing factor or the biometric information of the user viewing the haptic VR content based on deep learning. Can be.

The analyzer may obtain reference information, configure the haptic VR content by reflecting the reference information in the original VR content, and analyze the at least one fatigue causing factor from the haptic VR content.

The analysis unit may analyze a spatial factor and a temporal factor that make a user feel fatigued from the haptic VR content based on deep learning.

The analysis unit uses a cost function between a provided evaluation quality index provided by a user subjectively evaluating the fatigue of the haptic VR content and a predicted quality index that quantitatively predicts the fatigue of the haptic VR content based on deep learning. I can learn.

The analysis unit may learn by using a cost function between the biometric information of the user viewing the haptic VR content and the measured biometric information measured for the haptic VR content based on deep learning.

According to an embodiment of the present invention, a method and apparatus for evaluating haptic quality of VR content according to an embodiment of the present invention may predict haptic quality index of VR content so as to prevent severe fatigue or cyber sickness that may occur to the viewer after viewing the VR content. Can be.

1 is a diagram illustrating an apparatus for evaluating haptic VR content quality according to an embodiment.
2 is a diagram for describing a deep learning based analysis unit, according to an exemplary embodiment.
3 is a diagram illustrating a process of acquiring haptic VR content by reflecting a user's head movement.
4 is a diagram for describing a process of acquiring haptic VR content by reflecting a playback speed of a VR playback device.
5 is a view for explaining a process of analyzing fatigue causing factors.
6 is a view for explaining a process of machine learning using the haptic quality index provided by the analysis unit.
7 is a view for explaining a process of machine learning using the haptic quality index and biometric information provided by the analysis unit.
8 and 9 are diagrams for explaining the process of predicting the haptic quality index based on the fatigue inducing factor.
FIG. 10 is a diagram for explaining a process of predicting a haptic quality index based on biometric information as well as a fatigue inducing factor.
FIG. 11 is a diagram for describing a structure of a predictor that predicts a sensory quality index in consideration of various fatigue causing factors.
12 is a flowchart illustrating a method of evaluating VR content sensation quality according to an embodiment.
13 is a flowchart for explaining a step (S1210) of analyzing a fatigue causing factor.

Hereinafter, specific embodiments of the various embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, these specific embodiments do not limit or limit the present invention. Regardless of the reference numerals in the drawings, the same reference numerals denote the same components, and redundant descriptions are omitted.

1 is a diagram illustrating an apparatus for evaluating haptic VR content quality according to an embodiment.

Referring to FIG. 1, the VR content sensation quality evaluation apparatus according to an embodiment of the present invention includes an analyzer 170 and a predictor 180.

The analysis unit 170 analyzes a factor causing fatigue among VR contents based on deep learning. The prediction unit 180 quantitatively predicts the fatigue level, ie, the haptic quality, experienced by the user with respect to the VR content based on deep learning.

The VR content sensation quality evaluation apparatus may obtain the information from the VR playback device 110, the user 120, and the original VR content 130 to perform machine learning to predict the haptic quality index for the VR content.

In the machine learning process 191 of the VR content sensation quality evaluation device, the user 120 may watch the original VR content 130 through the VR playback device 110.

The VR playback device 110 may send reference information 140 about the original VR content 130 to the VR content sensation quality evaluation device. The VR playback device 110 may be, for example, a head mounted display (HMD). Reference information 140 is information that is reflected in the original VR content 130 is used to configure the haptic VR content that the user experiences. The reference information 140 may include, for example, the degree of head movement when the user watches VR content, the user's viewing posture, the degree of movement of the user's focus, and the playback speed (frames per second) of playing the original VR content 130. This may be a time delay between the original VR content and the haptic VR content, the viewing angle, and the VR content rendering information.

The biometric information 150 that is observed while the user 120 views the original VR content 130 through the VR playback device 110 may be sent to the VR content sensation quality evaluation device. The biometric information 150 is information used to determine how much fatigue the user 120 feels with respect to the original VR content 130, and may be used to predict a haptic quality index. The biometric information 150 may be brain waves, heart rate, body temperature, respiratory rate, and the like.

The user 120 may provide a haptic quality index with respect to the fatigue degree of VR content felt through the VR playback device 110, and the provided haptic quality index 160 may be sent to the VR content haptic quality evaluation device. The provided haptic quality index 160 serves as a reference for determining how accurate the haptic quality index predicted by the VR content haptic quality evaluation device is predicted. The VR content haptic quality evaluation apparatus may perform machine learning to minimize the difference between the haptic quotient 160 provided and the predicted haptic quality index.

In the machine learning process 191, the analyzer 170 uses reference information 140, biometric information 150, provided haptic quality index 160, and original VR content 130 to analyze fatigue-inducing factors. Machine learning based on deep learning. The fatigue inducing factor is a factor that causes the fatigue felt by the user 120 while watching the original VR content 130.

In the machine learning process 191, the prediction unit 180 uses the fatigue causing factor and / or the biometric information 150 to predict the quality of experience index as close as possible to the sensory quality index provided by the user 120. Based on the machine learning.

The evaluation process 192 of the VR content haptic quality evaluation device may be a process after the haptic quality evaluation device obtains a reliability above a threshold value through the machine learning process 191. For example, when the predicted haptic quality index predicted by the haptic quality evaluation device shows a difference of about 5% from the haptic quality index provided through the machine learning process 191, the evaluation process 192 may be performed.

The learned haptic quality evaluation device may obtain the original VR content 130 and predict and output the fatigue degree, that is, the haptic quality index, to be experienced by the user.

Viewers can easily identify VR contents that are likely to cause fatigue or cyber sickness through the predicted haptic quality index, and VR content creators can quantitatively understand how fatigue or cyber sickness their VR contents cause. In other words, the predicted haptic quality index may be used as a guideline for VR content creation that helps to produce safer VR content.

2 is a diagram for describing a deep learning based analysis unit, according to an exemplary embodiment.

Referring to FIG. 2, the analysis unit 170 receives information from the VR playback apparatus 110, the user 120, and the original VR content 130, and machine learning through an artificial neural network and analyzes fatigue inducing factors. The fatigue inducing factor is a factor that causes the fatigue felt by the user 120 while watching the original VR content 130. The fatigue-inducing factor may be, for example, image distortion due to the moving speed of the object in the VR content, the rotational movement of the VR content image, and the image stitching.

The analyzer 170 may analyze the fatigue causing factor according to predetermined algorithms. Each of the algorithms is determined to analyze different fatigue inducing factors, and allows the analyzer to machine learn about each fatigue inducing factor. Each algorithm is, for example, a convolutional neural netwrok (CNN), a recurrent neural network (RNN), a long short term memory (LSTM), a gated recurrent unit (GRU), a fully convolutional network (FCN), and the like. It may include at least one of an artificial neural network (ANN).

When the user 120 wears the HMD which is the VR playback device 110 and watches the original VR content 130, the analyzer 170 receives the head movement information of the viewer from the VR playback device 110, and the viewer's gaze. According to the configuration of the haptic VR content reproduced, and can be analyzed based on the fatigue causing factors.

The analysis unit 170 may perform machine learning to analyze the fatigue causing factors more accurately by comparing the relationship between the analyzed fatigue causing factors or the user's biometric information 150 and the haptic quality index 160 provided.

When the user 120 views the original VR content 130 through the VR playback device 110, the haptic VR content may be different from the original VR content 130 due to the viewing environment. For example, when the user 120 views the VR content through the head mounted display (HMD), when the user 120 moves the head, the haptic VR content viewed from the user 120's perspective is the original. Compared to the VR content 130, the field of view, angle, focus, and the like may vary. As another example, if the original VR content 130 has a playback speed of 90 Hz while the VR playback device 110 plays the original VR content 130 at a playback speed of 30 Hz or 60 Hz, the haptic VR content may be the original VR content ( Playback speed may be slow compared to 130).

The haptic quality of the VR content is determined by the haptic VR content that the actual viewer is watching. Accordingly, the VR content sensation quality evaluation device receives the reference information 140 of the VR content through the VR playback device 110, and acquires the haptic VR content by reflecting the reference information 140 on the original VR content 130. Afterwards, it is necessary to analyze what factors cause fatigue based on the acquired VR content.

3 is a diagram illustrating a process of acquiring haptic VR content by reflecting a user's head movement.

Referring to FIG. 3, the analyzer 170 may acquire the haptic VR content 310 by reflecting the head motion information of the user 120 wearing the VR playback device 110 on the head, to the original VR content 130. Can be.

The coordinates of a specific point of a specific frame of the original VR content may be represented as VR (x, y, z), and the vector coordinates of the user 120 moving his head while watching the original VR content (mx, my, mz) If it can be expressed as, the coordinates of a specific point in the viewer's gaze may be represented as VR '(x, y, z) = VR (x-mx, y-my, z-mz).

For example, if a soccer ball is located in the center of a specific frame of soccer-related original VR content, the user moves his head to move his gaze upward. It is located at the bottom.

4 is a diagram for describing a process of acquiring haptic VR content by reflecting a playback speed of a VR playback device.

Referring to FIG. 4, the analyzer 170 may acquire the haptic VR content 310 by reflecting the playback speed of the original VR content 130 and the playable speed of the VR playback device 110.

If the VR playback device 110 provides the VR content to the user 120 at a lower playback speed, even if the frame rate per second of the original VR content 130 is high, the haptic VR that the user 120 experiences The playback speed of the content 310 is different from the original VR content 130.

For example, even if the playback speed of the original VR content 130 is 90 Hz, when the playback speed of the VR playback device 110 is only 30 Hz, the playback speed of the haptic VR content 310 is the playback of the original VR content 130. 30Hz slower than the speed. At this time, the analysis unit 170 may analyze the fatigue causing factor in consideration of the sampling interval 90Hz / 30Hz = 3.

In addition to the playback speed, the analyzer 170 may acquire the haptic VR content 310 by reflecting reference information 160 such as image rendering and time delay in the original VR content 130.

5 is a view for explaining a process of analyzing fatigue causing factors.

Referring to FIG. 5, the analysis unit 170 analyzes the fatigue causing factor 550, and performs a spatial algorithm 530 on each frame 510a to 510n of the haptic VR content 310 reflecting the reference information 140. And temporal algorithm 540 may be applied. The fatigue or cybersickness that the user 120 experiences by watching the VR content is closely related to the spatial and temporal factors.

Spatial factors are fatigue-inducing factors that have spatial characteristics and cause fatigue, such as image distortion or viewing angle problems caused by stitching processes that connect images of various angles. It is a fatigue-inducing factor that has temporal characteristics and causes fatigue.

The spatial and temporal factors are related to each other and cause fatigue to the user 120. For example, if image distortion or rapid movement occurs only in one frame of VR content, the user is unlikely to experience extreme fatigue or cyber sickness. However, if image distortion or rapid movement occurs in several consecutive frames, the quality of the VR content experienced by the user will be very low. Therefore, the analyzer 170 should consider both spatial and temporal factors that can cause fatigue in the haptic VR content 310.

In FIG. 5, a convolutional neural netwrok (CNN) is used as an algorithm 530 for extracting spatial factors of haptic VR content, but a deep learning algorithm useful for learning spatial features such as an auto encoder or short temporal features. You can also use

In FIG. 5, although Long Short Term Memory (LSTM) is used as an algorithm 540 for extracting temporal factors of haptic VR content, learning of temporal features such as Recurrent Neural Network (RNN) and Gated Recurrent Unit (GRU) is performed. You can also use deep learning algorithms to help you.

6 is a view for explaining a process of machine learning using the haptic quality index provided by the analysis unit.

Referring to FIG. 6, the analyzer 170 may perform machine learning to more accurately predict the haptic quality index by using the haptic quality index 160 provided by the user to view and provide VR content. For example, the analyzer 170 may machine learn in a direction of minimizing the cost function based on the provided haptic quality index and the predicted haptic quality index.

를 최소화하는 방향으로 기계 학습할 수 있다.The analysis unit 170 is the cost function of relation 1

Machine learning in a way that minimizes

[Relationship 1]

는 VR 컨텐츠 체감 품질 평가 장치가 예측한 체감 품질 지수이고,

는 사용자(120)가 VR 컨텐츠를 시청하고 제공한 체감 품질 지수이고,

는 예를 들어, cross-entropy loss, L1- norm 또는 L2-norm등의 에러 계산 방법일 수 있으나 이에 국한되지 않는다.In relation 1

Is a haptic quality index predicted by the VR content haptic quality evaluation device,

Is a haptic quality index provided by the user 120 for watching and providing VR content,

For example, may be an error calculation method such as cross-entropy loss, L1-norm or L2-norm, but is not limited thereto.

For example, if it is possible to evaluate the haptic quality index with a score between 0 and 10, and the lower the index, the more severe the fatigue, the user watched VR content and provided a haptic quality index of 3, but the VR content haptic quality evaluation device If the predicted haptic quality index of 5, the analysis unit 170 can be machine learning by looking at 3 and 5 as a variable of the cost function.

7 is a view for explaining a process of machine learning using the haptic quality index and biometric information provided by the analysis unit.

If n total frames of haptic VR content are n, the biometric information 150 synchronized with the time of each frame may be used for machine learning to predict the haptic quality index. The biometric information 150 may be brain wave intensity, heart rate, body temperature, respiratory rate, and the like.

는 매 프레임 또는 짧은 시간의 시각적 특징을 기계 학습할 수 있도록 도와줄 수 있다.At this time, the cost function of relation 2 using the biometric information 150

Can help machine learn every frame or short time visual feature.

[Relationship 2]

는 VR 컨텐츠 체감 품질 평가 장치가 예측한 t번 째 프레임에서의 생체 정보,

는 t번 째 프레임에서 측정된 사용자(120)의 생체 정보(150)이다.In relation 2, n is the total number of frames,

Biometric information in the t-th frame predicted by the VR content sensation quality evaluation apparatus,

Is the biometric information 150 of the user 120 measured in the t th frame.

Performing machine learning on the biometric information may help the analyzer 170 to accurately predict the biometric information when the user 120 views the VR content. Machine learning using the biometric information 150 is intended to improve the quality of each frame of VR content, rather than to improve the overall quality of the VR content.

은 관계식 1 및 관계식 2의 비용 함수인

,

, 가중치 파라미터인

를 사용하여 정의할 수 있다.The analyzer 170 may machine learn based on the biometric information 150 and the provided haptic quality index 160. For example, the cost function in relation 3

Is the cost function of relations 1 and 2

,

, The weight parameter

Can be defined using

[Relationship 3]

As illustrated in FIG. 8 or 9, the prediction unit may predict the haptic quality index from one fatigue causing factor or several fatigue causing factors. Predicting the haptic quality index by considering only one fatigue factor reduces the time spent on machine learning and reduces the cost of constructing the algorithm. However, the accuracy may be inferior to that of predicting the haptic quality index by considering various fatigue inducing factors. Therefore, it is necessary to design in consideration of the trade-off relationship between accuracy and cost.

FIG. 10 is a diagram for explaining a process of predicting a haptic quality index based on biometric information as well as a fatigue inducing factor.

The fatigue inducing factor analyzed by the analysis unit 170 is machine learning to obtain a high-quality sensory quality index, and the biometric information 150 indicates objective fatigue information on the haptic VR content 310, and thus causes fatigue. Machine learning using factor and biometric information can predict the high quality of experience quality index.

FIG. 11 is a diagram for describing a structure of a predictor that predicts a sensory quality index in consideration of various fatigue causing factors.

The structure of the prediction unit may be, for example, a deep learning structure such as a deep neural network (DNN).

Each fatigue induction factor information 1110a to 1110n includes, for example, vector data in which weights of weights of bodily movements, rotational movements of VR content images, image distortions due to image stitching, etc. are taken into consideration for the bodily quality index. It may include. As the data included in each fatigue-inducing factor information 1110a to 1110n passes through the deep neural network, the sensory quality index for the original VR content may be predicted. If it is determined that the weight of the specific fatigue causing factor information on the haptic quality index is small, the performance of the measurement unit 180 may be improved through regularization techniques such as drop out and pruning.

12 is a flowchart illustrating a method of evaluating VR content sensation quality according to an embodiment.

Referring to FIG. 12, the VR content sensation quality evaluation method may include analyzing at least one fatigue causing factor from haptic VR content based on deep learning (S1210).

In one example, the VR content sensation quality evaluation method may analyze the spatial and temporal factors as fatigue-inducing factors.

Spatial factors are fatigue-inducing factors that have spatial characteristics and cause fatigue, such as image distortion or viewing angle problems caused by stitching processes that connect images of various angles. It is a fatigue-inducing factor that has temporal characteristics and causes fatigue.

The spatial and temporal factors are related to each other and cause fatigue to the user 120. For example, if image distortion or rapid movement occurs only in one frame of VR content, the user is unlikely to experience extreme fatigue or cyber sickness. However, if image distortion or rapid movement occurs in several consecutive frames, the quality of the VR content experienced by the user will be very low. Therefore, both spatial and temporal factors that can cause fatigue in haptic VR content should be considered.

As an algorithm for extracting spatial factors of haptic VR content, a deep learning algorithm useful for learning spatial or short temporal features such as a convolutional neural netwrok (CNN) and an auto encoder may be used.

An algorithm for extracting temporal factors of haptic VR content is to use deep learning algorithms useful for learning temporal features such as Long Short Term Memory (LSTM), Recurrent Neural Network (RNN), and Gated Recurrent Unit (GRU). Can be.

In order to machine learn the step S1210, the provided haptic quality index and the predicted haptic quality index that the user watches and provides the VR content may be used. For example, machine learning may be directed toward minimizing the cost function based on the provided haptic quality index and the predicted haptic quality index.

를 최소화하는 방향으로 기계 학습할 수 있다.For example, the cost function of relation 1

Machine learning in a way that minimizes

In order to machine learn the step S1210, biometric information may be used.

When n total frames of haptic VR content are present, biometric information synchronized with the time of each frame may be used for machine learning to predict the haptic quality index. The biometric information may be brain wave intensity, heart rate, body temperature, respiratory rate, and the like.

Step S1210 may be learned to minimize the difference between the measured biometric information and the predicted biometric information.

와 같을 수 있다.At this time, the cost function using the biometric information 150 is, for example,

May be the same as

[Relationship 2]

는 VR 컨텐츠 체감 품질 평가 장치가 예측한 t번 째 프레임에서의 생체 정보,

는 t번 째 프레임에서 사용자로부터 측정된 생체 정보이다.In relation 2, n is the total number of frames,

Biometric information in the t-th frame predicted by the VR content sensation quality evaluation apparatus,

Is biometric information measured from the user in the t th frame.

은 관계식 1 및 관계식 2의 비용 함수인

,

, 가중치 파라미터인

를 사용하여 정의할 수 있다.In operation S1210, the machine learning may be performed based on the biometric information 150 and the haptic quality index 160 provided. For example, the cost function in relation 3

Is the cost function of relations 1 and 2

,

, The weight parameter

Can be defined using

[Relationship 3]

The VR content sensation quality evaluation method includes predicting a haptic quality index for VR content based on deep learning (S1220).

In step S1220, it is possible to predict the haptic quality index from one fatigue trigger factor or several fatigue trigger factors. Deep learning machine learning the relationship between fatigue-inducing factors and the sensory quality index can predict relatively accurate sensory quality index.

Predicting the haptic quality index by considering only one fatigue factor reduces the time spent on machine learning and reduces the cost of constructing the algorithm. However, the accuracy may be inferior to that of predicting the haptic quality index by considering various fatigue inducing factors. Therefore, it is necessary to design in consideration of the trade-off relationship between accuracy and cost.

The fatigue inducing factor analyzed in step S1210 is machine learning to obtain a high-quality sensory quality index, and since the biometric information represents objective fatigue information on haptic VR content, the machine learning is performed using the fatigue inducing factor and biometric information. We can predict the high quality of experience quality index.

In other words, if the machine learning through the deep learning relationship between the biometric information and the sensory quality index can be predicted more accurately than the fatigue induction factor.

In order to machine learn S1220, a deep learning structure such as a deep neural network (DNN) may be used.

Each fatigue induction factor information 1110a to 1110n includes, for example, vector data in which weights of weights of bodily movements, rotational movements of VR content images, image distortions due to image stitching, etc. are taken into consideration for the bodily quality index. It may include. As the data included in each fatigue-inducing factor information 1110a to 1110n passes through the deep neural network, the sensory quality index for the original VR content may be predicted. If it is determined that the weight of the specific fatigue inducing factor information on the haptic quality index is small, the prediction performance can be improved through regularization techniques such as drop out and pruning.

13 is a flowchart for explaining a step (S1210) of analyzing a fatigue causing factor.

The haptic quality of the VR content is determined by the haptic VR content that the actual viewer is watching. Therefore, the reference information 140 of the VR content is obtained from the VR playback apparatus 110 (S1310), and the reference information 140 is reflected on the original VR content 130 to construct the haptic VR content (S1320). Based on the haptic VR content, it is necessary to analyze what factors cause fatigue (S1330).

Therefore, analyzing the fatigue inducing factor (S1210) may include obtaining reference information (S1310).

Reference information 140 is information that is reflected in the original VR content 130 is used to configure the haptic VR content that the user experiences. The reference information 140 may be obtained from the VR playback device 110. The reference information 140 may include, for example, the degree of head movement when the user watches VR content, the user's viewing posture, the degree of movement of the user's focus, and the playback speed (frames per second) of playing the original VR content 130. This may be a time delay between the original VR content and the haptic VR content, the viewing angle, and the VR content rendering information.

After the step S1310, it may include the step of configuring the haptic VR content based on the reference information (S1320).

For example, the haptic VR content 310 may be configured by reflecting the head motion information of the user 120 wearing the VR playback device 110 on the head to the original VR content 130.

The coordinates of a specific point of a specific frame of the original VR content may be represented as VR (x, y, z), and the vector coordinates of the user 120 moving his head while watching the original VR content (mx, my, mz) If it can be expressed as, the coordinates of a specific point in the viewer's gaze can be expressed as VR '(x, y, z) = VR (x-mx, y-my, z-mz) to construct haptic VR content. .

In addition to the head movement, the haptic VR content 310 may be configured by reflecting reference information 160 such as a playback speed, an image rendering, and a time delay in the original VR content 130.

After the step S1320, it is possible to analyze the fatigue inducing factor from the haptic VR content (S1330). Analyzing the fatigue causing factor is omitted as described above.

Although the present invention has been described in detail with reference to the limited embodiments as described above, the present invention is not limited to the above-described embodiments. Those skilled in the art to which the present invention pertains can see and change the claims and the description of the invention, and can be easily changed, modified and carried out to the implementation within the scope of the claims of the present invention.

Claims (12)

Analyzing at least one fatigue inducing factor from haptic VR content based on deep learning in the analysis unit; And
Quantitatively predicting a viewer's haptic quality index for the haptic VR content from the analyzed at least one fatigue causing factor based on deep learning in a predictor;
Including,
Analyzing the fatigue inducing factor
Acquiring one or more pieces of reference information among a head movement degree, a viewing position, a moving degree of focus, a reproduction speed, a time delay, a viewing angle, and rendering information;
Configuring the haptic VR content by reflecting the reference information on the original VR content; And
Analyzing the at least one fatigue causing factor from the configured haptic VR content
VR content haptic quality evaluation method comprising a. delete The method of claim 1,
Analyzing the fatigue inducing factor
Analyzing spatial and temporal factors that make a user feel fatigued from the haptic VR content based on deep learning;
VR content sensation quality evaluation method comprising a. The method of claim 3,
Analyzing the spatial and temporal factors
Analyzing spatial factors based on a convolutional neural network (CNN) or auto encoder, and analyzing temporal factors based on a recurrent neural network (RNN) or long short term memory (LSTM)
VR content sensation quality evaluation method comprising a. The method of claim 1,
Analyzing the fatigue inducing factor
Learning using a cost function between a provided evaluation quality index provided by a user subjectively evaluating the fatigue of the haptic VR content and a predicted quality index quantitatively predicting the fatigue of the haptic VR content based on deep learning
VR content sensation quality evaluation method comprising a. The method of claim 1,
Analyzing the fatigue inducing factor
Learning by using a cost function between biometric information of the user viewing the haptic VR content and measured biometric information measured for the haptic VR content based on deep learning
VR content sensation quality evaluation method comprising a. The method of claim 1,
Quantitatively predicting the haptic quality index
Quantitatively predicting the viewer's haptic quality index for the haptic VR content from the biometric information of the user viewing the haptic VR content
VR content sensation quality evaluation method further comprising. An analysis unit analyzing at least one fatigue causing factor from haptic VR content based on deep learning; And
A predicting unit quantitatively predicting the viewer's haptic quality index for the haptic VR content from the analyzed at least one fatigue-inducing factor or the biometric information of the user viewing the haptic VR content;
Including,
The analysis unit
Acquiring reference information of at least one of head movement degree, viewing posture, focus movement degree, playback speed, time delay, field of view, and rendering information, and constructing haptic VR content by reflecting the reference information in the original VR content. VR content sensation quality evaluation device for analyzing the at least one fatigue causing factor from the configured haptic VR content. delete The method of claim 8,
The analysis unit
The VR content sensation quality evaluation apparatus based on deep learning, analyzing the spatial and temporal factors that make the user feel fatigued from the haptic VR content. The method of claim 8,
The analysis unit
Learning using a cost function between the provided evaluation quality index provided by the user subjectively evaluating the fatigue of the haptic VR content and the predicted quality index quantitatively predicting the fatigue of the haptic VR content based on deep learning. VR content experience quality evaluation device characterized in that. The method of claim 8,
The analysis unit
VR content sensation quality evaluation apparatus, characterized in that the learning using the cost function between the biometric information of the user viewing the haptic VR content and the measured biometric information measured for the haptic VR content based on deep learning.

Images