KR20230090861A - A method and apparatus for evaluating VR and AR game contents using user's human body information - Google Patents

Abstract

The present invention relates to a method and device for evaluating a VR and AR game content using user's biometric information. According to an embodiment of the present invention, provided is a method for evaluating a VR and AR game content using user's biometric information, which includes the steps of: (a) receiving at least one of user's gaze information on virtual reality content and user's biometric information; and (b) visualizing at least one of the user's gaze information and biometric information.

Description

A method and apparatus for evaluating VR and AR game contents using user's biometric information {A method and apparatus for evaluating VR and AR game contents using user's human body information}

The present invention relates to a method and apparatus for evaluating VR and AR game contents using user's biometric information, and more particularly, to a method and apparatus for processing, visualizing, and utilizing user's gaze information and biometric information.

With the development of ICT technology, high-capacity transmission of 5G, and high-speed network services starting commercialization, the application of virtual reality contents is expanding in fields such as smart factories, drones, concerts, and sports broadcasting services.

Virtual reality technology has the characteristics of maximizing a sense of immersion and presence through the interaction of devices with a display environment similar to the real world through HMD (Head Mounted Display).

In addition, through the feature of being able to experience indirectly in a virtual environment, it is possible to conduct training in high-risk industries without exposure to risk, so the ability to cope with disasters and accidents through virtual experience of disaster safety education, which is difficult to experience directly can improve Research is being actively conducted in various fields by utilizing the characteristics of these virtual reality technologies.

In addition, content based on virtual reality technology is not limited to content focused on interest, but can be used in various fields such as information delivery, education, safety, and treatment, so advanced research is being conducted by converging virtual reality.

However, in many studies, it is necessary to derive research results by measuring the response and usability or effectiveness of users or test subjects experiencing virtual reality contents, but there is a lack of indicators or technologies to measure them.

[Patent Document 1] Korean Patent Publication No. 10-2020-0012561

The present invention was created to solve the above problems, and an object of the present invention is to provide a method and apparatus for evaluating VR and AR game contents using user's biometric information.

Another object of the present invention is to provide a method and apparatus for collecting and visualizing at least one of gaze information and biometric information (eg, heart rate) of virtual reality content of a user wearing a virtual reality terminal.

The objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood from the description below.

In order to achieve the above objects, a method for evaluating VR and AR game contents using user's biometric information according to an embodiment of the present invention includes (a) gaze information for virtual reality contents of a user wearing a virtual reality terminal. and receiving at least one of the biometric information of the user. and (b) visualizing at least one of gaze information and biometric information of the user.

In an embodiment, the method for evaluating VR and AR game contents using the user's biometric information, after the step (a), based on at least one of the user's gaze information and biometric information, the virtual reality content of the user Determining a quality evaluation value for the user evaluation item for the; may further include.

In an embodiment, the VR and AR game content evaluation method using the user's biometric information, after the step (a), based on at least one of the user's gaze information and biometric information, the above for the virtual reality content determining an object to be watched and a behavior pattern of the user; and adjusting execution information for the virtual reality content based on the user's gaze target object and behavior pattern.

In an embodiment, the VR and AR game content evaluation method using the user's biometric information, after the step (a), based on at least one of the user's gaze information and the user's biometric information, the user's dangerous situation is generated. determining whether; and transmitting, to the virtual reality terminal, at least one of an execution stop notification of the virtual reality content and a situation notification when the dangerous situation occurs.

In an embodiment, the method for evaluating VR and AR game contents using the user's biometric information, after the step (a), based on at least one of the user's gaze information and biometric information, the virtual corresponding to the user. determining reality content; and transmitting virtual reality content corresponding to the user to the virtual reality terminal.

In an embodiment, an apparatus for evaluating VR and AR game content using user's biometric information includes: a communication unit that receives at least one of user's gaze information for virtual reality content and the user's biometric information; and an output unit that visualizes at least one of the user's gaze information and biometric information.

In an embodiment, the apparatus for evaluating VR and AR game contents using the user's biometric information is based on at least one of the user's gaze information and biometric information, and the quality of the user evaluation item for the user's virtual reality content. A control unit for determining an evaluation value may be further included.

In an embodiment, the apparatus for evaluating VR and AR game contents using the user's biometric information is based on at least one of the user's gaze information and biometric information, and the user's gaze target object and behavior for the virtual reality content It may further include a control unit that determines a pattern and adjusts execution information for the virtual reality content based on the user's gaze target object and behavior pattern.

In an embodiment, the apparatus for evaluating VR and AR game contents using the user's biometric information includes a control unit that determines whether or not a dangerous situation of the user has occurred based on at least one of the user's gaze information and biometric information. The communication unit may transmit at least one of an execution stop notification of the virtual reality content and a situation notification to the virtual reality terminal when the dangerous situation occurs.

In an embodiment, the apparatus for evaluating VR and AR game contents using the user's biometric information includes a control unit that determines virtual reality content corresponding to the user based on at least one of the user's gaze information and biometric information; Further comprising, the communication unit may transmit virtual reality content corresponding to the user to the virtual reality terminal.

Specific details for achieving the above objects will become clear with reference to embodiments to be described later in detail in conjunction with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below, and may be configured in a variety of different forms, so that the disclosure of the present invention is complete and those of ordinary skill in the art to which the present invention belongs ( It is provided hereafter to fully inform the "ordinary skilled person") of the scope of the invention.

According to an embodiment of the present invention, based on the user's biometric information, a user's consumption pattern or content usage tendency may be analyzed to recommend content suitable for a field of interest.

In addition, according to an embodiment of the present invention, user's biometric information is collected, data is accumulated, processed into information, visualized, and used for usability evaluation.

The effects of the present invention are not limited to the above-mentioned effects, and the potential effects expected by the technical features of the present invention will be clearly understood from the description below.

1A and 1B are diagrams illustrating a VR and AR game content evaluation system using biometric information of a user according to an embodiment of the present invention.
2a and 2b are diagrams illustrating a user information visualization solution process according to an embodiment of the present invention.
3A to 3C are diagrams illustrating examples of user gaze information and heart rate according to an embodiment of the present invention.
4 is a diagram illustrating an example of determining a quality evaluation value for a user evaluation item according to an embodiment of the present invention.
5 is a diagram illustrating a VR and AR game content evaluation method using biometric information of a user according to an embodiment of the present invention.
6 is a diagram showing a functional configuration of an apparatus for evaluating VR and AR game contents using biometric information of a user according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail.

Various features of the invention disclosed in the claims may be better understood in consideration of the drawings and detailed description. Devices, methods, manufacturing methods, and various embodiments disclosed in the specification are provided for illustrative purposes. The disclosed structural and functional features are intended to enable a person skilled in the art to specifically implement various embodiments, and are not intended to limit the scope of the invention. The disclosed terms and phrases are intended to provide an easy-to-understand description of the various features of the disclosed invention, and are not intended to limit the scope of the invention.

In describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

Hereinafter, a method and apparatus for evaluating VR and AR game contents using biometric information of a user according to an embodiment of the present invention will be described.

1A and 1B are diagrams illustrating a VR and AR game content evaluation system 100 using biometric information of a user according to an embodiment of the present invention.

Referring to FIGS. 1A and 1B , the content evaluation system 100 may include a virtual reality terminal 110 , a server 120 and a manager server 130 .

For example, the virtual reality terminal 110 may include a Head Mounted Display (HMD). In one embodiment, the virtual reality terminal 110 may include an eye tracking module capable of tracking the pupil of the user therein. In this case, the virtual reality terminal 110 uses the image of the user obtained from the camera included in the virtual reality terminal 110 to track the 2D pupil and the 3D viewpoint vector using the result to track the gaze information. can be calculated.

In addition, the virtual reality terminal 110 extracts the shape of the lips with the edge information of the area where the conversion from the bright area to the dark area is found based on the edge appearing in the center of the image to detect the shape of the lips Human body By extracting signal data, it is possible to calculate interest level information by predicting a user's facial expression change for virtual reality content.

In one embodiment, the virtual reality terminal 110 may include a heart rate sensor. Accordingly, the virtual reality terminal 110 may collect two kinds of data of the user's gaze information and heart rate.

Thereafter, the virtual reality terminal 110 may transmit the collected data to at least one of the server 120 and the manager terminal 130 . The server 120 may store the received data in a database of the server 120 .

In one embodiment, the virtual reality terminal 110 may perform a visualization solution by analyzing data for visualization.

In one embodiment, the manager terminal 130 can import and parse data that has been converted into a DB, process it into data suitable for expressing a chart, and select and visualize data according to a timeline to be displayed.

In one embodiment, the manager terminal 130 may select an appropriate chart for displaying data in which eye gaze information and biometric information (eg, heart rate) are mixed and display time and pulse information changing according to a timeline through the chart. there is.

In one embodiment, for accurate index measurement and analysis, gaze information and biometric information of a user of virtual reality content may be visualized and used to determine a usability evaluation value.

The user's eye gaze information and biometric information can be used to implement a natural UI/UX using eye gaze and screen reproduction of virtual reality content in a manner similar to that seen by humans.

In addition, the user's gaze information and biometric information may be used to recommend virtual reality content suitable for a field of interest by analyzing the user's consumption pattern or content use tendency.

In one embodiment, for various virtual reality contents such as education, research, and games, gaze information and biometric information of the user while using the contents may be used to check the user's attention target or behavior pattern in the progress of the contents, and the virtual reality It is possible to check whether the contents operate in accordance with the intended situation and to adjust the settings of the virtual reality contents so that the user can have more interest.

2a and 2b are diagrams illustrating a user information visualization solution process according to an embodiment of the present invention.

Referring to FIGS. 2A and 2B , gaze information and heart rate may be collected through eye tracking of a user for virtual reality content.

Through eye tracking and heart rate measurement, biometric information can be collected in real time while the user is experiencing virtual reality content, analyzed and visualized, and used for usability evaluation.

In one embodiment, an interface layer can be designed through coupling with an external electronic device to be extended to measure more biometric information.

The server 120 stores data collected while the user experiences virtual reality content in real time in units of time, continuously classifies and collects data, and proceeds with database formation.

For example, data stored in the database of the server 120 in real time may be represented as shown in <Table 1>.

Type Key Columns User
Biosignal
data ID User Number
Eye tracking info
Pupil Rate
- Max Heart Rate
-Min Heart Rate...

3A to 3C are diagrams illustrating examples of user gaze information and heart rate according to an embodiment of the present invention.

Referring to FIGS. 3A to 3C , eye tracking means tracking and recording movements related to the gaze of a person's eyeball, and using computer vision technology and hardware to recognize a user's object, recognize information, preferences, etc. of biometric information can be collected.

Referring to FIG. 3A , through a visualization solution for usability evaluation, the administrator terminal 130 may configure data stored in a database based on a timeline.

In this case, data including gaze information and biometric information may be changed in real time, and may be stored as data in the form of CSV, JSON, or XML, for example.

The manager terminal 130 may retrieve and parse data that has been converted into a DB, process it into data suitable for expressing a chart, and select and provide data suitable for a timeline to be displayed.

The manager terminal 130 may check information on time and heart rate changes according to a timeline through a chart as shown in FIG. 3A so that eye tracking and heart rate data can be visually expressed in real time.

Referring to FIG. 3B , the manager terminal 130 may test the user's gaze and bio signals together by measuring the user's gaze.

The manager terminal 130 may extract a pupil area of the user through eye-tracking of the user and determine a 2D ellipse value for the pupil area.

Thereafter, the manager terminal 130 may determine a 3D pupil center based on the 2D ellipse value, thereby calculating a direction vector for the user's gaze as the user's gaze information.

Referring to FIG. 3C , the heart rate is data recording the number of heart beats per unit time and may be expressed in a unit of beats per minute (BPM). Heart rate is data used to measure and examine the user's physical condition in the medical field, and can be used as an index to measure fatigue and stress in behavior according to changes in heart rate.

In one embodiment, in the case of the game content field, with respect to the relationship between dynamic interaction and game experience, the manager terminal 130 measures the degree of arousal by measuring the heart rate while the user is playing the game, and measures the degree of attention through the heart rate. and can derive analysis results.

Users using virtual reality contents have more interactions than general contents, and many situations requiring physical movements may occur.

In order to evaluate the usability of such virtual reality content, the manager terminal 130 may collect data on human body reactions such as stress and fatigue of the user according to each situation and measure the changing information of the user.

In an embodiment, the manager terminal 130 may obtain the heartbeat signal of the user and calculate the heartbeat rate based on the waveform of the heartbeat signal.

Specifically, the manager terminal 130 measures the heart rate of the user wearing the virtual reality terminal 110 in real time and transmits biometric data to at least one of the server 120 and the manager terminal 130 to collect the data. can

The manager terminal 130 may calculate the heart rate by distinguishing the rise and fall based on each predetermined time interval of the heart rate, and measure the change of the heart rate in real time.

4 is a diagram illustrating an example of determining a quality evaluation value for a user evaluation item according to an embodiment of the present invention.

Referring to FIG. 4 , the manager server 130 may determine a quality evaluation value for a user evaluation item for the user's virtual reality content based on at least one of the user's gaze information and biometric information.

Since virtual reality contents are rapidly evolving by convergence with hardware, the ISO/IEC 9126 standard or the revised ISO/IEC 25010 standard can be used for quality evaluation.

ISO/IEC 9126 is an international standard that defines software quality characteristics and evaluation metrics. It describes the quality characteristics of software from the perspective of users and developers. Characteristics can be defined.

Since virtual reality content is also included in the software category, the manager terminal 130 may determine a quality evaluation value for the usability evaluation item by partially utilizing the criteria of the usability evaluation item of the ISO/IEC standard to suit the virtual reality content.

In one embodiment, the manager terminal 130 considers the software evaluation criteria, psychological experience elements (presence, interaction) and human factor elements (dizziness, motion sickness) of virtual reality content in the user experience element part to evaluate usability. It is possible to determine the quality evaluation value for

For example, usability evaluation items may include functionality, reliability, usability, efficiency, maintainability, and portability.

In an embodiment, the manager terminal 130 may determine a quality evaluation value corresponding to each usability evaluation item according to values of the user's gaze information and biometric information for each usability evaluation item.

In an embodiment, the manager terminal 130 may determine a quality evaluation value corresponding to each usability evaluation item by comparing values of the user's gaze information and biometric information with each threshold value for each usability evaluation item.

5 is a diagram illustrating a VR and AR game content evaluation method using biometric information of a user according to an embodiment of the present invention. In one embodiment, each step of FIG. 5 may be performed by the server 120 or the manager terminal 120 of FIG. 1 .

Referring to FIG. 5 , step S501 is a step of receiving at least one of gaze information about virtual reality content of a user wearing the virtual reality terminal 110 and biometric information of the user. For example, the biometric information may include the user's heart rate, but is not limited thereto and may include various user biometric information.

Step S503 is a step of visualizing at least one of the user's gaze information and biometric information. For example, at least one of the user's gaze information and biometric information may be expressed in a timeline-type chart or graph form.

In an embodiment, a quality evaluation value for a user evaluation item for the user's virtual reality content may be determined based on at least one of the user's gaze information and biometric information.

In one embodiment, based on at least one of the user's gaze information and biometric information, the user's gaze target object and behavior pattern for the virtual reality content are determined, and the user's gaze target object and behavior pattern are determined based on the virtual reality content. You can adjust the execution information for .

For example, the execution information may include at least one of difficulty, atmosphere, and field of view for virtual reality content, but may include various setting-related information without being limited thereto.

In one embodiment, based on at least one of the user's eye gaze information and biometric information, it is determined whether a dangerous situation occurs for the user, and when a dangerous situation occurs, at least one of a virtual reality content execution stop notification and a situation notification. may be transmitted to the virtual reality terminal 110.

In one embodiment, virtual reality content corresponding to the user may be determined based on at least one of the user's gaze information and biometric information, and the virtual reality content corresponding to the user may be transmitted to the virtual reality terminal 110 .

That is, according to the present invention, user-customized virtual reality content may be provided to the user based on at least one of the user's gaze information and biometric information.

In one embodiment, when the user's gaze information is directed to a first object among a plurality of objects included in the virtual reality content, when the user's heart rate becomes greater than a threshold value, the first object corresponding to the user's gaze information is virtualized. Excluded from the real content, instead of the excluded first object, a second object to which the user's gaze information is directed the most among a plurality of objects for a certain period of time and the heart rate does not exceed the threshold value is displayed on the virtual reality content. can be controlled to

For example, when the user's gaze information stays on the first object in the virtual reality content, if the heart rate rises excessively, the risk such as discomfort or shock is prevented, and the second object that the user's interest is considered to be the highest among the other objects excluding this is prevented. By displaying objects, the user experience can be improved.

In this case, the displayed second object may be determined to be the same type of object as the excluded first object in order to reduce a sense of heterogeneity in the virtual reality content. For example, the types of objects may include various types of characteristics such as people, animals, and plants, but are not limited thereto and may be set in various ways.

In one embodiment, when an object is displayed in virtual reality content, gaze information for each part (eg, arm, leg, shoulder) of the user's object may be detected. In this case, the user's interest score for each part of the object may be calculated according to the amount of change in the user's biometric information at the point in time when the gaze information corresponds to each part of the object.

If the interest score is higher than the threshold value, the characteristics (eg size, color) of the corresponding part of the object are controlled to be displayed in the virtual reality content, and the degree of interest according to the amount of change in the user's biometric information for the adjusted and displayed part The score may be recalculated, and the characteristics of the corresponding part of the object may be re-adjusted through feedback, and displayed in virtual reality content.

In one embodiment, motion information of a user calculated through a sensor included in the virtual reality terminal 110 may be received, and a variation amount due to the motion information of the user among variations in the user's biometric information may be corrected. That is, since the change in heart rate according to motion information such as the user's exercise, not the change in heart rate according to the user's gaze information, is irrelevant to the gaze information, accuracy can be improved by correcting it.

Through this, the user's gaze information and biometric information for each part of the object displayed in the virtual reality content is detected, and the characteristics of each part of the object are adjusted using this feedback as feedback, thereby providing the user with the object that the user is most interested in. can do.

For example, if the user has a high interest score for the long legs of the object character, the legs of the object are adjusted to be longer and displayed, and if the interest score is lowered again for the legs that are too long, the legs of the object are adjusted to be shorter again. By displaying, the user's favorite character can be customized and optimized.

6 is a diagram showing a functional configuration of an apparatus 600 for evaluating VR and AR game contents using biometric information of a user according to an embodiment of the present invention. In one embodiment, the content evaluation device 600 of FIG. 6 may be implemented by the server 120 or the manager terminal 120 of FIG. 1 .

In one embodiment, the content evaluation device 700 of FIG. 6 may be implemented in an integrated form of the server 120 and the manager terminal 120 of FIG. 1 .

Referring to FIG. 6 , the content evaluation device 600 may include a communication unit 610 , a display unit 620 , a control unit 630 and a storage unit 640 .

The communication unit 610 may receive at least one of eye gaze information of the user wearing the virtual reality terminal 110 and biometric information of the user.

In one embodiment, the communication unit 610 may include at least one of a wired communication module and a wireless communication module. All or part of the communication unit 610 may be referred to as a 'transmitter', a 'receiver' or a 'transceiver'.

The display unit 620 may visualize at least one of the user's gaze information and biometric information.

In one embodiment, the display unit 620 may display information processed by the content evaluation device 600 . For example, the display unit 620 may include a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, an Organic LED (OLED) display, and a Micro Electro Mechanical System (MEMS). It may include at least one of a mechanical systems display and an electronic paper display.

The controller 630 may determine a quality evaluation value for a user evaluation item for the user's virtual reality content based on at least one of the user's gaze information and biometric information.

In one embodiment, the controller 630 determines the user's gaze target object and behavior pattern for virtual reality content based on at least one of the user's gaze information and biometric information, and determines the user's gaze target object and behavior pattern. Based on this, it is possible to adjust execution information for virtual reality content.

In an embodiment, the controller 630 may determine whether a danger situation occurs for the user based on at least one of the user's gaze information and biometric information.

In one embodiment, the controller 630 may determine virtual reality content corresponding to the user based on at least one of the user's gaze information and biometric information.

In one embodiment, the controller 630 may include at least one processor or microprocessor, or may be part of a processor. Also, the controller 630 may be referred to as a communication processor (CP). The controller 630 may control the operation of the content evaluation device 600 according to various embodiments of the present disclosure.

The storage unit 640 may store at least one of user gaze information and biometric information.

In one embodiment, the storage unit 640 may be composed of volatile memory, non-volatile memory, or a combination of volatile and non-volatile memory. Also, the storage unit 640 may provide stored data according to the request of the control unit 630 .

Referring to FIG. 6 , the content evaluation device 600 may include a communication unit 610 , a display unit 620 , a control unit 630 and a storage unit 640 . In various embodiments of the present invention, the components described in FIG. 6 are not essential for the content evaluation device 600, so it can be implemented with more or fewer components than those described in FIG. 6. there is.

The above description is only illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention.

The various embodiments disclosed herein may be performed out of order, concurrently or separately.

In one embodiment, at least one step may be omitted or added in each figure described herein, may be performed in reverse order, or may be performed concurrently.

The embodiments disclosed herein are not intended to limit the technical spirit of the present invention, but are intended to explain, and the scope of the present invention is not limited by these embodiments.

The protection scope of the present invention should be interpreted according to the claims, and all technical ideas within the equivalent range should be understood to be included in the scope of the present invention.

100: content rating system
110: virtual reality terminal
120: server
130: manager server
600: content evaluation device
610: communication department
620: display unit
630: control unit
640: storage unit

Claims (10)

(a) receiving at least one of gaze information about virtual reality content of a user wearing a virtual reality terminal and biometric information of the user; and
(b) visualizing at least one of gaze information and biometric information of the user;
including,
A method for evaluating VR and AR game contents using user's biometric information.
According to claim 1,
After step (a),
determining a quality evaluation value for a user evaluation item for virtual reality content of the user based on at least one of gaze information and biometric information of the user;
Including more,
A method for evaluating VR and AR game contents using user's biometric information.
According to claim 1,
After step (a),
determining an object to be watched and a behavior pattern of the user for the virtual reality content, based on at least one of gaze information and biometric information of the user; and
adjusting execution information for the virtual reality content based on the user's gaze target object and behavior pattern;
Including more,
A method for evaluating VR and AR game contents using user's biometric information.
According to claim 1,
After step (a),
determining whether a dangerous situation of the user has occurred, based on at least one of gaze information and biometric information of the user; and
transmitting at least one of an execution stop notification of the virtual reality content and a situation notification to the virtual reality terminal when the dangerous situation occurs;
Including more,
A method for evaluating VR and AR game contents using user's biometric information.
According to claim 1,
After step (a),
determining virtual reality content corresponding to the user based on at least one of gaze information and biometric information of the user; and
Transmitting virtual reality content corresponding to the user to the virtual reality terminal;
Including more,
A method for evaluating VR and AR game contents using user's biometric information.
a communication unit configured to receive at least one of gaze information of a user wearing a virtual reality terminal and biometric information of the user; and
an output unit that visualizes at least one of gaze information and biometric information of the user;
including,
VR and AR game content evaluation device using user's biometric information.
According to claim 6,
a controller that determines a quality evaluation value for a user evaluation item for virtual reality content of the user based on at least one of gaze information and biometric information of the user;
Including more,
VR and AR game content evaluation device using user's biometric information.
According to claim 6,
Based on at least one of gaze information and biometric information of the user, determining an object to be watched and a behavior pattern of the user for the virtual reality content;
a control unit that adjusts execution information for the virtual reality content based on the user's gaze target object and behavior pattern;
Including more,
VR and AR game content evaluation device using user's biometric information.
According to claim 6,
a controller configured to determine whether or not a dangerous situation of the user has occurred based on at least one of gaze information and biometric information of the user;
Including more,
The communication unit transmits at least one of an execution stop notification of the virtual reality content and a situation notification to the virtual reality terminal when the dangerous situation occurs.
VR and AR game content evaluation device using user's biometric information.
According to claim 6,
a controller configured to determine virtual reality content corresponding to the user based on at least one of gaze information and biometric information of the user;
Including more,
The communication unit transmits virtual reality content corresponding to the user to the virtual reality terminal,
VR and AR game content evaluation device using user's biometric information.

Images