KR102389686B1 - Apparatus and method for switching mixed reality contents - Google Patents

Abstract

According to an embodiment of the present disclosure, a system for providing mixed reality (MR) content includes: an output device; at least one or more sensors; and at least one control unit electromagnetically connected to the output device and the at least one sensor, wherein the at least one control unit reproduces virtual reality (VR) content and responds to detection of a first input to play an augmented reality (AR) content, detect a second input associated with the AR content while playing the AR content, and compare the second input with a predetermined threshold for the AR content and calculate a result value, wherein the detection of the first input is performed within a predetermined playback period of the VR content.

Description

Apparatus and method for switching mixed reality contents

Various embodiments of the present disclosure relate to a technology for switching between mixed reality (MR) content, for example, between virtual reality (VR) content and augmented reality (AR) content. An apparatus or method for conversion is provided.

Mixed reality (MR) includes the meaning of augmented reality (AR) in which virtual information is added based on reality and augmented virtuality (AV) in which real information is added to a virtual environment. In other words, by providing a smart environment in which reality and virtual are naturally connected, users can have a rich experience. For example, you can interact with a virtual pet placed on the user's palm, or play a game by building a virtual game environment in a real room. You can also virtually rearrange furniture in your home, or create an environment where remote people gather and work together. The implementation of mixed reality usually goes through a four-step process consisting of real world recognition, virtual object creation, real and virtual fusion (synthesis), and mixed image viewing, and the technology used according to these processes is the real world. It includes vision technology for recognition, 3D modeling and authoring technology, real-time image and virtual image synthesis technology, real-time rendering technology, and interface technology. MR content goes beyond simply being a generic term for VR content and AR content, VR content and AR content form a single theme, and by providing a series of content services, for the purpose of delivering a rich user experience, It is evolving according to the continuous demand.

The present disclosure provides an apparatus for effectively switching between virtual reality (VR) content and augmented reality (AR) content reproduced in a system for providing mixed reality (MR) content, and provide a way

According to an embodiment of the present disclosure, a system for providing mixed reality (MR) content includes: an output device; at least one or more sensors; and at least one control unit electromagnetically connected to the output device and the at least one sensor, wherein the at least one control unit reproduces virtual reality (VR) content and responds to detection of a first input to play an augmented reality (AR) content, detect a second input associated with the AR content while playing the AR content, and compare the second input with a predetermined threshold for the AR content and calculate a result value, wherein the detection of the first input is performed within a predetermined playback period of the VR content.

The apparatus and method according to various embodiments of the present disclosure may reduce resources consumed by efficiently performing a content conversion in a process of providing mixed reality (MR) content.

Effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be.

1 illustrates an exemplary HMD in accordance with embodiments of the present disclosure in which embodiments are performed and in accordance with embodiments of the present disclosure;
2 is a flowchart illustrating an operation of switching between contents in a system for providing mixed reality contents according to an embodiment of the present disclosure.
3 is a configuration diagram illustrating an operation of a control unit according to an embodiment of the present disclosure.
4 is a flowchart of an operation for resetting content difficulty according to an embodiment of the present disclosure.

Before entering the detailed description below, it may be helpful in understanding the present invention to set forth definitions of certain words and phrases used throughout this patent document. The term “couple” and its derivatives may refer to any direct or indirect communication between two or more components, or whether these elements are in physical contact with each other. The terms “transmit”, “receive” and “communicate” as well as their derivatives include direct/indirect communication thereof. The terms "include" and "comprise", as well as their derivatives, mean including without limitation. The term “or” is inclusive and means 'and/or'. The phrase "associated with" as well as its derivatives include, be included within, interconnect with, contain , be contained within, connect to or with, couple to or with, be communicable with, cooperate with , interleave, juxtapose, be proximate to, be bound to or with, have, have a property of), to have a relationship to or with, etc. The term “controller” means any device, system, or part thereof that controls at least one operation. Such a controller may be implemented in hardware or a combination of hardware and software and/or firmware. Functions related to a particular controller may be centralized or distributed locally or remotely. When the phrase “at least one” is used with a listed item, it is meant that different combinations of one or more of the listed items may be used. For example, “at least one of A, B, C” includes any of A, B, C, A and B, A and C, B and C, and A and B and C bonds.

In addition, various functions described below may be implemented or supported by one or more computer programs, each computer program being formed of computer readable program code and implemented in a computer readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, related data, or portions thereof, adapted for implementation in a suitable computer-readable program. . The phrase "computer readable program code" includes computer code of any type, including source code, object code and executable code. The phrase "computer readable medium" means read only memory (ROM), random access memory (RAM), hard disk drive, compact disc (CD), digital, etc., by a computer, such as It includes any type of media that can be accessed. It may be a digital video disc (DVD), or any other type of memory. "Non-transitory" computer-readable media excludes wired, wireless, optical, or other communication links that transmit transitory electrical or other signals. Non-transitory computer-readable media includes media in which data can be permanently stored and media in which data can be stored and later overwritten, such as a rewritable optical disk or a removable memory device.

Definitions of specific words and phrases are provided throughout this disclosure, and one of ordinary skill in the art should understand that in most cases, these definitions may apply to past as well as future uses of the defined words and phrases. do.

1 to 4 discussed below and the various embodiments used to explain the principles of the present disclosure are for illustrative purposes only and should not be construed as limiting the scope of the present disclosure in any way. Those skilled in the art will appreciate that the principles of the present disclosure may be implemented in any suitably deployed wireless communication system.

1 illustrates an exemplary HMD 100 in accordance with embodiments of the present disclosure in which embodiments of the present disclosure may be practiced. The embodiment of the HMD 100 shown in FIG. 1 is for illustrative purposes only, the HMD 100 is made of a variety of different configurations, and FIG. 1 limits the scope of the present disclosure to any specific implementation of the HMD. I never do that.

In various embodiments, HMD 100 may have different shapes, and the present disclosure is not limited to any particular shape. For example, HMD 100 may be mounted in a headset for virtual reality (VR) and/or augmented reality (AR) applications, user equipment, mobile station, television It may be a mobile communication device such as a television, a subscriber station, a wireless terminal, a smart phone, a tablet, and the like. As another example, HMD 100 may include a headset and take the form of a wearable electronic device such as glasses, goggles, helmet, etc. for VR and/or AR applications.

1 , the HMD 100 includes an antenna 105 , a radio frequency (RF) transceiver 110 , a transmit (TX) processing circuit 115 , a microphone 120 , and a receive (receive, RX) processing circuitry 125 . In addition, the HMD 100 includes a speaker 130 , a processor 140 , an input / output (I / O) interface (interface, IF) 145 , a touch screen 150 , a display 155 , memory 160 , and one or more sensors 165 . Memory 160 includes a basic operating system (OS) 161 and one or more applications 162 .

The RF transceiver 110 or the receiver is a network (eg, WiFi, Bluetooth, cellular, 5G, long term evolution (LTE), LTE-A (advanced), worldwide interoperability for microwave access (WiMAX), or any other of a wireless network) type) receives an inbound RF signal transmitted by an access point (AP) (eg, a base station, a WiFi router, a Bluetooth device) from the antenna 105 . The RF transceiver 110 or receiver down-converts the inbound RF signal to generate an intermediate frequency (IF) or baseband signal. The IF or baseband signal is sent to RX processing circuitry 125, which generates a processed baseband signal by filtering, decoding, and/or digitizing the baseband or IF signal. The RX processing circuitry 125 transmits the processed baseband signal to the speaker 130 (eg, voice data), or to the processor 140 for further processing (eg, web browsing data).

The TX processing circuitry 115 receives analog or digital voice data from the microphone 120 or other outgoing baseband data (eg, web data, e-mail, or interactive video game data) from the processor 140 . . TX processing circuitry 115 encodes, multiplexes, and/or digitizes the outgoing baseband data to generate a processed baseband or IF signal. The RF transceiver 110 receives the outgoing processed baseband or IF signal from the TX processing circuitry 115 , and up-converts the baseband or IF signal into an RF signal that is transmitted via the antenna 105 .

The processor 140 may include one or more processors or other processing devices, and may control the overall operation of the HMD 100 by executing the OS 161 stored in the memory 160 . For example, processor 140 controls reception of forward channel signals and transmission of reverse channel signals by RF transceiver 110 , RX processing circuitry 125 , and TX processing circuitry 115 according to well-known principles. can do. In some embodiments, processor 140 includes at least one microprocessor or microcontroller. In another embodiment, the processor 140 may be implemented as a processing circuit. The processor 140 may perform the operations or instructions of any process disclosed herein.

In addition, the processor 140 may execute other processes and programs residing in the memory 160 . Processor 140 may move data into or out of memory 160 as required by an executing process. In some embodiments, processor 140 is configured to execute applications 162 based on OS 161 or in response to signals received from eNBs or an operator. The processor 140 is also coupled to an I/O interface 145 that provides the HMD 100 the ability to connect to other devices, such as laptop computers and handheld computers. The I/O interface 145 is a communication path between these peripherals and the processor 140 .

The processor 140 is also coupled to the touchscreen 150 and the display 155 . A user of the HMD 100 may input data into the HMD 100 using the touch screen 150 . Display 155 may be, for example, a liquid crystal display, light-emitting diode (LED) display, OLED capable of rendering text or graphics from video, games, websites, or a combination thereof. (optical LED), active matrix OLED (AMOLED), or other display.

Memory 160 is coupled to processor 140 . Portions of memory 160 may include random access memory (RAM), and other portions of memory 160 may include flash memory or other read-only memory (ROM). there is.

HMD 100 further includes one or more sensor(s) 165 capable of measuring a physical quantity or detecting an activity state of HMD 100 and converting the measured or detected information into an electrical signal. For example, the sensor 165 may include a headset or HMD 100, one or more buttons for touch input in a camera, a gesture sensor, a gyroscope or gyro sensor, an air pressure sensor, a magnetic sensor or magnetometer, an accelerometer or accelerometer, grip sensor, proximity sensor, color sensor (165)H (eg red green blue (RGB) sensor), bio-physical sensor, temperature / humidity sensor, light sensor (165)K, ultraviolet (ultraviolet, UV) sensor , an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an infrared (IR) sensor, an ultrasonic sensor, an iris sensor, a fingerprint sensor, and the like. The sensor(s) 165 may additionally include control circuitry for controlling at least one of the sensors included therein. Any one of these sensor(s) 165 may be located inside the HMD 100 , a headset configured to secure the HMD 100 , or both the headset and the HMD 100 . For example, in some embodiments, HMD 100 may include a headset.

The touch screen 150 may include a touch panel, a (digital) pen sensor, a key, or an ultrasonic input device. The touch screen 150 may recognize a touch input using, for example, at least one of a capacitive method, a pressure reduction method, an infrared method, and an ultrasonic method. Touchscreen 150 may also include control circuitry. In a capacitive manner, touchscreen 150 may recognize touch or proximity.

Although FIG. 1 shows an example of an HMD 100 , various changes may be made to FIG. 1 . For example, the various components of FIG. 1 may be combined, further subdivided, or omitted, and additional components may be added according to specific needs. As one specific example, the processor 140 may be divided into a plurality of processors, such as one or more central processing units (CPUs) and one or more graphics processing units (GPUs). . Also, although FIG. 1 depicts an HMD 100 configured like a mobile phone or smart phone, the HMD 100 may be configured to operate as other types of mobile or stationary devices. In other embodiments, where HMD 100 is a television, HMD 100 may not include a transceiver, touchscreen, or microphone. HMD 100 may include a receiver or decoder without a transmitter or encoder.

2 is a flowchart illustrating an operation of switching between contents in a system for providing mixed reality contents according to an embodiment of the present disclosure.

A system for providing mixed reality content may include an output device, a server, and a network. The output device is a device for receiving VR content or AR content from a server and outputting it to a user, and more specifically, may be defined as the HDM 100 . The network establishes a wireless communication connection between the server and the output device and enables data transmission and reception. The system for providing mixed reality content may further include at least one sensor. The sensor may be included in the output device or configured independently of the output device, and may be operatively connected to at least one of a server, a network, or an output device to enable data transmission/reception. Each of the steps shown in FIG. 2 may be understood as an operation of a system for providing mixed reality content, or an operation of at least one of an output device, a server, a network, and a sensor included as a component of the system. A system for providing mixed reality content may be referred to as a 'system' hereinafter.

In step 201s, the system plays the VR content.

VR contents may be displayed by an output device included as one component of the system. The VR content is pre-stored in a storage device included as a component of the system, and may be reproduced based on content data transmitted to an output device operatively connected to the storage device. According to another embodiment, VR content may be played based on content data transmitted based on a wireless communication connection established in a relationship with a server or an external server included as a component of the system. VR contents are contents related to virtual reality, and may be understood to include only virtual reality contents or dynamic images whose output is implemented in time series by an output means. According to an embodiment of the present disclosure, a user can enjoy VR content transmitted from a server through a wireless communication connection while wearing an output device such as an HMD. VR contents reproduced in the system may be audiovisual materials for the purpose of delivering information. Specifically, as content related to a historical event, it may be cultural education content for a user who views the content to experience it more realistically.

In step 203s, the system may detect a first input.

The first input may be detected by a sensor included as one component of the system. In addition, the sensor unit may be electrically connected or operatively connected to an output device that may be worn or used by the user to receive a signal associated with the first input detected from the sensor. The sensor for detecting the first input may perform the same or similar function as the sensor illustrated in FIG. 1 . The sensor may detect a user's motion, shout, body temperature, touch, etc. as a first input. The first input may be defined according to a preset rule, and may be a value stored in advance by the server. The first input may be input by a user using the system or may be input by a third party from the outside. The first input may be input by an input means operable through a method such as touch, tap, click, gesture, etc. In this case, a signal related to the first input input by the input means is directly transmitted to an output device or control unit can be When the first input is input through the input means, the detection of the first input is not necessarily made by the sensor. Although expressed as the first input, it should not be construed as meaning the most preceding input among the inputs applied from the user or the outside.

In step 205s, the system plays the AR content in response to detection of the first input. AR content may be performed by an output device included as a component of the system. AR content is augmented reality-related content, and may mean only augmented reality-related content or AR content mixed with VR content. When the AR content is reproduced through the output device, the output device may play only the AR content or the AR content together with the VR content. When the first input is detected by the system, the output device may immediately play AR content, and may play AR content at regular time intervals. Also, even if the first input is detected, AR content may be reproduced after determining whether a condition predefined in the system is satisfied. The first input may be made while the VR content is being played. Accordingly, the first input may not be detected in a situation where AR content is being reproduced. The predefined condition may be divided into i) a content condition and ii) a time condition.

The content condition may mean a condition determined based on the VR content being reproduced. Specifically, at a point where the reproduced VR content provides predefined specific information to a user wearing an output device, the corresponding condition may be satisfied. For example, in VR content, the first input may be detected only when a mountain appears as a background. In this case, in a situation in which a mountain does not appear in the background, even if the user performs an operation corresponding to the first input or makes a sound, the AR content corresponding to the detection of the first input may not be reproduced. Also, it may be determined that the first input itself is not detected.

The time condition may be that whether or not the condition is satisfied is determined based on the frame of the VR content reproduced by the system. Accordingly, the AR content may be reproduced only when the first input is detected at a specific point in the entire frame or at a specific section in the entire frame. For example, AR content playback may be performed only when a first input is sensed between 20.5 and 25.5 seconds from a playback start time of VR content. When both of the above-described conditions are satisfied, the AR content may be reproduced in response to the detection of the first input. During the reproduction of the VR content, an operation in which the AR content is reproduced in response to detection of the first input may be referred to as content switching. However, the meaning of these terms should not be construed as limiting that the reproduction of the VR content is stopped from the point in time when the first input is detected. Since the VR content and AR content can be dynamically changed or reproduced in response to detection of the first input, the first input may be understood as an input for switching between the VR content and AR content reproduced in the system for generating the MR content. can For example, assuming that medical training content developed based on historical events is reproduced as VR content, when an image corresponding to the enemy is reproduced, the user may call up AR content by applying a first input. . In this case, the enemy may be set as a target on the AR content, and the user may enjoy the AR content by applying a subsequent input to the target through an input means such as a sensor. When a time condition is added in this situation, even when an enemy appears, the first input must be made within a predetermined time, and the first input condition may be limited.

In step 207s, the system detects a second input.

The second input may be understood as a subsequent input applied while the AR content is being reproduced when the AR content is reproduced according to the first input. Like the first input, the second input may be input through an output device or a sensor attached to or operatively connected to the output device. In addition, it may be input by a third party through an external server, or may be input or corrected according to preset conditions. The first input may be understood as an input preceding the second input, but the meaning of the first input and the second input should not be understood as a term for forcing a chronological order.

In a situation where the second input is applied while the AR content is being reproduced, the second input may be an input related to the AR content. The second input may include user feedback information on a target, message, situation, sound, etc. implemented on AR content. For example, if the AR content to be reproduced includes a target that appears suddenly or a message requesting a specific action from the user, the user may perform an archery action for the target or a subsequent action corresponding to the message, which is It may be understood as a second input including feedback information. Furthermore, the second input may be detected through a sensor attached to the user's body or an input means operable by the user. By sensing the user's heart rate, body temperature, etc., the user's health and psychological state may be detected as feedback information, which may be included in the second input. Also, the second input may be sound information detected through a microphone.

In step 209s, the system may calculate a result value based on a preset threshold in response to detection of the second input.

The preset threshold may be understood as a target value that is recommended to be achieved by a user who experiences AR content. When the second input is detected as feedback information in response to actions (target, message, etc.) included in the AR content and requesting feedback information from the user, it may be determined whether the second input is equal to or greater than a target value. Through this, it is possible to individually determine whether the user achieves a goal for each action. The target value corresponding to the preset threshold is determined to be changeable by the initial setting. The user can adjust the target value by individually selecting the difficulty level, chapter, and the like. The result value may be information on whether the second input is detected to be greater than or equal to a preset threshold. A result value may be individually calculated according to the number of actions for requesting feedback information from the user on the AR content. In addition, by synthesizing whether the target value for each second input has been achieved, whether the target for the corresponding AR content has been achieved can be calculated in the form of a result value. Information on the calculated result value may be displayed through the user's output device or uploaded through an external server.

3 is a configuration diagram illustrating an operation of a control unit according to an embodiment of the present disclosure.

Referring to FIG. 3 , the controller 300 may hierarchically classify the detected second input. For example, when the second input is the user's voice, the decibels (dB) of the voice, the sound pressure, etc. may be measured, digitized, and classified into a plurality of groups. When the second input is a posture or movement of the user's body, it is measured by a sensor attached to the user's body or a sensor attached to an output device, and based on data related to a pre-stored reference posture or reference movement, speed, etc., A plurality of groups for evaluating whether the movement is accurate may be generated, and it may be determined which group the second input belongs to. When the second input is the user's action on the target implemented on the AR content, when the hit rate for the target is determined according to the user's corresponding action, a plurality of groups are created based on the target hit rate, and through this, the target is compared with the target. You can check whether the goal has been achieved.

The controller 300 may calculate a degree of goal achievement based on the detected second input. The goal achievement degree is determined based on the second input and a preset threshold, and may be divided into an external goal achievement degree and an internal goal achievement degree. Whether or not the external goal is achieved may be determined based on the behavior index detected in the system. The behavior index may be determined based on the user's movement, reaction speed, posture, target hit rate, size of shout, etc. measured by a sensor included as one component of the system. The controller 300 may determine that the external goal has been achieved when the behavioral index is equal to or greater than a preset behavioral index threshold. Whether to achieve the internal goal may be determined based on the psychological index detected in the system. The psychological index may be determined based on data about the user's heartbeat, body temperature, pupil movement, tremor, electrocardiogram, etc. measured by a sensor included as one component of the system. When the psychological index is equal to or greater than a preset behavior index threshold, the controller 300 may determine that the internal goal has been achieved. The controller 300 may determine the degree of achievement of the user who experiences the AR content by synthesizing the degree of achievement of the external goal and the degree of achievement of the internal goal. Data on goal achievement is transmitted through the server or stored in the system and can be viewed by the user. When it is determined that the user who experiences the AR content has achieved the goal, a message corresponding to the achievement of the goal may be displayed on the output device. For example, when it is determined that all specified goals have been achieved, a user using MR content for medical training may check an authentication screen related to a medical officer or the like.

4 is a flowchart of an operation for resetting content difficulty according to an embodiment of the present disclosure.

The system may classify the degree of difficulty of goal achievement based on data provided by a plurality of users who use AR content. In addition, based on the pre-stored data, it is possible to suggest and determine the user's tendency or difficulty according to the preference.

In step 401s , the system may evaluate the user's degree of achievement of the goal based on the detection of the user's second input. The evaluation of the user's goal achievement may be performed after the provision of the AR content to a specific user is finished, or may be performed in real time while the AR content is being played. When the degree of goal achievement corresponding to the user's second input does not continuously fall below the predetermined threshold value, the system may determine that the target achievement degree of the corresponding user is low. The evaluation may be made individually for each action that requests feedback information from the user, and may be determined based on a total sum or averaged value thereof.

In step 403s, the system may reset the difficulty level based on the assessed degree of achievement of the goal. The difficulty level may be classified into a plurality of groups. The system may classify and modify the difficulty level based on goal achievement data about a plurality of users using AR content. By comparing the target achievement degree of a specific user with the target achievement degree of other users existing in the database, a difficulty level corresponding to the target achievement degree of the specific user may be suggested. After the provision of AR content is finished, the user's degree of goal achievement is evaluated and quantified, and a higher difficulty level can be suggested when a threshold required to achieve the goal is exceeded. In addition, when the user's goal achievement level does not reach the threshold required to achieve the goal, a lower difficulty level may be suggested. When the user uses the MR content again, it is possible to suggest a difficulty reset on the initial setting screen based on the data on the target achievement level of the user. In addition, without a proposal to reset the difficulty, the system may reset the difficulty corresponding to the user's past goal achievement level by itself, based on the user's usage record, and provide MR content based on the reset difficulty level.

In step 405s, the system may update the reset difficulty level.

Through a server included in the system or an external server operatively connected to the system, it is possible to upload or update difficulty information for a specific reset user. When a specific user wants to use the MR content again, an appropriate difficulty level may be suggested to the user based on the updated difficulty level information.

According to an embodiment of the present disclosure, a system for providing mixed reality (MR) content includes: an output device; at least one or more sensors; and at least one control unit electromagnetically connected to the output device and the at least one sensor, wherein the at least one control unit reproduces virtual reality (VR) content and responds to detection of a first input to play an augmented reality (AR) content, detect a second input associated with the AR content while playing the AR content, and compare the second input with a predetermined threshold for the AR content and to calculate a result value, and the detection of the first input may be performed within a predetermined playback period of the VR content.

Preferably, the second input includes a third input related to the user's psychology and a fourth input related to the user's behavior, wherein the third input is related to at least one of sound, impedance, vibration period, and temperature. based on the detection, and the fourth input may be detected based on at least one of a speed, an angle, and a moving direction.

Preferably, the predetermined playback period may be at least one of a threshold period for the amount of packets of content data output from the VR content or a time period of the VR content to be reproduced.

Preferably, the at least one sensor may detect the second input based on a layered threshold.

Preferably, the at least one processor may be further configured to determine whether the user has achieved a goal for AR content based on the result value, and output a message indicating whether the determined goal has been achieved.

Preferably, the result value may be determined according to whether a value detected by the second input is equal to or greater than the pre-stored threshold value.

According to another embodiment of the present disclosure, a method of operating a system for providing mixed reality (MR) content includes a process of reproducing virtual reality (VR) content, and a response to detection of a first input Thus, a process of playing an augmented reality (AR) content, a process of detecting a second input associated with the AR content while playing the AR content, and comparing the second input with a predetermined threshold, and calculating a result value for the AR content, and the detection of the first input may be performed within a predetermined playback section of the VR content.

Preferably, the second input includes a third input related to the user's psychology and a fourth input related to the user's behavior, wherein the third input is related to at least one of sound, impedance, vibration period, and temperature. based on the detection, and the fourth input may be detected based on at least one of a speed, an angle, and a moving direction.

Preferably, the predetermined playback period may be at least one of a threshold period for the amount of packets of content data output from the VR content or a time period of the VR content to be reproduced.

Preferably, the process of detecting the second input may be performed based on a layered threshold.

Preferably, the method may further include, based on the result value, determining whether the user has achieved a goal for the AR content, and outputting a message indicating whether the determined goal has been achieved.

Preferably, the result value may be determined according to whether a value detected by the second input is equal to or greater than the pre-stored threshold value.

The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more items, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C" each may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as “first”, “second”, or “first” or “second” may simply be used to distinguish the component from other components in question, and may refer to components in other aspects (e.g., importance or order) is not limited. that one (eg first) component is "coupled" or "connected" to another (eg, second) component with or without the terms "functionally" or "communicatively" When referenced, it means that one component can be coupled to another component directly (eg, by wire), wirelessly, or through a third component.

As used herein, the term “module” may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic block, component, or circuit. A module may be an integrally formed part or a minimum unit of a part or a part thereof that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document include software (eg, a program) including one or more instructions stored in a storage medium (eg, internal memory or external memory) readable by a machine (eg, an electronic device). can be implemented as For example, a device (eg, a processor (eg, a processor) of an electronic device) may invoke and execute at least one of one or more instructions stored from a storage medium. In accordance with , the one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A device-readable storage medium includes: It may be provided in the form of a non-transitory storage medium, where 'non-transitory' means that the storage medium is a tangible device and does not include a signal (eg, EM wave). However, this term does not distinguish between a case in which data is semi-permanently stored in a storage medium and a case in which data is temporarily stored.

According to an embodiment, the method according to various embodiments disclosed in this document may be provided by being included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store™) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a device-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, a module or a program) of the described components may include a singular or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to integration. According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, omitted, or Or one or more other operations may be added.

HMD 100
RF transceiver 110
TX processing circuit 115
microphone 120
RX processing circuit 125
speaker 130
processor 140
I/O IF 145
touch screen 150
display 155
memory 160
operating system 161
Applications 162
sensor(s) 165

Claims (12)

A system for providing mixed reality (MR) content, the system comprising:
output device;
at least one or more sensors; and
At least one control unit electromagnetically connected to the output device and the at least one sensor,
The at least one control unit,
Play virtual reality (VR) content,
in response to detection of the first input, play augmented reality (AR) content;
detecting a second input associated with the AR content while playing the AR content;
and calculate a result value for the AR content by comparing the second input with a predetermined threshold,
The detection of the first input is performed within a predetermined playback period of the VR content,

The predetermined playback section is,
The system is at least one of a threshold period for the amount of content data packets output from the VR content or a time period of the VR content to be reproduced.
The method according to claim 1,
The second input is
a third input associated with the user's psychology and a fourth input associated with the user's behavior;
The third input is detected based on at least one of sound, impedance, vibration period, or temperature,
The fourth input is detected based on at least one of a speed, an angle, and a moving direction.
delete The method according to claim 1,
The at least one or more sensors,
A system for detecting the second input based on a layered threshold.
The method according to claim 1,
The at least one or more processors,
Based on the result value, it is determined whether the user's goal for AR content is achieved,
The system further configured to output a message indicating whether the determined goal has been achieved.
6. The method of claim 5,
wherein the result value is determined according to whether a value detected by the second input is greater than or equal to the predetermined threshold.
A method of operating a system for providing mixed reality (MR) content, the method comprising:
The process of playing virtual reality (VR) content, and
A process of reproducing augmented reality (AR) content in response to detection of a first input;
detecting a second input associated with the AR content while playing the AR content;
Comparing the second input with a predetermined threshold and calculating a result value for the AR content,
The detection of the first input is performed within a predetermined playback period of the VR content,

The predetermined playback section is,
The method is at least one of a threshold period for the amount of content data packets output from the VR content or a time period of the VR content to be reproduced.
8. The method of claim 7,
The second input is
a third input associated with the user's psychology and a fourth input associated with the user's behavior;
The third input is detected based on at least one of sound, impedance, vibration period, or temperature,
The fourth input is detected based on at least one of a speed, an angle, and a moving direction.
delete 8. The method of claim 7,
The process of detecting the second input is
How it is done based on stratified thresholds.
8. The method of claim 7,
Based on the result value, the process of determining whether the user's goal for AR content is achieved;
The method further comprising the step of outputting a message indicating whether the determined goal has been achieved.
12. The method of claim 11,
wherein the result value is determined according to whether a value detected by the second input is equal to or greater than the predetermined threshold.

Images