KR20220013657A - Recording medium for recording by axis synchronization program - Google Patents

Abstract

Provided are an extended reality service device including an axis synchronization function of synchronizing the axes of sensors inside and outside a terminal employed in an extended reality system, an axis synchronization method thereof, a program thereof, and a recording medium for recording the program. The extended reality service device, which provides extended reality-based content to a client including a terminal and a tracker, comprises an axis synchronizer for synchronizing axes of a first sensor installed in the terminal and a second sensor installed in the tracker. The axis synchronizer includes: a rotation information receiving unit for receiving respective rotation information from the terminal and the tracker; an offset calculation unit for calculating an offset for synchronizing the axes of the first sensor and the second sensor based on the rotation information of the terminal and the rotation information of the tracker; and a synchronization unit for synchronizing the axes of the first sensor and the second sensor based on the calculated offset. According to the present invention, the axis synchronization between the sensors inside and outside the terminal can be easily achieved even if a general terminal is used instead of a specific terminal, thereby providing high versatility in choosing a terminal for the extended reality system.

Description

Axis synchronization program recording medium {Recording medium for recording by axis synchronization program}

The present invention relates to an axis synchronization program recording medium, and more particularly, to a computer readable recording medium in which a program for executing a method of synchronizing an axis of a sensor outside a terminal in an extended reality system and an axis of a sensor inside the terminal is recorded. it's about

With the dramatic increase in computing power and the advancement of graphic processing technology and network technology, virtual reality, augmented reality, and mixed reality are being implemented everywhere in our lives.

Virtual reality (VR) refers to a reality that does not actually exist, but appears to exist. Virtual reality refers to an artificial world or such technology in which objects or backgrounds of the real world are artificially implemented using a computer.

Augmented reality (AR) is a technology that reflects or expands the virtual world in the real world. Augmented reality is a concept of adding an augmented information service by synthesizing related information implemented as a virtual object with an object in the real world.

Mixed reality (MR) is a method that integrates virtual reality (VR) and augmented reality (AR) and further strengthens interaction with users. Mixed reality (MR) has similarities to augmented reality in that it shows virtual objects by mixing and combining them in the real world.

However, in augmented reality, the distinction between real and virtual objects is clear and virtual objects are used in a form that complements real objects, whereas in mixed reality virtual objects appear with the same characteristics as real objects and are operated in an independent form. It is clearly distinguished from augmented reality in that it is viewed as immersive as virtual reality.

As described above, technologies that extend the experience of reality have emerged thanks to computing power and graphic processing technology, and these technologies are called eXtended Reality (XR).

Extended reality (XR) refers to ultra-realistic technologies and services that encompass mixed reality (MR) technologies that encompass virtual reality (VR) and augmented reality (AR). In this case, X means a variable, and it is a term that can encompass not only VR, AR, and MR, but also other forms of reality that will appear in the future.

In order to implement extended reality (XR), after a virtual object (and background) is created by a computer, movement is given to the virtual object and background while tracking the change or movement of the user's gaze, and synchronized with the user.

In the case of experiencing such extended reality (XR)-based content, conventionally, only a specific terminal into which an absolute value is input is used.

In addition, when several positioning systems operate together, the same tracking result cannot be derived because the reference points of each other do not match.

Prior Art 1: Republic of Korea Patent Registration No. 10-0953931 (Mixed Reality Realization System and Method) Prior art 2: Republic of Korea Patent Publication No. 10-2019-0118373 (Virtual reality experience system and method) Prior Art 3: Republic of Korea Patent Registration No. 10-0900806 (System and method for providing context-aware augmented reality)

The present invention has been proposed to solve the above-described problems in the prior art, and an extended reality service apparatus including an axis synchronization function for synchronizing the axes of sensors inside and outside a terminal employed in an extended reality system, an axis synchronization method thereof, and a method thereof Its purpose is to provide a program and a recording medium for executing the program on a computer.

In order to achieve the above object, the axis synchronization method program recording medium of the extended reality service device including the axis synchronization function according to a preferred embodiment of the present invention provides extended reality-based content to a client including a terminal and a tracker A method in which the extended reality service device synchronizes the axes of the first sensor in the terminal and the second sensor in the tracker. , measuring the rotated rotation angle to obtain and record the difference between the rotation angles of each axis; Accumulating and recording the difference between the rotation angles of the respective axes of the terminal and the tracker, and obtaining a new rotation angle difference (Cn) between the axis of the terminal and the axis of the tracker; Axis synchronization comprising a shaft synchronization step of reflecting the new rotation angle difference (Cn) to the previous rotation angle difference (C) to obtain and store the offset value to be currently applied, and also reflect the offset value in the axis information of the terminal to synchronize the axis A computer-readable recording medium on which a program for executing a method is recorded is provided.

The axis synchronization method of the extended reality service device including the axis synchronization function of the present invention described above can be implemented as computer-readable codes on a computer-readable recording medium. In addition, the above-described axis synchronization method may be implemented as a program to be executed by a computer, and may be implemented as a recording medium in which the program is recorded.

According to the present invention of this configuration, even if a normal terminal is used instead of a specific terminal, axis synchronization between the internal sensor and the external sensor of the terminal is easily achieved, so that the versatility of the terminal employed in the extended reality system can be provided. In other words, since it becomes easier to use terminals not envisaged by the developer or manufacturer, users can experience extended reality-based content at any time using a terminal not envisaged by the developer or manufacturer.

In addition, even when multiple users experience extended reality-based content in the same space, it is very easy to share the same virtual coordinates with each other, so that the same tracking result can be derived. In addition, it is possible to easily recognize each other between users, and it is possible to eliminate bumps and awkwardness that come from a heterogeneous sense of space.

1 is a block diagram of an extended reality system to which an extended reality service device including an axis synchronization function is applied according to an embodiment of the present invention.
FIG. 2 is an enlarged view of the HMD, the terminal, and the tracker shown in FIG. 1 .
3 is a conceptual diagram of an axis synchronization device of an extended reality service device according to the present invention.
FIG. 4 is an internal configuration diagram of the axis synchronizer shown in FIG. 1 .
5 is a flowchart schematically illustrating an axis synchronization method in an extended reality service device including an axis synchronization function according to an embodiment of the present invention.
FIG. 6 is a flowchart for explaining the step of calculating the offset shown in FIG. 5 in more detail.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In describing the present invention, in order to facilitate the overall understanding, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

1 is a block diagram of an extended reality system to which an extended reality service device including an axis synchronization function is applied according to an embodiment of the present invention, and FIG. 2 is an enlarged view of the HMD, the terminal, and the tracker shown in FIG. 1 .

The extended reality system of FIG. 1 may include a server 50 , a client 100 , and a network 200 .

When the server 50 sends extended reality-based content (eg, content related to games, education, tourism, etc.) to the client 100 through the network 200 , the extended reality-based content through the display unit of the terminal of the client 100 . image is displayed. Accordingly, the user manipulates the controller held in his hand to enjoy extended reality-based contents (eg, games).

The client 100 is used by a user and may be provided in a predetermined game room.

1 shows only one client 100, which is only simplified to help the understanding of the drawing, and in reality there will be a plurality of clients. For example, there may be more than one client 100 per game room.

The client 100 may include a controller 10 , an HMD 20 equipped with a terminal 30 , and a tracker 40 .

The controller 10 may reflect the user's action and transmit the user's will.

For example, the controller 10 may include a motion recognition-based controller, a physical controller, an attached motion recognition-based controller, an EEG-based controller, and the like.

The motion recognition-based controller is preferably used when the location of the users is accurately identified. First, motions of various objects are measured through infrared or image recognition methods. In addition, the user's will can be reflected by comparing the positions of the objects in which the motion has occurred with the previously obtained positions and substituting the motion information into the server 50 .

3DOF, 6DOF physical controller refers to a controller that has a physical button and can be held in a hand. The user's will can be reflected by transmitting information of various embedded chips (gyro, acceleration, compass, image sensor, etc.) to the user or the server 50 .

The attached motion recognition-based controller is a system that is attached to the user's wrist, waist, knee, and foot to position the user. do.

The brain wave recognition-based controller is a form in which a module for detecting brain waves is combined or additionally attached to the terminal 30, and it does not externally judge the user's behavior, but only determines the result and body state of the brain wave to reflect the user's will. have.

A head mounted display (HMD) 20 is worn on the user's head, and a terminal 30 capable of acquiring actual image information by photographing an image around the user is mounted (see FIG. 2 ). Although it is referred to as an HMD in FIG. 1, it may be a face mounted display (FMD).

The terminal 30 may include a sensor 30a to detect a user's movement and display a 3D virtual object on the displayed real image information.

Here, the sensor 30a may detect the moving position and direction of the corresponding terminal 30, the moving speed, and the like. Since the HMD 20 equipped with the terminal 30 is worn on the user's head, on the other hand, it can be seen that the sensor 30a senses the user's moving position and direction, moving speed, etc. according to the user's movement. That is, the movement of the user and the movement of the terminal 30 may be regarded as the same. The sensor 30a may be referred to as an internal sensor.

For example, the sensor 30a may use a sensor fusion value of 9-AXIS (Gyroscope, Accelerometer, Compass) as 6-DOF sensing.

6-DOF (Degree of Freedom) means 6 degrees of freedom, and 3D coordinates made up of XYZ are called 3DOF. In general, 3DOF means that the rotation value of the terminal 30 can be used, and 6DOF means that the rotation value and position value can be used.

9-AXIS refers to 9 axes, and one axis such as X, Y, Z, W is called 1AXIS. In general, xyz of Gyroscope, xyz of Accelerometer, and xyz of Compass are collectively called 9AXIS.

The terminal 30 may connect to the server 50 and express information such as contents and users provided by the server 50 in extended reality (XR).

The terminal 30 is capable of two-way communication with the server 50, and projection of real information and information of users playing together is possible.

For example, the terminal 30 may be a mobile device such as a smart phone.

In the above description, the HMD 20 and the terminal 30 are respectively illustrated as the terminal 30 is mounted on the HMD 20, but the HMD 20 on which the terminal 30 is mounted may be collectively referred to as a terminal. . If the HMD 20 on which the terminal 30 is mounted is collectively referred to as a terminal, the terminal 30 may be referred to as a mobile device.

Although not shown in Figure 1, the terminal 30 is other than the sensor 30a, a communication unit that performs a communication function with the server 50, a camera that captures the user's surroundings, a virtual object or terminal 30 rendered in three dimensions A virtual object DB in which a three-dimensional virtual object image corresponding to the movement coordinates of It may include a processor and the like.

The tracker 40 may be installed in the HMD 20 as shown in FIG. 2 .

The tracker 40 may receive a predetermined radio wave from a transmitter (not shown) installed inside the game space. It is preferable that a plurality of transmitters are installed inside the game space.

For example, the transmitter may emit an infrared laser. In this case, the transmitter includes a motor rotating vertically and a motor rotating horizontally. A horizontal radiation type infrared laser is attached to the vertical rotation motor, and a vertical radiation type infrared laser is attached to the horizontal rotation motor. Infrared laser emits horizontally and vertically at regular intervals, and when the radiation is finished, it emits all infrared rays to notify that one type of radiation is finished.

In this way, the radio wave (infrared laser) transmitted from the transmitter is received by the tracker 40 . About 15 to 100 sensors 40a may be radially located in the tracker 40 . The sensors 40a may determine the location of the corresponding tracker 40 by calculating the difference in time to receive the infrared lasers. Since the target that emits the infrared rays is the same, the trackers 40 can calculate all information by using the infrared transmitter as a target, so that the position in space can be derived at the same time. Here, the sensor 40a may be referred to as an external sensor.

In this way, the tracker 40 receives the radio wave from the transmitter, measures the movement (position value, rotation value) coordinates of the terminal 30 according to time, and sends the measured movement coordinates of the terminal 30 to the server 50 can be sent to

That is, in order to more accurately calculate the movement coordinates of the terminal (ie, the user), in the embodiment of the present invention, the movement coordinates of the terminal 30 are measured through an internal sensor (ie, the sensor 30a of the terminal 30 ). and also measures the movement coordinates of the terminal 30 through an external sensor (ie, the sensor 40a of the tracker 40).

The server 50 may distribute an identification code for each terminal and automatically update content (program) based on extended reality.

The server 50 may control the provision and execution of extended reality-based content to the client 100 , control the connection between the controller 10 and the terminal 30 , and control the state of the terminal 30 . .

The server 50 may control recording/recording of the terminal screen, streaming of the terminal screen, and the camera of the terminal.

The server 50 calculates the current movement coordinates of the terminal 30 based on the movement coordinates of the terminal (ie, the user) from the terminal 30 and the terminal (ie, the user) movement coordinates from the tracker 40, It can be transmitted to the terminal 30 .

Of course, the server 50 is based on connecting users, and can perform overall system settings such as game execution, suspension, team assignment, game type selection, and controller designation.

In particular, the server 50 includes an axis synchronizer 60 that synchronizes the axes of the sensor 30a inside the terminal 30 and the sensor outside the terminal (ie, the sensor 40a of the tracker).

The necessity of the axis synchronization unit 60 will be described as follows.

When the rotation angles of the terminal 30 and the tracker 40 are compared with each other, it may be as illustrated in FIG. 2 .

In FIG. 2 , the three axes displayed on the HMD 20 are rotation angles of the sensor 30a of the terminal 30 , and when the gamer wears the HMD 20 , the lower surface of the terminal 30 becomes the front, and the terminal ( 30) shows the rotation angle of a general VR terminal with the side facing up.

On the other hand, the three axes displayed on the tracker 40 show the rotation angle when the sensor of the tracker 40 is observed from a server (eg, PC) (not shown).

As illustrated in FIG. 2 , a physical angle in the real world is visible and determined, but in actual data, the rotation angle of the sensor 30a and the rotation angle of the sensor 40a are not clear and will not coincide with each other. That is, when the tracker 40 is viewed as a center, the angle of the sensor 30a of the terminal 30 and the angle of the sensor 40a of the tracker 40 will be different from the difference between the rotation angles in the real world. In other words, the two independent sensors 30a and 40a will have different position axes and reference points from each other.

Therefore, in order to solve this problem, the axis synchronization unit 60 for synchronizing the axes of the sensor 30a inside the terminal 30 and the sensor outside the terminal (ie, the sensor 40a of the tracker) is required.

When the axis synchronizer 60 is used, the position axis and the reference point of the two independent sensors 30a and 40a can be kept the same, so that space can be shared between users and a sense of reality can be increased. In particular, even when multiple users experience extended reality-based content in the same space, it is very easy to share the same virtual coordinates with each other, and the same tracking result can be derived. Due to this, it is possible to easily recognize each other between users, and it is possible to eliminate bumps and awkwardness caused by a heterogeneous sense of space.

The internal configuration of the shaft synchronizer 60 will be described later.

The above-described server 50 may be an example of the extended reality service device described in the claims of the present invention.

In FIG. 1 , the network 200 may include three schemes.

For example, the network 200 may include three systems: a general network (general response), a high-speed network (fast response), and a large-capacity network (fast speed, slow response). The three network schemes have independent network TCP/IP sockets.

Here, the general network refers to a network configuration of an online server that is generally configured, and refers to a network in which the server 50 and the client 100 can perform buffering in a range that is not heterogeneous. Buffering refers to a method of storing and transmitting data as one data in order to reduce the load on the network 200 when transmitting data too frequently or when transmitting a small amount of data.

An embodiment of the present invention may be configured based on the maximum packet amount (MTU) in a general network, and at this time, data may be transmitted after waiting for data transmission for up to 0.05 seconds. Data transfer is made at least 20 times per second, and this includes game data (eg, "3D object A is created at coordinates (0,1,0)", "Your A was attacked by opponent B", "Your HP has reached 50%", etc.

Unlike general networks, high-speed networks do not perform data buffering, and the key is to deliver input data as quickly as possible. A high-speed network uses the TCP/IP protocol the same as a general network. Therefore, it is not used well because a load may occur on the network 200 due to the transmission of a large amount of small packets.

In a high-speed network, if it is transmitted at any time when necessary, like a general network, the network load increases. Therefore, it is desirable to implement the transmission logic in a system that transmits periodically according to a required schedule. For example, in a high-speed network, it would be good to limit transmission of only a certain type, such as sensor data and identification number.

There is no significant meaning when using a wide-area network, but in a high-speed network, a meaningful speed can be expressed in a situation where the server 50 and the client 100 are directly connected (router, WIFI internal network).

A high-speed network can show a data transmission speed of about 0~1ms in the case of a wired network until transmission/reception, and may show a data transmission speed of about 1ms~30ms in a WIFI network. In the WIFI network, the average transmission speed will be maintained at about 4ms.

The concept of high-speed network described above may not be used as necessary because it is a concept of a primitive network, but it can be understood that it is applied as an essential element by adding the specificity of VR and the specificity of an internal network.

A large-capacity network is a concept opposite to a high-speed network, and follows a general web file large-capacity transmission protocol. The high-capacity network is a means for transmitting data generated or held by the server 50 or the client 100, and enables data transmission and reception in a variety of ways, from execution file version management to internal content data.

A large-capacity network attempts transmission with a larger buffer than a general network, and the transmission speed is faster as the transmission buffer becomes larger.

In the case of general online PC games and mobile games, it is often composed of only one network. Otherwise, it is preferable to form a combination of a general network and a large-capacity network.

Therefore, it is necessary to reflect the weight in consideration of the network error, which is the data difference caused by the delay of the network during the shaft synchronization process.

3 is a conceptual diagram of an axis synchronization device of an extended reality service device according to the present invention. As shown in Figure 3, the terminal 30 receives the rotation information of the sensor 30a inside the terminal and the sensor outside the terminal (ie, the sensor 40a of the tracker) and offset using the difference in rotation angle between the tracker and the terminal an axis synchronization unit 60 for detecting and synchronizing the axes by reflecting the offset value; A rotation operation unit 70 that receives rotation information of the terminal synchronized with the rotation axis of the tracker in the axis synchronization unit 60 and information of the sensor 40a of the tracker to calculate rotation information, and the rotation operation unit 70 and a final state value generating unit 80 for generating a final state value based on the rotation operation information.

As described above, in the present invention, the rotation value of the terminal internal sensor 30a and the rotation value of the external sensor (sensor of the tracker) 40a are received and the shaft synchronization unit 60 synchronizes the shaft. Shaft synchronization obtains the angular difference (C) from the rotation angle (B) of the tracker sensor (40a) to the rotation angle (A) of the terminal sensor (30a). Calculate the rotation angle (B) of the tracker sensor and the angle difference (C) to obtain an expected terminal rotation value (D). Practically, the expected terminal rotation value (D) and the terminal rotation value (A) are the same angle.

For shaft synchronization, the rotation angle (A) of the terminal and the rotation angle (B) of the tracker are measured after time passes, by measuring the rotation angle, the difference between each rotation angle is obtained and recorded, and the recorded rotation angle difference is continuously recorded Accumulate the new rotation angle difference (Cn) of the two rotations.

The new rotation angle difference (Cn) is reflected in the previous rotation angle difference (C) using a specific weight to obtain a rotation angle difference (C) to be used newly. That is, the rotation angle difference C is obtained as an offset value to be newly applied, and this is reflected in the rotation value of the terminal to synchronize the axis. Then, for the next calculation, the current rotation angles (A, B) and the rotation angle difference (C) are accumulated and recorded and used to detect the next offset value.

As such, by performing shaft synchronization according to the rotation angle difference through the shaft synchronization unit 60, the reference coordinates of the internal sensor and the external sensor are matched, and the rotation calculation unit 70 rotates the tracker based on the rotation angle difference according to the shaft synchronization Rotation information is calculated by inputting a value. After synchronizing the axes and matching the reference coordinates in this way, the rotation information for the actual movement is calculated, and the final state value generation unit 80 generates and provides final state information for the object of the content by the calculated rotation information, It is possible to reflect accurate tracking results in the extended reality service.

FIG. 4 is an internal configuration diagram of the axis synchronizer 60 shown in FIG. 1 .

The axis synchronizer 60 may include a rotation information receiving unit 61 , an offset calculating unit 62 , a synchronizing unit 63 , a storage unit 64 , and a control unit 65 .

The rotation information receiving unit 61 may receive rotation information (rotation value) of the terminal 30 and rotation information (rotation value) of the tracker 40 in real time or periodically.

The offset calculator 62 calculates the angular difference between the rotation axes of the sensors 30a and 40a of the terminal 30 and the tracker 40 based on the received rotation information of the terminal 30 and the rotation information of the tracker 40 . and, based on the calculated angular difference, an offset (that is, an adjustment value) used for axis synchronization of the sensor 30a and the sensor 40a may be calculated.

As described above, in the present invention, an extended reality service device that provides extended reality-based content to a client including a terminal and a tracker synchronizes the axes of a first sensor in the terminal and a second sensor in the tracker,

After the set time elapses for the rotation angle (A) of the terminal and the rotation angle (B) of the tracker for shaft synchronization, measuring the rotated rotation angle to obtain and record the difference between the rotation angles of each axis;

Accumulating and recording the difference between the rotation angles of the respective axes of the terminal and the tracker, and obtaining a new rotation angle difference (Cn) between the axis of the terminal and the axis of the tracker;

It may include a shaft synchronization step of reflecting the new rotation angle difference (Cn) to the previous rotation angle difference (C) to obtain an offset value to be currently applied and storing it, and also reflecting the offset value in the axis information of the terminal to synchronize the axis. .

As a specific embodiment of the offset calculator 62 , the offset may be calculated according to the following steps 1 to 10 . Then, the offset calculator 62 will repeat the following steps 1 to 10. The terminal 30 in the following may refer to only the terminal 30 purely, but since the terminal 30 is mounted on the HMD 20 and moves together, the HMD 20 equipped with the terminal 30 will be referred to as a terminal. may be

1. Based on the previous rotation angle of the terminal 30 and the current rotation angle of the terminal 30, a difference in the rotation angle of the terminal 30 (headsetDifference) at the current time is calculated. If this is expressed programmatically, for example, "Quaternion headsetDifference = beforeHeadsetRotation.Inverse() * headsetRotation" can be expressed.

2. Based on the calculated difference in rotation angle of the terminal 30 (headsetDifference), the rotated axis Aa of the terminal 30 is obtained. If this is expressed programmatically, for example, "headsetDifference.ToAngleAxis(out_, out var headsetRotatedAxis)" can be expressed. Here, the rotated axis Aa of the terminal 30 may mean the axis of the internal sensor 30a.

3. Convert the rotated axis Aa of the terminal 30 from a linear vector to a quaternion rotation angle. If this is expressed programmatically, for example, "Quaternion headsetAxisLook = Quaternion.LookRotation(headsetRotatedAxis)" can be expressed. A quaternion can be said to represent the rotation of space about a particular axis of rotation (ie, the axis of rotation) in three-dimensional space. As described above, when the rotated axis Aa of the terminal 30 is expressed in a quaternion system, it can represent that the terminal 30 rotates on an arbitrary axis with respect to the ground surface.

4. Calculate the previous expected rotation angle (beforeExpectHeadsetRotation) of the terminal 30 based on the previous rotation angle of the tracker 40 and the previous offset (Offset 1) (ie, the previously calculated offset). If this is expressed programmatically, for example, "Quaternion beforeExpectHeadsetRotation = (beforeTrackerRotation * offset)" can be expressed.

Then, the current expected rotation angle (currentExpectHeadsetRotation) of the terminal 30 is calculated based on the current rotation angle of the tracker 40 and the previous offset (Offset 1). If this is expressed programmatically, for example, "Quaternion currentExpectHeadsetRotation = (TrackerRotation * offset)" can be expressed.

Next, a difference in rotation angle (trackerDifference) of the tracker 40 at the current time is calculated based on the previous expected rotation angle of the terminal 30 and the current expected rotation angle of the terminal. If this is expressed programmatically, for example, "Quaternion trackerDifference = beforeExpectHeadsetRotation.Inverse() * currentExpectHeadsetRotation" can be expressed.

5. Based on the calculated difference in the rotation angle of the tracker 40 (trackerDifference), the rotation axis Da of the tracker 40 is obtained. If this is expressed programmatically, for example, "trackerDifference.ToAngleAxis(out_, out var trackerRotatedAxis)" can be expressed. Here, the rotated axis Da of the tracker 40 may mean the axis of the external sensor 40a.

6. Convert the rotated axis Da of the tracker 40 from a linear vector to a quaternion rotation angle. If this is expressed programmatically, for example, "Quaternion trackerAxisLook = Quaternion.LookRotation(trackerRotatedAxis)" can be expressed. As such, when the rotated axis Da of the tracker 40 is expressed in a quaternion system, it may represent that the tracker 40 rotates on an arbitrary axis with respect to the ground surface.

7. Calculate the angular difference (axisDifference) between the two rotation axes, that is, the rotation axis of the sensor 30a of the terminal 30 and the rotation axis of the sensor 40a of the tracker 40 . If this is expressed programmatically, for example, "Quaternion axisDifference = trackerAxisLook * headsetAxisLook.Inverse()" can be expressed.

8. Calculate a new offset (newOFFset) based on the calculated angle difference (axisDifference) between the two rotation axes and the previous offset (Offset 1). If this is expressed programmatically, it can be expressed as, for example, "Quaternion newOFFset = Offset * axisDifference". Here, Offset may be Offset 1 as an offset calculated immediately before.

9. Based on the previous offset (Offset 1), the new offset (newOFFset), and a specific weight (addFactor), an offset (Offset 2) to be used is calculated. For example, by applying the previous offset (Offset 1), the new offset (newOFFset), and a specific weight (addFactor) to quaternary linear interpolation (LERP), an offset (Offset 2) to be newly used may be calculated. If this is expressed programmatically, for example, "Offset = Quaternion.Lerp(Offset, newOFFset, addFactor)" can be expressed. Here, Offset in () may be the previous offset (Offset 1), newOFFset in () may be a new offset (newOFFset), and addFactor in () may be a specific weight.

Since the offset (Offset 2) to be newly used (ie, Offset outside ( ) in the calculation formula) can be regarded as an error caused by rotation, the offset (Offset 2) is an adjustment used for axis synchronization of the sensor 30a and the sensor 40a can be a value. Then, the offset (Offset 2) is stored in the storage unit 64 . Here, the offset (Offset 2) will be used by the synchronizer 63 .

10. For subsequent calculation, the current rotation angle of the terminal 30 and the current rotation angle of the tracker 40 are stored in the storage unit 64 .

In FIG. 4 , the synchronization unit 63 adjusts the axes of the two sensors 30a and 40a based on the offset 2 to be used newly calculated in real time or periodically by the offset calculator 62 to adjust the axes of the two sensors 30a and 40a. ) can be synchronized in real time or periodically. For example, the synchronizer 63 may synchronize the rotation angles of the axes of the two sensors 30a and 40a by applying the offset 2 to the current rotation angle of the tracker 40 . If necessary, the synchronizer 63 may synchronize the rotation angles of the axes of the two sensors 30a and 40a by applying the offset 2 to the current rotation angle of the terminal 30 .

The storage unit 64 stores various types of information generated in the calculation process by the offset calculation unit 62 . As described above, the storage unit 64 may store an offset (Offset 2) to be newly used, a current rotation angle of the terminal 30, a current rotation angle of the tracker 40, and the like.

The control unit 65 controls the overall operation of the axis synchronization unit 60 .

5 is a flowchart schematically illustrating an axis synchronization method in an extended reality service device including an axis synchronization function according to an embodiment of the present invention. It should be understood that the axis synchronization method in the extended reality service device including the axis synchronization function described below is performed by the axis synchronization unit 60 in the server 50 .

First, the rotation information receiving unit 61 of the shaft synchronizer 60 receives rotation information (rotation value) of the terminal 30 and rotation information (rotation value) of the tracker 40 in real time or periodically (S10).

Next, the offset calculation unit 62 of the axis synchronization unit 60 is based on the received rotation information of the terminal 30 and the rotation information of the tracker 40, the sensor 30a of the terminal 30 and the tracker 40, 40a) calculates the angular difference of the axis of rotation, and based on the calculated angular difference, an offset (that is, an adjustment value) used for axis synchronization of the sensor 30a and the sensor 40a is calculated (S20) .

Then, the synchronizer 63 of the axis synchronizer 60 adjusts the axes of the two sensors 30a and 40a with the offset (ie, Offset 2) calculated in real time or periodically by the offset calculator 62 to adjust the two sensors The rotation angles of the axes of (30a, 40a) are synchronized in real time or periodically (S30). For example, the synchronization unit 63 applies the offset (Offset 2) from the offset calculator 62 to the current rotation angle of the tracker 40 to synchronize the rotation angles of the axes of the two sensors 30a and 40a. have. If necessary, the synchronization unit 63 applies the offset (Offset 2) from the offset calculator 62 to the current rotation angle of the terminal 30 to synchronize the rotation angles of the axes of the two sensors 30a and 40a. have.

FIG. 6 is a flowchart for explaining the step of calculating the offset shown in FIG. 5 ( S20 ) in more detail.

First, the offset calculator 62 calculates a difference (headsetDifference) of the rotation angle of the terminal 30 at the current time based on the previous rotation angle of the terminal 30 and the current rotation angle of the terminal 30 ( S210 ).

Next, the offset calculator 62 obtains the rotated axis Aa of the terminal 30 based on the calculated difference in the rotation angle of the terminal 30 (headsetDifference) (S220).

Then, the offset calculator 62 converts the rotated axis Aa of the terminal 30 from a linear vector to a quaternion rotation angle (S230). As described above, when the rotated axis Aa of the terminal 30 is expressed in a quaternion system, it can represent that the terminal 30 rotates on an arbitrary axis with respect to the ground surface.

On the other hand, the offset calculation unit 62 calculates a rotation angle difference (trackerDifference) of the tracker 40 (S240). That is, the offset calculator 62 calculates a previous expected rotation angle (beforeExpectHeadsetRotation) of the terminal 30 based on the previous rotation angle of the tracker 40 and the previous offset (Offset 1). Then, the offset calculator 62 calculates the current expected rotation angle (currentExpectHeadsetRotation) of the terminal 30 based on the current rotation angle of the tracker 40 and the previous offset (Offset 1). Next, the offset calculator 62 calculates a difference in the rotation angle of the tracker 40 at the current time based on the previous expected rotation angle of the terminal 30 and the current expected rotation angle of the terminal.

Then, the offset calculator 62 obtains the rotated axis Da of the tracker 40 based on the calculated difference in the rotation angle of the tracker 40 (trackerDifference) (S250).

Thereafter, the offset calculator 62 converts the rotated axis Da of the tracker 40 from a linear vector to a quaternion rotation angle (S260). As such, when the rotated axis Da of the tracker 40 is expressed in a quaternion system, it may represent that the tracker 40 rotates on an arbitrary axis with respect to the ground surface.

Then, the offset calculation unit 62 calculates the angular difference (axisDifference) between the two rotation axes, that is, the rotation axis of the sensor 30a of the terminal 30 and the rotation axis of the sensor 40a of the tracker 40 (S270) .

Thereafter, the offset calculator 62 calculates a new offset (newOFFset) based on the calculated angular difference (axisDifference) between the two rotation axes and the previous offset (Offset 1) (S280).

Next, the offset calculator 62 calculates an offset (Offset 2) to be newly used based on the previous offset (Offset 1), the new offset (newOFFset), and a specific weight (addFactor) (S290). For example, the offset calculator 62 applies the previous offset (Offset 1), the new offset (newOFFset), and a specific weight (addFactor) to the linear interpolation (LERP) of the quaternion to use the offset (Offset 2) to be newly used. can be calculated Here, the offset (Offset 2) to be newly used can be regarded as an error caused by rotation, and can be an adjustment value.

Then, the offset calculator 62 stores the offset (Offset 2) in the storage unit 64 and transmits it to the synchronization unit 63 .

In addition, the axis synchronization method of the extended reality service device including the axis synchronization function of the present invention described above can be implemented as a computer readable code on a computer readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium is distributed in a computer system connected to a network, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the art to which the present invention pertains.

As described above, the best embodiment has been disclosed in the drawings and the specification. Although specific terms are used herein, they are used only for the purpose of describing the present invention, and are not used to limit the meaning or scope of the present invention described in the claims. Therefore, it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the present invention should be defined by the technical spirit of the appended claims.

10: controller 20: HMD
30: terminal 40: tracker
50: server 60: axis synchronization unit
61: rotation information receiving unit 62: offset calculating unit
63: synchronization unit 64: storage unit
65: control unit 100: client
200: network 30a: sensor (internal sensor)
40a: sensor (external sensor)

Claims (4)

A computer readable program is recorded for executing a method in which an extended reality service device that provides extended reality-based content to a client including a terminal and a tracker synchronizes the axes of the first sensor in the terminal and the second sensor in the tracker As a recording medium,
After a set time elapses for the rotation angle (A) of the terminal and the rotation angle (B) of the tracker for shaft synchronization, measuring the rotated rotation angle to obtain and record the difference between the rotation angles of each axis;
Accumulating and recording the difference between the rotation angles of the respective axes of the terminal and the tracker, and obtaining a new rotation angle difference (Cn) between the axis of the terminal and the axis of the tracker;
A shaft synchronization step of reflecting the new rotation angle difference (Cn) to the previous rotation angle difference (C) to obtain an offset value to be currently applied and storing it, and also reflecting the offset value in the axis information of the terminal to synchronize the axis
A computer-readable recording medium in which a program for executing the axis synchronization method of the extended reality service device is recorded. The method of claim 1,
The step of obtaining and recording the difference in the rotation angle of the shaft comprises:
receiving the rotation value of the terminal, calculating the current rotation angle from the previous rotation angle of the terminal, and obtaining the rotated axis of the terminal based on this;
converting the rotated axis information of the terminal into rotation angle information of the axis from a linear vector;
receiving the rotation value of the tracker, calculating the rotation difference according to the rotation angle of the previous tracker and the rotation angle of the current tracker, and obtaining the rotated axis of the tracker based on this;
converting the rotation axis information of the tracker into rotation angle information of the axis in a linear vector;
Storing the rotation angle of the axis of rotation of the terminal and the rotation angle of the axis of rotation of the tracker as well as calculating the difference in angle between the two rotation axes; including
A computer-readable recording medium in which a program for executing the axis synchronization method of the extended reality service device is recorded. According to claim 1,
The shaft synchronization step is
After calculating a specific weight on the new rotation angle difference (Cn) of the two rotation axes, a new offset value to be applied is calculated and stored by reflecting the previous offset value, and the new offset value is applied to the rotation angle information of the terminal to synchronize the axes. letting
A computer-readable recording medium in which a program for executing the axis synchronization method of the extended reality service device is recorded. 4. The method of claim 3,
The specific weight is
Characterized in that the weight is set by reflecting the data difference generated due to the delay of the network.
A computer-readable recording medium in which a program for executing the axis synchronization method of the extended reality service device is recorded.

Images