KR102032425B1 - Virtual reality game system using fiber bragg grating sensor and positional tracking sensor - Google Patents

Abstract

The present invention relates to a virtual reality game system. The purpose of the present invention is to provide the virtual reality game system for improving motion and positional tracking performance by using a fiber Bragg gating (FBG) sensor and a positioning sensor. For example, the virtual reality game system comprises: motion capture sensor modules each attached to the body of a user, measuring the moving directions and bending angles of joints through FBG sensors, and transmitting the measured values to an external device; a plurality of positioning sensor modules installed in the actual game space of the user, measuring positions of the user, to whom the motion capture sensor modules are attached, through a positioning sensor, and transmitting the measured position to the external device; a central processing terminal implementing a virtual reality game image based on joint motion data received through the motion capture sensor modules and position measurement data received through the positioning sensor modules, and controlling the motion of the avatar within the virtual reality game image; and an head mounted display (HMD) worn on the user′s body to receive the virtual image from the central processing terminal and output the same. The motion capture sensor modules include elasticity providing units installed on one sides of the FBG sensors, respectively, to provide resiliency to the FBG sensors when tension is generated by the movements and bending of the joints of the user.

Description

VIRTUAL REALITY GAME SYSTEM USING FIBER BRAGG GRATING SENSOR AND POSITIONAL TRACKING SENSOR}

An embodiment of the present invention relates to a virtual reality game system using an FBG sensor and a position tracking sensor.

Currently, active research is being conducted in response to optical fiber sensors, and a representative example thereof is an optical fiber Bragg grating (FBG) sensor.

These FBG sensors are used for fiber lasers, filters, and pulse compression. The advantages of FBG sensors are not only the measurement using a single sensor, but also the measurement of physical changes at multiple points by inserting several sensors into a single optical fiber. It is easy to measure the reflected wavelength back.

In addition, the FBG sensor generates a Bragg grating that reflects a specific wavelength on the optical cable, so that the reflected wavelength changes according to the tension-compression or temperature change. It can be utilized.

In addition, FBG sensors can be installed multiple sensors of different wavelengths on one cable at the same time, enabling multiplexing. Since the light is a source, even if the cable length is long, noise and distortion are not generated in the signal. The advantage is that the signal can be delivered without an amplifier. In addition, since it has little influence on electromagnetic waves and is made of glass, it is hardly affected by corrosion due to moisture, and thus has excellent long-term durability.

In addition, the FBG sensor is constructed by forming a periodic refractive index in the optical fiber core, and the central wavelength of the optical signal reflected by the FBG sensor is changed by the physical change amount by an external physical change. The amount of phosphorus change can be calculated.

Patent Registration No. 10-1501415 (Registration Date: March 04, 2015)

An embodiment of the present invention provides a virtual reality game system having improved motion and position tracking performance using an FBG sensor and a position tracking sensor.

The virtual reality game system according to an embodiment of the present invention, the motion capture sensor module is attached to the body to the user, respectively, and measures the movement direction and the bending angle of the joint through the Fiber Bragg Grating (FBG) sensor to the external device ; A location tracking sensor module installed in a plurality of real game spaces of the user and measuring the location of the user to which the motion capture sensor module is attached through a location tracking sensor to an external device; Central processing for implementing a virtual reality game image based on joint motion data received through the motion capture sensor module and position measurement data received through the position tracking sensor module, and controlling the motion of the avatar in the virtual reality game image. Terminal; A head mounted display (HMD) which is worn on a user's body and receives and outputs the virtual image from the central processing terminal; And the motion capture sensor module is provided at one side of the FBG sensor and provides an elastic providing unit for providing a restoring force to the FBG sensor when tension is generated due to movement and bending of a joint of a user.

In addition, the motion capture sensor module, at least one FBG shape sensor for detecting a shape in which the user joint is bent in accordance with the movement of the user joint; At least one FBG angle sensor for detecting an angle at which the user joint is bent according to the movement of the user joint; At least one FBG torsion sensor for detecting a torsion of the user joint according to the movement of the user joint; A light source transceiver configured to transmit an optical signal to the FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor, respectively, and to receive an optical signal reflected from the FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor; A connector connecting the FBG shape sensor, the FBG angle sensor, the FBG torsion sensor, and the light source transceiver; A first data converter configured to analyze the wavelength of the optical signal received through the light source transceiver and convert the wavelength into three-dimensional coordinate data about a shape, an angle, and a twist of a user joint; And a first wireless communication unit transmitting the 3D coordinate data converted through the first data conversion unit to the central processing terminal.

In addition, the FBG shape sensor, the FBG angle sensor and the FBG torsion sensor are attached to the user's finger and arm portion to track the motion of the corresponding body part, for other body parts except the user's finger and arm, The virtual reality game system may further perform motion capture using at least one of an infrared optical camera tracking and a marker.

The elastic providing unit may be provided at the connector side to provide a restoring force to the FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor, in which tension is generated due to movement and bending of a user joint.

The position tracking sensor module may include: at least one first position tracking sensor installed at an outer portion of the real game space and configured to sense a position of a user moving in the real game space; At least one second position tracking sensor installed in the real game space and configured to sense a position of a user moving in the real game space; A second data converter configured to convert signals measured by the first lidar sensor and the second lidar sensor into three-dimensional coordinate data; And a second wireless communication unit transmitting the 3D coordinate data generated by the second data conversion unit to the central processing terminal.

The first position tracking sensor and the second position tracking sensor may include at least one of a LiDAR sensor, an ultrasonic sensor, an infrared optical marker tracking sensor, and a GPS sensor.

In addition, the central processing terminal may transmit the virtual image to the HMD through a 5G wireless communication video streaming method.

According to the present invention, it is possible to provide a virtual reality game system having improved motion and position tracking performance using an FBG sensor and a position tracking sensor.

1 is a block diagram showing the overall configuration of a virtual reality game system according to an embodiment of the present invention.
2 is a block diagram showing the overall configuration of a motion capture sensor module according to an embodiment of the present invention.
3 is a block diagram showing the overall configuration of the position tracking sensor module according to an embodiment of the present invention.
4 is a diagram illustrating an example of wearing a motion capture sensor module according to an embodiment of the present invention.
5 is a diagram illustrating a virtual reality game image including an avatar displayed on the HMD according to an embodiment of the present invention.
6 is a view showing for explaining the type and configuration of the FBG sensor according to an embodiment of the present invention.
7 is a view showing for explaining the configuration concept of the first elastic providing unit according to an embodiment of the present invention.
8 is a view illustrating a configuration concept of the second elastic providing unit according to the embodiment of the present invention.
9 is a view showing for explaining the configuration arrangement and operation of the position tracking sensor module according to an embodiment of the present invention.

Terms used herein will be briefly described and the present invention will be described in detail.

The terms used in the present invention have been selected as widely used general terms as possible in consideration of the functions in the present invention, but this may vary according to the intention or precedent of the person skilled in the art, the emergence of new technologies and the like. In addition, in certain cases, there is also a term arbitrarily selected by the applicant, in which case the meaning will be described in detail in the description of the invention. Therefore, the terms used in the present invention should be defined based on the meanings of the terms and the contents throughout the present invention, rather than the names of the simple terms.

When any part of the specification is to "include" any component, this means that it may further include other components, except to exclude other components unless otherwise stated. In addition, the terms "... unit", "module", etc. described in the specification mean a unit for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software. .

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

1 is a block diagram showing the overall configuration of a virtual reality game system according to an embodiment of the present invention, Figure 2 is a block diagram showing the overall configuration of a motion capture sensor module according to an embodiment of the present invention, Figure 3 is 4 is a block diagram showing the overall configuration of the position tracking sensor module according to an embodiment of the present invention, Figure 4 is a view showing an example of wearing a motion capture sensor module according to an embodiment of the present invention, Figure 5 is an embodiment of the present invention FIG. 6 is a view illustrating a virtual reality game image including an avatar displayed on an HMD, and FIG. 6 is a view illustrating a type and configuration of an FBG sensor according to an embodiment of the present invention, and FIG. 7 is an embodiment of the present invention. 8 is a view illustrating a configuration concept of a first elastic providing unit according to an example, and FIG. 8 is a view illustrating a configuration concept of a second elastic providing unit according to an embodiment of the present invention. A diagram for the explanation of the construction and operation of the arrangement position tracking sensor module according to an embodiment of the invention.

1 to 9, the virtual reality game system 1000 according to an embodiment of the present invention includes a motion capture sensor module 100, a position tracking sensor module 200, a central processing terminal 300, and an HMD (Head). It may include at least one of the Mounted Display (400).

The motion capture sensor module 100 is attached to the body to the user, respectively, by measuring the direction of movement and the bending angle of the joint through the FBG (Fiber Bragg Grating) sensor to wireless or wired communication to the central processing terminal 300 Can be sent through. This motion capture refers to a technology of recording a human body's movement in a digital form by attaching a sensor to a user's body or using a camera. In this embodiment, the motion capture is implemented in the form of a sensor attachment method. For this purpose, the motion capture sensor module 100 can be basically manufactured in the form of a chute, such a configuration can be omitted, FBG shape sensor 110, FBG angle sensor 120, FBG torsion sensor 130, light source It may include a transceiver 140, a connector 150, a first data converter 160, a first wireless communication unit 170, and a first elastic providing unit 180, and a first elastic providing unit 180. In place of the second elastic providing unit 180 may be included, the first and second elastic providing unit 180, 190 may be implemented together.

The optical fiber Bragg grating called FBG (Fiber Bragg Grating) applied in this embodiment has a structure in which gratings having different refractive indexes are arranged at regular intervals on a core of the optical fiber. This is to use the phenomenon that light is reflected by the difference of refractive index. More specifically, only light of a specific wavelength is reflected by the interval of the grating, and the wavelength is shifted while the distance of the grating is changed by external force and temperature change. One fiber can be engraved with several FBGs, and one fiber can be used to manufacture several sensors, and the same sensor can be distributed to measure multiple areas simultaneously.

In addition, the FBG sensor receives a light source through an interrogator and measures the wavelength of the light reflected from the sensor. Two types of interrogators may be applied. First, an interrogator of the wavelength-position conversion method has a limited resolution and an SNR ratio by distributing and transmitting the reflected light waves to the CCD sensor simultaneously using a broadband light source. Secondly, the Tunable laser type interrogator uses a filter to generate a swept laser, and uses a single light wave transmitted and reflected to the sensor, which provides high resolution.

The FBG shape sensor 110 is an FBG sensor for detecting the shape along with the direction in which the user joint is bent according to the movement of the user joint. As shown in FIG. It can be produced by placing parallel to the interval. Through the FBG shape sensor 110, the shape and position of the body part according to the movement of the user joint may be measured through a process of peak detection, strain conversion, and shape reconstruction.

The FBG angle sensor 120 is a sensor for detecting the angle at which the user joint is bent according to the movement of the user joint, as shown in FIG. 6 (b), to place one optical fiber on one strand of epoxy. Can be produced.

The FBG torsion sensor 130 may be manufactured in the same form as the FBG shape sensor 110 as an FBG sensor for detecting the twist of the user joint according to the movement of the user joint.

As described above, the FBG sensor of the present embodiment may be formed of a combination of a shape, an angle, and a torsion sensor, and may be designed in various combinations according to body parts, and if necessary, at least one of three FBG sensors may be omitted. It is also possible. In addition, the FBG sensor may be implemented together with the chute as described above and attached to the body of the user.

Meanwhile, an IMU sensor is used as a conventional user motion sensor. In this embodiment, a sensing error occurs frequently due to an obstacle caused by a magnetic field, but in this embodiment, FBG sensors 110, 120, and 130 are applied to move a finger joint. In addition to detailed motion tracking up to and including high durability, there is almost no performance degradation even during long-term use (about 8,000 bends).

The light source transceiver 140 transmits an optical signal of a specific frequency band to the FBG shape sensor 110, the FBG angle sensor 120, and the FBG torsion sensor 130, respectively, and the FBG shape sensor 110 and the FBG angle sensor. The reflected light wave may be measured by receiving the light signal reflected from the 120 and the FBG torsion sensor 130. The light source transceiver 140 may be an FBG interrogator. As described above, the FBG interrogator of the present embodiment may be implemented by applying at least one of a wavelength-position conversion method and a tunable laser method. Gators can be applied.

The connector 150 may connect the FBG shape sensor 110, the FBG angle sensor 120, the FBG torsion sensor 130, and the light source transceiver 140.

The first data converter 160 analyzes the wavelength of the optical signal received through the light source transceiver 140, that is, the shape of the user joint through the joint coordinate system transformation based on the measured strain value. Three-dimensional coordinate data about angle and torsion can be generated.

The first wireless communication unit 170 may transmit the 3D coordinate data converted through the first data conversion unit 160 to the central processing terminal 300 through wireless communication such as Wi-Fi.

On the other hand, the motion capture sensor module 100 of the present embodiment may further include an elastic providing unit, the elastic providing unit is installed on the connector 150 side, the FBG shape sensor is generated tension due to the movement and bending of the user joint. 110, the FBG angle sensor 120, and the FBG torsion sensor 130 may be provided to provide a restoring force. The elastic providing part may include a first elastic providing part 180 and a second elastic providing part 190 according to an embodiment.

The first elastic providing unit 180 is directly connected to the connector 150 side, the FBG sensor 110, 120, 130 when the user joint is bent (when a high curvature occurs) is pulled FBG By providing elasticity to the sensors 110, 120, and 130, the movement of the joints is made more natural, and when the joints are stretched, the end positions of the FBG sensors 110, 120, and 130 may be quickly returned to their original positions.

For example, as shown in FIG. 7, the first elastic providing unit 180 may include an elastic member 181 (spring, rubber band, etc.) and a guide member 182 connected to the rear end of the connector 150. Can be. When the FBG sensors 110, 120, and 130 are pulled out, the first elastic providing part 180 moves the connector 150 along the guide member 182 as shown in FIG. 7B. At the point where the connector 150 is stopped, the restoring force by the elastic member 181 is applied, and then, when the force pulled by the FBG sensors 110, 120, and 130, that is, the tension is removed, FIG. 7 (c). As shown in FIG. 5, the connector 150 is returned to its original position along the guide member 182 by the elastic member 181.

The second elastic providing unit 190 is installed at the end side of the FBG sensors 110, 120, and 130 connected to the connector 150, and according to the bending of the user joint (when a high curvature occurs), the FBG sensor By providing elasticity to the FBG sensors 110, 120, and 130 when the 110, 120, and 130 are pulled out, the movement of the joint is more natural, and when the joints are stretched, the FBG sensors 110, 120, and 130 are stretched. The end of can be quickly returned to its original position.

The second elastic providing unit 190 has a configuration installed at the end side of the FBG sensors 110, 120, 130 connected with the connector 150, and may include a guide roller 191 and an elastic rail 192. have.

The guide roller 191 may include a first guide roller 191a and a second guide roller 191b. However, in the present embodiment, two guide rollers are illustrated to facilitate understanding, and two or more guide rollers. May be applied. The first guide roller 191a and the second guide roller 191b are installed at right angles as shown in FIG. 8, and the end side lines of the FBG sensors 110, 120, and 130 may be wound in a zigzag form. have.

The elastic rail 192 may include a first elastic rail 192a connected to a lower portion of the first guide roller 1912a and a second elastic rail 192b connected to a lower portion of the second guide roller 191b. have. The number of the first elastic rail 192a and the second elastic rail 192b may be installed in an appropriate number depending on the number of the guide rollers 191. Here, the central axis of the guide roller 191 is fixed to the end of the upper portion of the elastic rail 192, each outer periphery of the guide roller 191 is rotatable so that the FBG sensors 110, 120, 130 can move Can be configured. An elastic member is formed at each end of the elastic rail 192 to provide elastic force and restoring force according to the movement of the elastic rail 192.

In addition to the above-described FBG sensor, the light source transceiver 140, the connector 150, the first data converter 160, the first wireless communication unit 170, the first elastic providing unit 180, and the second elastic agent are combined with the FBG sensor. It can be implemented as a module with study 180.

On the other hand, the above-described FBG shape sensor 110, FBG angle sensor 120 and FBG torsion sensor 130 is attached and applied to the user's finger and arm portion can be used for motion tracking for the body part, For motion capture on other body parts, infrared optical camera tracking (joint camera tracking) and body attachment markers (vibration motion trackers) may be mixedly applied.

The position tracking sensor module 200 is installed in a plurality of real game space 10 of the user, and measures the position of the user to which the motion capture sensor module is attached via a LiDAR sensor to the central processing terminal ( 300). To this end, the position tracking sensor module 200 includes at least one of the first lidar sensor 210, the second lidar sensor 220, the second data converter 230, and the second wireless communication unit 240. can do.

As shown in FIG. 9, the first lidar sensor 210 may be installed at an outer portion of the real game space 10, and detect a location of a user moving in the real game space 10. Can be made of dogs.

As shown in FIG. 9, the second lidar sensor 220 may be installed inside the real game space 10, and may detect a location of a user moving in the real game space 10. Can be done.

The first lidar sensor 210 and the second lidar sensor 220 emits a laser pulse, and receives the light reflected from the surrounding target object and measures the distance to the object. This device draws precisely and can rotate 360 degrees.

In the conventional virtual reality game system for tracking the location of the user, but using a technique such as supersonic, triangulation, GPS, but has a large error, in particular referring to Figure 9, the first lidar sensor 210 ) Can be recognized by user 1, but user 2 is on the same line as user 1, and user 2, which is behind user 1, is difficult to accurately track by user 1.

As such, in order to reduce the error of the existing position tracking and increase the tracking performance, the present invention minimizes the tracking blind spots by installing lidar sensors on the outer and inner sides of the actual game space 10, respectively. Accordingly, as shown in FIG. 9, although the first lidar sensor 210 recognizes only the user 1 and the user 2 located behind the second lidar sensor 210 is not recognized, the second lidar sensor ( 220, the tracking blind spot in the actual game space 10 can be reduced as much as possible.

In the above description, the position tracking of the user is performed using only the lidar sensor. However, in addition, at least one of the triangulation method using the ultrasonic sensor and the triangulation method using the GPS sensor may be additionally applied. .

The second data converter 230 may convert signals measured by the first lidar sensor 210 and the second lidar sensor 220 into three-dimensional coordinate data. However, if necessary, conversion to two-dimensional coordinate data is possible according to the system design specification.

The second wireless communication unit 240 may transmit the 3D coordinate data generated by the second data converter 230 to the central processing terminal through wireless communication such as Wi-Fi.

The central processing terminal 300 implements a virtual reality game image based on the joint motion data received through the motion capture sensor module 100 and the position measurement data received through the position tracking sensor module 200. It is possible to control the motion of the avatar in the real game image. The central processing terminal 300 provides a basic three-dimensional virtual reality game video and content, although not shown, in conjunction with a controller (mock firearms, etc.) necessary for the user's game execution and progression receives signals generated from the controller. The solution can be reflected in the virtual reality game video and game process to proceed with the game and perform overall control. The central processing terminal 300 may transmit virtual image data to the HMD 400 in real time wirelessly or by wire using 5G wireless communication video streaming technology.

The head mounted display (HMD) 400 may be worn on a user's head and may receive and output a virtual image from the central processing terminal 300. When worn on the user's head, the HMD has a small display in close proximity to both eyes, so that three-dimensional (3D) images using parallax are projected, and a gyro sensor that tracks the user's movements and a rendering function that creates images according to the movements. Allows you to experience virtual reality (VR) or augmented reality (AR) as if it were in space.

What has been described above is just one embodiment for implementing a virtual reality game system using the FBG sensor and the lidar sensor according to the present invention, the present invention is not limited to the above embodiment, it is claimed in the claims As will be apparent to those skilled in the art to which the present invention pertains without departing from the gist of the present invention, the technical spirit of the present invention will be described to the extent that various modifications can be made.

1000: virtual reality system 100: motion capture sensor module
110: FBG shape sensor 120: FBG angle sensor
130: FBG torsion sensor 140: light source transceiver
150: connector 160: first data conversion unit
170: first wireless communication unit 180: first tension providing unit
181: elastic member 182: guide member
190: second tension providing unit 191: guide roller
191a: first guide roller 191b: second guide roller
192: elastic rail 192a: first elastic rail
192b: second elastic rail 200: position tracking sensor module
210: first lidar sensor 220: second lidar sensor
230: second data converter 240: second wireless communication unit
300: central processing terminal 400: HMD
10: Real Game Space

Claims (7)

A motion capture sensor module attached to the body to the user and measuring the movement direction and the bending angle of the joint through FBG (Fiber Bragg Grating) sensor, respectively, and transmitting the motion capture sensor module to an external device;
A location tracking sensor module installed in a plurality of real game spaces of the user and measuring the location of the user to which the motion capture sensor module is attached through a location tracking sensor to an external device;
Central processing for implementing a virtual reality game image based on joint motion data received through the motion capture sensor module and position measurement data received through the position tracking sensor module, and controlling the motion of the avatar in the virtual reality game image. Terminal; And
It is worn on the user's body, and includes a head mounted display (HMD) for receiving and outputting the virtual reality game image from the central processing terminal,
The motion capture sensor module,
At least one FBG shape sensor for detecting a shape in which the user joint is bent according to the movement of the user joint, at least one FBG angle sensor for detecting an angle at which the user joint is bent according to the movement of the user joint, and the user joint An FBG sensor including at least one FBG torsion sensor for detecting a torsion of a user joint according to the movement of the;
A light source transceiver for transmitting optical signals to the FBG sensors and receiving optical signals reflected from the FBG sensors, respectively;
A connector connecting the FBG sensor and the light source transceiver;
A first data converter configured to analyze the wavelength of the optical signal received through the light source transceiver and convert the wavelength into three-dimensional coordinate data about a shape, an angle, and a twist of a user joint;
A first wireless communication unit transmitting the 3D coordinate data converted through the first data conversion unit to the central processing terminal; And
It is provided on each side of the FBG sensor and includes an elastic providing unit for providing a restoring force to the FBG sensor when tension is generated due to the movement and bending of the user's joints,
The elastic providing unit,
A first elastic providing part including an elastic member connected to a rear end of the connector and a guide member for guiding the movement of the connector when the connector moves according to the movement of the FBG sensor; And
It is installed on the end side of the FBG sensor connected to the connector, the end of the FBG sensor is wound in a zigzag form and comprises a plurality of guide rollers installed at mutually opposite positions and a plurality of elastic rails respectively connected to the guide rollers At least one second elastic providing part,
The central axis of the guide roller is fixed to the elastic rail, respectively
The guide roller is a virtual reality game system, characterized in that to move along the elastic rail while rotating the movement of the FBG sensor.
delete According to claim 1,
The FBG shape sensor, the FBG angle sensor and the FBG torsion sensor are attached to the user's finger and arm portion to track the motion of the body part,
For other body parts except the user's finger and arm, the virtual reality game system further uses at least one of an infrared optical camera tracking and a marker.
Virtual reality game system, characterized in that the motion capture.
According to claim 1,
The elastic providing unit,
The virtual reality game system is installed on the connector side, and provides a restoring force to the FBG shape sensor, the FBG angle sensor and the FBG torsion sensor generated tension due to the movement and bending of the user joint.
According to claim 1,
The position tracking sensor module,
At least one first position tracking sensor installed at an outer portion of the real game space and configured to sense a position of a user moving in the real game space;
At least one second position tracking sensor installed in the real game space and configured to sense a position of a user moving in the real game space;
A second data converter configured to convert signals measured by the first position tracking sensor and the second position tracking sensor into 3D coordinate data; And
And a second wireless communication unit which transmits the 3D coordinate data generated by the second data conversion unit to the central processing terminal.
The method of claim 5,
The first position tracking sensor and the second position tracking sensor,
Virtual reality game system comprising at least one of a LiDAR sensor, an ultrasonic sensor, an infrared optical marker tracking sensor and a GPS sensor.
According to claim 1,
The central processing terminal is a virtual reality game system, characterized in that for transmitting the virtual reality game video to the HMD via 5G wireless communication video streaming.

Images