WO2020060086A1

WO2020060086A1 - Virtual reality game system using fiber bragg grating sensor and position tracking sensor

Info

Publication number: WO2020060086A1
Application number: PCT/KR2019/011581
Authority: WO
Inventors: 옥철식
Original assignee: 옥철식
Priority date: 2018-09-18
Filing date: 2019-09-06
Publication date: 2020-03-26
Also published as: KR102032425B1

Abstract

The present invention relates to a virtual reality game system, and a technical problem to be solved is to provide a virtual reality game system with improved motion and position tracking performance by using an FBG sensor and a position tracking sensor. For example, disclosed is a virtual reality game system comprising: a motion capture sensor module each attached to a user's body, measuring each of a movement direction and a bending angle of a joint via a fiber Bragg grating (FBG) sensor, and transmitting the measured movement direction and bending angle to an external device; a plurality of position tracking sensor modules installed in an actual game space of the user, measuring a position of the user, to which the motion capture sensor module is attached, via a position tracking sensor, and transmitting the measured position to the external device; a central processing terminal for implementing a virtual reality game image based on joint motion data received from the motion capture sensor module and position measurement data received from the position tracking sensor module, and controlling a motion of an avatar in the virtual reality game image; and a head mounted display (HMD) worn on the user's body, receiving the virtual image from the central processing terminal, and outputting the received virtual image, wherein the motion capture sensor module includes an elasticity providing unit each installed on one side of the FBG sensor to provide a restoring force to the FBG sensor when tension is generated due to movement and bending of the user's joint.

Description

Virtual reality game system using fiber Bragg grating sensor and position tracking sensor

An embodiment of the present invention relates to a virtual reality game system using an FBG sensor and a location tracking sensor.

Currently, active research is being conducted in response to an optical fiber sensor, and a representative example is an optical fiber Bragg grating (hereinafter referred to as FBG) sensor.

These FBG sensors are used in optical fiber lasers, filters, pulse compression, etc.The advantage of FBG sensors is not only the measurement using a single sensor, but also the insertion of multiple sensors in a single optical fiber to measure the amount of physical change at multiple points. It is easy to measure the reflected wavelength.

In addition, the FBG sensor generates a Bragg grating that reflects a specific wavelength on an optical cable, so that the reflected wavelength varies according to tensile-compression or temperature change, so as to convert the amount of reflected wavelength changed from the initial wavelength into tensile-compression. Can be utilized.

In addition, the FBG sensor can be installed multiple sensors with different wavelengths on one cable at the same time, and multi-flashing is possible. Because the light is a source, noise and distortion do not occur in the signal even if the cable length is long, up to several tens of kilometers It has the advantage of being able to transmit signals without an amplifier. In addition, since it has little effect on electromagnetic waves and is made of glass, it is a sensor with excellent long-term durability because there is little effect of corrosion due to moisture or the like.

In addition, the FBG sensor is constructed by making a periodic refractive index in the optical fiber core, and the center wavelength of the optical signal reflected by the FBG sensor is changed by the physical change by the external physical change. The amount of phosphorus change can be calculated.

An embodiment of the present invention provides a virtual reality game system with improved motion and location tracking performance using an FBG sensor and a location tracking sensor.

The virtual reality game system according to an embodiment of the present invention is a motion capture sensor module that is attached to a user's body and measures the motion direction and bending angle of a joint through an FBG (Fiber Bragg Grating) sensor, respectively, and transmits it to an external device. ; A location tracking sensor module installed in the user's actual game space and measuring the location of the user with the motion capture sensor module attached to the external device through a location tracking sensor; Based on the joint motion data received through the motion capture sensor module and the location measurement data received through the location tracking sensor module, a virtual reality game image is implemented, but central processing to control the motion of the avatar in the virtual reality game image Terminal; A head mounted display (HMD) worn on the user's body and receiving and outputting the virtual image from the central processing terminal; And the motion capture sensor module, which is installed on one side of the FBG sensor, and includes an elasticity providing unit that provides resilience to the FBG sensor when tension is generated due to movement and bending of a user's joint.

In addition, the motion capture sensor module, at least one FBG shape sensor for detecting the shape of the user joint bent according to the movement of the user joint; At least one FBG angle sensor for detecting an angle at which the user joint is bent according to the movement of the user joint; At least one FBG twist sensor for detecting twist of the user joint according to the movement of the user joint; A light source transmitting and receiving unit that transmits optical signals to the FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor, and receives optical signals reflected from the FBG shape sensor, the FBG angle sensor, and the FBG twist sensor; A connector for connecting the FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor and the light source transceiver; A first data conversion unit that analyzes the wavelength of the optical signal received through the light source transmission / reception unit and converts the shape, angle, and twist of the user joint into 3D coordinate data; And it may include a first wireless communication unit for transmitting the three-dimensional coordinate data converted through the first data conversion unit to the central processing terminal.

In addition, the FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor are attached to the user's finger and arm parts to track motion for the corresponding body part, and for other body parts except the user's finger and arm parts, The virtual reality game system may further perform motion capture using at least one of an infrared optical camera tracking and a marker.

In addition, the elastic providing unit may be provided on the connector side and provide resilience to the FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor where tension is generated due to movement and bending of the user joint.

In addition, the position tracking sensor module is installed on the outer portion of the real game space, at least one first position tracking sensor for detecting the user's location moving in the real game space; At least one second position tracking sensor installed inside the real game space and detecting a user's location moving within the real game space; A second data converter configured to convert signals measured through the first lidar sensor and the second lidar sensor into three-dimensional coordinate data; And it may include a second wireless communication unit for transmitting the three-dimensional coordinate data generated by the second data conversion unit to the central processing terminal.

Further, the first position tracking sensor and the second position tracking sensor may include at least one of a LiDAR sensor, an ultrasonic sensor, an infrared optical marker tracking sensor, and a GPS sensor.

In addition, the central processing terminal may transmit the virtual image to the HMD through a 5G wireless communication video streaming method.

According to the present invention, it is possible to provide a virtual reality game system with improved motion and location tracking performance using an FBG sensor and a location tracking sensor.

1 is a block diagram showing the overall configuration of a virtual reality game system according to an embodiment of the present invention.

2 is a block diagram showing the overall configuration of a motion capture sensor module according to an embodiment of the present invention.

3 is a block diagram showing the overall configuration of a position tracking sensor module according to an embodiment of the present invention.

4 is a view showing an example of wearing a motion capture sensor module according to an embodiment of the present invention.

5 is a diagram showing a virtual reality game image including an avatar displayed on an HMD according to an embodiment of the present invention.

6 is a view showing to explain the type and configuration of the FBG sensor according to an embodiment of the present invention.

7 is a view showing to explain the configuration concept of the first elastic providing unit according to an embodiment of the present invention.

8 is a view showing to explain the configuration concept of the second elastic providing unit according to an embodiment of the present invention.

9 is a view showing to explain the configuration arrangement and operation of the position tracking sensor module according to an embodiment of the present invention.

Terms used in the present specification will be briefly described, and the present invention will be described in detail.

The terminology used in the present invention was selected from the general terms that are currently widely used while considering the functions in the present invention, but this may vary according to the intention or precedent of a person skilled in the art or the appearance of new technologies. In addition, in certain cases, some terms are arbitrarily selected by the applicant, and in this case, their meanings will be described in detail in the description of the applicable invention. Therefore, the terms used in the present invention should be defined based on the meanings of the terms and the contents of the present invention, not simply the names of the terms.

When a certain part of the specification "includes" a certain component, this means that other components may be further included instead of excluding other components unless specifically stated otherwise. In addition, terms such as “... unit” and “module” described in the specification mean a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software. .

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains may easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

1 is a block diagram showing the overall configuration of a virtual reality game system according to an embodiment of the present invention, FIG. 2 is a block diagram showing the overall configuration of a motion capture sensor module according to an embodiment of the present invention, and FIG. FIG. 4 is a block diagram showing the overall configuration of a position tracking sensor module according to an embodiment of the present invention, and FIG. 4 is a diagram showing an example of wearing a motion capture sensor module according to an embodiment of the present invention, and FIG. FIG. 6 is a view showing a virtual reality game image including an avatar displayed on an HMD, and FIG. 6 is a view illustrating a type and configuration of an FBG sensor according to an embodiment of the present invention, and FIG. 7 is an embodiment of the present invention 8 is a view illustrating a configuration concept of a first elastic providing unit according to an example, and FIG. 8 is a diagram illustrating a configuration concept of a second elastic providing unit according to an embodiment of the present invention. A diagram for the explanation of the construction and operation of the arrangement position tracking sensor module according to an embodiment of the invention.

1 to 9, the virtual reality game system 1000 according to an embodiment of the present invention includes a motion capture sensor module 100, a location tracking sensor module 200, a central processing terminal 300 and a HMD (Head) Mounted Display) (400).

The motion capture sensor module 100 is attached to the user's body, and measures the movement direction and bending angle of the joint through the FBG (Fiber Bragg Grating) sensor, respectively, to perform wireless or wired communication to the central processing terminal 300. Can be transmitted through. The motion capture refers to a technique of recording the movement of the human body in a digital form by attaching a sensor to the user's body or using a camera, and is implemented in the form of a sensor attachment method in this embodiment. For this, the motion capture sensor module 100 may be basically manufactured in a suit form, and such a configuration may be omitted, and the FBG shape sensor 110, the FBG angle sensor 120, the FBG torsion sensor 130, and the light source Transceiver 140, the connector 150, the first data conversion unit 160, may include a first wireless communication unit 170 and the first elastic providing unit 180, the first elastic providing unit 180 In the alternative, the second elastic providing unit 180 may be included, and the first and second elastic providing

units

180 and 190 may be implemented together.

An optical fiber Bragg grating called FBG (Fiber Bragg Grating) applied in this embodiment has a structure in which a grating having a different refractive index is disposed at regular intervals at a core of an optical fiber, and this refractive index ( It is to use the phenomenon that light reflects due to the difference in refractive index. More specifically, only light of a specific wavelength is reflected according to the interval of the grating, and the wavelength of the grating changes as the interval of the grating changes due to external force and temperature change. Multiple FBGs can be engraved on a single fiber, and multiple sensors can be manufactured with a single fiber, and multiple regions can be measured simultaneously by dispersing the same sensor.

In addition, the FBG sensor receives a light source through an interrogator and measures the wavelength of light reflected from the sensor. Two types of interrogator may be applied. First, the wavelength-position conversion type interrogator is a method of dispersing and transmitting the reflected light waves simultaneously to a CCD sensor by using a broadband light source, and has characteristics of limited resolution and SNR ratio. Second, the tunable laser type interrogator uses a filter to generate a sweeping laser, and a single light wave is transmitted and reflected by a sensor, and has a property of providing high resolution.

The FBG shape sensor 110 is an FBG sensor for detecting the shape of the user joint along with the bending direction of the user joint according to the movement of the user joint, as shown in FIG. It can be produced by arranging in parallel at even intervals. Through the FBG shape sensor 110, the shape and position of a part of the body according to the movement of a user joint may be measured through a process of peak detection, strain conversion, and shape reconstruction.

The FBG angle sensor 120 is a sensor for detecting the angle at which the user's joint is bent according to the movement of the user's joint, as shown in FIG. 6 (b), placing one optical fiber on one strand of epoxy Can be produced.

The FBG twist sensor 130 is an FBG sensor for detecting twist of the user joint according to the movement of the user joint, and may be manufactured in the same form as the FBG shape sensor 110.

As described above, the FBG sensor of the present embodiment may be formed of a combination of a shape, an angle, and a torsion sensor, and may be designed in various combinations according to body parts. If necessary, at least one of the three FBG sensors may be omitted. It is also possible. In addition, the FBG sensor may be implemented with a suit as described above and attached to the user's body.

On the other hand, as an existing user motion sensor, an IMU sensor is used, and the IMU sensor frequently experiences a sensing error due to a magnetic field failure, but in this embodiment, the

FBG sensor

110, 120, 130 is applied to move the finger joint. Not only is it possible to track motion in detail, but it also has the advantage that there is little performance degradation even for long-term use (about 8,000 bends) due to its high durability.

The light source transceiving unit 140 transmits optical signals of a specific frequency band to the FBG shape sensor 110, the FBG angle sensor 120, and the FBG torsion sensor 130, respectively, and the FBG shape sensor 110 and the FBG angle sensor The reflected light wave may be measured by receiving the light signal reflected from the 120 and the FBG torsion sensor 130. The light source transmitting and receiving unit 140 may be an FBG interrogator. As described above, the FBG interrogator of this embodiment may be implemented by applying at least one of a wavelength-position conversion method and a tunable laser method, and an interro of a method suitable for each characteristic according to a position attached to the body Gator can be applied.

The connector 150 may connect the FBG shape sensor 110, the FBG angle sensor 120 and the FBG twist sensor 130, and the light source transceiver 140.

The first data converter 160 analyzes the wavelength of the optical signal received through the light source transceiving unit 140, that is, based on the measured strain value, converts it into a joint coordinate system (including direction) of the user joint, Three-dimensional coordinate data for angle and torsion can be generated.

The first wireless communication unit 170 may transmit the 3D coordinate data converted through the first data conversion unit 160 to the central processing terminal 300 through wireless communication such as Wi-Fi.

On the other hand, the motion capture sensor module 100 of the present embodiment may further include an elastic providing portion, the elastic providing portion is installed on the connector 150 side, FBG shape sensor that generates tension due to movement and bending of the user joint 110, the FBG angle sensor 120 and the FBG torsion sensor 130 may be configured to provide resilience. The elastic providing part may include a first elastic providing part 180 and a second elastic providing part 190 according to an embodiment.

The first elastic providing unit 180 is directly connected to the connector 150 side, and when the user joint is bent (when high curvature occurs), the

FBG sensors

110, 120, and 130 are pulled when the FBG By providing elasticity to the

sensors

110, 120, 130, the movement of the joint is made more natural, and when the joint is stretched, the end positions of the

FBG sensors

110, 120, 130 can be quickly returned to the original position.

For example, as shown in FIG. 7, the first elastic providing unit 180 includes an elastic member 181 (spring, rubber band, etc.) and a guide member 182 connected to the rear end of the connector 150. You can. When the

FBG sensors

110, 120, and 130 are pulled from the first elastic providing unit 180, the connector 150 moves forward along the guide member 182, as shown in FIG. 7 (b). When the connector 150 is stopped, the restoring force by the elastic member 181 acts, and then, when the force pulled by the

FBG sensors

110, 120, 130, that is, the tension is removed, FIG. 7 (c) As shown in the connector 150 is returned to the original position along the guide member 182 by the elastic member 181.

The second elastic providing unit 190 is installed on the end side of the

FBG sensors

110, 120, and 130 connected to the connector 150, and the user's joint is bent (when high curvature occurs). When the (110, 120, 130) is pulled, by providing elasticity to the FBG sensor (110, 120, 130), it makes the movement of the joint more natural, and when the joint is stretched, the FBG sensor (110, 120, 130) The end of can be quickly returned to the original position.

The second elastic providing unit 190 is configured to be installed on the end side of the

FBG sensors

110, 120, and 130 connected to the connector 150, and may include a guide roller 191 and an elastic rail 192. have.

The guide roller 191 may include a first guide roller 191a and a second guide roller 191b, but in this embodiment, two guide rollers are illustrated to help understanding, and two or more guide rollers are provided. May be applied. The first guide roller 191a and the second guide roller 191b are installed at opposite positions as shown in FIG. 8, and the end side lines of the

FBG sensors

110, 120, 130 can be wound in a zigzag form. have.

The elastic rail 192 may include a first elastic rail 192a connected to a lower portion of the first guide roller 1912a and a second elastic rail 192b connected to a lower portion of the second guide roller 191b. have. The number of the first elastic rail 192a and the second elastic rail 192b may be installed in an appropriate number according to the number of guide rollers 191. Here, the central axis of the guide roller 191 is fixed to the end of the upper portion of the elastic rail 192, each outer periphery of the guide roller 191 is rotatable so that the FBG sensor (110, 120, 130) can move Can be configured. An elastic member is formed at each end of the elastic rail 192 to provide elastic force and restoring force according to the movement of the elastic rail 192.

In addition to the above-described FBG sensor, the light source transmitting and receiving unit 140, the connector 150, the first data conversion unit 160, the first wireless communication unit 170, the first elastic providing unit 180 and the second elastic agent It may be implemented as a module with the study (180).

Meanwhile, the above-described FBG shape sensor 110, FBG angle sensor 120, and FBG torsion sensor 130 may be attached and applied to a user's finger and arm parts to be used for motion tracking for a corresponding body part, For the motion capture of other parts of the body, infrared optical camera tracking (joint tracking camera for joint recognition) and body attachment markers (vibe motion tracker) can be mixed.

The location tracking sensor module 200 is installed in a plurality of users in the actual game space 10, and measures the location of the user to which the motion capture sensor module is attached through a LiDAR sensor to perform a central processing terminal ( 300). To this end, the position tracking sensor module 200 includes at least one of the first lidar sensor 210, the second lidar sensor 220, the second data conversion unit 230, and the second wireless communication unit 240. can do.

The first lidar sensor 210 is installed on the outer portion of the real game space 10, as shown in Figure 9, it is possible to detect the location of the user moving in the real game space 10, a number Can consist of dogs.

As illustrated in FIG. 9, the second lidar sensor 220 is installed inside the real game space 10 and can detect the location of a user moving in the real game space 10, and a plurality of It can be done.

The first lidar sensor 210 and the second lidar sensor 220 emit a laser pulse and receive the light reflected from the surrounding object and return, thereby measuring a distance to the object and the like. It is a device that accurately draws and can rotate 360 degrees.

In the existing virtual reality game system, a device for tracking a user's location uses technologies such as subsonic, triangulation, and GPS, but has a large error. In particular, referring to FIG. 9, the first lidar sensor 210 ) User 1 can be recognized, but it is on the same line as User 1, and in the case of User 2 located behind User 1, accurate tracking was difficult by User 1.

As described above, in order to reduce the error for the existing location tracking and increase the tracking performance, in this embodiment, the lidar sensors are respectively installed in the outer part and the inside of the actual game space 10 to minimize the existing blind spot for tracking. Accordingly, as illustrated in FIG. 9, even if the first lidar sensor 210 recognizes only the user 1 and the user 2 located thereafter is not recognized, the second lidar sensor located inside the actual game space 10 ( 220), the tracking blind spot in the real game space 10 can be reduced as much as possible.

On the other hand, in the above, it has been described that the user's position is tracked using only the lidar sensor, but it is also possible to additionally apply at least one of a triangulation method using an ultrasonic sensor and a triangulation method using a GPS sensor. .

The second data converter 230 may convert the signal measured through the first lidar sensor 210 and the second lidar sensor 220 into three-dimensional coordinate data. However, if necessary, conversion to 2D coordinate data is also possible according to system design specifications.

The second wireless communication unit 240 may transmit the 3D coordinate data generated through the second data conversion unit 230 to a central processing terminal through wireless communication such as Wi-Fi.

The central processing terminal 300 implements a virtual reality game image based on the joint motion data received through the motion capture sensor module 100 and the position measurement data received through the position tracking sensor module 200 and performs virtualization. You can control the motion of the avatar in the reality game video. The central processing terminal 300 provides basic 3D virtual reality game images and contents, and although not shown, receives signals generated from the controller in connection with a controller (simulated firearm, etc.) necessary for the user's game execution and progress. The game can be reflected in the virtual reality game video and game process, and overall control can be performed. The central processing terminal 300 may transmit virtual image data to the HMD 400 in real-time wireless or wired by using 5G wireless communication image streaming technology.

The HMD (Head Mounted Display) 400 may be worn on the user's head and receive and output a virtual image from the central processing terminal 300. When worn on the user's head, the HMD has a small display near the eyes of both eyes to project a three-dimensional (3D) image using parallax, a gyro sensor that tracks the user's movement, and a 3D rendering function that creates the image according to the movement. It allows you to realize virtual reality (VR) or augmented reality (AR) as if you are in space.

What has been described above is only one embodiment for implementing the virtual reality game system using the FBG sensor and the lidar sensor according to the present invention, the present invention is not limited to the above embodiment, and is claimed in the following claims As described above, any person having ordinary knowledge in the field to which the present invention belongs, without departing from the gist of the present invention, will have the technical spirit of the present invention to the extent that various changes can be implemented.

Claims

A motion capture sensor module that is attached to the user's body and measures the motion direction and bending angle of the joint through an FBG (Fiber Bragg Grating) sensor, and transmits it to an external device;

A location tracking sensor module installed in the user's actual game space and measuring the location of the user with the motion capture sensor module attached to the external device through a location tracking sensor;

Based on the joint motion data received through the motion capture sensor module and the position measurement data received through the location tracking sensor module, a virtual reality game image is implemented, but central processing to control the motion of the avatar in the virtual reality game image Terminal;

A head mounted display (HMD) worn on the user's body and receiving and outputting the virtual image from the central processing terminal; And

The motion capture sensor module is installed on one side of the FBG sensor, a virtual reality game characterized in that it includes an elasticity providing unit that provides resilience to the FBG sensor when tension is generated due to movement and bending of the user's joint. system.
According to claim 1,

The motion capture sensor module,

At least one FBG shape sensor for detecting a shape in which the user joint is bent according to the movement of the user joint;

At least one FBG angle sensor for detecting an angle at which the user joint is bent according to the movement of the user joint;

At least one FBG twist sensor for detecting twist of the user joint according to the movement of the user joint;

A light source transmitting and receiving unit that transmits optical signals to the FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor, and receives optical signals reflected from the FBG shape sensor, the FBG angle sensor, and the FBG twist sensor;

A connector for connecting the FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor and the light source transceiver;

A first data conversion unit that analyzes the wavelength of the optical signal received through the light source transmission / reception unit and converts the shape, angle, and twist of the user joint into 3D coordinate data; And

And a first wireless communication unit for transmitting the 3D coordinate data converted through the first data conversion unit to the central processing terminal.
According to claim 2,

The FBG shape sensor, the FBG angle sensor, and the FBG torsion sensor are attached to the user's finger and arm parts to track motion for the corresponding body part,

For other body parts except the user's finger and arm, the virtual reality game system further uses at least one of infrared optical camera tracking and a marker.

A virtual reality game system characterized by performing motion capture.
According to claim 2,

The elastic providing unit,

It is installed on the connector side, the virtual reality game system characterized in that it provides a restoring force to the FBG shape sensor, the FBG angle sensor and the FBG torsion sensor that generates tension due to movement and bending of the user joint.
According to claim 1,

The position tracking sensor module,

At least one first location tracking sensor installed on the outer portion of the real game space and detecting a user's location moving in the real game space;

At least one second position tracking sensor installed inside the real game space and detecting a user's location moving within the real game space;

A second data converter configured to convert signals measured through the first lidar sensor and the second lidar sensor into three-dimensional coordinate data; And

And a second wireless communication unit transmitting the 3D coordinate data generated through the second data conversion unit to the central processing terminal.
The method of claim 5,

The first position tracking sensor and the second position tracking sensor,

A virtual reality game system comprising at least one of a LiDAR sensor, an ultrasonic sensor, an infrared optical marker tracking sensor, and a GPS sensor.
According to claim 1,

The central processing terminal is a virtual reality game system, characterized in that for transmitting the virtual image to the HMD through a 5G wireless communication video streaming method.