KR20240046352A - Metaverse based golf training system - Google Patents

Abstract

The present invention relates to a metaverse-based golf training system. The metaverse-based golf training system according to an embodiment of the present invention can provide golf training in a metaverse environment implemented on a metaverse server, and generates the user's joint information based on images captured of the user's movements. and Kinect device; A head mounted device that outputs virtual reality to the user; A user terminal that transmits the joint information to a metaverse server, receives virtual reality information from the metaverse server, transmits the virtual reality information to the head mounted device, and controls virtual reality output from the head mounted device; and providing a virtual golf environment on the metaverse, expressing the user's avatar in 3D in the golf environment based on the joint information transmitted from the user terminal, and providing virtual reality information in which the avatar is expressed. a metaverse server that transmits data to a user terminal and provides a service that allows the user to receive golf lessons in the virtual reality; may include.

Description

Metaverse-based golf training system {METAVERSE BASED GOLF TRAINING SYSTEM}

The present invention relates to a metaverse-based golf training system. More specifically, it is about a golf training system that allows you to meet an instructor in the virtual reality of the metaverse and receive golf lessons, or practice golf on your own using the artificial intelligence of the metaverse.

As the time spent indoors increases due to the coronavirus pandemic, people's demand for outdoor activities and interactions with other people is increasing. In particular, people who like to exercise often experience increased stress due to being confined to the house because they are unable to exercise with others.

VR (virtual reality) technology, which has been researched and developed for a long time, has evolved into a metaverse in the Corona era. Beyond simply experiencing 3D virtual reality, social functions for interacting with people have been strengthened.

You can have non-face-to-face video conferences in the virtual reality provided by Metaverse, and you can also play games with people. Metaverse can take advantage of virtual reality to replace existing face-to-face activities and provide additional services.

For example, during a non-face-to-face meeting, each person can use their own blackboard to write down what they are explaining, and multimedia such as videos can be played, making the meeting more dynamic.

We are actively trying to introduce this metaverse into sports. Among them, golf was not easily accessible to users because the cost of learning golf or playing golf in the field was expensive.

The demand to solve these problems in golf with the metaverse is increasing.

Korean Patent No. 10-10-1270489

The technical problem that the present invention seeks to solve is to provide a system for receiving golf lessons in the virtual reality of the metaverse.

The technical problem that the present invention seeks to solve is to provide a system that allows golf training alone with the help of artificial intelligence in the virtual reality of the metaverse.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

The metaverse-based golf training system according to an embodiment of the present invention to achieve the above technical problem can provide golf training in a metaverse environment implemented on a metaverse server, and can provide golf training based on video captured of the user's movements. A Kinect device that generates joint information of the user; A head mounted device that outputs virtual reality to the user; A user terminal that transmits the joint information to a metaverse server, receives virtual reality information from the metaverse server, transmits the virtual reality information to the head mounted device, and controls virtual reality output from the head mounted device; and providing a virtual golf environment on the metaverse, expressing the user's avatar in 3D in the golf environment based on the joint information transmitted from the user terminal, and providing virtual reality information in which the avatar is expressed. a metaverse server that transmits data to a user terminal and provides a service that allows the user to receive golf lessons in the virtual reality; Can include.Can include.

According to the present invention as described above, there is an effect of being able to access the metaverse at home, meet an instructor in virtual reality, and receive a golf wrestling match without having to go to a driving range to learn golf.

In addition, according to the present invention as described above, when you meet an instructor in virtual reality on the metaverse, the instructor can point out swing problems while watching the movements of an avatar that copies the user's movements, which has a similar effect to teaching golf by meeting in real life. .

In addition, according to the present invention as described above, when making a golf swing alone on the metaverse, training information regarding golf posture can be received with the help of artificial intelligence, which has the effect of allowing golfers of intermediate level or higher to train alone.

Figure 1 is a configuration diagram of a metaverse-based golf training system according to an embodiment of the present invention.
Figure 2 is an example of a user's avatar expression in virtual reality on the metaverse, according to an embodiment of the present invention.
Figure 3 is an example of recognizing a user's posture using a Kinect device and copying it to represent an avatar in virtual reality on the metaverse, according to an embodiment of the present invention.
Figure 4 is an example of the Kinect device recognizing the user's joints, according to an embodiment of the present invention.
Figure 5 is an example of a UI for selecting an instructor in virtual reality on the metaverse, according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely intended to ensure that the disclosure of the present invention is complete and to provide common knowledge in the technical field to which the present invention pertains. It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings that can be commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

Figure 1 is a configuration diagram of a metaverse-based golf training system according to an embodiment of the present invention. Referring to FIG. 1, a metaverse-based golf training system (hereinafter referred to as “golf training system”) according to an embodiment of the present invention will be described.

The golf training system 10 according to an embodiment of the present invention may include a Kinect device 110, a head mounted device 120, a user terminal 100, and a metaverse server 200.

The Kinect device 110 may generate joint information about the user based on images captured of the user's movements. The Kinect device 110 may include a video camera, an infrared transmitter, and an infrared sensor. The Kinect device 110 can capture depth images by measuring the distance to the object by detecting the infrared rays emitted from the infrared transmitter when they are reflected by an object and returning back. The Kinect device 110 can capture RGB images using a video camera. The Kinect device 110 can extract the human skeleton and joints by combining the RGB image and the depth image. The Kinect device 110 may transmit information about the extracted fracture to the user terminal 100.

The head mounted device 120 can output virtual reality to the user. The head mounted device 120 may include two displays, and each display may be located in front of the user's eyes, allowing the user to perceive 3D virtual reality.

The head mounted device 120 may include commercial products such as Oculus Rift, Vive, Playstation VR, and OSVR, but this is only an example and is not limited thereto.

The user terminal 100 may transmit joint information transmitted from the Kinect device 110 to the metaverse server 200. The joint information may be used to recognize the user's motion or posture, or may be used as an input for interaction with virtual objects existing in virtual reality on the metaverse.

When the user terminal 100 receives virtual reality information of the metaverse from the metaverse server 200, the user terminal 100 may transmit the virtual reality information to the head mounted device 120, and the head mounted device 120 may transmit the virtual reality information to the head mounted device 120. It can be controlled so that is output.

For example, the user terminal 100 may be a PC (Personal Computer) including a high-performance graphics card, and renders the virtual reality information as a 3D image and outputs it to the head mounted device 120. can do.

The metaverse server 200 can provide a virtual golf environment on the metaverse. The metaverse server 200 may represent the user's avatar in 3D in the golf environment based on the joint information transmitted from the user terminal 100. The metaverse server 200 may display one or more avatars in the golf environment based on each user's joint information from one or more user terminals. The avatar has an appearance defined in advance by each user, and the posture of the avatar is determined by the joint information. The metaverse server 200 may create an avatar in the virtual reality that is in the same posture as the user. The metaverse server 200 may transmit virtual reality information representing the avatar to the user terminal 100.

The metaverse server 200 can simulate the interactions of each user in the virtual reality. The metaverse server 200 can match users and golf instructors.

The metaverse server 200 can provide golf lessons to users in the virtual reality of the golf environment. The metaverse server 200 matches a user with a golf instructor, and the golf instructor can provide golf lessons to the user by watching the user's avatar make a golf swing.

The metaverse server 200 may provide an avatar that imitates the user's movements based on the user's fracture information.

The metaverse server 200 according to an embodiment of the present invention allows the golf instructor to recognize the user's movements just by looking at the avatar in the virtual reality of the metaverse, and customize golf to the user according to the recognized movements. Lessons can be provided.

The metaverse server 200 according to another embodiment of the present invention can use artificial intelligence to analyze the golf swing based on the user's movements and provide customized golf lessons to the user. The metaverse server 200 can calculate a motion matching rate by comparing it with the user's golf swing based on a predefined standard golf swing, and transmit golf lesson information according to the motion matching rate to the user terminal 100. there is.

Although not shown in FIG. 1, the golf training system 10 according to another embodiment of the invention may further include a haptic device.

The haptic device may include an acceleration sensor, a vibration device, and a gripper. The acceleration sensor may be a sensor that measures the acceleration at which the haptic device moves. The vibration device may be a device that generates vibration for haptic output. The grip part may be a part that a user can hold with his or her hand.

The user terminal 100 may transmit joint information recognized by the Kinect device 110 and acceleration information output from the haptic device to the metaverse server 200 when the user makes a swing motion. The metaverse server 200 may analyze the user's swing motion based on the joint information and the acceleration information and transmit golf lesson information according to the analyzed swing motion to the user terminal 100.

The metaverse server 200 can analyze the swing motion and calculate vibration according to the hit when hitting a golf ball by swinging, and generate vibration information according to the calculated vibration to the user terminal 100. Can be transmitted.

When the user terminal 100 receives vibration information from the metaverse server 200, it can control the vibration of the vibration device according to the vibration information.

Figure 2 is an example of a user's avatar expression in virtual reality on the metaverse, according to an embodiment of the present invention.

Referring to FIG. 2, the metaverse server 200 can implement virtual reality on the metaverse as a golf driving range or golf field. The metaverse server 200 may receive the user's joint information from the user terminal 100 and display an avatar posing according to the joint information on the virtual reality. Figure 2 is only an example, and the present invention is not limited thereto.

Figure 3 is an example of recognizing a user's posture using a Kinect device and copying it to represent an avatar in virtual reality on the metaverse, according to an embodiment of the present invention. Figure 4 is an example of the Kinect device recognizing the user's joints, according to an embodiment of the present invention. Based on Figures 3 and 4, the user's joint information recognized by the Kinect device will be described.

As shown on the left side of FIG. 3, the Kinect device 110 can extract the user's joint information from the image based on depth images and artificial intelligence and transmit the joint information to the metaverse server 200.

As shown on the right side of FIG. 3, when the Kinect device 110 receives the joint information, it can create an avatar in virtual reality posing according to the joint information.

Figure 4 shows joints recognized by the Kinect device 110 and their names.

Among the joint information, the metaverse server 200 converts the relative positions of the joints of the shoulder, elbow, wrist, pelvis, and knee into vectors and then recognizes the motion. If the motion is recognized as a swing start motion, it is converted into a swing completion motion. The mid-swing motion is recognized at regular time intervals until recognized, the mid-swing motion is compared with a predetermined reference mid-motion to calculate a motion match rate, and golf lesson information is sent to the user terminal 100 based on the motion match rate. can be transmitted to.

The metaverse server 200 may use a machine learning algorithm learned to recognize the user's motion based on the vector values of the joints. The metaverse server 200 may convert the relative positions of the joints of the shoulder, elbow, wrist, pelvis, and knee among the joint information received from the user terminal 100 into vectors. By inputting the converted vector value into the machine learning algorithm, the motion matching rate with predefined golf motions can be derived.

When a motion matching a predefined swing start motion is recognized, the metaverse server 200 may recognize that the user is starting a golf swing, and may perform the user's motion at regular time intervals until the swing completion motion is recognized. can be compared with a predefined mid-swing motion. For example, the swing start motion may be a general golf address posture, the swing completion motion may be a motion of holding a golf club and stopping for a certain period of time, and swing motions can be compared at 0.2 second intervals, but this is only an example and is limited to this. It doesn't work.

Figure 5 is an example of a UI for selecting an instructor in virtual reality on the metaverse, according to an embodiment of the present invention. The operation of selecting an instructor will be described with reference to FIG. 5 .

The metaverse server 200 may synthesize an instructor selection user interface (UI) with virtual reality and transmit the synthesized virtual reality information to the user terminal 100.

The user terminal 100 may output the virtual reality information to the head mounted device 120. The user can select one of the instructors included in the instructor selection UI by moving his or her hand.

The user terminal 100 may recognize the instructor selection based on the joint information recognized by the Kinect device 110. For example, if the position of the wrist joint included in the joint information is within the position of one instructor among the instructor selection UI, it may be determined to select the instructor.

The user terminal 100 may transmit the selected instructor to the metaverse server 200.

When receiving the selected instructor, the metaverse server 200 transmits information indicating selection to the user terminal indicated by the selected instructor, and when information indicating acceptance is received from the user terminal of the selected instructor, the metaverse server 200 selects the instructor. The user's avatar and the selected instructor's avatar can be placed in one metaverse space. The user and instructor can meet each other in one space and conduct golf lessons through interaction with each other.

Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will be able to understand it. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

10 Golf Training Systems on the Metaverse
User terminal 100
Kinect device 110
Head mount device 120

Claims (4)

In a system that provides golf training in a metaverse environment implemented on a metaverse server,
A Kinect device that generates the user's joint information based on the video captured of the user's movements;
A head mounted device that outputs virtual reality to the user;
A user terminal that transmits the joint information to a metaverse server, receives virtual reality information from the metaverse server, transmits the virtual reality information to the head mounted device, and controls virtual reality output from the head mounted device; and
Providing a virtual golf environment on the metaverse, expressing the user's avatar in 3D in the golf environment based on the joint information transmitted from the user terminal, and providing virtual reality information in which the avatar is expressed to the user a metaverse server that transmits data to a terminal and provides a service that allows the user to receive golf lessons in the virtual reality; Including,
The metaverse server is,
For golf lessons, the user and a golf instructor can meet on the metaverse, and the golf swing can be recognized based on the joint information of the user so that the user can give golf lessons on his own, and the recognized golf swing and Comparing a predetermined reference swing and transmitting golf lesson information according to the comparison to the user terminal,
Metaverse-based golf training system. According to paragraph 1,
The user terminal is,
Upon receiving the instructor selection UI, the instructor selection UI is output to the head-mounted device, and based on the joint information extracted by the Kinect device, it is determined which one of the instructors included in the instructor selection UI is selected, and the selected instructor is selected. is transmitted to the metaverse server,
The metaverse server is,
When the selected instructor is received, information indicating selection is transmitted to the user terminal indicated by the selected instructor, and when information indicating acceptance is received from the user terminal of the selected instructor, the avatar of the user who selected the instructor and the selected instructor are transmitted. Placing avatars in one metaverse space,
Metaverse-based golf training system. According to paragraph 1,
The metaverse server is,
Among the joint information, the relative positions of the joints of the shoulder, elbow, wrist, pelvis, and knee are converted into vectors and then the motion is recognized. If the motion is recognized as a swing start motion, a certain time interval is provided until it is recognized as a swing completion motion. Recognizes the mid-swing motion, calculates a motion matching rate by comparing the mid-swing motion with a predetermined reference mid-motion, and transmits golf lesson information to the user terminal based on the motion matching rate.
Metaverse-based golf training system. According to paragraph 1,
It further includes a haptic device including an acceleration sensor, a vibration device, and a gripper,
The user terminal is,
When the user makes a swing motion, the joint information recognized by the Kinect device and the acceleration information output from the haptic device are transmitted to the metaverse server, and when vibration information is received from the metaverse server, the vibration information is transmitted according to the vibration information. Controls the vibration of the vibrating device,
The metaverse server is,
The user's swing motion is analyzed based on the joint information and the acceleration information, vibration information according to hitting the golf ball during the swing motion is generated and transmitted to the user terminal, and golf according to the analyzed swing motion is generated. Transmitting lesson information to the user terminal,
Metaverse-based golf training system.