KR20240113684A - Human body muscle learning system by metaverse interaction - Google Patents

Abstract

The present invention relates to a human muscle learning system through metaverse interaction. When a user selects a muscle in the metaverse system, the control unit presents the name of the muscle in voice or text and also presents the method of using the muscle. In other words, the metaverse system presents exercise methods appropriate for the muscles in question and predicted results according to the exercise, and allows the user to operate as an avatar in the metaverse. The movements are performed together by the user's avatar and the character.
The present invention provides the name of the muscle so that it can be easily remembered, and the user's avatar and character share the exercise method and prediction results according to the muscle in the metaverse, making muscle exercise interesting, increasing the user's motivation, and providing a remarkable effect that increases the goal achievement rate. It works.

Description

Human body muscle learning system by metaverse interaction}

The present invention relates to a human muscle learning system through metaverse interaction. More specifically, it provides the name of the muscle so that it can be easily remembered, and the user avatar and character share the exercise method and predicted result according to the muscle in the metaverse. It is about a human muscle learning system using metaverse interaction that makes muscle exercise interesting, increases user motivation, and increases goal achievement rate.

In general, in modern times, people sit in front of their desks for a long time or have less free time, so the time to exercise becomes shorter, and the body generally becomes weaker. And because they did not learn the correct exercise posture, exercise often became the cause of their health. Therefore, it is necessary to exercise correctly and efficiently, and as an example of a conventional patent technology, registration number 10-2461484 includes a method performed by one or more processors of a computing device, obtaining information on the user's exercise ability corresponding to the user terminal. steps; generating a virtual character based on the motor ability information; and providing a training service based on the virtual character. The step of acquiring the exercise ability information includes: body size of the user based on body part information to which each of the plurality of sensor devices provided on the user's body is attached and distance information between the plurality of sensor devices Obtaining information; Including, providing the training service includes: identifying a type of exercise performed by the user based on movement information of the user obtained from the plurality of sensor devices; Counting the user's movements according to the identified type of exercise to generate the user's momentum sensing information, but not counting movements unrelated to the identified type of exercise; Includes, wherein identifying the type of exercise includes: acquiring an exercise rule setting signal from the user; generating a new exercise rule based on the user's movement information; and sharing the generated exercise rules with other users. Further comprising: providing the training service, comprising: acquiring posture information of the user based on information obtained from the plurality of sensor devices; Generating device control information for adjusting the intensity of an exercise device used by the user and exercise guide information for the user based on the user's posture information; and transmitting the device control information and the exercise guide information to the exercise device. A method of providing metaverse training services, including, is disclosed.

In addition, Publication No. 10-2012-0003588 discloses a healthcare system equipped with a metaverse interconnection function including a health server.

And registration number 10-2445543 discloses a system for providing metaverse training services, and registration number 10-2406852 discloses a user health management system using wearable devices and metaverse.

Therefore, the present invention was created to solve the above problems. It provides the name of the muscle so that it can be easily remembered, and the user's avatar and character share the exercise method and predicted result according to the muscle in the metaverse to enable muscle exercise. The goal is to provide a human muscle learning system using metaverse interaction that is interesting and increases user motivation and goal achievement rate.

The present invention relates to a human muscle learning system through metaverse interaction. When a user selects a muscle in the metaverse system, the control unit presents the name of the muscle in voice or text and also presents the method of using the muscle. In other words, the metaverse system presents exercise methods appropriate for the relevant muscles and predicted results according to the exercise, and allows the user to operate as an avatar in the metaverse. The movement is characterized by having the user's avatar and the character perform together.

Therefore, the present invention informs the name of the muscle in question so that it can be easily remembered, and the exercise method and prediction results according to the muscle are shared with the user's avatar and character in the metaverse, so that muscle exercise is interesting, increases the user's motivation, and increases the goal achievement rate. It has a remarkable effect.

Figure 1 is a metaverse connection screen of the present invention.
Figure 2 is a metaverse character screen of the present invention
Figure 3 is an AI hand implementation screen of the present invention
Figure 4 is a screen implemented with an AI hand in the metaverse of the present invention.
Figure 5 is a screen of the AI body implementation of the present invention

The present invention relates to a human muscle learning system through metaverse interaction. When a user selects a muscle in the metaverse system, the control unit presents the name of the muscle in voice or text and also presents the method of using the muscle. It is characterized by

In addition, the metaverse system presents exercise methods appropriate for the muscles and the results obtained according to the exercise, and allows the user to operate as an avatar in the metaverse.

Additionally, the operation is characterized in that the user avatar and the character are together.

The present invention will be described in detail with the accompanying drawings as follows. Figure 1 is the metaverse connection screen of the present invention, Figure 2 is the metaverse character screen of the present invention, Figure 3 is the AI hand implementation screen of the present invention, and Figure 4 is The screen of the AI hand implemented in the metaverse of the present invention, Figure 5 is the screen of the AI body implemented of the present invention.

In the human muscle learning system based on the metaverse interaction of the present invention, when the user selects the corresponding muscle, such as the thigh muscle, in the metaverse system, the control unit presents the name of the corresponding muscle in voice or text. Also, the method of using the muscle, for example, relaxing, stretching, or rotating the muscle, is presented.

In addition, the metaverse system suggests exercise methods and exercises suitable for the relevant muscles, such as soccer for the thigh muscles, and also presents the results obtained accordingly. The development and qualitative improvement of the quadriceps muscles are presented. Then, the user becomes an avatar in the metaverse and operates.

Additionally, the control unit causes the user avatar and the character to perform the above actions together. Characters are operated by keys and mouse.

Meanwhile, the muscles are assembled into a single object in the corresponding area, and in the case of leg muscles, several muscles such as the fetus fetus are combined into a bone shape. Then, the character disassembles and reassembles the various muscles mentioned above. Multi-user access is possible through the Photon Network, so when a teacher and students connect together, the teacher demonstrates first and the students follow. Each of the muscles has a different color, and this is implemented by applying material or text to match the corresponding color. And when the corresponding muscles are combined in the right position, a bright lamp material operates, making it appear as if a light is turned on. Turning on the lamp is controlled by the metaverse control unit by a trigger. Meanwhile, if the correct answer is turned on as shown above through the com3 serial port, the actual LED connected to the com3 serial port via Arduino can also be turned on. In order to make the class more efficient, each muscle shows its own characteristics when placed in a box object, so it can be stretched and stretched at different degrees, allowing the class to be taught as if actual human muscles reacted to the human body's electrical signals. This can be implemented with animation effects, and in the case of Uniter, it is controlled by inserting the relevant animation clip into the animator. Each operation is controlled through an on-click operation on the button controlled by the network manager.

The technical gist of the present invention is to show and follow movements performed in the real world within the metaverse. Metaverse is produced using Unit and Unreal Engine. In addition, multiplayer is also possible by connecting to Photon so that multiple players can access and exercise and watch at the same time in one projector scene. At this time, components according to Hierarchy such as Photon View, Photon Rigid Body View, and Photon Transformer View are imported. Therefore, teachers and students can exercise in one scene, increasing educational effectiveness.

Soon, you will be able to actually experience the desired movement through video in reality and within the metaverse, increasing the educational effectiveness. Within the metaverse, movement is implemented by applying scripts to characters. Scripts are built in C# language for units. And the backgrounds are created in 3D using Blender and Maya programs and then imported. Import is mainly done by converting to an Fbx file to suit Unity. For the character's complex movements, access a site that provides automatic rigging, such as Mixamo, upload the character, implement the desired movements such as running and throwing, and then import them into Unity. You can build assets yourself, but you can reduce production costs by using free assets. Recognition with an object object can be done automatically by using a collision trigger and activating a trigger using collide when the character collides with the object object.

For exercise in reality, if an AI module is installed in the reality interaction metaverse exercise system of the present invention using an AI model such as a media pipe, all images captured by a camera in reality are displayed on the metaverse screen through the image processing unit and AI module. It will appear. The image processing unit uses OPENCV to create clearer images through filtering. In the metaverse, characters exercise together, and real-life avatars exercise with characters in the metaverse. Users become characters or avatars. Meanwhile, once the exercise muscle is determined, the control unit instructs the character to perform exercises that are helpful for that muscle. In the case of hand movements, the character folds and unfolds the fingers one by one in order, and the user follows suit, and the avatar in the metaverse does the same. Also, by carrying the block object together, the distance it can be carried without dropping it to a certain distance is measured. At this time, the user avatar hand uses only one finger. Another method is to measure the time it takes to carry a block object to a certain distance. The data is recorded online and offline and compared in the next round to see how much progress has been made. Soon, balance, finger muscle flexibility, etc. will develop and you will get good results. The weight and size of a block object, or cube, can be changed in the component, and sometimes a rotational force is generated by a script so that the character and user avatar resist and do not drop the cube. Because multiplayers connect simultaneously through the Photon Network, multiple user avatars can work together to carry the cube. At this time, the wind strength can be adjusted in advance on Terran, and the terrain can be designed to be curved.

In the case of exercise in reality according to the present invention, for example, if the hand is made into an avatar in the metaverse,

Use the PyCharm program.

Go to the Settings tab of the PyCharm program.

Install CVZONE PAKAGE from the python interpreter of the Project in the settings tab of the PyCharm program.

In CVZONE PAKAGE

Includes CVZONE, OPENCV-PYTHON, NUMPY, PIP, SETUPTOOLS, and WHEEL.

CVZONE is COMPUTER VISION ZONE and is the area displayed on the computer.

Install MEDIAPIPE from the project's python interpreter in the PyCharm program settings tab.

MEDIAPIPE is about recognizing objects.

Below are instructions on how to install PyCharm by writing in the PyCharm notebook.

code

# pip install cvzone

# pip install mediapipe

import cv2

from cvzone.HandTrackingModule import HandDetector

import socket

#Parameters

width, height = 1850, 1025

#Webcam

cap = cv2.VideoCapture(0)

cap.set(3, width)

cap.set(4, height)

#Hand Detector

detector = HandDetector(maxHands=1, detectionCon=0.8)

#Communication

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

serverAddressPort = ('127.0.0.1', 5052)

whileTrue:

success, img = cap.read()

data = []

#Hands

hands, img = detector.findHands(img)

# Landmark values - (x,y,z) *21

if hands:

# Get the first hand detected

hand = hands[0]

# Get the landmark list

lmList = hand['lmList']

#print(lmList)

for lm in lmList:

data.extend([lm[0], height - lm[1], lm[2]])

print(data)

sock.sendto(str.encode(str(data)), serverAddressPort)

img = cv2.resize(img, (0, 0), None, 0.5, 0.5)

cv2.imshow('Image', img)

cv2.waitKey(1)

cv2.destroyAllWindows()

Import the opencv library into PyCharm notes.

Import the webcam from the opencv library in PyCharm notes.

The width and height sizes of the window that outputs the video captured by the webcam are set to 1250 and 720.

In the PyCharm note, the HandDetector is imported from the HandTrackingModule, one of the functions of MEDIAPIPE.

Specifies the number of hands to be tracked by the tracking module as 1.

Accordingly, when video data captured by the webcam is input, the object is recognized by HandDetector using the HANDTRACKINGMODULE module of MEDIAPIPE, and the video captured by the webcam is output to the output window.

The output hand shape consists of 21 spheres and a band connecting each sphere and is imported into Unity. Meanwhile, when using the above hand, a picture can be drawn on a plate object. The plate to which Unity's INDENT DENT script is added has a groove formed by a pencil-shaped object to which the INDENT ACTOR script has been added. Soon, a groove will be drawn according to the movement of the pencil-shaped 3D object held by the character by adding a script. Add a script to grab one of the real hand-shaped spheres. Therefore, as you move your hand, text or pictures are drawn on the plate object in the metaverse.

For the real body, import the opencv library into PyCharm notes.

Import the webcam from the opencv library in PyCharm notes.

The width and height sizes of the window that outputs the video captured by the webcam are set to 1250 and 720.

In the PyCharm note, the body detector (PoseDetector) is imported from PoseModule (body tracking module), one of the functions of MEDIAPIPE.

Specifies the number of bodies to be tracked by the tracking module as 1.

Accordingly, when video data captured by a webcam is input, the body is recognized as an object using the PoseDetector of PoseModule, one of the functions of MEDIAPIPE, and the video captured by the webcam is output to the output window.

The output body shape consists of 32 spheres and a band connecting each sphere and is imported into Unity.

And the present invention outputs a voice instructing movement, and when a character or a real avatar reaches a certain exercise course, a signal is transmitted to the audio of the camera object in the metaverse mainly by using the step taken by the collision value. Instructions such as how to do the exercise and how many times to repeat are given by voice. Or, they are displayed as text.

There is a mirror object in the metaverse, so when a character or avatar stands in front of a mirror and clicks the body type button they want, an exercise method suitable for their body type is presented.

The system of the present invention consists of an image providing unit, a storage unit, and a control unit.

The control unit motivates users by displaying photos of famous athletes performing exercises suitable for the selected body type within the metaverse.

Metaverse is produced using Unit and Unreal Engine. In addition, multiplayer is also possible by connecting to Photon so that multiple players can access and exercise and watch at the same time in one scene. Therefore, teachers and students can exercise in one scene, increasing educational effectiveness. The metaverse content is also produced as an app through Unity, so users can always exercise and practice through the smartphone app.

Claims (3)

When the user selects the muscle in the metaverse system, the control unit presents the name of the muscle in voice or text and also presents the method of using the muscle. A human muscle learning system based on metaverse interaction. The human muscle learning system based on metaverse interaction according to claim 1, wherein the metaverse system presents exercise methods appropriate for the relevant muscles and results obtained according to the exercises, and allows the user to operate as an avatar in the metaverse. The human muscle learning system according to claim 2, wherein the movement is performed by a user avatar and a character together.