KR102171319B1 - Appratus for writing motion-script, appratus for self-learning montion and method for using the same - Google Patents

Abstract

A motion textbook authoring apparatus, a motion self-learning apparatus, and a method are disclosed. In the motion self-learning method according to an embodiment of the present invention, in the motion self-learning method using a motion textbook authoring device and motion self-learning device, the motion textbook authoring device generates a motion script based on the expert motion of a first user. Step to do; Analyzing, by the motion self-learning device, a second user's motion learning the expert motion based on the motion script, and outputting, by the motion self-learning device, a result of analyzing the motion of the second user. do.

Description

Motion textbook writing device, motion self-learning device and method {APPRATUS FOR WRITING MOTION-SCRIPT, APPRATUS FOR SELF-LEARNING MONTION AND METHOD FOR USING THE SAME}

The present invention relates to a motion learning system framework, and more particularly, to a motion recognition-based service and motion self-learning technology.

There has always been a demand to learn the exemplary postures required for sports movements such as yoga, taekwondo and table tennis, or for professionals such as flight attendants and hoteliers.

One of the best ways to learn specific movements is to repeatedly imitate the movements of the experts through one-on-one lessons with experts, and continuously correct one's posture by receiving immediate feedback.

However, it is difficult for a general individual to visit a specialized facility and receive instruction from a professional instructor due to restrictions such as location, time, and economic cost. Therefore, requirements for a self-learning method that can effectively learn specific movements without meeting an expert and receiving direct instruction are increasing, and the related base industry is also developing.

Currently, the most common method for learners to self-learn movements such as yoga, gymnastics, and dance is by imitating video. This is a method in which the learner purchases a teaching video or downloads an appropriate video from the Internet, sees and follows the instructor's actions on the screen. Existing learning methods that simply imitate these videos are difficult to be seen as an effective self-learning method because there is no accurate feedback on the accuracy of the learner's actions, error correction and improvement, and learning achievement. In addition, since it relies solely on the willpower of the learner without information on the current status of learning, the learner's ability to sustain learning is also significantly lowered.

With the rapid development of IT technology in recent years, a variety of content services based on motion recognition sensors and motion recognition technologies have been provided.

Participatory game platforms that respond to user actions such as Microsoft's Xbox and Sony's PlayStation are gaining worldwide success. These kinds of game platforms are mainly implemented with a technology that recognizes the position of a user's body part and a designated posture and motion by analyzing a depth image input through a low-cost 3D camera.

Following the success of participatory game contents based on simple motion recognition technology, motion recognition technology is being applied for the purpose of coaching on accurate exercise posture even in sports areas consisting of relatively complex motions such as golf and baseball. In recent years, a sports simulator platform including a hardware device through which the learner can ride directly along with a technology that provides feedback to the learner through motion analysis has been developed. In the case of sports such as horseback riding or yachting, where it is not easy to receive instruction from a professional instructor, a sports simulator platform that simulates the real environment can be a good alternative.

Although the above-mentioned participatory game platform, sports motion and posture analysis system, and sports simulator platform including equipment provide services based on the technology that analyzes the user's motion, all of them support the self-learning of individual learners. It is not a system for doing.

Motion-based game platforms are mainly developed for entertainment or fitness purposes, and do not provide diagnostic services for the user's posture.

The sports posture analysis system provides a detailed analysis of a player's posture and a diagnosis service based on the analysis. However, these diagnostic services are specialized and developed for the sport by some specific operators that specialize in selling sports simulators, and are built in a sports-only business site equipped with related facilities such as screen golf and screen ballpark. This coaching system is not aimed at self-learning, but aims to secure expertise in sports simulators, increase interest and immersion, and serve as an auxiliary analysis tool for professional leaders.

In the point that the above-described requirements for self-learning are "providing a universal learning method for sample movements in various fields without constraints on time and space", the existing coaching system does not meet these requirements at all.

Meanwhile, Korean Patent Laid-Open Publication No. 10-2010-0132322 “A motion learning system and method using motion capture and three-dimensional graphics” describes one or more three-dimensional renderings simultaneously using a virtual character to which motion capture and three-dimensional graphic technology are applied. By simultaneously outputting graphic motion image information and one or more three-dimensional graphic step image information to one or more display devices, a motion learning system that can accurately observe and understand human motion information obtained in a three-dimensional space through image information from various viewpoints. It is starting.

An object of the present invention is to provide a universal learning method for sample movements in various fields without constraints on time and space in a method for learners to learn movements and postures by themselves.

In addition, the present invention allows motion experts in a specific field to directly author, edit, and publish motion scripts as their performances, and allow a number of motion learners to self-learn correct motions with the published motion texts. It aims to provide a generalized system framework that supports.

In addition, an object of the present invention is to support the production of motion-based services and teaching content, which dramatically reduces the cost of content production by allowing motion-based content creators to easily secure motion data from a large-capacity motion textbook database.

In addition, an object of the present invention is to improve content satisfaction so that self-learners can expand the demand level for self-learning by receiving various motion-based content and services developed by content creators from a motion textbook store server.

In addition, an object of the present invention is to overcome the limitations of the existing self-learning method that simply imitates the teaching video and contribute to the creation of an ecosystem of content services related to self-learning movements.

An operation textbook authoring apparatus according to an embodiment of the present invention for achieving the above object comprises: an authoring unit for generating attribute information corresponding to a user's operation; A sensor unit obtaining sensor data from the user's motion; And a motion script generating unit generating a motion script based on the attribute information and the sensor data, and an editing publishing unit distributing the motion script.

In this case, the authoring unit comprises: a header information setting unit for generating motion script header information including at least one of a file property for the motion script and tag information for searching for the motion script; A motion attribute mapping unit for mapping the user's motion and the attribute information, and an additional information setting unit for setting whether to provide additional information used for motion learning.

In this case, the attribute information may include key pose setting information corresponding to the user's key pose; Key motion setting information corresponding to the user's key motion; It may include at least one of operation setting information corresponding to the speed and strength of the key motion, and key body part setting information corresponding to the key body part of the user based on the key pose and the key motion.

In this case, the motion script generation unit is a motion descriptor generation unit that determines the type and characteristic of the motion acquisition device and generates motion descriptor information from the acquired expert sensor data based on the identified type and characteristic of the motion acquisition device. ; And a motion property generator generating motion property information from the acquired expert sensor data based on the property information.

In this case, the motion script generator may generate the motion script including at least one of the motion script header information, the motion attribute information, the motion descriptor information, and the additional information.

In this case, the editing and publishing unit may perform at least one of textbook image editing, detailed operation correction, detailed attribute correction, and additional information editing on the generated motion script to distribute the edited final motion script to the operation textbook store server. have.

In addition, a motion self-learning apparatus according to an embodiment of the present invention for achieving the above object includes: a learning unit for learning the expert motion to a user by outputting an expert motion of a motion script; A sensor unit that acquires sensor data from the user's motion using a motion acquisition device; And a motion evaluation unit that generates final motion evaluation information that evaluates the user's motion based on the Moseng script and sensor data, and an output unit that outputs the final motion evaluation information and calibration information for correcting the user's motion.

In this case, the learning unit includes an expert motion script setting unit for generating learning attribute information using attributes selected by the user for motion learning in the motion script; A learning environment setting unit for generating learning environment information based on the learning environment set by the user, and a learner motion script setting unit for generating the user's motion as a learner motion script.

In this case, the motion evaluation unit is a motion analysis unit that generates posture accuracy information and dynamic expressive power information of the user based on the motion script and the sensor data, and final motion evaluation information based on the posture accuracy information and the dynamic expressive power information. It may include an operation evaluation calculation unit that generates.

In this case, the motion analysis unit may analyze motion descriptor information and motion attribute information by using sensor data obtained from a motion acquisition device of the same type as the motion acquisition device used in the motion script.

In this case, the motion analysis unit generates posture accuracy information including posture accuracy values obtained by analyzing the similarity of the user's motions for each result of analyzing the motion descriptor information and motion attribute information based on the motion script and the sensor data. can do.

In this case, the motion analysis unit analyzes the similarity of at least one of the speed, intensity, and motion interval of the user's motion for each result of analyzing the motion descriptor information and motion attribute information based on the motion script and the sensor data. It is possible to generate dynamic expressiveness information including one dynamic expressiveness value.

In this case, the motion evaluation calculator may determine a weight for each of the posture accuracy values and each of the dynamic expressive power values based on at least one of the number, type, and characteristic of the motion acquisition device.

In this case, the motion evaluation calculator may determine a sum weight for summing the weighted posture accuracy values and the dynamic expressive power values for each of the user's motions.

In this case, the motion evaluation calculator may generate final motion evaluation information including final motion evaluation values calculated by summing the posture accuracy values and the dynamic expressive power values based on the sum weight determined for each of the user's motions. have.

In this case, the motion evaluation calculator may set a high sum of the dynamic expression power values in proportion to the user's learning ability setting value, and set the posture accuracy value to be low.

In this case, the output unit may edit the learner's motion script based on the final evaluation result information and the correction information and distribute it to the motion textbook store server.

In this case, the output unit may further output learning evaluation information obtained by evaluating a score for each evaluation element and a motion accuracy for each body part of the motion learner based on the final motion evaluation information.

In this case, the output unit may further output detailed evaluation information including a score trend for each evaluation element of the expert action learned by the user.

In addition, in the motion self-learning method according to an embodiment of the present invention for achieving the above object, in the motion self-learning method using the motion textbook authoring device and the motion self-learning device, the motion textbook authoring device Generating a motion script based on an expert motion; Analyzing, by the motion self-learning device, a second user's motion learning the expert motion based on the motion script, and outputting, by the motion self-learning device, a result of analyzing the motion of the second user. do.

The present invention can provide a universal learning method for sample movements in various fields without constraints on time and space in a method for a learner to learn movements and postures on their own.

In addition, the present invention allows motion experts in a specific field to directly author, edit, and publish motion scripts as their performances, and allow a number of motion learners to self-learn correct motions with the published motion texts. A generalized system framework that supports can be provided.

In addition, according to the present invention, motion-based content creators can easily secure motion data from a large-capacity motion textbook database, thereby supporting the production of motion-based services and teaching content, which significantly reduces content production costs.

In addition, the present invention can improve content satisfaction so that self-learners can expand the demand level for self-learning by receiving various motion-based content and services developed by content creators from a motion textbook store server.

In addition, the present invention overcomes the limitations of the existing self-learning method that simply imitates the teaching video, and contributes to the creation of an ecosystem of content services related to self-learning movement.

1 is a diagram showing an operation self-learning system according to an embodiment of the present invention.
2 is a block diagram showing an operation textbook authoring apparatus according to an embodiment of the present invention.
3 is a diagram showing in detail an example of the operation textbook authoring apparatus shown in FIG. 2.
4 is a diagram showing a motion script according to an embodiment of the present invention.
5 is a diagram showing an operation self-learning device according to an embodiment of the present invention.
6 is a diagram showing in detail an example of the motion self-learning device shown in FIG. 5.
7 is a diagram showing in detail an example of a motion evaluation process performed by the motion evaluation unit illustrated in FIG. 6.
8 and 9 are diagrams showing an output screen of a final motion evaluation result according to an embodiment of the present invention.
10 is an operation flow diagram showing a method for self-learning motion according to an embodiment of the present invention.
11 is an operation flow diagram showing in detail an example of the operation textbook authoring step shown in FIG.
12 is an operation flow diagram showing in detail an example of the operation self-learning step shown in FIG. 10.
13 is an operation flow diagram showing in detail an example of the operation evaluation step shown in FIG. 12.
14 is a block diagram showing a computer system according to an embodiment of the present invention.

The present invention will be described in detail with reference to the accompanying drawings as follows. Here, repeated descriptions, well-known functions that may unnecessarily obscure the subject matter of the present invention, and detailed descriptions of configurations are omitted. Embodiments of the present invention are provided to more completely explain the present invention to those with average knowledge in the art. Accordingly, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

Throughout the specification, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing an operation self-learning system according to an embodiment of the present invention.

Referring to FIG. 1, in the motion self-learning system according to an embodiment of the present invention, the motion textbook authoring apparatus 100 may generate a motion script based on the motion of a motion expert.

The motion textbook authoring apparatus 100 may capture an expert performance using a 3D camera, a smart phone, and a motion motion acquisition device capable of measuring various positions/speeds/angles.

In this case, the motion textbook authoring apparatus 100 may generate a motion script corresponding to the motion textbook by receiving the image and sensor data of the motion motion acquisition device, describing the motion of the expert and assigning motion properties.

At this time, the motion textbook authoring apparatus 100 may upload the motion script to the motion textbook store server 10.

The motion learner can learn exemplary motions by using the motion self-learning device 200 by receiving a motion script that they want to learn directly from a specific motion expert or downloading it from the motion textbook store server 10 that sells motion scripts. .

The motion self-learning device 200 may compare and analyze a sample motion of a motion script written by an expert with a learner's current motion to provide an appropriate diagnosis and learning method to the learner, and support the learner to continuously correct the posture. .

In addition, motion learners can produce motion scripts from learning actions taken in the course of their learning, and publish them on the store site.

At this time, the motion script written by the motion learner may have a level difference from the model motion of the expert, but it can be a useful motion textbook for another learner with a lower level than the learner, and the content creators choose the motion of interest. Can provide a variety of.

In addition, various motion scripts published by an unspecified number of experts (or learners) may be accumulated in the motion textbook store server 10. A set of motion scripts describing motions in various fields performed by a large number of people can be excellent data for creating content based on human motions.

Conventionally, in order for content creators to secure human motion data, there is a method of direct production or requesting a producer who specializes in motion data.

However, in the related art, since motion data is produced spontaneously when necessary, there is a disadvantage in that production cost is high and production time is required. However, the content creator on the system framework proposed in the present invention is the operation textbook store server 10 It has the advantage of easily securing motion data through the store site of

2 is a block diagram showing an operation textbook authoring apparatus according to an embodiment of the present invention. 3 is a diagram showing in detail an example of the operation textbook authoring apparatus shown in FIG. 2.

2 and 3, the operation textbook authoring apparatus 100 according to an embodiment of the present invention includes an authoring unit 110, a sensor unit 120, a motion script generation unit 130, and an editing publishing unit 140. Includes.

The authoring unit 110 may generate attribute information for a motion script corresponding to a user's motion.

That is, the authoring unit 110 may correspond to an authoring user interface (UI) for generating and setting overall properties of a motion script that a motion expert wants to generate.

In this case, the authoring unit 110 may include a header information setting unit 111, an operation attribute mapping unit 112, and an additional information setting unit 113.

In this case, the header information setting unit 111 may generate motion script header information including at least one of a file property for the motion script and tag information for searching for the motion script.

That is, the header information setting unit 111 may describe properties of a motion script file such as author, authoring time, sensor type, operation description, and operation category.

The motion attribute mapping unit 112 may assign attribute information to the motion of the motion expert.

That is, the motion attribute mapping unit 112 may give information on the attribute of the motion itself.

At this time, the motion attribute mapping unit 112 explicitly provides information on matters to be learned, such as key poses, key motions, and importance of key body parts, that the learner should focus on when learning a specific motion, before the expert captures the motion. Can be given by.

The attribute information includes key pose setting information corresponding to key poses, which are important postures among movements of a motion expert; Key motion setting information corresponding to key motions, which are important motion sections among motion experts' motions; Key motion setting information corresponding to the speed and intensity of the key motion for setting the speed and intensity of important motions in important motion sections, and the user who sets the importance for each body part of the motion expert for the important postures and the important motions It may include at least one of body importance setting information corresponding to the key body part.

The additional information setting unit 113 may set whether to provide additional information that may help a learner learn movement.

That is, the additional information setting unit 113 may set whether to provide additional information that may help learning, such as an audio sequence such as background music and an expert's explanation voice.

The sensor unit 120 may acquire expert sensor data from the motion of the motion expert using the low-action motion acquisition device.

The low-action motion acquisition device may include at least one of a camera, a smartphone, a motion sensor, a position sensor, an acceleration sensor, an inertial sensor, a multi-sensor, and a Kinect device capable of obtaining the expert sensor data.

The motion script generation unit 130 may generate a motion script based on set attribute information and the acquired sensor data.

That is, the motion script generation unit 130 is a sensor data sequence for the performance of the expert acquired by the operation operation acquisition device (low-priced 3D camera, smartphone, various sensors, etc.) and set and given by the authoring unit 110. A motion script can be generated from the header and property information of the motion script.

The motion script generator 130 may further include a motion descriptor generator 131 and a motion attribute generator 132.

The motion descriptor generator 131 may determine the type and characteristic of the motion acquisition device, and generate motion descriptor information from the acquired expert sensor data based on the identified type and characteristic of the motion acquisition device.

That is, the motion descriptor generation unit 131 may receive expert sensor data capturing the expert's performance motion and describe the exemplary motion.

In this case, the motion descriptor generation unit 131 may grasp the type and characteristic of the low-action motion acquisition device currently connected to the system, and obtain expert sensor data extractable from the device.

For example, when a 3D camera such as Kinect is connected, it is possible to input a depth image sequence, a 2D image sequence, and an audio sequence, and expert sensor data corresponding thereto may be stored as a component of an operation engineer.

In addition, the motion descriptor generator 132 may generate an additional motion descriptor component by extracting the position of the body part from the depth image sequence.

The motion attribute generator 132 may generate motion attribute information from the acquired expert sensor data based on the attribute information.

That is, the motion attribute generator 132 may store the motion attribute assigned by the expert in the authoring unit 110 and automatically extract the attribute from the expert's motion.

While the motion properties that can be generated in the above-mentioned authoring UI are knowledge-based properties through expert experience, the properties generated by the motion property generator 132 may correspond to properties based on statistical analysis of motion.

For example, the motion attribute generator 132 may extract a positional shift, velocity, acceleration, angular velocity, and the like for the whole body, upper body, lower body, arms, and both legs.

In this case, the motion attribute generator 132 may generate motion attribute information such as size, speed, complexity, power, and flexibility for the entire or unit motion of the corresponding motion based on the extracted numerical information.

Unlike a static posture such as a pose, the motion attribute information is a characteristic related to the dynamic quality of a continuous motion, and may serve as a standard for providing correction information for a delicate expression method of motion during self-learning.

In this case, the motion script generation unit 130 may generate a motion script including at least one of the motion script header information, the motion attribute information, the motion descriptor information, and the additional information.

The editing publishing unit 140 may distribute the generated motion script to the operation textbook store server 10.

That is, the editing publishing unit 140 may be configured as a user interface capable of editing the generated motion script and uploading the generated motion script to the operation text store server 10.

In this case, the letter publishing unit 140 may include a textbook image editing, detailed operation and attribute correction, additional information editing, and a store site publishing UI.

In this case, the editing publishing unit 140 distributes the edited final motion script to the motion text store server by performing at least one of textbook image editing, detailed motion correction, detailed attribute correction, and additional information editing on the generated motion script. I can.

4 is a diagram showing a motion script according to an embodiment of the present invention.

Referring to FIG. 4, a motion script according to an embodiment of the present invention includes a motion script header, a motion attribute, a motion descriptor, and additional information.

The motion script header may include overall motion script header information of the motion script file.

The motion script header information may include information such as author, authoring time, sensor type used for authoring, a simple description of an action and action keyword, a category to which an action belongs, and length of an action.

In this case, the motion script header information may be used as tag information for effectively searching for a motion script required by a learner.

The motion property may include motion property information about the property of the motion itself, not the motion script file.

In the motion attribute, information on which part the learner focuses on when learning a specific motion is given is given.

For example, setting a key pose indicating where the most important posture is among a series of movements, setting a key motion indicating the most important movement section, setting the importance of each body part in taking a specific pose, setting the speed and intensity of important movements Motion property information such as, etc. may be included in the motion property. These motion properties may be explicitly generated by an expert in the motion textbook authoring apparatus 100 described in FIGS. 2 and 3, or may be automatically generated by the motion textbook authoring apparatus 100.

The motion descriptor is a part that stores motion descriptor information corresponding to an image sequence extracted from a series of motions taken by an expert and sensor data itself. When a motion learner who has downloaded a motion script learns, motion descriptor information can be an exemplary motion for the learner's motion.

That is, the motion descriptor information may be a reference motion for providing feedback and correction information on the motion learner's motion through the motion self-learning device 200.

For example, the elements of motion descriptor information are different depending on what kind of sensor the expert used to create a motion script and what information the sensor can provide.

When a motion script is authored with a smart phone, speed/acceleration information about a 2D video sequence or hand motion can be stored. In the case of authoring through a low-cost 3D camera such as Kinect, more diverse and specific information can be obtained than a smartphone, and the specific information is skeleton data composed of position information of each joint, 3D depth image sequence, 2D image sequence, etc. May correspond.

If an expert wants to provide more precise and detailed motion information, position, acceleration, and IMU (inertial) sensors can be attached to the desired body part and used.

At this time, operation technician information may be generated from acquired expert sensor data. In this way, an expert can describe a motion using a low-action motion acquisition device that combines various sensors capable of detecting motion, and the quality of a motion script increases as the types of data constituting motion descriptor information increase.

Additional information is information that helps learners to self-learn well. The author may additionally provide an audio sequence such as background music that can increase learning concentration, an explanation narration of an expert who specifically guides the operation, and a motion description subtitle that can be played along with the 2D video.

5 is a diagram showing an operation self-learning device according to an embodiment of the present invention. 6 is a diagram showing in detail an example of the motion self-learning device shown in FIG. 5.

5 and 6, the motion self-learning device 200 according to an embodiment of the present invention includes a learning unit 210, a sensor unit 220, a motion evaluation unit 230, and an output unit 240. Include.

The learning unit 210 may output an expert motion of a motion script to allow the user to learn the expert motion.

In this case, the learning unit 210 may directly receive the motion script acquired by the learner or download the motion script from the motion textbook store server 10.

In this case, the learning unit 210 may use a display device or the like or the operation self-learning device 200 may be connected to a display device or the like to output an expert motion of a motion script.

In this case, the motion learner may learn the expert motion by following the expert motion output from the display device.

In this case, the learning unit 210 may set the motion script according to the motion that the motion learner wants to learn before performing the learning.

In this case, the learning unit 210 may include an expert motion script setting unit 211, a learning environment setting unit 212, and a learner motion script setting unit 213.

The expert motion script setting unit 211 may set learning attribute information by selecting required attributes for a motion that the motion learner wants to learn from motion attribute information included in the motion script.

The learning environment setting unit 212 may set learning environment information for the motion that the motion learner wants to learn.

The learner motion script setting unit 213 may set a header and properties of a learner motion script in order to generate a motion learner's motion as a learner motion script.

The sensor unit 220 may acquire learner sensor data from the motion learner's motion using the learning motion acquisition device.

The learning motion acquisition device may include at least one of a camera, a smartphone, a motion sensor, a position sensor, an acceleration sensor, an inertial sensor, a multi-sensor, and a Kinect device for acquiring the learner sensor data from the motion learner's motion. .

The motion evaluation unit 230 may generate final motion evaluation information by evaluating the acquired learner sensor data based on a set motion script.

That is, the motion evaluation unit 230 compares and analyzes the motion learner's performance acquired by the learning motion acquisition device (low-priced 3D camera, smartphone, various sensors) and the motion described in the expert motion script, and evaluates appropriate evaluation information for the user. Can provide.

At this time, the motion evaluation unit 230 may compare/analyze the motion descriptor and motion attribute of the expert in the motion script with the motion descriptor and motion attribute acquired from a series of motion sequences of the current learner, and provide the evaluation result. . When comparing motions, a result can be calculated by comparing motion descriptors and attributes extracted from sensor data acquired from the same device. Although it is possible to compare sensor data obtained from different devices, the reliability of the evaluation result may be lowered because the operation descriptor and operation attribute are different.

In this case, the motion evaluation unit 230 may include a motion analysis unit 231 and a motion evaluation calculation unit 232.

The motion analysis unit 231 may generate posture accuracy information and dynamic expression power information of the motion learner obtained based on the motion script and the learning sensor data.

That is, the motion analysis unit 231 may compare/analyze the motion of the expert in the motion script and the motion of the learner who is currently taking the motion.

In this case, the motion analysis unit 231 may calculate a posture accuracy value through comparison between motion technicians, and calculate a dynamic expression power analysis result through comparison between motion attributes.

The motion analysis unit 231 first compares/analyzes motion descriptor information and motion attribute information generated from learner sensor data acquired using the same type of learning motion acquisition device as the low-action motion acquisition device used to generate the motion script. I can.

At this time, the motion analysis unit 231 includes posture accuracy values obtained by analyzing the similarity of postures of the motion learner for each result of analyzing the motion descriptor information and the motion attribute information of the motion script and the acquired learner sensor data. Can generate posture accuracy information.

In this case, the motion evaluation calculation unit 232 may generate final motion evaluation information based on the posture accuracy information and the dynamic expression power information.

In this case, the motion evaluation calculator 232 may determine a weight for each of the posture accuracy values and each of the dynamic expression power values based on at least one of the number, type, and characteristic of the learning motion acquisition device.

In this case, the motion evaluation calculator 232 may determine a sum weight between the posture accuracy values for which the weight is determined and the dynamic expressive power values for each motion learner's motion.

At this time, the motion evaluation calculation unit 232 is the final motion evaluation information including final motion evaluation values calculated by summing the posture accuracy values and the dynamic expressive power values based on the sum determined for each motion of the motion learner. Can be created.

In this case, the motion evaluation calculation unit 232 may set the sum of the dynamic expression power values high in proportion to the learning ability setting value of the motion learner and set the posture accuracy value low.

For example, when the motion evaluation calculation unit 232 sets the learning ability setting value of the motion learner to be low and corresponds to the beginner, it sets the summation weight of the posture accuracy values high, and sums the dynamic expression power values. The specific gravity can be set high.

In addition, the motion evaluation calculation unit 232 sets the sum of the dynamic expressive power values high, and the sum weight of the posture accuracy values when the motion learner's learning ability set value is set high and corresponds to the skilled person. Can be set lower.

The output unit 240 may output the final motion evaluation information and motion correction information for correcting the motion learner's motion.

That is, the output unit 240 may receive the final evaluation result information, correct the operation in real time, and provide a comprehensive evaluation result of the learner and detailed evaluation for each detailed item.

In this case, the output unit 240 may support the learner to continuously correct his/her own motion based on the learning result through the learning history management.

In this case, the output unit 240 may further output learning evaluation information obtained by evaluating a score for each evaluation element and a motion accuracy for each body part of the motion learner based on the final motion evaluation information.

In this case, the output unit 240 may further output detailed evaluation information including a score trend for each evaluation element of the expert action learned by the user.

In addition, the output unit 240 may include a UI capable of publishing a learner's motion script.

That is, the output unit 240 may edit the learner's motion script based on the final evaluation result information and the correction information and distribute it to the operation textbook store server.

7 is a diagram showing in detail an example of a motion evaluation process performed by the motion evaluation unit illustrated in FIG. 6.

As mentioned above, there are various devices such as cameras and smart phones, as well as various sensors as low-action motion acquisition devices capable of capturing the motions of experts. The motion expert can select an appropriate low-action motion acquisition device to capture their performance, and various expert sensor data obtained at this time are stored in the motion script. Like a motion expert, a motion learner can learn a motion by selecting various kinds of motion acquisition devices for learning. For example, if the motion script to be learned includes expert motions captured with a 3D camera and a smart phone, the learner can capture one's motion by selecting one of the 3D camera and a smartphone as a motion acquisition device, It is also possible to use both devices at the same time. That is, self-learning can be performed by acquiring the learner's actions by using some or all of the same apparatus as the apparatus for acquiring an actionable action used when creating a motion script.

In the motion evaluation process, an evaluation result may be provided by comparing/analyzing the motion descriptor and motion attribute of an expert in the motion script with the motion descriptor and motion attribute acquired from a series of motion sequences of the current learner. When comparing motions, a result can be calculated by comparing motion descriptors and attributes extracted from sensor data acquired from the same device. Although it is possible to compare sensor data obtained from different devices, the reliability of the evaluation result may be lowered because the operation descriptor and operation attribute are different.

Motion descriptors and attribute comparison/analysis results can be largely divided into posture accuracy and dynamic expression.

The motion is composed of a set of consecutive postures (poses), and posture accuracy may correspond to an analysis result of how similar a learner's static posture at a specific time point (or section) is to that of an expert. On the other hand, dynamic expressiveness, unlike posture, may correspond to the analysis result of how similar the quality of continuous movements such as size, speed, power, and flexibility of learners' movements to expert movements. This is an analysis result related to the quality of the motion and provides evaluation information on whether the motion expression method is appropriate.

In the present invention, various conventional techniques that can analyze the similarity between two postures/motions for each sensor data can be applied.

For example, the similarity analysis method of the present invention is a method of analyzing the similarity between two postures/motions using body part position data extracted from a 3D camera, and calculating the similarity of the entire posture by measuring the relative position and angle of each joint. In the case of a method and a depth image and a 2D image, a method of measuring similarity through matching and shape comparison of the silhouette template of the entire body may be used.

That is, the similarity analysis method of the present invention can apply all existing similarity analysis methods to the present invention.

Posture accuracy information and dynamic expressiveness information, which are the results of comparing/analyzing motion descriptors and motion properties for each type of input sensor data, can be calculated as the final motion evaluation result.

In this case, the final motion evaluation result may be calculated as in Equation 1.

[Equation 1]

In Equation 1, n is the number of matched sensor data and i may correspond to the index (type) of the matched sensor data. Pi represents the posture accuracy analysis result obtained by comparison between motion technicians extracted from the i-th sensor data. Wi is a weight for the accuracy analysis result. The more precise and more information provided by the sensor data captured by the motion acquisition device is, the greater the possibility of producing a more reliable result in the posture accuracy analysis, so a higher weight can be given to the posture accuracy analysis result. Ej represents the dynamic expressiveness analysis result obtained by comparing the motion attributes extracted from the j-th sensor data. Wj is the weight for the dynamic expressiveness result. That is, Wj is a weight for reflecting the difference in quality in dynamic information provided according to the characteristics of the motion acquisition device.

Finally, the final motion evaluation result of how similar the learner's motion is to the expert's motion described in the motion script can be expressed as the sum of posture accuracy and dynamic expression. In the final result, the weight of posture accuracy and dynamic expression can be adjusted with α. If the movement to be learned is a movement whose posture accuracy is more important than expressiveness, the value of the posture accuracy analysis result in the final result value can be increased by setting the α value higher. On the other hand, for artistic movements, such as dance, where expressive power is important, the proportion of dynamic expression can be increased by setting the α value low. As another example, since accuracy is more important than expressiveness in most cases for beginners, in this case, set the α value higher, and in the case of advanced people who have almost reached the level of expert level and have no problems with posture, the proportion of expressiveness is more. Can be set high.

8 and 9 are diagrams showing an output screen of a final motion evaluation result according to an embodiment of the present invention.

Referring to FIG. 8, in the output screen of the final motion evaluation result according to an embodiment of the present invention, it can be seen that scores are evaluated for each attribute such as posture, timing, power expression, and movement. At this time, it is also possible to output the total score for each attribute score, as well as the calories and time consumed while performing motion learning.

Scores for each attribute can be displayed in various graphs as well as bar graphs and radial graphs.

In addition, by outputting the accuracy of the posture learning motion for each body part, correction information may be provided to the motion learner.

Referring to FIG. 9, a score trend graph for each motion attribute during a motion learning time period may be output as a line graph. In addition, it is also possible to output the captured data image to output the model action of the expert and the action taken by the learner.

10 is an operation flow diagram showing a method for self-learning motion according to an embodiment of the present invention.

Referring to FIG. 10, in the method of self-learning motions according to an embodiment of the present invention, first, a motion textbook is written (S310).

That is, in step S310, the motion textbook authoring apparatus 100 may generate a motion script based on the motion of the motion expert.

In this case, in step S310, first, attribute information of the motion script may be generated (S311).

That is, in step S311, the motion expert may generate property information by setting the overall properties of the motion script to be generated.

In this case, in step S311, motion script header information including at least one of a file property for a motion script and tag information for searching for the motion script may be generated.

At this time, step S311 may describe properties of the motion script file such as author, authoring time, sensor type, operation description, and operation category.

In addition, in step S311, attribute information may be assigned to the motion of the motion expert.

In this case, in step S311, information on the properties of the operation itself may be provided.

At this time, in step S311, information on matters to be learned, such as key pose, key motion, key body part importance, etc., that the learner should focus on when learning a specific movement is explicitly given before the expert captures the movement. I can.

The attribute information includes key pose setting information corresponding to key poses, which are important postures among movements of a motion expert; Key motion setting information corresponding to key motions, which are important motion sections among motion experts' motions; Key motion setting information corresponding to the speed and intensity of the key motion for setting the speed and intensity of important motions in important motion sections, and the user who sets the importance for each body part of the motion expert for the important postures and the important motions It may include at least one of body importance setting information corresponding to the key body part.

In addition, in step S311, it is possible to set whether to provide additional information that may help the learner to learn movements.

In this case, in step S311, it is possible to set whether to provide additional information that can help learning, such as an audio sequence such as background music and an expert's explanation voice.

In addition, in step S310, expert sensor data may be collected (S312).

That is, in step S312, expert sensor data may be obtained from the motion of the motion expert using the low-action motion acquisition device.

The low-action motion acquisition device may include at least one of a camera, a smartphone, a motion sensor, a position sensor, an acceleration sensor, an inertial sensor, a multi-sensor, and a Kinect device capable of obtaining the expert sensor data.

In addition, in step S310, a motion script may be generated (S313).

That is, in step S313, a motion script may be generated based on the set attribute information and the acquired sensor data.

At this time, step (S313) is a sensor data sequence for the expert's performance acquired by the low-action motion acquisition device (low-priced 3D camera, smartphone, various sensors, etc.) and a motion script set and given by the authoring unit 110 It is possible to generate a motion script from the header and attribute information of.

In this case, in step S313, the type and characteristic of the motion acquisition device may be identified, and motion descriptor information may be generated from the acquired expert sensor data based on the identified type and characteristic of the motion acquisition device.

In this case, in step S313, an exemplary operation may be described by receiving expert sensor data capturing the expert's performance motion.

In this case, in step S313, the type and characteristic of the low-action operation acquisition device currently connected to the system may be identified, and expert sensor data extractable from the device may be obtained.

For example, when a 3D camera such as Kinect is connected, it is possible to input a depth image sequence, a 2D image sequence, and an audio sequence, and expert sensor data corresponding thereto may be stored as a component of an operation engineer.

In this case, in step S313, an additional motion descriptor component may be generated by extracting the position of the body part from the depth image sequence.

Further, in step S313, motion attribute information may be generated from the acquired expert sensor data based on the attribute information.

In this case, in step S313, the operation attribute assigned by the expert in the authoring unit 110 may be stored and the attribute may be automatically extracted from the expert's operation.

For example, in this case, the step S313 may extract positional displacement, velocity, acceleration, angular velocity, etc. for the whole body, upper body, lower body, both arms, and both legs.

In this case, in step S313, operation attribute information such as size, speed, complexity, power, and flexibility for the entire or unit operation of the corresponding operation may be produced based on the extracted numerical information.

Unlike a static posture such as a pose, the motion attribute information is a characteristic related to the dynamic quality of a continuous motion, and may serve as a standard for providing correction information for a delicate expression method of motion during self-learning.

In this case, in step S313, a motion script including at least one of the motion script header information, the motion attribute information, the motion descriptor information, and the additional information may be generated.

In addition, the motion self-learning method according to an embodiment of the present invention may publish a motion textbook (S320).

That is, in step S320, the motion textbook authoring apparatus 100 may distribute the motion script to the motion textbook store server 10.

In this case, step S320 may perform textbook image editing, detailed operation and attribute correction, additional information editing, and store site publishing functions through the UI.

In this case, in step S320, the edited final motion script may be distributed to the motion text store server by performing at least one of textbook image editing, detailed motion correction, detailed attribute correction, and additional information editing on the generated motion script. have.

In addition, the motion self-learning method according to an embodiment of the present invention may self-learn a motion textbook (S330).

In step S330, first, a motion script may be set (S331).

That is, in step S331, the motion script downloaded from the motion textbook store server 10 may be set according to the motion that the motion learner wants to learn.

In this case, in step S331, the motion script acquired by the learner may be directly input or downloaded from the motion textbook store server 10.

In this case, in step S331, the learning attribute information may be set by selecting required attributes for the motion that the motion learner wants to learn from motion attribute information included in the motion script.

In this case, in step S331, the motion learner may set learning environment information for the motion to be learned.

In this case, in step S331, a header and property of the learner motion script may be set in order to generate the motion learner's motion as a learner motion script.

In addition, in step S330, learner sensor data may be obtained (S332).

That is, in step S332, the learner sensor data may be obtained from the motion learner's motion using the learning motion acquisition device.

The learning motion acquisition device may include at least one of a camera, a smartphone, a motion sensor, a position sensor, an acceleration sensor, an inertial sensor, a multi-sensor, and a Kinect device for acquiring the learner sensor data from the motion learner's motion. .

In addition, in step S330, a motion learner's motion may be evaluated (S333).

In step S333, first, the motion descriptor and the motion attribute may be compared (S333a).

That is, step S333a compares and analyzes the performance of the motion learner acquired by the learning motion acquisition device (low-priced 3D camera, smartphone, various sensors) and the motion described in the expert motion script to provide appropriate evaluation information to the user. can do.

In this case, in step S333a, an evaluation result may be provided by comparing/analyzing the motion descriptor and motion attribute of the expert in the motion script with the motion descriptor and motion attribute acquired from a series of motion sequences of the current learner. When comparing motions, a result can be calculated by comparing motion descriptors and attributes extracted from sensor data acquired from the same device. Although it is possible to compare sensor data obtained from different devices, the reliability of the evaluation result may be lowered because the operation descriptor and operation attribute are different.

In this case, in step S333a, motion descriptor information and motion attribute information generated from learner sensor data acquired using the same type of learning motion acquisition device as the low-action motion acquisition device used to generate the motion script may be compared.

In addition, in step S333, posture accuracy and dynamic expression power may be calculated (S333b).

That is, in step S333b, the motion learner's posture accuracy information and dynamic expression power information acquired based on the motion script and the learning sensor data may be generated.

At this time, step S333b may compare/analyze the motion of the expert in the motion script and the motion of the learner who is currently taking the motion.

In this case, in step S333b, a posture accuracy value may be calculated through comparison between motion technicians, and a dynamic expressive power analysis result may be calculated through a comparison between motion attributes.

At this time, step (S333b) first compares/analyzes motion descriptor information and motion attribute information generated from learner sensor data acquired using the same type of learning motion acquisition device as the low-action motion acquisition device used to generate the motion script. can do.

In this case, step S333b includes posture accuracy values obtained by analyzing the similarity of the postures of the motion learner for each result of analyzing the motion descriptor information and motion attribute information of the motion script and the acquired learner sensor data. Can generate accuracy information.

In this case, step S333b may generate final motion evaluation information based on the posture accuracy information and the dynamic expressive power information.

In this case, in step S333b, a weight for each of the posture accuracy values and each of the dynamic expression power values may be determined based on at least one of the number, type, and characteristic of the learning motion acquisition device.

In this case, in step S333b, for each of the motion learner's motions, the weighted weighted posture accuracy values and the summation weight between the dynamic expressive power values may be determined.

In addition, in step S333, final motion evaluation information may be generated (333c).

That is, in step S333c, the final motion evaluation information including final motion evaluation values calculated by summing the posture accuracy values and the dynamic expressive power values based on the sum determined for each motion of the motion learner may be generated. have.

In this case, in step S333c, in proportion to the learning ability setting value of the motion learner, the sum of the dynamic expression power values may be set to be high, and the posture accuracy value may be set to be low.

In this case, in step S333c, in proportion to the learning ability setting value of the motion learner, the sum of the dynamic expression power values may be set high, and the posture accuracy value may be set low.

For example, in step S333c, when the learning ability setting value of the motion learner is set low and corresponds to a beginner, the summation weight of the posture accuracy values is set high, and the summation weight of the dynamic expression power values is increased. Can be set.

At this time, in step S333c, when the learning ability setting value of the motion learner is set high and corresponds to the skilled person, the summation weight for the dynamic expression power values is set high, and the summation weight for the posture accuracy values is set low. I can.

In addition, the motion self-learning method according to an embodiment of the present invention may output a final motion evaluation result (S340).

That is, the step S340 may output the final motion evaluation information and motion correction information for correcting the motion learner's motion.

At this time, in step S340, the final evaluation result information is received, the motion is corrected in real time, and a comprehensive evaluation result of the learner and detailed evaluation for each detailed item may be provided.

At this time, step S340 may support the learner to continuously correct his or her own movement based on the learning result through the learning history management.

In this case, step S340 may further output learning evaluation information obtained by evaluating a score for each evaluation element and a motion accuracy for each body part of the motion learner based on the final motion evaluation information.

In this case, step S340 may further output detailed evaluation information including a score transition for each evaluation element of the expert's motion learned by the user.

In this case, in step S340, the learner's motion script may be edited based on the final evaluation result information and the correction information using a UI capable of publishing the learner's motion script and distributed to the motion textbook store server. .

11 is an operation flow diagram showing in detail an example of the operation textbook authoring step shown in FIG.

Referring to FIG. 11, in step S311, the motion expert may generate property information by setting an overall property of a motion script to be generated.

In this case, in step S311, motion script header information including at least one of a file property for a motion script and tag information for searching for the motion script may be generated.

At this time, step S311 may describe properties of the motion script file such as author, authoring time, sensor type, operation description, and operation category.

In addition, in step S311, attribute information may be assigned to the motion of the motion expert.

At this time, in step S311, information on the properties of the operation itself may be provided.

At this time, in step S311, information on matters to be learned, such as key pose, key motion, key body part importance, etc., that the learner should focus on when learning a specific movement is explicitly given before the expert captures the movement. I can.

The attribute information includes key pose setting information corresponding to key poses, which are important postures among movements of a motion expert; Key motion setting information corresponding to key motions, which are important motion sections among motion experts' motions; Key motion setting information corresponding to the speed and intensity of the key motion for setting the speed and intensity of important motions in important motion sections, and the user who sets the importance for each body part of the motion expert for the important postures and the important motions It may include at least one of body importance setting information corresponding to the key body part.

In addition, in step S311, it is possible to set whether to provide additional information that may help the learner to learn movements.

In this case, in step S311, it is possible to set whether to provide additional information that can help learning, such as an audio sequence such as background music and an expert's explanation voice.

In addition, in step S310, expert sensor data may be collected (S312).

That is, in step S312, expert sensor data may be obtained from the motion of the motion expert using the low-action motion acquisition device.

The low-action motion acquisition device may include at least one of a camera, a smartphone, a motion sensor, a position sensor, an acceleration sensor, an inertial sensor, a multi-sensor, and a Kinect device capable of obtaining the expert sensor data.

In addition, in step S310, a motion script may be generated (S313).

That is, in step S313, a motion script may be generated based on the set attribute information and the acquired sensor data.

At this time, step (S313) is a sensor data sequence for the expert's performance acquired by the low-action motion acquisition device (low-priced 3D camera, smartphone, various sensors, etc.) and a motion script set and given by the authoring unit 110 It is possible to generate a motion script from the header and attribute information of.

In this case, in step S313, the type and characteristic of the motion acquisition device may be identified, and motion descriptor information may be generated from the acquired expert sensor data based on the identified type and characteristic of the motion acquisition device.

In this case, in step S313, an exemplary operation may be described by receiving expert sensor data capturing the expert's performance motion.

In this case, in step S313, the type and characteristic of the low-action operation acquisition device currently connected to the system may be identified, and expert sensor data extractable from the device may be obtained.

For example, when a 3D camera such as Kinect is connected, it is possible to input a depth image sequence, a 2D image sequence, and an audio sequence, and expert sensor data corresponding thereto may be stored as a component of an operation engineer.

In this case, in step S313, an additional motion descriptor component may be generated by extracting the position of the body part from the depth image sequence.

Further, in step S313, motion attribute information may be generated from the acquired expert sensor data based on the attribute information.

In this case, in step S313, the operation attribute assigned by the expert in the authoring unit 110 may be stored and the attribute may be automatically extracted from the expert's operation.

For example, in this case, the step S313 may extract positional displacement, velocity, acceleration, angular velocity, etc. for the whole body, upper body, lower body, both arms, and both legs.

In this case, in step S313, operation attribute information such as size, speed, complexity, power, and flexibility for the entire or unit operation of the corresponding operation may be produced based on the extracted numerical information.

Unlike a static posture such as a pose, the motion attribute information is a characteristic related to the dynamic quality of a continuous motion, and may serve as a standard for providing correction information for a delicate expression method of motion during self-learning.

In this case, in step S313, a motion script including at least one of the motion script header information, the motion attribute information, the motion descriptor information, and the additional information may be generated.

12 is an operation flow diagram showing in detail an example of the operation self-learning step shown in FIG.

Referring to FIG. 12, in step S331, a motion script downloaded from the motion textbook store server 10 may be set according to the motion the motion learner wants to learn.

In this case, in step S331, the motion script acquired by the learner may be directly input or downloaded from the motion textbook store server 10.

In this case, in step S331, the learning attribute information may be set by selecting required attributes for the motion that the motion learner wants to learn from motion attribute information included in the motion script.

In this case, in step S331, the motion learner may set learning environment information for the motion to be learned.

In this case, in step S331, a header and property of the learner motion script may be set in order to generate the motion learner's motion as a learner motion script.

In addition, in step S330, learner sensor data may be obtained (S332).

That is, in step S332, the learner sensor data may be obtained from the motion learner's motion using the learning motion acquisition device.

The learning motion acquisition device may include at least one of a camera, a smartphone, a motion sensor, a position sensor, an acceleration sensor, an inertial sensor, a multi-sensor, and a Kinect device for acquiring the learner sensor data from the motion learner's motion. .

In addition, in step S330, a motion learner's motion may be evaluated (S333).

In step S333, first, the motion descriptor and the motion attribute may be compared (S333a).

That is, step S333a compares and analyzes the performance of the motion learner acquired by the learning motion acquisition device (low-priced 3D camera, smartphone, various sensors) and the motion described in the expert motion script to provide appropriate evaluation information to the user. can do.

In this case, in step S333a, an evaluation result may be provided by comparing/analyzing the motion descriptor and motion attribute of the expert in the motion script with the motion descriptor and motion attribute acquired from a series of motion sequences of the current learner. When comparing motions, a result can be calculated by comparing motion descriptors and attributes extracted from sensor data acquired from the same device. Although it is possible to compare sensor data obtained from different devices, the reliability of the evaluation result may be lowered because the operation descriptor and operation attribute are different.

In this case, in step S333a, motion descriptor information and motion attribute information generated from learner sensor data acquired using the same type of learning motion acquisition device as the low-action motion acquisition device used to generate the motion script may be compared.

In addition, in step S333, posture accuracy and dynamic expression power may be calculated (S333b).

That is, in step S333b, the motion learner's posture accuracy information and dynamic expression power information acquired based on the motion script and the learning sensor data may be generated.

At this time, step S333b may compare/analyze the motion of the expert in the motion script and the motion of the learner who is currently taking the motion.

In this case, in step S333b, a posture accuracy value may be calculated through comparison between motion technicians, and a dynamic expressive power analysis result may be calculated through a comparison between motion attributes.

At this time, step (S333b) first compares/analyzes motion descriptor information and motion attribute information generated from learner sensor data acquired using the same type of learning motion acquisition device as the low-action motion acquisition device used to generate the motion script. can do.

In this case, step S333b includes posture accuracy values obtained by analyzing the similarity of the postures of the motion learner for each result of analyzing the motion descriptor information and motion attribute information of the motion script and the acquired learner sensor data. Can generate accuracy information.

In this case, step S333b may generate final motion evaluation information based on the posture accuracy information and the dynamic expressive power information.

In this case, in step S333b, a weight for each of the posture accuracy values and each of the dynamic expression power values may be determined based on at least one of the number, type, and characteristic of the learning motion acquisition device.

In this case, in step S333b, for each of the motion learner's motions, the weighted weighted posture accuracy values and the summation weight between the dynamic expressive power values may be determined.

In addition, in step S333, final motion evaluation information may be generated (333c).

That is, in step S333c, the final motion evaluation information including final motion evaluation values calculated by summing the posture accuracy values and the dynamic expressive power values based on the sum determined for each motion of the motion learner may be generated. have.

In this case, in step S333c, in proportion to the learning ability setting value of the motion learner, the sum of the dynamic expression power values may be set high, and the posture accuracy value may be set low.

In this case, in step S333c, in proportion to the learning ability setting value of the motion learner, the sum of the dynamic expression power values may be set high, and the posture accuracy value may be set low.

For example, in step S333c, when the learning ability setting value of the motion learner is set low and corresponds to a beginner, the summation weight of the posture accuracy values is set high, and the summation weight of the dynamic expression power values is increased. Can be set.

At this time, in step S333c, when the learning ability setting value of the motion learner is set high and corresponds to the skilled person, the summation weight for the dynamic expression power values is set high, and the summation weight for the posture accuracy values is set low. I can.

13 is an operation flow diagram showing in detail an example of the operation evaluation step shown in FIG. 12.

13, step S333a compares and analyzes the performance of the learner's motion acquired by the learning motion acquisition device (low-priced 3D camera, smartphone, various sensors) and the motion described in the expert motion script, and is appropriate for the user. Evaluation information can be provided.

In this case, in step S333a, an evaluation result may be provided by comparing/analyzing the motion descriptor and motion attribute of the expert in the motion script with the motion descriptor and motion attribute acquired from a series of motion sequences of the current learner. When comparing motions, a result can be calculated by comparing motion descriptors and attributes extracted from sensor data acquired from the same device. Although it is possible to compare sensor data obtained from different devices, the reliability of the evaluation result may be lowered because the operation descriptor and operation attribute are different.

In this case, in step S333a, motion descriptor information and motion attribute information generated from learner sensor data acquired using the same type of learning motion acquisition device as the low-action motion acquisition device used to generate the motion script may be compared.

In addition, in step S333, posture accuracy and dynamic expression power may be calculated (S333b).

That is, in step S333b, the motion learner's posture accuracy information and dynamic expression power information acquired based on the motion script and the learning sensor data may be generated.

At this time, step S333b may compare/analyze the motion of the expert in the motion script and the motion of the learner who is currently taking the motion.

In this case, in step S333b, a posture accuracy value may be calculated through comparison between motion technicians, and a dynamic expressive power analysis result may be calculated through a comparison between motion attributes.

At this time, step (S333b) first compares/analyzes motion descriptor information and motion attribute information generated from learner sensor data acquired using the same type of learning motion acquisition device as the low-action motion acquisition device used to generate the motion script. can do.

In this case, step S333b includes posture accuracy values obtained by analyzing the similarity of the postures of the motion learner for each result of analyzing the motion descriptor information and motion attribute information of the motion script and the acquired learner sensor data. Can generate accuracy information.

In this case, step S333b may generate final motion evaluation information based on the posture accuracy information and the dynamic expressive power information.

In this case, in step S333b, a weight for each of the posture accuracy values and each of the dynamic expression power values may be determined based on at least one of the number, type, and characteristic of the learning motion acquisition device.

In this case, in step S333b, for each of the motion learner's motions, the weighted weighted posture accuracy values and the summation weight between the dynamic expressive power values may be determined.

In addition, in step S333, final motion evaluation information may be generated (333c).

That is, in step S333c, the final motion evaluation information including final motion evaluation values calculated by summing the posture accuracy values and the dynamic expressive power values based on the sum determined for each motion of the motion learner may be generated. have.

In this case, in step S333c, in proportion to the learning ability setting value of the motion learner, the sum of the dynamic expression power values may be set high, and the posture accuracy value may be set low.

In this case, in step S333c, in proportion to the learning ability setting value of the motion learner, the sum of the dynamic expression power values may be set high, and the posture accuracy value may be set low.

For example, in step S333c, when the learning ability setting value of the motion learner is set low and corresponds to a beginner, the summation weight of the posture accuracy values is set high, and the summation weight of the dynamic expression power values is increased. Can be set.

At this time, in step S333c, when the learning ability setting value of the motion learner is set high and corresponds to the skilled person, the summation weight for the dynamic expression power values is set high, and the summation weight for the posture accuracy values is set low. I can.

14 is a block diagram showing a computer system according to an embodiment of the present invention.

Referring to FIG. 14, an embodiment of the present invention may be implemented in a computer system 1100 such as a computer-readable recording medium. As shown in FIG. 14, the computer system 1100 includes one or more processors 1110, memory 1130, user input devices 1140, user output devices 1150, and storage devices that communicate with each other through a bus 1120. (1160) may be included. Also, the computer system 1100 may further include a network interface 1170 connected to the network 1180. The processor 1110 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 1130 or the storage 1160. The memory 1130 and the storage 1160 may be various types of volatile or nonvolatile storage media. For example, the memory may include a ROM 1131 or a RAM 1132.

As described above, the operation textbook authoring apparatus, the operation self-learning apparatus and method according to the present invention are not limited to the configuration and method of the embodiments described above, but various modifications may be made to the embodiments. All or part of each of the embodiments may be selectively combined and configured.

10: operation manual store server 100: operation manual authoring device
110: authoring unit 111: header information setting unit
112: motion attribute mapping unit 113: additional information setting unit
120: sensor unit 130: motion script generation unit
131: motion descriptor generator 132: motion attribute generator
140: editorial publishing unit 200: motion self-learning device
210: learning unit 211: expert motion script setting unit
212: learning environment setting unit 213: learner motion script setting unit
220: sensor unit 230: operation evaluation unit
231: motion analysis unit 232: motion evaluation calculation unit
240: output
1100: computer system 1110: processor
1120: bus 1130: memory
1131: ROM 1132: RAM
1140: user input device 1150: user output device
1160: storage 1170: network interface
1180: network

Claims (20)

An authoring unit that generates attribute information corresponding to a user's motion;
A sensor unit that acquires sensor data from the user's motion using a motion acquisition device;
A motion script generator for generating a motion script based on the attribute information and the sensor data; And
An editing publishing unit that distributes the motion script;
Including,
The authoring department
A header information setting unit for generating motion script header information including at least one of a file property for the motion script and tag information for searching for the motion script;
An operation attribute mapping unit for mapping the user's operation and the attribute information; And
An additional information setting unit that sets whether to provide additional information used for motion learning;
Operation textbook authoring apparatus comprising a. delete The method according to claim 1,
The above attribute information is
Key pose setting information corresponding to the user's key pose;
Key motion setting information corresponding to the user's key motion;
Motion setting information corresponding to the speed and strength of the key motion; And
Key body part setting information corresponding to the user's key body part based on the key pose and the key motion;
An operation textbook authoring apparatus comprising at least one of. The method of claim 3,
The motion script generation unit
A motion descriptor generator configured to determine the type and characteristic of the motion acquisition device, and generate motion descriptor information from the acquired expert sensor data based on the identified type and characteristic of the motion acquisition device; And
A motion attribute generator for generating motion attribute information from the acquired expert sensor data based on the attribute information;
Operation textbook authoring device, characterized in that it further comprises. The method of claim 4,
The motion script generation unit
And generating the motion script including at least one of the motion script header information, the motion attribute information, the motion descriptor information, and the additional information. The method of claim 5,
The editorial publishing department
And distributing the edited final motion script to an operation text store server by performing at least one of textbook image editing, detailed operation correction, detailed attribute correction, and additional information editing on the generated motion script. A learning unit that outputs an expert action of a motion script to learn the expert action to a user;
A sensor unit that acquires sensor data from the user's motion using a motion acquisition device;
A motion evaluation unit generating final motion evaluation information for evaluating the user's motion based on the motion script and sensor data; And
An output unit outputting the final motion evaluation information and correction information for correcting the user's motion;
Including,
The learning unit
An expert motion script setting unit configured to set learning attribute information using attributes selected by the user for motion learning in the motion script;
A learning environment setting unit configured to set learning environment information based on the learning environment set by the user; And
A learner motion script setting unit that generates the user's motion as a learner motion script;
Operation self-learning device comprising a. delete The method of claim 7,
The motion evaluation unit
A motion analysis unit for generating posture accuracy information and dynamic expression power information of the user based on the motion script and the sensor data; And
A motion evaluation calculator configured to generate final motion evaluation information based on the posture accuracy information and the dynamic expression power information;
Operation self-learning device comprising a. The method of claim 9,
The motion analysis unit
A motion self-learning device, characterized in that the motion descriptor information and motion attribute information are analyzed using sensor data obtained from the motion obtaining device of the same type as the motion obtaining device used in the motion script. The method of claim 10,
The motion analysis unit
And generating posture accuracy information including posture accuracy values obtained by analyzing the similarity of the user's motion for each result of analyzing the motion descriptor information and the motion attribute information based on the motion script and the sensor data Self-learning device. The method of claim 11,
The motion analysis unit
For each result of analyzing the motion descriptor information and motion attribute information based on the motion script and the sensor data, including dynamic expressive power values obtained by analyzing at least one similarity among speed, intensity, and motion section of the user's motion. Motion self-learning device, characterized in that generating dynamic expressive power information. The method of claim 12,
The motion evaluation calculation unit
And determining a weight for each of the posture accuracy values and each of the dynamic expressive power values based on at least one of the number, type, and characteristic of the motion acquisition device. The method of claim 13,
The motion evaluation calculation unit
And determining a summation weight for summing the weighted posture accuracy values and the dynamic expressive power values for each of the user's motions. The method of claim 14,
The motion evaluation calculation unit
And generating final motion evaluation information including final motion evaluation values calculated by summing the posture accuracy values and the dynamic expressive power values based on the sum weight determined for each of the user's motions. The method of claim 15,
The motion evaluation calculation unit
The motion self-learning device, characterized in that, in proportion to the user's learning ability setting value, the sum of the dynamic expression power values is set high and the posture accuracy value is set low. The method of claim 16,
The output unit
The motion self-learning device, characterized in that the learner motion script is edited and distributed to a motion textbook store server based on the final motion evaluation information and the correction information. The method of claim 17,
The output unit
And further outputting learning evaluation information obtained by evaluating a score for each evaluation element and a motion accuracy for each body part of the user based on the final motion evaluation information. The method of claim 14,
The output unit
A motion self-learning device, characterized in that further outputting detailed evaluation information including a score transition for each evaluation element of the expert motion learned by the user. In the motion self-learning method using the motion textbook authoring device and the motion self-learning device,
Generating, by the motion textbook authoring apparatus, a motion script based on an expert motion of a first user;
Analyzing, by the motion self-learning device, a motion of a second user who has learned the expert motion based on the motion script; And
Outputting, by the motion self-learning device, a result of analyzing the motion of the second user;
Including,
Analyzing the motion of the second user
Outputting the expert action of the first user in the motion script to learn the expert action from the second user;
Including,
The learning step
In the motion script, learning attribute information is set using attributes selected by the second user for motion learning,
Set learning environment information based on the learning environment set by the second user,
The motion self-learning method, characterized in that generating the second user's motion as a learner motion script.

Images