KR20200098970A - Smart -learning device and method based on motion recognition - Google Patents

Abstract

The present invention relates to smart beauty technique learning device and method based on motion recognition, and in particular, to a smart beauty technique learning device based on motion recognition, comprising: an image acquisition unit acquiring an image of a learner, photographed through a motion recognition sensor; an image processing unit extracting an object from the photographed image of the learner and/or an input image, and calculating one or more posture information data of the object; a motion script generation unit including continuous motion data of the object based on the calculated posture information data; a motion determination unit for determining the degree of error between motion data of the object in the input image and motion data of the object in the image of the learner; and a learning support unit for outputting instruction for posture correction of the object in the photographed image of the learner when the degree of error of the motion determination unit exceeds a set numerical range. According to the present invention, when the motion data of the object in the input image and the motion data of the learner do not match, it is determined so that the learner can determine a motion state by oneself and correct a wrong posture according to the output instruction.

Description

Smart beauty technology learning device and method based on motion recognition {Smart -learning device and method based on motion recognition}

The present invention relates to a motion recognition-based smart beauty technology learning device and method that supports the acquisition of beauty technology through learning through video, and more particularly, a learner watches a lecture of a hair designer or makeup artist in the video. At the same time, it relates to a motion recognition-based smart beauty technology learning device and method that enables self-learning about professional skills such as dyeing method or makeup method by identically following the movements of experts in images.

There has always been a need to familiarize yourself with the correct posture and the correct use of professional tools to learn the correct posture for various sports or the expertise of various professional workers such as hair designers and makeup artists. One of the best ways to do this is to repeat the expert's actions repeatedly through a one-on-one class with an expert, receive immediate feedback, and continuously correct incorrect usage.

However, it is not easy to receive individual lessons from professional instructors due to restrictions such as location, time, and economic costs. Therefore, requirements for self-learning methods that can learn specific movements, techniques, etc. through image learning such as the Internet are increasing, and the related infrastructure industry is also developing.

However, the current self-learning method through video is a method of seeing and following the actions taken by the instructor on the screen, and is effective because there is no accurate feedback on the accuracy, error correction, improvement, and learning achievement of the learner. It is difficult to say that it is a self-learning method.

With the rapid development of IT technology in recent years, motion recognition sensors and sports posture analysis systems based on motion recognition technology provide detailed analysis of the player's posture and diagnosis services based on this, but these diagnosis services There is a problem that it is difficult to be applied to various fields in that it has been developed specially for the sport by several specific companies that professionally sell sport simulators.

In consideration of these problems, it can be applied to the non-sports field, and while it can be easily used at home, the motion recognition-based smart beauty that provides accurate feedback on the accuracy, error correction and improvement of the learner's actions, and learning achievement. Technology learning devices are in demand.

Republic of Korea Patent Publication No. 10-2016-0139932 Korean Patent Registration No. 10-1414744

The present invention was conceived to solve the above-described problem, in order to learn the precise movements of experts in the field of makeup and beauty, such as a maker-up artist, it is possible to use more easily at a remote location, such as at home, and the accuracy of the movements that the learner is taking, Its purpose is to provide a motion recognition-based smart beauty technology learning device that enables accurate feedback on error correction and improvement and learning achievement.

However, the technical problem to be achieved by the present invention is not limited to the technical problems mentioned above, and other technical problems not mentioned are clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. Can be.

In order to achieve the above-described object of the present invention, a motion recognition-based smart beauty technology learning device capable of smart learning by following the motion of an expert in a pre-input image is obtained, and acquires an image of a learner captured through a motion recognition sensor. An image acquisition unit; An image processing unit that extracts an object from the captured image of the learner and/or the input image to calculate attitude information data of at least one object; A motion script generator that includes (generates) continuous motion data of the object based on the calculated posture information data; A motion determination unit determining an error degree between motion data of the object in the input image and motion data of the object in the learner's image; And a learning support unit that outputs an instruction for correcting the posture of the object in the image of the imaged learner when the error degree of the motion determination unit exceeds a set numerical range, wherein the motion data of the object in the input image and the When the learner's motion data is inconsistent, it is determined so that the learner can determine the state of the motion and correct the wrong posture according to the output instruction.

In one embodiment of the present invention, the motion recognition-based smart beauty technology learning device includes the learner and/or the learning tool used by the learner from a sensor attached to the learner and/or the learning tool used by the learner. It is characterized in that it further includes a sensor unit capable of acquiring position coordinate data, and more precisely senses the movement of the learner and/or the learning tool used by the learner.

In one embodiment of the present invention, the motion script generation unit generates the generated motion script in units of an image sequence, wherein the motion data includes a position of the posture information data, a movement coordinate, a number of rotations, and/or a translational motion coordinate. It is characterized in that it is possible to increase the learning efficiency of the learner by generating a motion script so that the expert's actions in the input video can be learned in units of video sequences.

In one embodiment of the present invention, in determining the degree of error, the motion determination unit compares and analyzes coordinates in motion data of a plurality of objects based on motion data coordinates of the object in the input image, / Or by measuring the error of the number of rotations, the learner can more accurately recognize the difference between the expert's motion and the learner's motion in following the expert's technique in the inputted image so that the learner can learn efficiently. .

In addition, in an embodiment of the present invention, the instruction may include a direction, a position, a speed, an angle of movement and/or a number of rotations in which the learner should move for posture correction. It is characterized in that it can be output in a manner, characterized in that the learner can learn a more accurate method in imitating the skill of the expert in the input image.

In addition, in an embodiment of the present invention, the learning support unit, when the error degree exceeds a set numerical range, automatically adjusts the play speed of the input video and/or replays the input video in sequence units. And when the learner does not follow the operation of the expert in the input video by more than a certain level, the play speed of the input video is adjusted so that the learner can learn more easily.

In addition, an object of the present invention is a motion recognition-based smart learning method capable of smart learning by identically following an expert's motion in an input image, the method comprising: acquiring an image of a learner captured through a motion recognition sensor; Extracting an object from the learner's image and/or the input image; Generating at least one posture information data of the object; Generating continuous motion data of the object based on the generated posture information data; Determining an error degree between motion data of the object in the input image and motion data of the object in the learner's image; And outputting an instruction for correcting the motion of the object in the image when the error degree exceeds a set numerical range, wherein determining whether the motion data matches, comprises determining the numerical range in which the error degree is set. If it exceeds, further comprising the step of automatically adjusting the play speed of the input video or repeatedly playing the input video in sequence units, when the motion of the object in the input video and the learner's motion are inconsistent It is possible to achieve the object of the present invention through a motion recognition-based smart beauty technology learning method, characterized in that the learner can determine the state of the motion by determining this and correct the wrong posture according to the output instruction. do.

According to the motion recognition-based smart beauty technology learning device of the present invention as described above, the following effects are provided.

According to an exemplary embodiment of the present invention, data obtained by analyzing object behavior in an output image and behavior data of a learner recognized through a sensor may be compared and analyzed.

In addition, according to the present invention, it is possible to mutually exchange between users and videos, such as playing the next video when a specific action is taken rather than unilateral content consumption, so that it can be used for educational programs requiring mutual interaction and programs requiring long-term management.

On the other hand, the effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a block diagram of a motion recognition-based smart beauty technology learning device according to an embodiment of the present invention.
2 is an exemplary view of an image obtained by calculating the learner's posture information data according to an embodiment of the present invention.
3 is an exemplary diagram of a motion script in which motion data of a learner is layered on a frame of an image photographing a learner's motion according to an embodiment of the present invention.
Figure 4 is a flow chart of a motion recognition-based smart beauty technology learning method according to an embodiment of the present invention

Hereinafter, the present invention will be described with reference to the accompanying drawings. However, the present invention may be implemented in various different forms, and therefore is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected (connected, contacted, bonded)" with another part, it is not only "directly connected", but also "indirectly connected" with another member in the middle. "Including the case. In addition, when a part "includes" a certain component, this means that other components may be further provided, not excluding other components, unless specifically stated to the contrary.

The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

For reference, among the configurations of the present invention to be described below, the above-described conventional technology will be referred to for the same configuration as the prior art, and a separate detailed description will be omitted.

1 is a block diagram for explaining a motion recognition-based smart beauty technology learning device 100 according to an embodiment of the present invention.

As shown in Fig. 1, the motion recognition-based smart beauty technology learning device 100 includes an image acquisition unit 110, an image processing unit 120, a sensor unit 130, a motion script generation unit 140, and an operation judgment. Including the unit 150, the learning support unit 160, and the display unit 170, it is possible to enable self-learning by following the same operation of the expert in the input image.

The components shown in FIG. 1 are not essential in implementing the motion recognition-based smart beauty technology learning device 100, so the motion recognition-based smart beauty technology learning device 100 described in this specification is listed above. It may have more or fewer components than components.

According to an embodiment of the present invention, the motion recognition-based smart beauty technology learning device 100 may be played by inputting an image in which expert skills or know-how are captured by the learner 200. For example, a hair designer's lecture on hair dyeing or a makeup artist's makeup lecture video can be input and played, and the learner 200 is a lecturer of an expert (ex. hair designer or makeup artist) in the video. While listening to, by following the same movement of the experts, you can learn how to dye or make makeup.

The image acquisition unit 110 according to an embodiment of the present invention may acquire the learner's image 111 captured through a motion recognition sensor. In more detail, if the motion recognition sensor can capture the learner 200 following the motion of the expert in the video played on the motion recognition-based smart beauty technology learning device 100, the learner 200 photographed in this way The learning image of may be acquired through the image acquisition unit 110. For example, if the learner 200 practices makeup in the same manner as the manager in the input video while watching a makeup course as a video, the process and the result may be photographed through a motion recognition sensor.

The motion recognition sensor may be installed at a position spaced a predetermined distance forward with respect to the front of the learner 200 so as to capture a depth image of the learner 200. In addition, the motion recognition sensor is preferably a device including a depth sensor, an RGB sensor, and a photographing camera to obtain a general image, a color image, and a depth image, and includes a Kinect sensor to determine the position and movement of the player. It can also detect and transmit 3D data. In addition, the learner 200 may be photographed by including an infrared emission and an infrared sensor. For example, when the learner 200 observes a course of a hair designer and practices it, an image of the learner 200's practice motion, that is, a motion using scissors and/or a dryer, may be acquired. However, this is only an exemplary embodiment and may be included in the motion recognition-based smart beauty technology learning device 100 in various forms.

The motion recognition-based smart beauty technology learning device according to an embodiment of the present invention may further include a sensor unit 130. More specifically, the learner may attach a sensor for acquiring data composed of the learner's location coordinates to the body, and more precisely obtain the learner's movement through the obtained location coordinates data. Data composed of position coordinates is information that can be calculated using x, y, and z coordinates, and may include information such as direction and speed. In addition, the sensor may be attached to a part of the learning tool used by the learner, and the sensor unit 130 may acquire data composed of the positional coordinates of the learning tool used by the learner through the sensor attached in this way. . That is, as in the case of the learner, information such as the direction and moving speed of the learning tool used by the learner can be obtained.

The image processing unit 120 according to an embodiment of the present invention may extract an object from the captured learner's image 111 and/or the input image. The object may be a learner 200 included in an image captured through the motion recognition sensor or a tool used by the learner 200 (eg, a tool for styling hair or a brush for makeup). In addition, the image processing unit 120 recognizes and extracts an expert in an image input to a motion recognition-based self-learning device and a tool used by the expert (ex. a tool for hair styling or a brush for makeup) as an object. You may.

2 is an exemplary diagram of an image obtained by calculating the posture information data 121 of the learner 200 according to an embodiment of the present invention.

According to an embodiment of the present invention, the image processing unit 120 may extract at least one object from an image acquired from a motion recognition sensor, and generate attitude information data 121 for the extracted object. Here, the posture information data 121 may include body information of learners 200 and experts, in particular, location information of fingers, wrists, and elbows of learners 200 and experts, and standing information of learners 200 and experts. have. For example, the body information may be information on the height, shoulder width, arm, etc. of the learner 200 by analyzing pixels of the depth image, and the standing information is the posture information of the learner 200 in the learning start stage. For example, the widths of the right and left feet of the learner 200, the right arm and the right hand of the learner 200, and the front-rear distance between the left arm and the left hand of the learner 200 may be included. In addition, in the process of calculating the standing information by the image processing unit 120, for example, the length can be determined from the number of vertical and horizontal pixels occupied by the body of the learner 200 in the captured image, and the resolution and angle of view You can know the angle the body occupies.

The image processing unit 120 according to an embodiment of the present invention, when the learner 200 uses tools for learning (ex. scissors, makeup tools, hair rolls, etc.), the position of the finger holding the tool May be measured through the number of vertical and horizontal pixels. In addition, according to another embodiment of the present invention, the image processing unit 120 may generate the posture information of the learner 200 as skeleton data based on joint points.

3 is an exemplary diagram of a motion script in which motion data of the operating learner 200 is layered on a frame of an image in which the motion of the learner 200 is captured according to an embodiment of the present invention.

The motion script generation unit 140 according to an embodiment of the present invention may generate a motion script including continuous motion data of the object based on the calculated posture information data 121. In more detail, the motion data 141 may be obtained by integrating data on a series of continuous motions of an object in an image based on the posture information data 121. For example, when the posture information data 121 is output as skeleton data connecting joint points, the motion data 141 may be a continuous output of skeleton data for a series of consecutive motions. Therefore, the image processing unit 120 extracts and stores the learner 200, the expert, the learning tools used by the learner, and the posture information data 121 of the teaching tool used by the expert, and stores each of these posture information data 121 Based on the individual motion data 141 for each object can be generated. A motion script may be generated by extracting the continuous motion of the object including the motion data 141 in units of frames per second. The motion data 141 may include the position, movement coordinates, rotation and/or translational coordinates of the object in the image based on the posture information data 121, whereby the learner 200 You can make it possible to judge the motion objectively.

In addition, the motion script generation unit 140 may include information data composed of the learner's position coordinates obtained by the sensor unit 130 when generating the learner's motion data. That is, the learner's motion data may be generated by combining the learner's posture information data and the acquired learner's position coordinate data. That is, more precise motion data of the learner may be obtained by integrating the posture information data and the learner's position coordinate data acquired through the sensor. In addition, when generating motion data of the learning tool used by the learner, it is also possible to generate motion data of the learning tool used by the learner by integrating the posture information data and position coordinate data of the learning tool used by the learner. .

In addition, the motion script generation unit 140 may generate a motion script by layering motion data of an object in the image on each image. The motion script means, for example, an image in which motion data of the learning field and learning tools used by the learner are layered on a captured learner's image, and the inputted expert's image is of the teaching aid used by the expert and the expert. By layering motion data, motion scripts for experts can be created. In another embodiment, another motion script may be generated by layering only the learner's motion data on the expert's motion script.

2 and 3, the posture information data 121 calculated from the learner's image 111 is skeleton data composed of a point corresponding to a joint and a line connecting the joint point as shown in FIG. The motion script generation unit 140 generates motion data 141 based on the posture information data 121 in a series of processes in which the learner 200 operates as shown in FIG. It is possible to create a motion script in which motion data of the operating learner 200 is layered on an image photographing the motion of.

The motion script generation unit 140 according to an embodiment of the present invention may generate the motion script in units of image sequences. Here, the sequence refers to the division of video paragraphs describing the beginning to the end of a specific situation. Accordingly, the motion script generation unit 140 detects the beginning and the end of a series of consecutive motions included in the video recorded in frames per second, divides and stores the sequence in units of a sequence, so that the action of the expert in the input video is imaged. Motion scripts can be created to learn in units of sequences. For example, if the hair designer in the video gives a lecture that creates curls on a part of the hair in a series of continuous motions without stopping during the lecture, a motion script based on the start and end of the motion can be generated. .

The motion determination unit 150 according to an embodiment of the present invention may determine an error degree between motion data of the object in the input image and motion data of the object in the learner's image 111. In more detail, the degree of inconsistency between the motion data of the object in the learner's image and the motion data of the object in the input image (ex. expert) may be calculated and displayed as a numerical value. For example, the degree of inconsistency may be calculated by comparing and analyzing the motion data of the learner 200 and the motion data of the expert. In this case, the degree of inconsistency between the motion data of the object in the learner image (eg, learner) and the motion data of the object in the input image (eg, expert) may be referred to as the degree of error. This degree of error can be considered including the direction of movement of the motion data (ex. joint point) of the object in the learner image, the number of rotations, and/or translation. In addition, since the object may include a tool used by the learner 200 or an expert, the degree of error may be calculated by including the position coordinate, rotation number, and translation coordinate of the tool.

The operation determination unit 150 according to an embodiment of the present invention uses a motion recognition sensor including an RGB sensor when color is important in the learning content (for example, when the input image is a makeup lecture by a makeup expert). Through this, it is possible to calculate the error degree by determining not only the operation of the learner 200 but also whether the makeup color used and the color of the makeup result match.

The learning support unit 160 according to an embodiment of the present invention corrects the posture of the object in the captured learner's image 111 when the determination result of the operation determination unit 150, that is, the error degree exceeds a set value. Instructions can be printed out. In more detail, the learner 200 using the smart beauty technology learning device 100 based on motion recognition according to the present invention provides a numerical value for outputting the determination result of the degree of inconsistency or the match between the expert's motion and his/her motion. Can be set. That is, when the error degree exceeds the set value, the error degree may be layered on the object in the input image and output.

In addition, the instruction may include a direction, a position, a speed, a moving angle, and/or a number of rotations in which the learner 200 should move for posture correction. In more detail, the instructions include coordinates, angles, rotation times, etc. to which the object (ex. learner) in the captured image should move based on the input motion data of the object (ex. expert) in the image. It can mean. The instructions may be layered on the reproduced image and output as a caption or an arrow, or may be supported as a voice, so that the learner 200 can learn a more accurate method for following the skill of an expert in the input image. have. However, the method of outputting the instructions is only an exemplary embodiment and is not limited thereto.

The learning support unit 160 according to another embodiment of the present invention automatically adjusts the play speed of the input image and/or repeats the input image in sequence units when the error degree exceeds a set numerical range. Can play. That is, the degree of inconsistency between the motion data 141 of the object in the input image and the motion data 141 of the learner 200 exceeds a certain value, so that the learner 200 performs the operation of the expert in the input image. If it is determined that it cannot be followed by a certain level or more, the learner 200 may more slowly follow the expert's motion by adjusting the playback speed of the input image. Alternatively, the input image may be repeatedly reproduced in sequence units until the error degree is within a set numerical range. In addition, as the lecture video of the expert is repeatedly played, the repeated playback may be stopped when it is determined whether the match between the learner and the expert's motion data is within a set numerical range. For example, the learner 200 may randomize and set the degree of agreement with the expert's motion to plan the learning progress. That is, when the error range is set to 10%, if the degree of error between the expert's motion data 141 and the learner's 200's motion data 141 exceeds 10% in an image sequence unit, the image is repeated in sequence units. Can play.

In addition, the learning support unit 160 according to another embodiment of the present invention may store an error degree of motion data of learners and experts, which is a learning result, so that a learning progress can be planned. In other words, it is possible to store values of the degree of error of the motion data at the time of initial learning and the degree of error of the motion data in an improved state, and record the sequence of images with a large degree of error in order to recommend videos that need to be reviewed to facilitate learning. Alternatively, it may be recommended to review the recorded video sequence at the next playback.

Learning support unit 160 according to another embodiment of the present invention, stored in its own DB (see Fig. 1) or from outside (a service company that provides weather information, etc.) in real time or at a set cycle external environment information (for example, temperature , Rainfall, wind speed, wind direction, humidity, illuminance, etc.) are received, and expert cosmetic methods related to maker-up and cosmetic methods optimized for external environment information can be implemented through the display unit 170, At the same time or according to the user's selection, conforming to the external environmental conditions (for example, when the weather is cloudy and windy, content for a darker makeup method is displayed through the display unit 170) or SPF ( Recommendations for maker-up products with enhanced functions such as Sun Protection Factor), water-proof, etc., and make-up methods based on these products are implemented through the display unit 170, and the user or learner can follow along and learn. Hazard learning support unit 160, through a wired or wireless network in advance (or in real time) from experts, etc. to receive content or information on the optimized cosmetic method according to external environmental information, makeup technology, information on recommended cosmetics, etc. It can be stored in the technology learning device 100 (it can be a DB).

Learning support unit 160 according to another embodiment of the present invention, the learner's makeup pattern collected through the image acquisition unit 110 (method of using various beauty devices such as brushes, eye makeup, lipstick use color, use method, etc. (May include a pattern) can be stored in the motion recognition-based smart beauty technology learning device 100 (for example, it may be a DB shown in FIG. 1), and thus stored (or proceeding in real time) Compare the make-up pattern of the learner with the make-up method (or make-up pattern) of a make-up artist such as a make-up artist previously input (stored) in the DB, and reproduce a mutual comparison image through the display unit 170 separately at a time preferred by the learner, or The comparison analysis result, etc. may be provided, and additionally, data may be transmitted to and compared to other terminals (devices connected through a wired or wireless network such as a PC or tablet PC) designated by the learner.

In addition, the motion recognition-based smart beauty technology learning device 100 according to another embodiment of the present invention receives makeup progress information such as real-time makeup of professionals such as makeup artists in real time through a communication network, and at the same time, an image acquisition unit ( 110) and the image processing unit 120 to compare and analyze the make-up progress information of the learner input and processed in real time, such as real-time make-up, so that the expert and the learner can perform make-up while comparing each other through the display unit 170 In addition, the expert and/or the learner's makeup pattern and makeup progress can be stored in a DB so that the expert and/or the learner can analyze it as necessary.

In addition, the motion recognition-based smart beauty technology learning device 100 according to another embodiment of the present invention may support an evaluation of acquisition of beauty technology through use of the device 100 for each individual learner who conducts learning. . To this end, the device 100 may present the individual learner's "make-up completeness" in the DB (for example, the evaluation of the completeness may be presented on a scale of 1 to 10, and whether or not the motion match/disagreement with the expert for which the comparative image analysis was performed, etc. This can be a measure of scoring), whether individual learners "achieve completeness (acquisition of beauty skills through learning, etc.)" and/or beauty of individual or group learners through relative comparisons with multiple other learners. Evaluation of whether or not the technology has been achieved can also be conducted through the learning support unit 160.

In addition, the display unit 170 according to another embodiment of the present invention may be a display device such as, for example, an LCD or a touch screen, and the display unit 170 responds to an external user's input or receives additional data. It can be composed of a touch screen that can be input.

The communication unit (not shown) provides a communication interface required to transmit a transmission/reception signal between the motion recognition-based smart beauty technology learning device 100 and the learner 200 and a sensor attached to the learning tool. In addition, the communication unit may be a device including hardware and software necessary to transmit and receive signals such as data signals through wired or wireless connection with other network devices.

The memory DB performs a function of temporarily or permanently storing data processed by the motion recognition-based smart beauty technology learning device 100. Here, the memory DB may include a magnetic storage medium or a flash storage medium, but the scope of the present invention is not limited thereto.

Figure 4 is a flow chart of a motion recognition-based smart beauty technology learning method according to an embodiment of the present invention

The motion recognition-based smart beauty technology learning method according to an embodiment of the present invention is an operation capable of smart learning by identically following the motion of an expert in an image input to the motion recognition-based smart beauty technology learning device 100 A recognition-based smart learning method, specifically including a step (S410) of acquiring the learner's image 111 captured through a motion recognition sensor.

Also, it includes the step of extracting the learner's image 111 and/or an object in the input image (S420). More specifically, the learner 200 captured through the motion recognition sensor and the learning tool used by the learner 200 may be recognized as objects and extracted. In addition, in the input image, experts and teaching tools used by experts may be recognized and extracted as objects.

A method of learning smart beauty technology based on motion recognition according to an embodiment of the present invention may include generating (S430) posture information data 121 of at least one or more objects.

The posture information data 121 is data about an object in the image, that is, a posture in a stationary state in an initial state of learning of the learner 200 and the expert, and is composed of joints or skeletons of the learner 200 and the expert in a more specific scale. It can mean skeleton data.

Next, in the motion recognition-based smart beauty technology learning method according to an embodiment of the present invention, the step of generating continuous motion data 141 of the object based on the generated posture information data 121 (S440) It may include.

The motion data 141 is stored by extracting and storing the image processing unit 120 of the learner's image 111 and the posture information data 121 of the object in the input image, based on the posture information data 121 A continuous motion of an object in an image is converted into data. For example, when the posture information is output as skeleton data connecting joint points, the motion data 141 may be a continuous output of the skeleton data. The skeleton data may be composed of a point corresponding to a joint and/or a line indicating a skeleton on the display.

The motion recognition-based smart beauty technology learning method according to an embodiment of the present invention is whether the motion data 141 of the object in the input image and the motion data 141 of the object in the learner's image 111 match. It may include determining (S450).

Whether the motion data 141 is matched is determined based on the object in the input image as an error in the motion data 141 of the object in the captured image, and the error is specifically the movement movement of the learner 200 , The number of rotations of the hand, the height of the arm, and the use of tools can be expressed numerically.

In the step of determining whether the motion data 141 is matched according to an embodiment of the present invention (S450), when the error exceeds a set numerical range as a result of the determination, the play speed of the input video is automatically adjusted or Alternatively, it may further include reproducing the input image in sequence units. For example, an error in the degree of correspondence between the motion data 141 of the object in the input image and the motion data 141 of the learner 200 exceeds a certain value, and the learner 200 When it is determined that the movement of the expert is not followed by a certain level or more, the learner 200 can more slowly follow the movement of the expert by adjusting the playback speed of the input image to increase the learning effect. have. Alternatively, the input image may be repeatedly reproduced in sequence units until the error is within a set numerical range. That is, when it is determined that the match is within the set numerical range by repeated reproduction, the repetition may be stopped.

A motion recognition-based smart beauty technology learning method according to an embodiment of the present invention includes a step (S460) of outputting an instruction for correcting the motion of the object in the image when the determination result exceeds a set numerical range. Thus, when the motion of the object in the input image and the motion of the learner 200 are inconsistent, it is determined that the learner 200 can grasp the state of the motion and correct the wrong posture according to the output instruction. .

More specifically, the result of determining whether the motion data 141 matches or not is converted into a numerical value and output in an image, thereby promoting motion correction of the learner 200. In addition, for more active motion correction, the motion data 141 of the learner 200 may be layered on the motion data 141 of the expert in the video being reproduced and output, and at this time, a specific direction of movement is indicated by arrows and/or It can also be displayed as text. At this time, these instructions may be output by voice.

The above methods may be implemented by the motion recognition-based smart beauty technology learning device 100 as shown in FIG. 1, and in particular, may be implemented as a software program that performs these steps, and in this case, The programs may be stored on a computer readable recording medium or transmitted by a computer data signal combined with a carrier wave in a transmission medium or a communication network. In this case, the computer-readable recording medium includes all types of recording devices in which data readable by the computer system is stored, for example, ROM, RAM, CD-ROM, DVD-ROM, DVD-RAM, magnetic tape. , A floppy disk, a hard disk, an optical data storage device, and the like.

Although an embodiment of the present invention has been described above, the spirit of the present invention is not limited to the embodiment presented in the present specification, and those skilled in the art who understand the spirit of the present invention can add components within the scope of the same idea. It will be possible to easily propose other embodiments by changing, deleting, adding, etc., but it will be said that this is also within the scope of the present invention.

As described above, although it has been described with reference to a preferred embodiment of the present invention, those skilled in the art will variously modify or modify the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. It can be done.

100: Motion recognition-based smart beauty technology learning device
110: image acquisition unit
120: image processing unit
121: posture information data
130: sensor unit
140: motion script generation unit
141: motion data
150: operation judgment unit
160: Learning Support Department
170: display unit
200: learner

Claims (5)

In the motion recognition-based smart beauty technology learning device capable of smart learning by identically following the motion of the expert in the pre-input video,
An image acquisition unit for acquiring an image of a learner captured through a motion recognition sensor;
An image processing unit that extracts an object from the captured image of the learner and/or the input image to calculate attitude information data of at least one object;
A motion script generator that generates continuous motion data of the object based on the calculated posture information data;
A motion determination unit determining an error degree between motion data of the object in the input image and motion data of the object in the learner's image; And
Including a learning support unit that outputs instructions for correcting the posture of the object in the captured learner's image when the error degree of the operation determination unit exceeds a set numerical range,
If the motion data of the object in the input image and the motion data of the learner are inconsistent, it is determined that the learner can determine the state of the motion and correct the wrong posture according to the output instruction. Smart beauty technology learning device based on motion recognition.
The method of claim 1,
The smart beauty technology learning device,
Further comprising a sensor unit capable of obtaining positional coordinate data of the learner and/or the learning tool used by the learner from a sensor attached to the learner and/or the learning tool used by the learner,
Motion recognition-based smart beauty technology learning device, characterized in that it can more precisely sense the movement of the learner and / or the learning tool used by the learner.
The method of claim 1,
The motion script generation unit,
Generate the generated motion script in units of video sequences,
The motion data may include a position, a movement coordinate, a number of rotations, and/or a translational motion coordinate of the posture information data,
Motion recognition-based smart beauty technology learning device, characterized in that it is possible to increase the learning efficiency of the learner by generating a motion script to learn the behavior of the expert in the input image in units of image sequences.
The method of claim 1,
The operation determination unit,
In determining the degree of error,
By comparing and analyzing the coordinates in the motion data of the plurality of objects based on the motion data coordinates of the object in the input image, measuring an error in the direction and/or the number of rotations,
When the learner follows the skill of the expert in the input image, the difference between the expert's movement and the learner's movement is more accurately recognized so that the learner can learn efficiently,
The above instructions are:
By including the direction, position, speed, movement angle and/or number of rotations that the learner should move for posture correction, so that it can be output in the manner of audio and/or subtitles,
It allows the learner to learn a more accurate method in imitating the skill of the expert in the input video,
The learning support unit,
If the error degree exceeds the set numerical range,
Automatically adjusting the play speed of the input video and/or repeatedly playing the input video in sequence units,
When the learner does not follow the expert's motion in the input video by a certain level or more, the learner can learn more easily by adjusting the playback speed of the input video. Smart Beauty Technology Learning Device.
In the motion recognition-based smart learning method that enables smart learning by identically following the motion of the expert in the input video,
Acquiring an image of a learner captured through a motion recognition sensor;
Extracting an object from the learner's image and/or the input image;
Generating at least one posture information data of the object;
Generating continuous motion data of the object based on the generated posture information data;
Determining an error degree between motion data of the object in the input image and motion data of the object in the learner's image; And
Including the step of outputting an instruction for correcting the motion of the object in the image when the error degree exceeds a set numerical range,
The step of determining the error degree of the motion data,
When the error degree exceeds a set numerical range, automatically adjusting the play speed of the input video or repeatedly playing the input video in sequence units,
Motion recognition, characterized in that when the motion of the object in the input image and the motion of the learner are inconsistent, it is determined that the learner can grasp the state of the motion and correct the wrong posture according to the output instruction. Based on smart beauty technology learning method.

Images