CN115035042A - Evaluation method and device for motion migration model, electronic device and storage medium - Google Patents
Evaluation method and device for motion migration model, electronic device and storage medium Download PDFInfo
- Publication number
- CN115035042A CN115035042A CN202210541723.7A CN202210541723A CN115035042A CN 115035042 A CN115035042 A CN 115035042A CN 202210541723 A CN202210541723 A CN 202210541723A CN 115035042 A CN115035042 A CN 115035042A
- Authority
- CN
- China
- Prior art keywords
- action
- video
- target
- evaluating
- migration model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The application provides an evaluation method and device of an action migration model, electronic equipment and a storage medium, wherein the method comprises the steps of inputting an acquired action video into an action migration model to be evaluated to obtain a target video; acquiring a first group of position information corresponding to a plurality of first skeleton points in the action video and a second group of position information corresponding to a plurality of second skeleton points in the target video, wherein the second skeleton points are skeleton points corresponding to the first skeleton points in the target video; and evaluating the action migration effect of the action migration model based on the first group of position information and the second group of position information, so that the action migration effect of the action migration model can be objectively and accurately evaluated.
Description
Technical Field
The present application relates to the field of model evaluation technologies, and in particular, to an evaluation method and apparatus for an action migration model, an electronic device, and a storage medium.
Background
This section is intended to provide a background or context to the embodiments of the application that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
The action migration is colloquially understood to be the migration of the user's actions, expressions and gestures to a virtual person. The method has rich application scenes in the modern network, such as real-person real-time driving of a virtual anchor, real-person real-time driving of virtual human customer service, real-person seats for bank transaction and the like. However, currently, most of the evaluation on the migration effect of the action migration model is based on real-time migration, that is, a tester artificially performs a series of actions, and then checks the accuracy, sensitivity, stability and the like of virtual human migration in real time through subjective checking effect. However, such evaluation results are affected by ambient light, shooting angle, distance, and subjective feeling of the user, and it is difficult to obtain objective evaluation.
Disclosure of Invention
In view of the above, an object of the present application is to provide a method and an apparatus for evaluating an operation migration model, an electronic device, and a storage medium.
In view of the above, the present application provides an evaluation method of an action migration model, including:
acquiring a motion video, and inputting the motion video into a motion migration model to be evaluated to obtain a target video;
acquiring a first group of position information corresponding to a plurality of first skeleton points in the action video and a second group of position information corresponding to a plurality of second skeleton points in the target video, wherein the second skeleton points are skeleton points corresponding to the first skeleton points in the target video;
and evaluating the action migration effect of the action migration model based on the first set of position information and the second set of position information.
In some embodiments, evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information specifically includes:
determining a displacement difference for each of the first and second bone points based on the first and second sets of location information;
and evaluating the action migration effect of the action migration model based on the displacement difference.
In some embodiments, before the evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information, the method further comprises:
and respectively carrying out normalization processing on the first group of position information and/or the second group of position information.
In some embodiments, evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information specifically includes:
determining a first number of bone points with displacement difference smaller than a preset distance in target bone points, and evaluating the action migration effect of the action migration model based on the first number and the total number of the target bone points;
wherein the target bone point is the first bone point or the second bone point.
In some embodiments, evaluating the motion migration effect of the motion migration model based on the displacement difference specifically includes:
and determining a preset weight corresponding to each first skeleton point and the corresponding second skeleton point, and evaluating the action migration effect of the action migration model based on the displacement difference and the preset weight.
In some embodiments, the evaluating the action migration effect of the action migration model based on the displacement difference specifically includes:
selecting a plurality of target distances within a preset distance range;
for each of the target distances, determining a second number of bone points of the target bone points for which the displacement difference is less than the target distance; wherein the target bone point is a first bone point or a second bone point;
evaluating the action migration effect of the action migration model based on all of the second quantities and the total number of the target bone points.
In some embodiments, evaluating the action migration effect of the action migration model based on all of the second number and the total number of the target bone points includes:
plotting a curve in a coordinate system based on the plurality of target distances and the ratio of the second number corresponding to each target distance to the total number of second bone points; wherein the horizontal axis of the curve represents distance and the vertical axis of the curve represents ratio;
and evaluating the action migration effect of the action migration model based on the curve.
In some embodiments, evaluating the action migration effect of the action migration model based on the curve specifically includes:
and evaluating the action migration effect of the action migration model based on the area of a graph formed by the curve and the horizontal axis of the coordinate system.
In some embodiments, before inputting the motion video to the motion migration model to be evaluated, the method further comprises:
preprocessing the action video to enable the action migration model to accurately identify the action to be migrated in the action video;
wherein the pre-processing comprises at least one of: video clipping, resolution adjustment, frame rate adjustment, and background replacement.
In some embodiments, after evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information, the method further comprises:
and adding interference elements in the action video, inputting the action video added with the interference elements into an action migration model to be evaluated, and evaluating the action migration effect of the action migration model again.
In some embodiments, the obtaining a first group of location information corresponding to a plurality of first bone points in the action video and a second group of location information corresponding to a plurality of second bone points in the target video specifically includes:
and acquiring a first group of position information corresponding to a plurality of first bone points in the action video and a second group of position information corresponding to a plurality of second bone points in the target video by a MediaPipe attitude estimation algorithm.
Based on the same inventive concept, an exemplary embodiment of the present application further provides an evaluation apparatus for an action migration model, including:
the input module is used for acquiring a motion video and inputting the motion video into a motion migration model to be evaluated to obtain a target video;
the acquisition module is used for acquiring a first group of position information corresponding to a plurality of first skeleton points in the action video and a second group of position information corresponding to a plurality of second skeleton points in the target video, wherein the second skeleton points are skeleton points corresponding to the first skeleton points in the target video;
and the evaluation module is used for evaluating the action migration effect of the action migration model based on the first group of position information and the second group of position information.
Based on the same inventive concept, the exemplary embodiments of the present application further provide an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable by the processor, and when the processor executes the program, the processor implements the evaluation method of the motion migration model as described above.
Based on the same inventive concept, the exemplary embodiments of the present application also provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the evaluation method of an action migration model as described above.
As can be seen from the above, according to the evaluation method, the evaluation device, the electronic device, and the storage medium of the motion migration model provided by the application, the obtained motion video is input into the motion migration model to be evaluated, so as to obtain the target video corresponding to the motion video; then acquiring a first group of position information corresponding to a plurality of first skeleton points in the action video and a second group of position information corresponding to a plurality of second skeleton points in the target video, wherein the second skeleton points are skeleton points corresponding to the first skeleton points in the target video; and finally, evaluating the action migration effect of the action migration model based on the first group of position information and the second group of position information, wherein the action migration effect of the action migration model is evaluated through the acquired position information of the first bone point and the second bone point, so that the action migration effect of the action migration model can be objectively and accurately evaluated.
Drawings
In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for the embodiments or related technologies in the following description are briefly introduced, and it is obvious that the drawings in the following description are only the embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic flowchart of an evaluation method of an action migration model according to an embodiment of the present application;
FIG. 2 is a diagram illustrating a motion migration performed by a motion migration model according to an embodiment of the present application;
FIG. 3 is a graph for evaluating the effect of motion migration according to an embodiment of the present application;
FIG. 4 is another graph for evaluating the effect of motion migration according to the embodiment of the present application;
fig. 5 is a schematic structural diagram of an evaluation apparatus for motion migration model according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a specific electronic device according to an embodiment of the present application.
Detailed Description
The principles and spirit of the present application will be described with reference to a number of exemplary embodiments. It should be understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the present application, and are not intended to limit the scope of the present application in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
According to an embodiment of the application, an evaluation method and device of an action migration model, an electronic device and a storage medium are provided.
In this document, it is to be understood that any number of elements in the figures are provided by way of illustration and not limitation, and any nomenclature is used for differentiation only and not in any limiting sense.
The principles and spirit of the present application are explained in detail below with reference to several representative embodiments of the present application.
Summary of The Invention
In the prior art, for the evaluation of the migration effect of the action migration model, a tester performs a series of actions artificially, and then a user subjectively checks the action migration effect, so as to check the accuracy, sensitivity, stability and the like of virtual human migration in real time. However, such evaluation results are affected by ambient light, shooting angle, distance, and subjective feeling of the user, and it is difficult to obtain objective evaluation.
In order to solve the above problem, the present application provides an evaluation method of an action migration model, specifically including:
the action videos directly obtained replace the performance of a tester in front of a real-time camera, and the obtained action videos can be videos shot in advance, so that all the action videos used for testing can be recorded at one time, a manual performance action link is replaced, the action videos can be repeatedly used for many times, the action migration evaluation efficiency is improved, meanwhile, when a plurality of action migration models are evaluated, the action videos input by all the action migration models can be completely the same, and the accuracy of the action migration models is further improved; after the acquired action video is input into an action migration model to be evaluated, a target video corresponding to the action video is obtained, a first group of position information corresponding to a plurality of first bone points in the action video and a second group of position information corresponding to a plurality of second bone points in the target video are respectively acquired, wherein the second bone points are the bone points corresponding to the first bone points in the target video, and the action migration effect of the action migration model is evaluated based on the first group of position information and the second group of position information, so that the action migration effect of the action migration model can be objectively and accurately evaluated.
Having described the basic principles of the present application, various non-limiting embodiments of the present application are described in detail below.
Application scene overview
In some specific application scenarios, the evaluation method of the action migration model can be applied to various evaluation systems and evaluation software. The action migration model may be an action migration model applied to each APP or website.
In some specific application scenarios, the evaluation method of the action migration model can be directly applied to local operation and can also be operated in a cloud server. When the cloud server runs, the acquired data to be processed are sent to the cloud server through the network, the server processes the data to be processed through the evaluation method of the action migration model, and the processing result is sent to the local through the network.
The following describes an evaluation method of a motion migration model according to an exemplary embodiment of the present application with reference to a specific application scenario. It should be noted that the above application scenarios are only presented to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect. Rather, embodiments of the present application may be applied to any scenario where applicable.
Exemplary method
Referring to fig. 1, a method for evaluating an action migration model is provided for an embodiment of the present application, including the following steps:
s101, obtaining a motion video, and inputting the motion video into a motion migration model to be evaluated to obtain a target video.
In a specific implementation, before the evaluation of the motion migration model, a motion video may be recorded, where the motion video may be obtained by shooting a test object, and optionally, the test object may be a human, an animal, a robot, or another object that can generate a motion. Optionally, in order to ensure the quality of the recorded motion video, when recording the video, the user can pay attention to the fact that the background is as clean as possible, the light is not too strong or too weak, the shooting camera is parallel to the top of the head of the shot object as much as possible, and the machine position shoots slightly downwards. After the recorded action video is obtained, the action video is obtained, and the action video is input into an action migration model to be evaluated, so that a target video corresponding to the recorded video can be obtained, optionally, the obtained action video can also be an action video stored in advance, or the action video is directly obtained from a network or a third party, and the method is not limited herein. The target video is a video generated by migrating the motion of the initial object in the motion video to the target object. That is, the target video is a virtual video generated by the motion migration model by simulating the motion of the initial object in the motion video, the virtual video includes a target object corresponding to the initial object in the motion video, and the virtual object repeats the motion of the initial object in the motion video. Referring to fig. 2, the left side is the target video output by the motion migration model, and the right bottom side is the recorded motion video, it can be seen that the target video is substantially consistent with the motion of the character in the motion video.
In some embodiments, in order to facilitate inputting the acquired motion video into the motion migration model to be evaluated, the motion video may be input into the motion migration model to be evaluated through a wecam video studio.
To avoid interference of some irrelevant content in the motion video with the motion migration model, in some embodiments, before inputting the motion video into the motion migration model to be evaluated, the method further comprises:
preprocessing the action video to enable the action migration model to accurately identify the action to be migrated in the action video;
wherein the pre-processing comprises at least one of: video clipping, resolution adjustment, frame rate adjustment, and background replacement.
In particular, the python tool may be used to perform video clipping on the motion video, delete useless time frames, and then unify the resolution and frame rate of the video, for example, the resolution of the video is set to 1080P, and the frame rate is set to 30 fps. Alternatively, for some complex backgrounds, they may be replaced with a single color simple background.
S102, acquiring a first group of position information corresponding to a plurality of first skeleton points in the action video and a second group of position information corresponding to a plurality of second skeleton points in the target video, wherein the second skeleton points are skeleton points corresponding to the first skeleton points in the target video.
In specific implementation, a first group of position information corresponding to a plurality of first skeleton points in the action video and a second group of position information corresponding to a plurality of second skeleton points in the target video are obtained. Wherein the first set of location information comprises location information of a plurality of first bone points and the second bone points comprise location information of a plurality of second bone points. Optionally, the number and the positions of the first bone points and the second bone points may be set according to needs, and are not limited herein. For example, 14 skeletal points may be set when performing the upper body motion migration of the human body, and 21 skeletal points may be set when performing the gesture migration. It should be noted that each of the first skeleton points corresponds to one of the second skeleton points, for example, if one of the first skeleton points of a human body in the motion video is a skeleton point of a finger tip of a left hand, the second skeleton point corresponding to the first skeleton point is a skeleton point of a finger tip of a left hand of the virtual character in the target video.
In some embodiments, the obtaining a first group of location information corresponding to a plurality of first bone points in the action video and a second group of location information corresponding to a plurality of second bone points in the target video specifically includes:
and acquiring a first group of position information corresponding to a plurality of first bone points in the action video and a second group of position information corresponding to a plurality of second bone points in the target video by a MediaPipe attitude estimation algorithm.
In particular, MediaPipe is a development framework for data stream processing machine learning applications developed and sourced by Google. The position information of each bone point in the video can be automatically acquired through a MediaPipe attitude estimation algorithm. Optionally, an openpos (human body posture recognition) posture estimation algorithm may be used to replace the MediaPipe posture estimation algorithm, where openpos is an open source library developed by the university of Camancylmenu (CMU) based on a convolutional neural network and supervised learning and using caffe as a framework, and can implement posture estimation of human body motion, facial expression, finger motion, and the like. Optionally, other existing neural network algorithms may also be used to determine a first set of location information of a plurality of first bone points in the motion video and a second set of location information of a plurality of second bone points in the target video, which is not limited herein.
S103, evaluating the action migration effect of the action migration model based on the first group of position information and the second group of position information.
In specific implementation, after the first group of location information and the second group of location information are determined, the action migration effect of the action migration model is evaluated according to the first group of location information and the second group of location information. Optionally, the relative position relationship between each first bone point and the corresponding second bone point may be determined through the first group of position information and the second group of position information, and the closer the position of each first bone point and the corresponding second bone point is, the better the motion migration effect of the motion migration model is.
In some embodiments, evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information specifically includes:
determining a displacement difference for each of the first bone points and its corresponding second bone point based on the first and second sets of location information;
and evaluating the action migration effect of the action migration model based on the displacement difference.
In some embodiments, before the evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information, the method further comprises:
and respectively carrying out normalization processing on the first group of position information and/or the second group of position information.
In a specific implementation, since the size and the position of the virtual object in the target video generated by the motion migration model may be different from the size and the position of the target object in the motion video, if the displacement difference between the first bone point and the second bone point is determined directly through the first set of position information and the second set of position information, a large error may occur. For example, when the initial object is a gesture, the sizes of the fingers and the palm of the target object need to be adjusted to be consistent with the sizes of the fingers and the palm of the initial object, the orientations of the fingers and the palm of the initial object are unified, and the spatial positions of the target object and the initial object in the respective videos are unified, optionally, the initial object may be kept unchanged, then the size and the position of the target object are adjusted according to the initial object, the target object may also be kept unchanged, then the size and the position of the initial object are adjusted according to the target object, or the sizes and the positions of the target object and the initial object are adjusted to be preset sizes and preset positions at the same time, which is not limited herein. After normalization is completed, determining the displacement difference between the first bone point and the second bone point according to the normalized first group of position information and the normalized second group of position information, and evaluating the action migration effect of the action migration model according to the displacement difference.
In some embodiments, the evaluating the action migration effect of the action migration model based on the displacement difference specifically includes:
determining a first number of bone points with displacement difference smaller than a preset distance in target bone points, and evaluating the action migration effect of the action migration model based on the first number and the total number of the target bone points;
wherein the target bone point is a first bone point or a second bone point.
In specific implementation, the displacement difference between each first bone point and the corresponding second bone point is determined in a plurality of first bone points, then a first number of bone points with displacement difference smaller than a preset distance in a target bone point is determined, and the motion migration effect of the motion migration model is evaluated according to the first number and the total number of the target bone points.
In some embodiments, evaluating the action migration effect of the action migration model based on the displacement difference specifically includes:
and determining a preset weight corresponding to each first bone point and the corresponding second bone point, and evaluating the action migration effect of the action migration model based on the displacement difference and the preset weight.
During specific implementation, the displacement difference of each first bone point and the second bone point corresponding to the first bone point is determined in a plurality of first bone points, then the preset weight corresponding to each first bone point and the second bone point corresponding to the first bone point is determined, and the action migration effect of the action migration model is evaluated according to the displacement difference of each first bone point and the second bone point corresponding to the first bone point and the preset weight. Optionally, the product of each displacement difference and the corresponding preset weight is obtained first, then all the products are summed, and the action migration effect of the action migration model is evaluated by using the summation result, generally speaking, the smaller the value of the summation result is, the better the action migration effect of the action migration model is.
Since the size and the figure proportion of the target object and the initial object in the motion video are different in many times during motion migration, and when the motion migration effect is observed, a situation that the shape is similar and the actual size is far different sometimes occurs, so that the motion migration effect cannot be determined by simply determining whether the displacements of corresponding bone points are identical, in consideration of these factors, in some embodiments, the evaluating the motion migration effect of the motion migration model based on the displacement difference specifically includes:
selecting a plurality of target distances within a preset distance range;
for each of the target distances, determining a second number of bone points of the target bone points for which the displacement difference is less than the target distance; wherein the target bone point is a first bone point or a second bone point;
evaluating the action migration effect of the action migration model based on all of the second quantities and the total number of the target bone points.
In specific implementation, the displacement difference between each first bone point and the corresponding second bone point is determined, and then a plurality of target distances are selected within a preset distance range, wherein the preset distance range can be set as required without limitation, for example, the preset distance range can be set to be 20-50mm, and the number of the selected target distances is not limited; for example, three target distances of 20mm, 30mm, and 50mm may be selected. Then, for each target distance, a second number of target bone points for which the displacement difference is smaller than the target distance is determined, for example, when three target distances of 20mm, 30mm and 50mm are selected, a second number of target bone points for which the displacement difference is smaller than 20mm needs to be determined, a second number of target bone points for which the displacement difference is smaller than 30mm, and a second number of target bone points for which the displacement difference is smaller than 50 mm. And then evaluating the action migration effect of the action migration model according to the three second quantities and the total quantity of the target bone points. Alternatively, the ratio of each of the second numbers to the total number of the target bone points may be determined, and then all the ratios are summed, and in general, the smaller the value of the result of the summation, the better the motion migration effect of the motion migration model.
In order to further ensure the accuracy of the evaluation of the motion migration model, and simultaneously consider the similar situation in motion migration and the different actual sizes, in some embodiments, the evaluation of the motion migration effect of the motion migration model based on all the second numbers and the total number of the target bone points specifically includes:
plotting a curve in a coordinate system based on the plurality of target distances and the ratio of the second number corresponding to each target distance to the total number of second bone points; wherein the horizontal axis of the curve represents distance and the vertical axis of the curve represents ratio;
and evaluating the action migration effect of the action migration model based on the curve.
In specific implementation, referring to fig. 3, the abscissa of the curve represents the distance, the preset distance range in fig. 3 is 20-50mm, and the ordinate of the curve is the ratio of the second number to the total number of the target bone points, wherein when the target distance is 20, the ratio of the corresponding second number to the total number of the target bone points is 0.3, when the target distance is 35, the ratio of the corresponding second number to the total number of the target bone points is 0.7, and when the target distance is 50, the ratio of the corresponding second number to the total number of the target bone points is 0.8. After the coordinate system draws the curve, the action migration effect of the action migration model can be evaluated according to the curve. Generally, the closer the curve is to the Y-axis, the better the motion migration effect of the motion migration model. Optionally, the curve drawn in the coordinate system may be a smooth curve, as shown in fig. 3, or may be a broken line, refer to fig. 4, and it should be noted that the meaning of each parameter in the coordinate system of fig. 4 is the same as that in fig. 3, and is not described herein again.
In some embodiments, evaluating the action migration effect of the action migration model based on the curve specifically includes:
and evaluating the action migration effect of the action migration model based on the area of a graph formed by the curve and the horizontal axis of the coordinate system.
In specific implementation, in order to further ensure the accuracy of the evaluation of the motion migration model, the motion migration effect of the motion migration model may be evaluated according to the area of a graph formed by a curve drawn in a coordinate system and a horizontal axis of the coordinate system. For example, in fig. 4, the graph formed by the curve and the horizontal axis (X axis) of the coordinate system is two right trapezoids, and the motion transition effect of the motion transition model can be evaluated by calculating the areas of the two right trapezoids. Generally, the larger the area of the graph formed by the curve and the horizontal axis of the coordinate system is, the better the motion transition effect of the motion transition model is.
In some embodiments, after evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information, the method further comprises:
and adding interference elements in the action video, inputting the action video added with the interference elements into an action migration model to be evaluated, and evaluating the action migration effect of the action migration model again.
In specific implementation, when comparing the action migration effects of a plurality of action migration models, if the action videos of the input action migration models are relatively simple, the action migration effects of the two action migration models may be the same, at this time, an interference element needs to be added to the action video, the action video with the added interference element is input into the action migration model to be evaluated, and the action migration effects of the action migration models are evaluated again, so that the action migration effects of the action migration models are better distinguished by increasing the recognition difficulty of the action videos input into the action migration models. Optionally, the added interference element may be various elements that are easily recognized as a target object motion by mistake, for example, a background person is added in the person motion video, or a color difference between the background and the motion object may be reduced, which is not limited herein.
According to the evaluation method of the action migration model, the obtained action video is input into the action migration model to be evaluated, and a target video is obtained; then acquiring a first group of position information corresponding to a plurality of first skeleton points in the action video and a second group of position information corresponding to a plurality of second skeleton points in the target video, wherein the second skeleton points are skeleton points corresponding to the first skeleton points in the target video; and finally, evaluating the action migration effect of the action migration model based on the first group of position information and the second group of position information, wherein the action migration effect of the action migration model is evaluated through the acquired position information of the first bone point and the second bone point, so that the action migration effect of the action migration model can be objectively and accurately evaluated.
Exemplary device
Based on the same inventive concept, the application also provides an evaluation device of the action migration model, which corresponds to any embodiment method.
Referring to fig. 5, the apparatus for evaluating an action migration model includes:
the input module 201 inputs the acquired action video into an action migration model to be evaluated to obtain a target video;
an obtaining module 202, configured to obtain a first set of location information corresponding to a plurality of first bone points in the action video and a second set of location information corresponding to a plurality of second bone points in the target video, where the second bone points are bone points in the target video corresponding to the first bone points;
the evaluation module 203 evaluates the action migration effect of the action migration model based on the first set of location information and the second set of location information.
For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, the functionality of the various modules may be implemented in the same one or more software and/or hardware implementations as the present application.
The apparatus of the foregoing embodiment is used to implement the evaluation method of the corresponding action migration model in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Based on the same inventive concept, corresponding to the method of any embodiment described above, the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the method for evaluating the action migration model described in any embodiment above is implemented.
Fig. 6 is a schematic diagram illustrating a more specific hardware structure of an electronic device according to this embodiment, where the electronic device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein the processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 are communicatively coupled to each other within the device via bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 1020 may store an operating system and other application programs, and when the technical solution provided by the embodiments of the present specification is implemented by software or firmware, the relevant program codes are stored in the memory 1020 and called to be executed by the processor 1010.
The input/output interface 1030 is used for connecting an input/output module to input and output information. The i/o module may be configured as a component within the device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and the output devices may include a display, speaker, vibrator, indicator light, etc.
The communication interface 1040 is used for connecting a communication module (not shown in the drawings) to implement communication interaction between the present apparatus and other apparatuses. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).
It should be noted that although the above-mentioned device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will appreciate that the above-described apparatus may also include only those components necessary to implement the embodiments of the present description, and not necessarily all of the components shown in the figures.
The electronic device of the foregoing embodiment is used to implement the evaluation method of the corresponding action migration model in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Exemplary program product
Based on the same inventive concept, corresponding to any of the above-described embodiment methods, the present application further provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the evaluation method of the action migration model according to any of the above embodiments.
Computer-readable media of the present embodiments, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
The computer instructions stored in the storage medium of the foregoing embodiment are used to enable the computer to execute the method for evaluating an action migration model according to any one of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which are not described herein again.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the context of the present application, features from the above embodiments or from different embodiments may also be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of brevity.
In addition, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown in the provided figures for simplicity of illustration and discussion, and so as not to obscure the embodiments of the application. Further, devices may be shown in block diagram form in order to avoid obscuring embodiments of the application, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the application are to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the application, it should be apparent to one skilled in the art that the embodiments of the application can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.
While the present application has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic ram (dram)) may use the discussed embodiments.
The present embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present application are intended to be included within the scope of the present application.
Claims (14)
1. A method for evaluating an action migration model, comprising:
acquiring a motion video, and inputting the motion video into a motion migration model to be evaluated to obtain a target video;
acquiring a first group of position information corresponding to a plurality of first skeleton points in the action video and a second group of position information corresponding to a plurality of second skeleton points in the target video, wherein the second skeleton points are skeleton points corresponding to the first skeleton points in the target video;
and evaluating the action migration effect of the action migration model based on the first set of position information and the second set of position information.
2. The method of claim 1, wherein evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information comprises:
determining a displacement difference for each of the first bone points and its corresponding second bone point based on the first and second sets of location information;
and evaluating the action migration effect of the action migration model based on the displacement difference.
3. The method of claim 1, wherein before evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information, the method further comprises:
and respectively carrying out normalization processing on the first group of position information and/or the second group of position information.
4. The method of claim 2, wherein the evaluating the action migration effect of the action migration model based on the displacement difference comprises:
determining a first number of bone points with displacement difference smaller than a preset distance in target bone points, and evaluating the action migration effect of the action migration model based on the first number and the total number of the target bone points;
wherein the target bone point is a first bone point or a second bone point.
5. The method of claim 2, wherein evaluating the motion migration effect of the motion migration model based on the displacement difference comprises:
and determining a preset weight corresponding to each first skeleton point and the corresponding second skeleton point, and evaluating the action migration effect of the action migration model based on the displacement difference and the preset weight.
6. The method according to claim 2, wherein the evaluating the action migration effect of the action migration model based on the displacement difference specifically comprises:
selecting a plurality of target distances within a preset distance range;
for each of the target distances, determining a second number of bone points of the target bone points for which the displacement difference is less than the target distance; wherein the target bone point is a first bone point or a second bone point;
evaluating the action migration effect of the action migration model based on all of the second quantities and the total number of the target bone points.
7. The method of claim 6, wherein evaluating the motion migration effect of the motion migration model based on all of the second number and the total number of the target bone points comprises:
plotting a curve in a coordinate system based on the plurality of target distances and a ratio of the second number corresponding to each target distance to the total number of second bone points; wherein the horizontal axis of the curve represents distance and the vertical axis of the curve represents ratio;
and evaluating the action migration effect of the action migration model based on the curve.
8. The method according to claim 7, wherein evaluating the action migration effect of the action migration model based on the curve specifically comprises:
and evaluating the action migration effect of the action migration model based on the area of a graph formed by the curve and the horizontal axis of the coordinate system.
9. The method of claim 1, wherein prior to inputting the motion video to a motion migration model to be evaluated, the method further comprises:
preprocessing the action video so that the action migration model can accurately identify the action to be migrated in the action video;
wherein the pre-processing comprises at least one of: video clipping, resolution adjustment, frame rate adjustment, and background replacement.
10. The method of claim 1, wherein after evaluating the action migration effect of the action migration model based on the first set of location information and the second set of location information, the method further comprises:
and adding interference elements in the action video, inputting the action video added with the interference elements into an action migration model to be evaluated, and evaluating the action migration effect of the action migration model again.
11. The method according to claim 1, wherein acquiring a first set of location information corresponding to a plurality of first bone points in the action video and a second set of location information corresponding to a plurality of second bone points in the target video specifically includes:
and acquiring a first group of position information corresponding to a plurality of first bone points in the action video and a second group of position information corresponding to a plurality of second bone points in the target video by a MediaPipe attitude estimation algorithm.
12. An evaluation device for an action migration model, comprising:
the input module is used for acquiring a motion video and inputting the motion video into a motion migration model to be evaluated to obtain a target video;
the acquisition module is used for acquiring a first group of position information corresponding to a plurality of first skeleton points in the action video and a second group of position information corresponding to a plurality of second skeleton points in the target video, wherein the second skeleton points are skeleton points corresponding to the first skeleton points in the target video;
and the evaluation module is used for evaluating the action migration effect of the action migration model based on the first group of position information and the second group of position information.
13. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable by the processor, the processor implementing the method of any one of claims 1 to 11 when executing the program.
14. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method according to any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210541723.7A CN115035042A (en) | 2022-05-17 | 2022-05-17 | Evaluation method and device for motion migration model, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210541723.7A CN115035042A (en) | 2022-05-17 | 2022-05-17 | Evaluation method and device for motion migration model, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115035042A true CN115035042A (en) | 2022-09-09 |
Family
ID=83120224
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210541723.7A Pending CN115035042A (en) | 2022-05-17 | 2022-05-17 | Evaluation method and device for motion migration model, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115035042A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116110080A (en) * | 2023-04-04 | 2023-05-12 | 成都新希望金融信息有限公司 | Switching method of real facial mask and virtual facial mask |
-
2022
- 2022-05-17 CN CN202210541723.7A patent/CN115035042A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116110080A (en) * | 2023-04-04 | 2023-05-12 | 成都新希望金融信息有限公司 | Switching method of real facial mask and virtual facial mask |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110532984B (en) | Key point detection method, gesture recognition method, device and system | |
| CN107251096B (en) | Image capturing apparatus and method | |
| US20190043216A1 (en) | Information processing apparatus and estimating method for estimating line-of-sight direction of person, and learning apparatus and learning method | |
| CN109376631B (en) | Loop detection method and device based on neural network | |
| CN111553282A (en) | Method and device for detecting vehicle | |
| CN109144252B (en) | Object determination method, device, equipment and storage medium | |
| CN114463825B (en) | Face prediction method based on multi-mode fusion and related equipment | |
| CN109215131B (en) | Virtual face driving method and device | |
| CN110910322B (en) | Picture processing method and device, electronic equipment and computer readable storage medium | |
| JPWO2013005815A1 (en) | Object detection apparatus, object detection method, and program | |
| US11276202B2 (en) | Moving image generation apparatus, moving image generation method, and non-transitory recording medium | |
| CN109271929B (en) | Detection method and device | |
| CN106406693B (en) | Image chooses method and device | |
| CN115035042A (en) | Evaluation method and device for motion migration model, electronic device and storage medium | |
| CN106484614A (en) | A kind of method of verification picture processing effect, device and mobile terminal | |
| EP3309750A1 (en) | Image processing apparatus and image processing method | |
| CN111539349A (en) | Training method and device of gesture recognition model, gesture recognition method and device thereof | |
| CN111126358A (en) | Face detection method, face detection device, storage medium and equipment | |
| CN110390344B (en) | Alternative frame updating method and device | |
| CN114120382B (en) | Face recognition system testing method and device, electronic equipment and medium | |
| CN110956131B (en) | Single-target tracking method, device and system | |
| CN115731341A (en) | Three-dimensional human head reconstruction method, device, equipment and medium | |
| CN109685015B (en) | Image processing method and device, electronic equipment and computer storage medium | |
| US10755088B2 (en) | Augmented reality predictions using machine learning | |
| CN115546408A (en) | Model simplifying method and device, storage medium, electronic equipment and product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |