Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides an intelligent dance pace learning device and method, which can pre-store a dance video wanted to be learned by a user, perform corresponding light guidance according to the video, and enhance the dance effect by atmosphere light after the user keeps up with the frequency, so as to help the learner to enjoy the process of learning dance.
The method is realized by the following technical scheme: an intelligent dance pace learning method comprises the following steps:
presetting a dance name for performance, searching out a dance video according with the dance name through the Internet, downloading the dance video to a local file, and numbering the dance video;
extracting the dance video from a local file, and starting to play the dance video;
dividing the dance video into N sections of videos averagely according to duration, and preloading the ith section of video while playing the ith-1 section of video, wherein i is 1, 2, 3 … N;
preprocessing each video segment to obtain decomposed dance movements, dividing the video frames into regions according to types, marking the divided regions and arranging the regions according to a time axis;
starting to play the dance video, and starting to display the dance special effect by using a time axis;
the dance special effect starts and stops following the dance video.
Preferably, the total time length of the dance video is set to be K, the unit is second, and the calculation formula according to the time length average is
N=K/L,
L={30,K≥300;15,180≤K<300;5,K<180},
Wherein N, K, L is a positive integer.
Preferably, the preprocessing of each video includes eliminating a video picture, proposing dance movements in the video, dividing the video picture into regions according to dance types, calculating the occurrence time of limbs in each region, and counting according to a time axis.
On the other hand, the intelligent dance pace learning device comprises a host, a display, a dance special effect display module, a music player and a video acquisition module,
the host is interconnected with the network end through the communication module, the display is provided with an input module, the output end of the host is respectively connected with the display, the dance special effect display module and the music player, the video acquisition module and the input module of the display are respectively connected with the input end of the host,
dance special effect display module is including diversified light efficiency module, the light efficiency module is including polytype light, the light setting is around dance training district, and every paces learning device is including dance training district, just dance training district is directly proportional with the size in dance training district with the distance of host computer.
Preferably, the video acquisition module is used for obtaining the action of the user in the dance training area, when the body action frequency of the user exceeds a preset dance video, the light effect module with the time axis as the standard is adjusted in an acceleration mode, characters input by the user in the input module of the display are selected according to the character, the dance video with the highest character matching degree is screened out through the host through networking, the preset dance video is determined through the selection of the user, and the host processes the preset video to obtain the lighting effect matched with the light effect module through the body action in the video.
Preferably, in step 3, the dance training area may have a polygonal shape, the video frame is divided into regions according to a preset dance video, the direction of the limb appearing in the frame is obtained according to a time axis, and the response of the light effect module is associated with the division of the video frame regions.
Preferably, contain judging module and light efficiency reinforcing module in the host computer, when user's limbs frequency of action surpassed predetermined dance video, when judging module judges that the frequency surpassed predetermined standard value, judging module drive the light efficiency reinforcing module, light efficiency reinforcing module is a plurality of enhancement atmosphere lamps, and atmosphere lamp is for setting up the inboard of light, works as when judging module judges that the frequency is less than predetermined standard value, atmosphere lamp is closed.
Preferably, the operation principle of the host comprises the following steps:
step 1: the host matches dance videos with high similarity to dance names of users through the network end, arranges the dance videos according to the similarity, and stores one or more dance videos into the user according to the requirement;
step 2: extracting the dance video from a local file, starting playing the dance video, equally dividing the dance video into N sections of videos according to duration, and preloading the ith section of video while playing the ith-1 section of video, wherein i is 1, 2, 3 … N;
and step 3: the dance training method includes the steps that each section of video is preprocessed to obtain decomposed dance actions, video frames are divided into regions according to types, the divided regions are marked and arranged according to a time axis, it is determined that a user is located in a dance training region, a display starts from dance video playing, and dance special effects are displayed through a dance special effect display module.
The invention has the beneficial effects that:
(1) when the dance training device is used for practicing dancing, the best lighting effect can be matched, and after the dance training device is used for practicing dancing, the stage atmosphere after enhancement can be obtained.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to fig. 1 to 5 of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other implementations made by those of ordinary skill in the art based on the embodiments of the present invention are obtained without inventive efforts.
In the description of the present invention, it is to be understood that the terms "counterclockwise", "clockwise", "longitudinal", "lateral", "upper", "lower", "front", "rear", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are used for convenience of description only, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present invention.
Example 1:
an intelligent dance pace learning method comprises the following steps:
presetting a dance name for performance, searching out a dance video according with the dance name through the Internet, downloading the dance video to a local file, and numbering the dance video;
extracting the dance video from a local file, and starting to play the dance video;
dividing the dance video into N sections of videos averagely according to duration, and preloading the ith section of video while playing the ith-1 section of video, wherein i is 1, 2, 3 … N;
preprocessing each video segment to obtain decomposed dance movements, dividing the video frames into regions according to types, marking the divided regions and arranging the regions according to a time axis;
starting to play the dance video, and starting to display the dance special effect by using a time axis;
the dance special effect starts and stops following the dance video. .
It is worth to be noted that the total time length of the dance video is set to be K, the unit is second, and the calculation formula according to the time length average is
N=K/L,
L={30,K≥300;15,180≤K<300;5,K<180},
Wherein N, K, L is a positive integer.
It is worth to be noted that the preprocessing of each video includes eliminating the video frame, proposing dance movements in the video, dividing the video frame into regions according to dance types, calculating the time of the limbs in each region, and counting with a time axis.
It should be noted that, in step S2, when the dance video starts playing, the dance special effect display device starts working, the dance special effect display device controls the change of the atmosphere lamp on the dance display device by acquiring the change of the learning pace of the user, the change frequency of the atmosphere lamp is associated with the working frequency of the user' S limb,
the dance special effect device processes the body posture of the user through a deep learning algorithm by recording the user, processing an RGB image generated by the recording, specifically, obtaining the working frequency of the body of the user through measuring and calculating the frequency of 2D coordinates on the body posture of the user within a threshold time, estimating the 2D posture (x, y) coordinates of each joint from the RGB image, and further estimating the 3D posture (x, y, z) coordinates from the RGB image.
The human body limb/skeleton is obtained by performing posture Estimation (position Estimation) through an RGB image, or directly obtained through a depth camera (e.g., Kinect), (position Estimation) refers to a computer vision technology for detecting human figures in images and videos, and can determine the position of a certain body part of a person appearing in the images, namely the positioning problem of human joints in the images and videos, and can also be understood as searching for a specific posture in the space of all joint postures. In short, the task of posture estimation is to reconstruct human joints and limbs, and the difficulty mainly lies in reducing the complexity of the model analysis algorithm and being able to adapt to various changeable conditions, environments (illumination, occlusion, etc.), and inputs: single frame image
And (3) outputting: a high-dimensional attitude vector represents the position of a joint point, but not a class mark of a certain class, so that the method needs to learn the mapping from a high-dimensional observation vector to a high-dimensional attitude vector, and the pedestrian is firstly identified by single-person attitude estimation, and then the required key point is found in the pedestrian region position. Common data sets are MPII, LSP, FLIC, LIP, each with different accuracy indicators. The MPII is the most common benchmark in the current single-person posture estimation, a PCKh index (the distance between a predicted key point and a GT-labeled key point after passing through a head size normal) is used, the accuracy of the existing algorithm can reach 93.9 percent, behavior identification can be realized by means of related research results of posture estimation, for example, a posture library such as HDM05 provides skeleton information of people in each frame of video, the motion type can be judged based on the skeleton information, the image processing is the prior art, and details are not repeated here.
It is worth explaining that the intelligent terminal part records dance action videos and sends captured dance action video information to the remote computing part through the network communication part, and the remote computing part identifies the collected dance action videos through the limb action identification algorithm part, generates limb tracks and motion parameters and then presents the limb tracks and the motion parameters through the human-computer interaction part; or the intelligent terminal part records dance action videos, sends the captured dance action video information to the limb action identification algorithm part, identifies limbs through the limb action identification algorithm part, generates limb tracks and motion parameters and sends the limb tracks and the motion parameters to the remote computing part through the network communication part, and the remote computing part displays the acquired limb tracks and the motion parameters through the human-computer interaction part;
the body movement identification algorithm component identifies the limbs of the dancer in the dance movement video through a deep learning algorithm, wherein the identification at least comprises the identification of upper limbs, lower limbs, fingers, feet, heads, trunks and included joints, and rectangular coordinates or polar coordinates parameters of the limbs and the joints and movement tracks of the limbs and the joints in the dance process, namely the limb tracks and the movement parameters, are generated;
the dance action evaluating component acquires a certain action segment in the limb track and the motion parameter generated by the limb action identification algorithm component, digitally matches the certain action segment with a dance standard diaphragm plate, gives action differences between the action segments in the current limb track and the motion parameter and the dance standard diaphragm plate after different action segments in the whole limb track and the motion parameter are matched with the dance standard diaphragm plate one by one, and sends the action differences to the human-computer interaction component;
the dance standard diaphragm plate comprises a plurality of standard dance action parameters in dance actions;
and the man-machine interaction part sets a dance standard diaphragm database which is suitable for the moment, displays the action difference between the action segment in the current limb track and the motion parameter and the dance standard diaphragm, and indicates the action difference to the dancer.
Preferably, the intelligent terminal part adopts any one of a smart phone, a tablet computer, a notebook computer and a desktop computer. Preferably, the network communication component adopts any one or any multiple of the following network communication modes: 3G, 4G or 5G mobile network, WIFI wireless network and wired network.
Each standard dance motion parameter comprises any one of the following information:
data flow information of motion speed, position, acceleration and angular acceleration of each joint axis according to a time reference;
data stream information of motion speed, position, acceleration and angular acceleration of each joint axis according to a time reference, and image, picture and music information corresponding to the data stream information.
It should be noted that, the processing procedure of each dance video includes, but is not limited to, processing each frame of dance motion by the number of frames, disassembling the dance motion, and playing the dance motion through a screen, and further, in order to facilitate the dance beginner to learn, the invention can highlight the places with large dance motion amplitude, for example, if the user clicks the belly dance teaching video, the dance video matches the dance type with complicated motion in the belly, the obtained video frame will perform three parts of left, middle, and right in the middle, 1 area is divided in the upper part, 1 area is also divided in the lower part, if the belly dance video counts 3 minutes, the upper area counts the motion of the upper half body of the dancer such as the time of hands and heads, the lower area counts the motion of the lower half body of the dancer such as the time of feet, and the middle area counts the motion of the lower half body of the dancer such as the time of feet, And the time of the motion of the hip, the chest and the thighs, the time of the motion is arranged in regions according to a time axis, the time of the motion is counted, and the processing result is matched with the dance motion of the dancer acquired in real time.
Example 2:
an intelligent dance pace learning device comprises a host, a display, a dance special effect display module, a music player and a video acquisition module,
the host computer passes through communication module and net end interconnection, be provided with input module on the display, the output of host computer is connected with display, dance special effect display module, audio player respectively, the input module of video acquisition module and display is connected with the input of host computer respectively, dance special effect display module is including diversified light efficiency module, light efficiency module is including polytype light, the light setting is around dance training district, and every paces learning device is including dance training district, just dance training district and the distance of host computer and the size in dance training district are directly proportional.
It is worth mentioning that the video acquisition module is used for acquiring the actions of the user in the dance training area, when the body action frequency of the user exceeds a preset dance video, the light effect module with a time axis as a standard is adjusted in an accelerated manner, according to characters input by the user in the input module of the display, the host screens out the dance video with the highest character matching degree through networking, the preset dance video is determined through the selection of the user, and the host processes the preset video to obtain the lighting effect matched with the body action and the light effect module in the video.
It is worth to be noted that the dance training area is of a polygonal structure, the video picture is divided into regions according to a preset dance video, the direction of the limb in the picture is obtained according to a time axis, and the response of the light effect module is related to the division of the video picture regions.
It is worth mentioning that the main frame comprises a judgment module and a light effect enhancement module, when the body action frequency of a user exceeds a preset dance video, the judgment module judges that the frequency exceeds a preset standard value, the judgment module drives the light effect enhancement module, the light effect enhancement module is a plurality of enhanced atmosphere lamps, the atmosphere lamps are arranged on the inner side of the illuminating lamp, and when the judgment module judges that the frequency is lower than the preset standard value, the atmosphere lamps are turned off.
It is worth to be noted that the dance motion matching algorithm is to segment a series of dance motions of dancers, match a mathematical model trained by the dance motion training algorithm with a standard dance motion retrieved by a dance standard diaphragm plate database interface, and the matching method includes but is not limited to joint angle parameter matching, motion tail end parameter matching, joint motion parameter matching, image matching and the like; the dance action evaluation algorithm is used for weighting and scoring matching results of dance actions of a certain section of the dancer and standard dance actions in the dance standard diaphragm database, indicating the standard degree of the dance actions of the dancer according to scores, and indicating the difference between each limb joint and the standard dance actions in the dance actions of the dancer.
It should be noted that, when the dance video is a multi-person dance, and a plurality of users simultaneously learn dance, the openpos algorithm is adopted to first detect joints (key points) of all persons in the image, and then assign the detected key points to each corresponding person, and the openpos network first extracts features from the image by using the previous network layers (VGG-19 is used in the above flowchart). These features are then passed to two parallel convolutional layer branches. The first branch is used to predict 18 confidence maps, each representing a joint in the human skeleton. The second branch predicts a set of 38 joint affine Fields (PAFs) describing the degree of connection between the joints, openpos uses a series of steps to optimize the prediction value of each branch. Using the joint confidence maps, bipartite graphs (as shown above) may be formed between each pair of joints. Using the PAF value, the weaker connection in the bipartite graph is removed. Through the steps, the human body posture skeletons of all the people in the image can be detected and distributed to the correct people.
Or a DeepCut algorithm is adopted to generate a candidate set consisting of D joint candidates. Referring to FIG. 5, the set represents possible positions of all joints of all persons in the image for the person classification of a single image. A subset is selected from the joint candidate set. A label is added to each selected human joint. The tag is one of the C joint classes. Each joint class represents a joint, such as an arm, a leg, a trunk, and the like. The marked joints are divided into each corresponding person, taking into account a triplet (x, y, z) of binary random variables whose fields are as follows,
considering two candidate joints D and D ' in the candidate set D and two classes C and C ' in the class set C, joint candidates are obtained by fast RCNN or dense CNN, and if x (D, C) ═ 1, it represents that the candidate joint D belongs to the class C, and likewise, x (D, D ') ═ 1 represents that the candidate joints D and D ' belong to the same person, and z (D, D ', C, C ') ═ x (D, C) × (D, D '). If the above equation value is 1, it represents that the candidate joint d belongs to the class c, the candidate joint d ' belongs to the class c ', and the candidate joints d and d ' belong to the same person, which can be expressed as a linear equation system with respect to (x, y, z). In this way, an Integer Linear Programming (ILP) model is built and multi-person pose estimation can be a problem to solve the set of linear equations.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.