CN110310351A - A kind of 3 D human body skeleton cartoon automatic generation method based on sketch - Google Patents
A kind of 3 D human body skeleton cartoon automatic generation method based on sketch Download PDFInfo
- Publication number
- CN110310351A CN110310351A CN201910597737.9A CN201910597737A CN110310351A CN 110310351 A CN110310351 A CN 110310351A CN 201910597737 A CN201910597737 A CN 201910597737A CN 110310351 A CN110310351 A CN 110310351A
- Authority
- CN
- China
- Prior art keywords
- network
- sketch
- human body
- data
- skeleton cartoon
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
- G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Human Computer Interaction (AREA)
- Social Psychology (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention relates to a kind of 3 D human body skeleton cartoon automatic generation method based on sketch, this system is to simplify the interactive mode of skeleton cartoon production as setting about a little, the head and the tail frame of animation to be generated is provided in a manner of the input of sketch image, realize that the sketch three-dimensional reconstruction of system and skeleton cartoon interpolation frame information are automatically synthesized two parts module respectively using tensorflow building neural network framework, it is final to realize the 3 D human body skeleton cartoon automatic generation method based on sketch.System front and back end code logic separation, flexibility with higher.The system can carry out pretreatment operation to user's input picture automatically, to meet each section network inputs format, can reduce the complexity in the interaction of whole system.
Description
Technical field
The present invention relates to Computer Animated Graph field more particularly to a kind of 3 D human body skeleton cartoons based on sketch certainly
Dynamic generation method.
Background technique
In computer graphics, dimensional Modeling Technology provides many necessary methods and is used to object in the real world
Body is converted into the mathematical expression form under three-dimensional system of coordinate, and is rendered by computer program, to realize virtual empty
Between simulate real world effect.The current 3 d modeling softwares for having many maturations, such as CAD, Maya, 3DsMax etc., and
The extensive use of every field has all been obtained in actual three-dimensional modeling.Although these 3 d modeling softwares are widely used for
Production environment, but generally require the training by profession in the use of software and there is the learning curve of steeper.
Traditional 3 d modeling software is emphasized to limit using input form of the cumbersome rules of interaction to user, builds to improve
Mould precision, therefore, even for profession modeling personnel for modeling task also can be costly time overhead.
In order to solve this problem, the dimensional Modeling Technology based on sketch is provided using cartographical sketching side as input
The effective ways of formula progress Geometric Modeling.Modeling technique based on sketch describe it is a kind of based on gesture for construction indicate
(CSG) rapid modeling for simple threedimensional model may be implemented in the method modeled, this method.Three-dimensional modeling based on sketch
Technology finally exports corresponding three-dimensional mould according to the sketch outline information of input mainly using two-dimentional sketch curve as input
Type.Modeling technique based on sketch is mainly to be the user with painting ability but shortage 3 d modeling software use experience and mention
Out and design.In recent years, the modeling technique based on sketch is used for three-dimensional modeling task more and more by people, and may be used
To be widely used in some special dimensions, such as animal modeling, game role modeling, dress designing, scalp electroacupuncture.Pass through letter
Single two-dimentional sketch outline stroke, user can be used sketch input interface and carry out the complex model pair with free geometric jacquard patterning unit surface
The modeling of elephant, so as to reduce the time overhead of modeling period in a more effective manner.
For three-dimensional modeling, Computer Animated Graph is always the key points and difficulties studied.Computer animation is
Existing in the form of animation sequence, need to summarize animated actions using storyboard on this basis, and in animation time
A fixed scene is drawn in detail in the specific key frame of axis, it is desirable that can reflect in each key frame entire
Multidate information of the animation in particular moment.In order to guarantee the continuity of animation sequence, needs to be added between a pair of of key frame and insert
It is worth frame, and the number density of interpolated frame determines the quality of animation.For relative two dimensional animation, three-dimensional animation needs more rule
It then limits, more there is challenge in the research of wherein 3 D human body skeleton cartoon.In 3 D human body skeleton cartoon, human body is benefit
It is modeled with articulated chain body, wherein particular moment skeletal joint point can be reflected in the world in each fixed key frame
Specific coordinate position in coordinate system.In complete three-dimensional skeleton cartoon manufacturing process, it is necessary first to skeleton structure
Three-dimensional modeling is carried out to need to model personnel during modeling to simulate the positional relationship of real human body skeletal joint point
The three-dimensional modeling and skeleton structure knowledge for grasping profession, need to put into biggish manpower and time cost；Followed by need
With reference to artis during real human body skeleton motion coordinate position variation track so that it is determined that key frame and interpolated frame model
And carry out necessary editing.Although entire manufacturing process can be realized by using animation soft, by
In the rules of interaction for needing a large amount of professional domain knowledge and complexity, therefore, it has become the bottlenecks that limitation user uses.
Summary of the invention
In view of this, the application provides a kind of 3 D human body skeleton cartoon automatic generation method based on sketch.The system
It is automatic by way of inputting any two human actions sketch image using sketch modeling technique and the solution of deep learning method
3 D human body skeleton cartoon is generated, to improve 3 D human body skeleton cartoon by the frequency of interaction and complexity of reduction system
Producing efficiency.
The application is achieved by the following technical solution:
A kind of 3 D human body skeleton cartoon automatic generation method based on sketch, this method comprises the following steps
Step 1, it realizes the interaction with user, receives the human action sketch image file of user's input；
Step 2, background model is called according to the sketch image file；
Step 3, the missing in completion animation sequence is carried out according to the human action information in animation sequence in head and the tail frame to insert
Value frame is automatically synthesized, and then realizes the generation of full animation sequence；
Step 4, by the generation data render of the full animation sequence to screen, visual 3 D human body bone is obtained
Animation.
Further, in the step 2, background model is called according to the sketch image file, is specifically included:
Step 201, the human action sketch image inputted according to user carries out image pre-processing method, to be met
The sketch image data of network inputs format；
Step 202, sketch image recognition web tab is formulated, is obtained for describing network to sketch recognition capability
Export result；
Step 203, sketch image recognition network training is carried out, specific sketch recognition result is obtained according to input sketch image
And realize the mapping for arriving three-dimensional space skeletal joint point coordinate information；
Step 204, the coordinate information of skeletal joint point in human body three-dimensional space is obtained.
Further, described image preprocess method in step 201, specifically includes:
The sketch image data that successively user is inputted using profile testing method, fill method, equal proportion Zoom method
Image transformation is carried out, to obtain meeting the network inputs of the 3 D human body skeleton model reconstruction model based on sketch image.
Further, the sketch successively user inputted using profile testing method, fill method, equal proportion Zoom method
Image data carries out image transformation, specifically includes:
Sketch image is inputted to user and carries out human body closed curve contour detecting, so that human body can be described by obtaining in image
The main region part of movement；
According to the obtained body curve's profile of the contour detecting to closing section realize fill, with improve image for
The descriptive power of human action；
Convert original image to the network inputs for meeting the 3 D human body skeleton model reconstruction model based on sketch image
Format, and shield the Unnecessary detail information in original image.
Further, in step 202, the formulation sketch image recognition web tab, specifically includes:
Label is divided into three levels according to the relationship between human action, respectively action classification, movement pattern classification and
Act frame category, three kinds of labels for motion images descriptive power from thick to thin, final action action frame class label be used to retouch
State the action message of single frames in specific animation sequence.
Further, in step 203, the progress sketch image recognition network training, specifically includes:
The identification of action sketch image is carried out using convolutional neural networks layering according to the formulation of sketch recognition web tab
With classification, comprising: training method, parameter adjustment and error function setting；
The training method uses tensorflow as deep learning tool, and gradually right using the mode of hierarchical classification
Network is trained, and decomposes the model of more difficult training using the mode of Model Fusion in training and carry out weak typing
The training of model, and every department pattern is merged, obtain final result.
For adjusting network parts parameter to be optimal effect, parameter includes: convolution kernel ruler for the parameter adjustment
Very little, weight and deviation Initialize installation, convolution layer number, optimizer setting and learning rate Initialize installation；
Wherein the convolutional layer quantity determines the dimension and network query function amount of character representation, convolutional layer more multiple features indicate more
Abstract, while calculation amount is also bigger, the fewer character representation of convolutional layer is smaller with hour operation quantity closer to initial data.
Further, in step 3, the human action information according in animation sequence in head and the tail frame complete dynamic
Being automatically synthesized for the missing interpolated frame in sequence is drawn, is specifically included:
When given any two act frame data, using data-oriented as the head of one section of Complete three-dimensional skeleton animation
Tail frame simultaneously automatically generates the interpolation frame data lacked between two frames, used method include skeleton cartoon feature extracting method and
Interpolated frame automatic synthesis method.
Further, the skeleton cartoon feature extracting method passes through coding and decoding using convolution autoencoder network structure
Operation undergoes data to regenerate process, and the input data of network is complete skeleton cartoon sequence data, network it is final
Output is the regeneration data of animation sequence, and optimisation strategy when network training is minimized between initial data and regeneration data
Variance distance, trained model can pass through coding and calculate the feature extraction for realizing original skeleton cartoon.
Further, the mode that the interpolated frame automatic synthesis method is combined using convolution feedforward network with interpolation arithmetic
The gradually variation tendency between recovery action ultimately generates complete skeleton cartoon sequence, includes using nearest in network layer structure
Interpolated layer, convolutional layer and the active coating of adjacent interpolation strategies.
Further, the interpolated layer using arest neighbors interpolation strategies, specifically includes: arest neighbors interpolation strategies can be to original
Beginning data carry out the amplification in size, and can retain primary data information (pdi), by being staged through interpolated layer in network query function
Calculate to meet final output data format size；
Data Jing Guo interpolated layer are abstracted and are reversed by the convolutional layer, realize the fitting to target output；
The active coating for increasing the non-linear of network, and reduces the relation of interdependence in network parameter, alleviates
The over-fitting of network.
Compared with the prior art, the advantages of the present invention are as follows:
1) it proposes to combine using arest neighbors interpolation with convolution strategy on carrying out the design that interpolated frame is automatically synthesized model
Network model, and according to the variance of the output of true animation sequence and network model output apart from step-up error function, so as to
To improve model level by minimizing error amount in the network training stage.
2) sketch Three-dimension Reconstruction Model and 3 D human body skeleton cartoon model is automatically synthesized to encapsulate using layer architecture
To specific functional module and realize complete interactive function.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present invention, constitutes part of this application, this hair
Bright illustrative embodiments and their description are used to explain the present invention, and are not constituted improper limitations of the present invention.
Fig. 1 is the composite structural diagram of automatic generation method of the invention；
Fig. 2 is the image list that three-dimensional skeletal joint dot position information rendering generates；
Fig. 3 is using two-dimentional sketch image recognition model schematic layered；
The trend chart of error and accuracy when Fig. 4 is trained；
Fig. 5 is the schematic diagram realizing three-dimensional skeleton model according to the Freehandhand-drawing action sketch of input and rebuilding；
Fig. 6 is that skeleton cartoon is automatically synthesized model structure schematic diagram；
Fig. 7 is the training Time Duration Error trend graph that animation feature extracts model；
Fig. 8 is that animation feature extracts model test results figure；
Fig. 9 is the error trend graph that interpolated frame is automatically synthesized network model t raining period；
Figure 10 is automatically generated the system sequence figure of method；
Figure 11 is automatically generated the operation result exemplary diagram of method.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistent with the application.On the contrary, they be only with it is such as appended
The example of the consistent device and method of some aspects be described in detail in claims, the application.
It is only to be not intended to be limiting the application merely for for the purpose of describing particular embodiments in term used in this application.
It is also intended in the application and the "an" of singular used in the attached claims, " described " and "the" including majority
Form, unless the context clearly indicates other meaning.It is also understood that term "and/or" used herein refers to and wraps
It may be combined containing one or more associated any or all of project listed.
Below in conjunction with attached drawing and example, the present invention is described in further detail.
In the manufacturing process of three-dimensional animation, compared with using traditional animation Software for producing, this mode of sketch drafting for
Simpler easy for user, user is not required to the excessive professional art knowledge of to master and rules of interaction, uses 2 D animation
This theory of production that input mode carries out three-dimensional animation can increase substantially entire production from the angle for simplifying input form
The efficiency in period.On the other hand, with deep learning especially convolutional neural networks (Convolutional Neural
Networks, CNN) Successful utilization and a large amount of online actions capture number of the model in computer graphics and computer vision
According to the presence in library, the production of skeleton animation is also advanced towards more intelligent direction.By using deep learning algorithm
The change in location relationship in motion capture data library between movement and artis is extracted and learns, and then automatic by computer program
Key frame and interpolated frame are generated to the manual skeleton cartoon editing for replacing tradition complicated, further reduced interactive answer
Miscellaneous degree, improves producing efficiency.
Fig. 1 shows automatic for the 3 D human body skeleton cartoon based on sketch described in 1 in accordance with an embodiment of the present disclosure
The frame of generation method.
It is given birth to automatically refering to what is shown in Fig. 1, embodiment of the disclosure 1 provides a kind of 3 D human body skeleton cartoon based on sketch
At method, it should be noted that step shown in the flowchart of the accompanying drawings can be in such as a group of computer-executable instructions
It is executed in computer system, but in some cases, it can be to be different from step shown or described by sequence execution herein
Suddenly.Refering to what is shown in Fig. 1, the 3 D human body skeleton cartoon automatic generation method based on sketch includes:
Head and the tail frame human action sketch image: the sketch image input by user that can describe human action is used respectively
It is acted to describe first frame and the tail frame of animation sequence to be generated；
Image preprocessing: original input picture is located in advance using contour detecting, filling and equal proportion scaling respectively
Reason, purpose meet subsequent network input size, and the crucial Pixel Information in enlarged drawing；
Type of action sorter network: the classification of action classification is carried out using the strategy of Model Fusion, wherein weak point of every part
Class device is all made of two convolutional layers and two full articulamentums, and wherein the number of convolution kernel is respectively 32 and 64 in convolutional layer,
For convolution kernel having a size of 3 × 3, full articulamentum interior joint number is 1024, and Model Fusion is using monolayer BP network as fusion device；
Movement pattern formal classification network: using the convolutional neural networks of single step mode, connected entirely by two layers of convolutional layer and two
Layer composition is connect, wherein the number of convolution kernel is respectively 32 and 64 in convolutional layer, and convolution kernel is having a size of 3 × 3, full articulamentum interior joint
Number is 1024, for identification the specific movement pattern form in certain action classification；
Crucial frame alignment network: using the convolutional neural networks of single step mode, by two layers of convolutional layer and two full articulamentum groups
At；Wherein the number of convolution kernel is respectively 32 and 128 in convolutional layer, and convolution kernel is having a size of 3 × 3, full articulamentum interior joint number
It is 2056, specifically acts frame information with identification；
3 D human body skeletal joint point coordinate information: point for finally entering sketch can be exported by crucial frame alignment network
Class is as a result, can obtain 3 D human body bone from classification results to the mapping relations three-dimensional skeletal joint point coordinate according to known
The coordinate information of bone artis；
3 D human body skeleton cartoon interpolated frame is automatically synthesized network: using arest neighbors interpolation before convolution strategy combines
Present network structure, with complete movement tendency information is automatically synthesized from known the first two frames action message；
Skeleton cartoon feature extraction and Restoration model: this department pattern both can obtain animation according to existing animation sequence
Characteristic information, on the contrary can also be according to the final complete animation sequence of animation feature Information recovering.It is automatically synthesized by interpolated frame
The input of network can obtain one section of complete motion characteristic information, and it is dynamic that complete bone is obtained by Restoration model
Draw sequence data.
Complete skeleton cartoon sequence data: network and skeleton cartoon feature extraction are automatically synthesized according to interpolated frame and restore mould
The calculating of type ultimately generates complete skeleton cartoon sequence data；
Three-dimensional rendering: it is sat according to the skeleton artis indicated frame by frame in obtained skeleton cartoon sequence data in three-dimensional
Absolute location information under mark system, uses OpenGL to carry out three-dimensional rendering as figure class libraries；
3 D human body skeleton cartoon: according to carrying out three-dimensional rendering frame by frame, and screen refresh frequency is controlled, finally obtains three-dimensional
Skeleton animation is simultaneously output on screen.
Refering to what is shown in Fig. 2, fixed in order to provide type of action sorter network, movement pattern formal classification network and key frame
Position network training when training dataset, use the three-dimensional animation sequence in human body motion capture data library as initial data simultaneously
Parsing obtains the skeletal structure under three-dimensional space frame by frame, converts two-dimensional frames for three-dimensional skeletal structure using the mode of rendering and acts
Image list and the mapping for establishing the corresponding location information to skeletal joint point each under three-dimensional system of coordinate.
Refering to what is shown in Fig. 3, giving the model structure of sketch sorter network, wherein action classification sorter network uses model
The mode of fusion, each weak typing network is made of two layers of convolutional layer and two full articulamentums, and is made using softmax
For classifier, use the intersection entropy function as shown in formula (1) as error function,
Loss (p, q)=- ∑j pj log qj (1)
Model Fusion is carried out according to the method as shown in formula (2) using monolayer BP network；
P '=∑i w i·p i+b (2)
Wherein piIndicate the classification results of i-th of Weak Classifier, the final classification of p ' expression strong classifier is as a result, wiIt indicates
Weak Classifier i exports the weight of result, and b indicates offset.
Movement pattern formal classification network and crucial frame alignment sorter network are that use is complete by two layers of convolutional layer and two
The convolutional neural networks composition of articulamentum composition, and use softmax as classifier, intersect using as shown in formula (1)
Entropy function is as error function.
Refering to what is shown in Fig. 4, in order to prove sorter network can convergence and monitor network in training process convergence variation,
The changing tendency of error and accuracy in three kinds of network training process is had recorded respectively.
Three kinds of sorter networks are tested by using test set data, obtain test result as shown in the table.
Refering to what is shown in Fig. 5, the validity in order to prove model, use OpenGL as tool storage room according to using input sketch
It identifies that obtained skeletal joint point carries out three-dimensional rendering in the specific co-ordinate position information of three-dimensional space, obtains visual three-dimensional
Skeleton model.
Refering to what is shown in Fig. 6, giving the structure that 3 D human body skeleton cartoon is automatically synthesized model.Entire model can be divided into
Two parts, wherein first part's (left part) is action message feature extraction and recovery unit, and middle process is special in this section
The raw motion data that sign is extracted can be indicated in hidden unit in the form of acting manifold.In addition, in order to may be implemented
From the characteristic in movement manifold to the recovery of raw motion data, this unit in design using can carry out simultaneously
The convolution autoencoder network of feature extraction and characteristic recovery bidirectional operation.Second part (right part) is that movement tendency information is extensive
Multiple unit, this part are connected with the top of action message extraction module, and loss movement is gradually completing in entire calculating process
The prediction and recovery of trend feature information, output result are mapped to hidden unit in a manner of acting manifold, this subnetwork makes
The feedforward network structure combined with arest neighbors interpolation with convolution strategy.By the characteristic information predicting and restore finally by
The action message recovery operation of a part of unit further restores complete action message, realizes the interpolation of animation sequence missing frame
With completion.
Wherein the coding in convolution autoencoder network is calculated as shown in formula (3):
Wherein weightOffsetm
=256 indicate the quantity of hidden unit, ω0Indicate 3 × 3 convolution kernel size, operationIndicate that convolution algorithm, Ψ indicate sampling
The core maximum pond operation that sliding step is 2 having a size of 3 and in the first dimension, operation result can carry out input data to drop and adopt
Sample simultaneously halves the length of the first dimension of input data, and Relu is finally used to increase the non-of network model as activation primitive
Linearly.Input data X can be abstracted by complete encoding operation EC, obtain movement manifoldAnd it deposits
Store up hidden unit.
Decoding in convolution autoencoder network is calculated as shown in formula (4):
In decoding calculates, the movement manifold H in hidden unit is as input, W0 TWith W1 TRespectively weight W0And W1Turn
It sets, is carrying out operationWhen be actually to have carried out de-convolution operation.
Motion characteristic based on convolution autoencoder network structure extracts shown in the error function such as formula (5) of network:
Wherein, X indicates original bone cartoon section.
Refering to what is shown in Fig. 7, giving the error tendency that the animation feature based on autoencoder network extracts model.
Refering to what is shown in Fig. 8, giving the test result that the animation feature based on autoencoder network extracts model.
Shown in the feedforward network calculation formula such as formula (6) that arest neighbors interpolation is combined with convolution strategy:
Wherein function NN (x, s) indicate using arest neighbors interpolation method by the size of the first dimension of x be amplified to s (such asThen)； WithRespectively indicate convolution meter in every layer
The weight of calculation, wherein m1=64, m2=64, m3=128, ω1And ω2Indicate 3 × 3 convolution kernel size, ω3And ω4Expression 5 ×
5 convolution kernel size, ω5Indicate 7 × 7 convolution kernel size；b01、b02、b03、b04And b05Respectively indicate each layer convolution algorithm
Deviation.By operationIn order to increase the non-linear of network, every layer of calculated result is all used
Relu carries out Nonlinear Processing as activation primitive.Shown in error function such as formula (7) when network training:
The formula is meant that by measuring the movement tendency information predicted by feedforward network to realistic operation trend
The variance of distance is carrying out acting quality when manifold is predicted between characteristic to react feedforward network.
Network is automatically synthesized with the interpolated frame that convolution strategy combines using arest neighbors interpolation refering to what is shown in Fig. 9, giving
The error tendency in model training period.
Refering to what is shown in Fig. 10, giving the system sequence of the 3 D human body skeleton cartoon automatic generation method based on sketch
Figure.
With reference to shown in Figure 11, the result example of system operation is given, may be implemented from given two human action sketches
Automatic conversion of the image to 3 D human body skeleton cartoon.
Those of ordinary skill in the art will appreciate that all or part of the steps in the above method can be instructed by program
Related hardware is completed, and described program can store in computer readable storage medium, such as read-only memory, disk or CD
Deng.Optionally, one or more integrated circuits also can be used to realize, accordingly in all or part of the steps of above-described embodiment
Ground, each module/unit in above-described embodiment can take the form of hardware realization, can also use the shape of software function module
Formula is realized.The present invention is not limited to the combinations of the hardware and software of any particular form.
It should be noted that the invention may also have other embodiments, without departing substantially from spirit of that invention and its essence
In the case of, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these are corresponding
Change and modification all should fall within the scope of protection of the appended claims of the present invention.
Claims (10)
1. a kind of 3 D human body skeleton cartoon automatic generation method based on sketch, which is characterized in that this method includes following step
Suddenly
Step 1, it realizes the interaction with user, receives the human action sketch image file of user's input；
Step 2, background model is called according to the sketch image file；
Step 3, the missing interpolated frame in completion animation sequence is carried out according to the human action information in animation sequence in head and the tail frame
Be automatically synthesized, and then realize full animation sequence generation；
Step 4, by the generation data render of the full animation sequence to screen, it is dynamic to obtain visual 3 D human body bone
It draws.
2. 3 D human body skeleton cartoon automatic generation method according to claim 1, which is characterized in that in the step 2
In, background model is called according to the sketch image file, is specifically included:
Step 201, the human action sketch image inputted according to user carries out image pre-processing method, to obtain meeting network
The sketch image data of input format；
Step 202, sketch image recognition web tab is formulated, is obtained for describing output of the network to sketch recognition capability
As a result；
Step 203, sketch image recognition network training is carried out, specific sketch recognition result and real is obtained according to input sketch image
Now arrive the mapping of three-dimensional space skeletal joint point coordinate information；
Step 204, the coordinate information of skeletal joint point in human body three-dimensional space is obtained.
3. 3 D human body skeleton cartoon automatic generation method according to claim 2, which is characterized in that in step 201
Described image preprocess method, specifically includes:
Successively the sketch image data that user inputs is carried out using profile testing method, fill method, equal proportion Zoom method
Image transformation, to obtain meeting the network inputs of the 3 D human body skeleton model reconstruction model based on sketch image.
4. 3 D human body skeleton cartoon automatic generation method according to claim 3, which is characterized in that successively use profile
Detection method, fill method, equal proportion Zoom method carry out image transformation to the sketch image data that user inputs, specific to wrap
It includes:
Sketch image is inputted to user and carries out human body closed curve contour detecting, so that human action can be described by obtaining in image
Main region part；
Closing section is realized according to the contour detecting obtained body curve's profile and is filled, to improve image for human body
The descriptive power of movement；
Convert original image to the network inputs format for meeting the 3 D human body skeleton model reconstruction model based on sketch image,
And shield the Unnecessary detail information in original image.
5. 3 D human body skeleton cartoon automatic generation method according to claim 3, which is characterized in that in step 202,
The formulation sketch image recognition web tab, specifically includes:
Label is divided into three levels, respectively action classification, movement pattern classification and movement according to the relationship between human action
Frame category, three kinds of labels for motion images descriptive power from thick to thin, final action action frame class label be used to describe to have
The action message of single frames in body animation sequence.
6. 3 D human body skeleton cartoon automatic generation method according to claim 2, which is characterized in that in step 203,
The progress sketch image recognition network training, specifically includes:
It is layered the identification for carrying out action sketch image using convolutional neural networks and divides according to formulating for sketch recognition web tab
Class, comprising: training method, parameter adjustment and error function setting；
The training method uses tensorflow as deep learning tool, and using the mode of hierarchical classification gradually to network
It is trained, and the model of more difficult training is decomposed using the mode of Model Fusion in training and carries out weak typing model
Training, and every department pattern is merged, obtains final result.
The parameter adjustment is optimal effect for adjusting network parts parameter, and parameter includes: convolution kernel size, power
Weight and deviation Initialize installation, convolution layer number, optimizer setting and learning rate Initialize installation；
Wherein the convolutional layer quantity determines the dimension and network query function amount of character representation, and convolutional layer more multiple features indicate more abstract
Change, while calculation amount is also bigger, the fewer character representation of convolutional layer is smaller with hour operation quantity closer to initial data.
7. 3 D human body skeleton cartoon automatic generation method according to claim 1, which is characterized in that in step 3, institute
State the missing interpolated frame carried out according to the human action information in animation sequence in head and the tail frame in completion animation sequence from dynamic circuit connector
At specifically including:
When given any two act frame data, using data-oriented as the head and the tail frame of one section of Complete three-dimensional skeleton animation
And the interpolation frame data lacked between two frames are automatically generated, used method includes skeleton cartoon feature extracting method and interpolation
Frame automatic synthesis method.
8. 3 D human body skeleton cartoon automatic generation method according to claim 7, which is characterized in that
The skeleton cartoon feature extracting method passes through coding and decoding operation experience one using convolution autoencoder network structure
At process, the input data of network is complete skeleton cartoon sequence data for data reproduction, and the final output of network is animation sequence
The regeneration data of column, optimisation strategy when network training are to minimize initial data and regenerate the variance distance between data,
Trained model can calculate the feature extraction for realizing original skeleton cartoon by coding.
9. 3 D human body skeleton cartoon automatic generation method according to claim 7, which is characterized in that
The mode gradually recovery action that the interpolated frame automatic synthesis method is combined using convolution feedforward network with interpolation arithmetic
Between variation tendency, ultimately generate complete skeleton cartoon sequence, include using arest neighbors interpolation strategies in network layer structure
Interpolated layer, convolutional layer and active coating.
10. 3 D human body skeleton cartoon automatic generation method according to claim 7, which is characterized in that
The interpolated layer using arest neighbors interpolation strategies, specifically includes: arest neighbors interpolation strategies can carry out ruler to initial data
Amplification on very little, and primary data information (pdi) can be retained, met in network query function by being staged through the calculating of interpolated layer
Final output data format size；
Data Jing Guo interpolated layer are abstracted and are reversed by the convolutional layer, realize the fitting to target output；
The active coating for increasing the non-linear of network, and reduces the relation of interdependence in network parameter, alleviates network
Over-fitting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910597737.9A CN110310351B (en) | 2019-07-04 | 2019-07-04 | Sketch-based three-dimensional human skeleton animation automatic generation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910597737.9A CN110310351B (en) | 2019-07-04 | 2019-07-04 | Sketch-based three-dimensional human skeleton animation automatic generation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110310351A true CN110310351A (en) | 2019-10-08 |
| CN110310351B CN110310351B (en) | 2023-07-21 |
Family
ID=68078118
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910597737.9A Active CN110310351B (en) | 2019-07-04 | 2019-07-04 | Sketch-based three-dimensional human skeleton animation automatic generation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110310351B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111401272A (en) * | 2020-03-19 | 2020-07-10 | 支付宝(杭州)信息技术有限公司 | Face feature extraction method, device and equipment |
| CN111862276A (en) * | 2020-07-02 | 2020-10-30 | 南京师范大学 | Automatic skeleton animation production method based on formalized action description text |
| CN113112576A (en) * | 2021-04-15 | 2021-07-13 | 华强方特（深圳）动漫有限公司 | Method for automatically disassembling Maya large bone total weight to fine differentiated bone |
| CN113505751A (en) * | 2021-07-29 | 2021-10-15 | 同济大学 | Human skeleton action recognition method based on difference map convolutional neural network |
| CN114067091A (en) * | 2022-01-17 | 2022-02-18 | 深圳慧拓无限科技有限公司 | Multi-source data labeling method and system, electronic equipment and storage medium |
| CN114842155A (en) * | 2022-07-04 | 2022-08-02 | 埃瑞巴蒂成都科技有限公司 | High-precision automatic bone binding method |
| CN116704553A (en) * | 2023-06-13 | 2023-09-05 | 长江大学 | Human body characteristic identification auxiliary system based on computer vision technology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101958007A (en) * | 2010-09-20 | 2011-01-26 | 南京大学 | Three-dimensional animation posture modeling method by adopting sketch |
| US20160232698A1 (en) * | 2015-02-06 | 2016-08-11 | Electronics And Telecommunications Research Institute | Apparatus and method for generating animation |
| WO2017084204A1 (en) * | 2015-11-19 | 2017-05-26 | 广州新节奏智能科技有限公司 | Method and system for tracking human body skeleton point in two-dimensional video stream |
| CN107968962A (en) * | 2017-12-12 | 2018-04-27 | 华中科技大学 | A kind of video generation method of the non-conterminous image of two frames based on deep learning |
-
2019
- 2019-07-04 CN CN201910597737.9A patent/CN110310351B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101958007A (en) * | 2010-09-20 | 2011-01-26 | 南京大学 | Three-dimensional animation posture modeling method by adopting sketch |
| US20160232698A1 (en) * | 2015-02-06 | 2016-08-11 | Electronics And Telecommunications Research Institute | Apparatus and method for generating animation |
| WO2017084204A1 (en) * | 2015-11-19 | 2017-05-26 | 广州新节奏智能科技有限公司 | Method and system for tracking human body skeleton point in two-dimensional video stream |
| CN107968962A (en) * | 2017-12-12 | 2018-04-27 | 华中科技大学 | A kind of video generation method of the non-conterminous image of two frames based on deep learning |
Non-Patent Citations (6)
| Title |
|---|
| HUANG HAIBIN，KALOGEＲAKIS E，YUMEＲ E，ET AL: "Shape synthesis from sketches via procedural models and convolutional networks", 《IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS》 * |
| SILLA C N，FREITAS A A: "A survey of hierarchical classification across different application domains", 《DATA MINING AND KNOWLEDGE DISCOVERY》 * |
| 刘旭等: "辅助室内定位的关键人体姿态识别", 《科学技术与工程》 * |
| 王俊岭等: "深层次特征学习的Adaboost大规模图像分类算法", 《电视技术》 * |
| 王富平等: "多尺度微分模式相似性角点检测算法", 《光电工程》 * |
| 马昊，李淑琴，丁濛，孟坤: "基于草图的三维建模技术综述", 《智能计算机与应用》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111401272A (en) * | 2020-03-19 | 2020-07-10 | 支付宝(杭州)信息技术有限公司 | Face feature extraction method, device and equipment |
| CN111401272B (en) * | 2020-03-19 | 2021-08-24 | 支付宝(杭州)信息技术有限公司 | Face feature extraction method, device and equipment |
| CN111862276A (en) * | 2020-07-02 | 2020-10-30 | 南京师范大学 | Automatic skeleton animation production method based on formalized action description text |
| CN111862276B (en) * | 2020-07-02 | 2023-12-05 | 南京师范大学 | Automatic skeletal animation production method based on formalized action description text |
| CN113112576A (en) * | 2021-04-15 | 2021-07-13 | 华强方特（深圳）动漫有限公司 | Method for automatically disassembling Maya large bone total weight to fine differentiated bone |
| CN113505751B (en) * | 2021-07-29 | 2022-10-25 | 同济大学 | Human skeleton action recognition method based on difference map convolutional neural network |
| CN113505751A (en) * | 2021-07-29 | 2021-10-15 | 同济大学 | Human skeleton action recognition method based on difference map convolutional neural network |
| CN114067091B (en) * | 2022-01-17 | 2022-08-16 | 深圳慧拓无限科技有限公司 | Multi-source data labeling method and system, electronic equipment and storage medium |
| CN114067091A (en) * | 2022-01-17 | 2022-02-18 | 深圳慧拓无限科技有限公司 | Multi-source data labeling method and system, electronic equipment and storage medium |
| CN114842155A (en) * | 2022-07-04 | 2022-08-02 | 埃瑞巴蒂成都科技有限公司 | High-precision automatic bone binding method |
| CN114842155B (en) * | 2022-07-04 | 2022-09-30 | 埃瑞巴蒂成都科技有限公司 | High-precision automatic bone binding method |
| CN116704553A (en) * | 2023-06-13 | 2023-09-05 | 长江大学 | Human body characteristic identification auxiliary system based on computer vision technology |
| CN116704553B (en) * | 2023-06-13 | 2024-01-26 | 长江大学 | Human body characteristic identification auxiliary system based on computer vision technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110310351B (en) | 2023-07-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110310351A (en) | A kind of 3 D human body skeleton cartoon automatic generation method based on sketch | |
| Xiang et al. | Deep learning for image inpainting: A survey | |
| CN110599573B (en) | Method for realizing real-time human face interactive animation based on monocular camera | |
| CN111553968B (en) | Method for reconstructing animation of three-dimensional human body | |
| Zhang et al. | Hair-GAN: Recovering 3D hair structure from a single image using generative adversarial networks | |
| Shen et al. | Deepsketchhair: Deep sketch-based 3d hair modeling | |
| CN113255457A (en) | Animation character facial expression generation method and system based on facial expression recognition | |
| CN111524226A (en) | Method for detecting key point and three-dimensional reconstruction of ironic portrait painting | |
| CN116385606A (en) | Speech signal driven personalized three-dimensional face animation generation method and application thereof | |
| Wang et al. | Computer-aided traditional art design based on artificial intelligence and human-computer interaction | |
| Kobayashi et al. | Motion capture dataset for practical use of AI-based motion editing and stylization | |
| Hu et al. | Pose-aware attention network for flexible motion retargeting by body part | |
| CN114170353A (en) | Multi-condition control dance generation method and system based on neural network | |
| Unlu et al. | Interactive sketching of mannequin poses | |
| Zhou et al. | Hierarchical learning recurrent neural networks for 3D motion synthesis | |
| Chang et al. | 3D hand reconstruction with both shape and appearance from an RGB image | |
| Zhang et al. | Hair-gans: Recovering 3d hair structure from a single image | |
| Victor et al. | Pose Metrics: a New Paradigm for Character Motion Edition | |
| Yu et al. | Application of Computer Graphics and Machine Learning in Computer Aided Design of Digital Sculpture | |
| Tejera et al. | Space-time editing of 3d video sequences | |
| Tian et al. | Augmented Reality Animation Image Information Extraction and Modeling Based on Generative Adversarial Network | |
| Park et al. | A feature‐based approach to facial expression cloning | |
| Jiang et al. | Animation scene generation based on deep learning of CAD data | |
| Lu et al. | Design and implementation of a virtual teacher teaching system algorithm based on facial expression recognition in the era of big data | |
| Li et al. | Pose-aware 3D talking face synthesis using geometry-guided audio-vertices attention |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |