CN102231209A - Two-dimensional character cartoon generating method based on isomerism feature dimensionality reduction - Google Patents

Abstract

The invention discloses a stylized two-dimensional cartoon generating method based on nonnegative style decomposition. In the method provided by the invention, the knowledge of machine learning is realized, thus the function of generating stylized two-dimensional cartoon based on nonnegative style decomposition is realized. The method comprises the following steps: inputting the two-dimensional cartoon keyframe sequence of two-dimensional cartoon characters, systematically extracting corresponding two-dimensional skeleton characteristic expression, and decomposing into style base vectors and hidden components of the two-dimensional cartoon actions; and combining the obtained style base vectors and the isomeric hidden components to regenerate the two-dimensional skeleton characteristic expression with the style of specific cartoon character, and driving the bodies of the two-dimensional cartoon characters to form a stylized two-dimensional cartoon keyframe. In the invention, the two-dimensional cartoon actions are decomposed into the style base vectors and the hidden components, thus reducing the problems in application caused by the separation action body and style in the traditional methods, solving the problem that cartoon data cannot be based on semantic coherence in the traditional nonnegative matrix decomposition method, improving the accuracy and enlarging the application range.

Description

Two-dimensional character cartoon generation method based on the heterogeneous characteristic dimensionality reduction

Technical field

The present invention relates to a kind of stylized two-dimensional cartoon generation method of decomposing based on non-negative style, be specifically related to a kind of non-negative style decomposition of two-dimentional skeleton character expression formula and the stylization generation method of solution and two-dimensional cartoon animation thereof, belong to the general field of computer animation and computer machine study.

Background technology

Along with a large amount of inputs application in industry, entertainment field about computer two-dimensional cartoon and Research of Machine Learning, the two-dimensional cartoon generation method of reusing based on existing two-dimensional cartoon character video data becomes important, a comprehensive research focus gradually.Although the research about this field still is in the unfashioned stage of exploration at present, has produced some and had very much the method for reference value.

The researcher has developed the synthetic method of some two-dimensional cartoon data reusings.Such as, being published in the method that the paper " Cartoon textures " on the meeting SIGGRAPH in 2004 proposed is, the user must at first provide first frame and the tail frame in the required synthetic new cartoon sequence, whole synthetic process obtains by the calculating of existing cartoon frame is reset, and the user can't be according to the synthetic result of demand control of oneself.Be published in the paper " Motion texture:a two-level statistical modelfor character motion synthesis " on the meeting SIGGRAPH in 2002, the researcher successfully obtains new three-dimensional cartoon motion by reusing the three-dimensional motion data.Above method has excited us to develop a kind of generation method based on two-dimensional character cartoon data reusing.

The two and three dimensions action stylization decomposition of some comparative maturities and the method for migration have been arranged at present, and these methods have been represented present popular stylized generation technique.Such as, be published in the paper " Style machines " on the meeting SIGGRAPH in 2000, utilize Hidden Markov Model (HMM) (HMM) to generate highly nonlinear some conventional action, such as this method as an example, by generating the validity that a series of dance movements with various otherness styles have represented method with dance movement.Be published in the paper " Style translation for human motion " on the periodical ACM ToG, the author has set up linear session variable (LTI) model and has described the exercises imported and the action style of output, and has proposed the different action of aliging of iterative motion distortion (IMW) algorithm.Foregoing these methods mainly are what to be used at the action of the true three-dimension that utilizes the three-dimensional motion capture technique to be obtained under the true environment, be not suitable for this method institute at have a two-dimensional cartoon action data of exaggerating characteristic.

Be published in the paper " Turning to the masters:motioncapturing cartoons " on the meeting SIGGRAPH in 2002, proposed a kind of method of getting in touch between the cartoon style of three-dimensional true action and exaggeration that makes up; The author is by combining the keyframe interpolation technology with the affined transformation technology, the body that will have non-rigid characteristic changes catches, and is redirected to simultaneously on the target body; But this technology still needs the some frames of the manual definition of user oneself to change the key frame of body.Similarly, being published in the paper " Stylizing motion with drawings " on the meeting SCA in 2003 has proposed a kind of with the style generting machanism of 3 D captured data tax with exaggerationization, comprise the conversion of bone and geometry in this generting machanism, thereby formed three-dimensional animation effect.

These years recently, the effective decomposition that utilizes the nonnegative matrix decomposition method to obtain the feature representation formula in conjunction with the L-1 normal form has also obtained researcher's attention, has formed some more representational methods.The paper of on periodical NeuralComputation, delivering " Separating style and content with bilinear models ", propose a kind of facial movement that will be obtained by the facial expression data capture technique, resolved into the method for actuator body and action style by bilinear model (BM).Be published in the paper " Style learning and transferring for facial animation editing " on the meeting SCA in 2009, the author has proposed to learn based on the Gaussian process model (CBGPM) of restriction the edit style of facial movement.More than at the method for facial movement style matrix decomposition, all be as illumination, learn for the influence of style by further investigation towards conversion factors such as, the colours of skin.Be published in the paper " Multifactorgaussian process models for style-content separation " on the meeting ICML in 2007 and proposed a kind of cyclical movement and style thereof of learning three-dimensional true person model based on multi-modal Gaussian process model (MGPM); In fact, this multi-modal Gaussian process model is a kind of version of Gaussian process hidden variable model in essence.Be published in the paper " Face poser:Interactive modeling of 3dfacial expressions using facial priors " on the meeting SIGGRAPH in 2009, the author proposes and has described in detail non-linear decomposition method based on kernel function, and with the expression motion of people's face be decomposed into the action entity and expression characterizes two interrelated factors.

Summary of the invention

The objective of the invention is provides a kind of stylized two-dimensional cartoon generation method of decomposing based on non-negative style in order to overcome the limitation of the two-dimensional cartoon synthetic method of separating with style based on body in the present stylization generation.

The step of the stylized two-dimensional cartoon generation method of decomposing based on non-negative style is as follows:

1) from certain two-dimensional cartoon animated character's two-dimensional cartoon video, extracts the video-frequency band that comprises the complete action of cartoon figure, after the technical finesse of video-frequency band process video image, utilize self-defining extraction method of key frame to extract cartoon figure's two-dimensional cartoon key frame sequence, and two-dimensional cartoon key frame sequence is carried out normalization and centralization processing; Treated two-dimensional cartoon key frame sequence is utilized self-defining feature extracting method, obtain corresponding two-dimentional skeleton character expression formula; Obtain the two-dimentional skeleton character expression formula of several two-dimensional cartoon animated character correspondences, and these two-dimentional skeleton character expression formulas are classified according to self-defining action classification, set up two-dimensional cartoon personage's action database;

2) at certain two-dimensional cartoon animated character's two-dimentional skeleton character expression formula, utilize the objective function and the solution thereof of self-defining non-negative style decomposition, obtain the style base vector and the latent assembly of this two dimension skeleton character expression formula correspondence by the acquiring method of iteration; Utilize acquired latent assembly,, obtain the dimensionality reduction matrix of corresponding maintenance based on semantic understanding by the mode of quadratic function differentiate;

3) specify the two-dimensional cartoon animated character that need generate the two-dimensional cartoon action according to the user, from two-dimensional cartoon animated character's action database, extract corresponding two-dimentional skeleton character expression formula, obtain corresponding latent assembly with non-negative style decomposition method, and overlap with the style base vector that non-negative style decomposition method obtains with other two-dimentional skeleton character expression formula, form the two-dimentional skeleton character expression formula of isomery; With the two-dimentional skeleton character expression formula of isomery according to self-defining unique point driving method, the stylized two-dimensional cartoon that obtains the two-dimensional cartoon key frame of certain two-dimensional cartoon animated character stylization and then obtain to decompose by the key point deformation techniques based on non-negative style.

Described step 1) comprises:

From certain two-dimensional cartoon animated character's two-dimensional cartoon video, extract the video-frequency band V that comprises the complete action of cartoon figure _Cart, right _VcartCarry out Fourier and change noise reduction process, eliminate the influence of background and video noise for later process; From V _CartIn the frame of video playing up out, utilize the Hausdorff distance algorithm to obtain the distance matrix M between the frame and frame in the cartoon video sequence _Cart=R ^{N * n}, wherein n is V _CartThe number of frames of playing up out,

Hausdorff distance in the expression cartoon video frame between i frame and the j frame, matrix M _CartIn each multiply by coefficient respectively

Finish the normalized of Hausdorff distance, d wherein _{Cart_max}Be matrix M _CartIn maximal value, obtaining through normalized M _CartAfterwards, according to preset threshold

Come the diagonal values in the filtered matrix, will obtain respectively

Pairing i frame obtains two-dimensional cartoon key frame sequence thus as key frame

Wherein m is a quantity of key frames;

At the two-dimensional cartoon key frame sequence that has obtained

Definition A _iBe each frame key frame

The corresponding region area that includes two-dimensional cartoon animated character integrity profile, defconstant C _a, for each the frame key frame in the two-dimensional cartoon key frame sequence Obtain the normalization size to C through following formula _aKey frame:

$I_{\mathrm{cart}}^i=I_{\mathrm{cart}}^i\×\frac{A_i}{C_a}---1$

Obtain through normalized key frame sequence by formula 1

At each the frame key frame in the sequence

Utilize the Sobel contour extraction method to obtain its corresponding A _iIn the complete contour edge of two-dimensional cartoon animated character, at t point of contour edge grab sample, calculate this t the point the geometric coordinate center

And the coordinate displacement between this geometric coordinate center and the image center

With A _iAccording to coordinate displacement

Carry out translation, thereby finish

Centralization handle, key frame successively through handling, is obtained the key frame sequence through centralization $\{I_{\mathrm{cart}}^1,I_{\mathrm{cart}}^2,...,I_{\mathrm{cart}}^m\};$

At the two-dimensional cartoon key frame sequence of finishing normalization and centralization

At each frame key frame According to two-dimensional cartoon animated character's limbs sign, to get key point at two-dimensional cartoon animated character's head, four limbs, trunk each several part and amount to 17, the two-dimentional skeleton character that obtains this key frame correspondence is expressed vector Wherein

With

It is key frame

The x of j key point and y coordinate; At two-dimensional cartoon key frame sequence

Obtain the two-dimentional skeleton character expression formula X=[X of its correspondence ₁, X ₂... Xm] ∈ R ^{M * d}, each frame key frame wherein

Two-dimentional skeleton character by correspondence is expressed vectorial X _iRepresent that d is the dimension of feature; Acquisition belongs to different two-dimensional cartoon animated characters' two-dimentional skeleton character expression formula X, forms two-dimentional bone expression formula set

Wherein variable char represents two-dimensional cartoon animated character's kind, and t represents two-dimensional cartoon animated character's quantity; With two-dimentional bone expression formula set

Difference according to the two-dimensional cartoon figure action is divided into the r class, thereby sets up two-dimensional cartoon personage's action database.

Described step 2) comprising:

For certain two-dimensional cartoon animated character's two-dimentional skeleton character expression formula X, carry out non-negative style decomposition and ask for the dimensionality reduction matrix W of maintenance based on semantic understanding according to following objective function:

$C(U,V,W)={\min }_{U,V,W}\frac{1}{2}{\left|\right|X-\mathrm{UV}\left|\right|}_F^2\mathrm{\α}{\mathrm{\Σ}}_{\mathrm{ij}}{V}_{\mathrm{ij}}+\mathrm{\β}{\left|\right|W\tilde{V}-Y\left|\right|}_F^2---2$

Wherein weights α 〉=0 and β 〉=0 is a heterogeneous equilibrium degree coefficient; Matrix U=[u ₁, u ₂..., u _d] ∈ R ^{M * d}With its each column vector as the style base vector, matrix V=[v ₁, v ₂..., v _m] ^T∈ R ^{D * d}Comprise the pairing latent assembly of each style base vector, the constraint condition that will follow in this objective function is U _Ij〉=0 and V _Ij〉=0, and ‖ u _i‖=1; Matrix Y=[y ₁, y ₂..., y _m] ^T∈ 0,1} ^{M * r}Be classified information actual value matrix, r wherein works as X for the quantity of classification _iWhen belonging to the l class, Y so _IlValue be 1, otherwise be 0; Matrix

Be matrix V=[v ₁, v ₂..., v _m] ^TRestructuring matrix, restructuring procedure is as follows: the definition matrix

Set

Be preceding r proper vector of matrix M, then matrix

Be set

Linear combination according to following definition mode:

Wherein

Be the non-negative weights in the linear combination;

Finding the solution optimum U and the solution of V need finish by iteration, and algorithm is as follows:

1. initialization: initial U ⁰And V ⁰, according to the constraint condition of front, U here ⁰And V ⁰Must keep non-negative, simultaneously ‖ u _i‖=1;

2. iteration: iteration U according to the following steps ^tAnd V ^tTo restraining:

a)U′＝U ^t-ε(U ^tV ^t-x)(V ^t) ^T

B) if U ' is arranged _Ij＜0, U ' is set so _Ij=0

C) with ‖ u ' _i‖ zooms to 1, and U is set simultaneously ^T+1=U '

d)V ^t+1＝V ^t.*((U ^t+1) ^TX)./((U ^t+1) ^T(U ^t+1)V ^t+ε)

E) increase t certainly

In the algorithm sign of operation .* and ./represent intermolecular multiplication and intermolecular division respectively.Obtaining optimum V and corresponding restructuring matrix by iterative algorithm

After, by the mode of quadratic function differentiate, obtaining the dimensionality reduction matrix W of corresponding maintenance based on semantic understanding, formula is as follows:

$W=Y{{\tilde{V}}^T\left(\tilde{V}{\tilde{V}}^T\right)}^{-1}---4.$

Described step 3) comprises:

Specify the two-dimensional cartoon animated character char=1 that need generate the two-dimensional cartoon action according to the user, at first according to extracting corresponding two-dimentional skeleton character expression formula X in the described two-dimensional cartoon animated character's of step 1) the action database _Char=1, according to step 2) and the corresponding latent assembly V of described non-negative style decomposition method acquisition _Char=1Should conceal assembly, with other two-dimentional skeleton character expression formula X _Char=2According to step 2) the style base vector U that obtains of described non-negative style decomposition method _Char=2Overlap, form the two-dimentional skeleton character expression formula of isomery ${\tilde{X}}_{\mathrm{char}=1}=U_{\mathrm{char}=2}{V}_{\mathrm{char}=1};$

Obtain consequent two-dimentional skeleton character expression formula

After, promptly obtained previously defined 17 key points and existed

Coordinate position in the defined m frame key frame sequence; Utilize following formula to come to obtain the coordinate position of 17 key points in complete n frame sequence based on the coordinate position of 17 key points in the m frame key frame:

$v_i^{t+1}=v_i^t+\frac{\left|\right|v_i^1-v_i^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_i+\frac{\left|\right|v_{i+1}^1-v_{i+1}^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_{i+1---5}$

$v_j^{t+1}=v_j^t+\frac{\left|\right|v_i^1-v_i^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_i+\frac{\left|\right|v_{i+1}^1-v_{i+1}^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_{i+1}---6$

Wherein

With

Be i and the coordinate position of j key point in the non-key frame of t frame, t=[1 wherein, 2 ..., (n-m)] and be the index value of all non-key frame;

Be the coordinate displacement absolute value of i key point between t and the non-key frame of t+1 frame, in like manner

Be the coordinate displacement absolute value of j key point between t and the non-key frame of t+1 frame; Thus, obtain the coordinate position of 17 key points in complete n frame sequence;

Adopt the constant deformation method of existing maintenance rigidity, with the drive point of 17 key points as deformation, utilize the drive source of the coordinate position of these key points in complete n frame sequence, thereby carry out the stylized two-dimensional cartoon that the deformation acquisition is decomposed based on non-negative style as deformation.

The present invention has reduced the performance difficulty that classic method is brought by the style and the body factor of machine learning model micromotion by reusing that existed, abundant two-dimensional cartoon animated video; By proposing the method based on the stylized two-dimensional cartoon generation of non-negative style decomposition, having solved classic method can only decompose the narrow problem of bringing of applicability at three-dimensional real motion simultaneously, has improved accuracy, has enlarged the scope of using.

Description of drawings

The present invention is further illustrated below in conjunction with the drawings and specific embodiments.

Fig. 1 is a method system flowchart of the present invention.

Embodiment

The step of the stylized two-dimensional cartoon generation method of decomposing based on non-negative style is as follows:

1) from certain two-dimensional cartoon animated character's two-dimensional cartoon video, extracts the video-frequency band that comprises the complete action of cartoon figure, after the technical finesse of video-frequency band process video image, utilize self-defining extraction method of key frame to extract cartoon figure's two-dimensional cartoon key frame sequence, and two-dimensional cartoon key frame sequence is carried out normalization and centralization processing; Treated two-dimensional cartoon key frame sequence is utilized self-defining feature extracting method, obtain corresponding two-dimentional skeleton character expression formula; Obtain the two-dimentional skeleton character expression formula of several two-dimensional cartoon animated character correspondences, and these two-dimentional skeleton character expression formulas are classified according to self-defining action classification, set up two-dimensional cartoon personage's action database;

2) at certain two-dimensional cartoon animated character's two-dimentional skeleton character expression formula, utilize the objective function and the solution thereof of self-defining non-negative style decomposition, obtain the style base vector and the latent assembly of this two dimension skeleton character expression formula correspondence by the acquiring method of iteration; Utilize acquired latent assembly,, obtain the dimensionality reduction matrix of corresponding maintenance based on semantic understanding by the mode of quadratic function differentiate;

3) specify the two-dimensional cartoon animated character that need generate the two-dimensional cartoon action according to the user, from two-dimensional cartoon animated character's action database, extract corresponding two-dimentional skeleton character expression formula, obtain corresponding latent assembly with non-negative style decomposition method, and overlap with the style base vector that non-negative style decomposition method obtains with other two-dimentional skeleton character expression formula, form the two-dimentional skeleton character expression formula of isomery; With the two-dimentional skeleton character expression formula of isomery according to self-defining unique point driving method, the stylized two-dimensional cartoon that obtains the two-dimensional cartoon key frame of certain two-dimensional cartoon animated character stylization and then obtain to decompose by the key point deformation techniques based on non-negative style.

Described step 1) comprises:

From certain two-dimensional cartoon animated character's two-dimensional cartoon video, extract the video-frequency band V that comprises the complete action of cartoon figure _Cart, to V _CartCarry out Fourier and change noise reduction process, eliminate the influence of background and video noise for later process; From V _CartIn the frame of video playing up out, utilize the Hausdorff distance algorithm to obtain the distance matrix M between the frame and frame in the cartoon video sequence _Cart=R ^{N * n}, wherein n is V _CartThe number of frames of playing up out,

Hausdorff distance in the expression cartoon video frame between i frame and the j frame, matrix M _CartIn each multiply by coefficient respectively

Finish the normalized of Hausdorff distance, d wherein _{Cart_max}Be matrix M _CartIn maximal value, obtaining through normalized M _CartAfterwards, according to preset threshold

Come the diagonal values in the filtered matrix, will obtain respectively

Pairing i frame obtains two-dimensional cartoon key frame sequence thus as key frame

Wherein m is a quantity of key frames;

At the two-dimensional cartoon key frame sequence that has obtained

Definition A _iBe each frame key frame

The corresponding region area that includes two-dimensional cartoon animated character integrity profile, defconstant C _a, for each the frame key frame in the two-dimensional cartoon key frame sequence

Obtain the normalization size to C through following formula _aKey frame:

$I_{\mathrm{cart}}^i=I_{\mathrm{cart}}^i\×\frac{A_i}{C_a}---1$

Obtain through normalized key frame sequence by formula 1

At each the frame key frame in the sequence

Utilize the Sobel contour extraction method to obtain its corresponding A _iIn the complete contour edge of two-dimensional cartoon animated character, at t point of contour edge grab sample, calculate this t the point the geometric coordinate center

And the coordinate displacement between this geometric coordinate center and the image center

With A _iAccording to coordinate displacement

Carry out translation, thereby finish

Centralization handle, key frame successively through handling, is obtained the key frame sequence through centralization $\{I_{\mathrm{cart}}^1,I_{\mathrm{cart}}^2,...,I_{\mathrm{cart}}^m\};$

At the two-dimensional cartoon key frame sequence of finishing normalization and centralization

At each frame key frame According to two-dimensional cartoon animated character's limbs sign, to get key point at two-dimensional cartoon animated character's head, four limbs, trunk each several part and amount to 17, the two-dimentional skeleton character that obtains this key frame correspondence is expressed vector

Wherein

With

It is key frame

The x of j key point and y coordinate; At two-dimensional cartoon key frame sequence

Obtain the two-dimentional skeleton character expression formula X=[X of its correspondence ₁, X ₂... X _m] ∈ R ^{M * d}, each frame key frame wherein Two-dimentional skeleton character by correspondence is expressed vectorial X _iRepresent that d is the dimension of feature; Acquisition belongs to different two-dimensional cartoon animated characters' two-dimentional skeleton character expression formula X, forms two-dimentional bone expression formula set

Wherein variable char represents two-dimensional cartoon animated character's kind, and t represents two-dimensional cartoon animated character's quantity; With two-dimentional bone expression formula set

Difference according to the two-dimensional cartoon figure action is divided into the r class, thereby sets up two-dimensional cartoon personage's action database.

Described step 2) comprising:

For certain two-dimensional cartoon animated character's two-dimentional skeleton character expression formula X, carry out non-negative style decomposition and ask for the dimensionality reduction matrix W of maintenance based on semantic understanding according to following objective function:

$C(U,V,W)={\min }_{U,V,W}\frac{1}{2}{\left|\right|X-\mathrm{UV}\left|\right|}_F^2+\mathrm{\α}{\mathrm{\Σ}}_{\mathrm{ij}}{V}_{\mathrm{ij}}+\mathrm{\β}{\left|\right|W\tilde{V}-Y\left|\right|}_F^2---2$

Wherein weights α 〉=0 and β 〉=0 is a heterogeneous equilibrium degree coefficient; Matrix U=[u ₁, u ₂..., u _d] ∈ R ^{M * d}With its each column vector as the style base vector, matrix V=[v ₁, v ₂..., v _m] ^T∈ R ^{D * d}Comprise the pairing latent assembly of each style base vector, the constraint condition that will follow in this objective function is U _Ij〉=0 and V _Ij〉=0, and ‖ u _i‖=1; Matrix Y=[y ₁, y ₂..., y _m] ^T∈ 0,1} ^{M * r}Be classified information actual value matrix, r wherein works as X for the quantity of classification _iWhen belonging to the l class, Y so _IlValue be 1, otherwise be 0;

Matrix

Be matrix V=[v ₁, v ₂..., v _m] ^TRestructuring matrix, restructuring procedure is as follows: the definition matrix

Set

Be preceding r proper vector of matrix M, then matrix

Be set

Linear combination according to following definition mode:

Wherein

Be the non-negative weights in the linear combination;

Finding the solution optimum U and the solution of V need finish by iteration, and algorithm is as follows:

1. initialization: initial U ⁰And V ⁰, according to the constraint condition of front, U here ⁰And V ⁰Must keep non-negative, simultaneously ‖ u _i‖=1;

2. iteration: iteration U according to the following steps ^tAnd V ^tTo restraining:

a)U′＝U ^t-ε(U ^tV ^t-x)(V ^t) ^T

B) if U ' is arranged _Ij＜0, U ' is set so _Ij=0

C) with ‖ u ' _i‖ zooms to 1, and U is set simultaneously ^T+1=U '

d)V ^t+1＝V ^t.*((U ^t+1) ^TX)./((U ^t+1) ^T(U ^t+1)V ^t+ε)

E) increase t certainly

In the algorithm sign of operation .* and ./represent intermolecular multiplication and intermolecular division respectively.Obtaining optimum V and corresponding restructuring matrix by iterative algorithm After, by the mode of quadratic function differentiate, obtaining the dimensionality reduction matrix W of corresponding maintenance based on semantic understanding, formula is as follows:

$W=Y{\tilde{V}}^T{\left(\tilde{V}{\tilde{V}}^T\right)}^{-1}---4.$

Described step 3) comprises:

Specify the two-dimensional cartoon animated character char=1 that need generate the two-dimensional cartoon action according to the user, at first according to extracting corresponding two-dimentional skeleton character expression formula X in the described two-dimensional cartoon animated character's of step 1) the action database _Char=1, according to step 2) and the corresponding latent assembly V of described non-negative style decomposition method acquisition _Char=1Should conceal assembly, with other two-dimentional skeleton character expression formula X _Char=2According to step 2) the style base vector U that obtains of described non-negative style decomposition method _Char=2Overlap, form the two-dimentional skeleton character expression formula of isomery ${\tilde{X}}_{\mathrm{char}=1}=U_{\mathrm{char}=2}{V}_{\mathrm{char}=1};$

Obtain consequent two-dimentional skeleton character expression formula

After, promptly obtained previously defined 17 key points and existed

Coordinate position in the defined m frame key frame sequence; Utilize following formula to come to obtain the coordinate position of 17 key points in complete n frame sequence based on the coordinate position of 17 key points in the m frame key frame:

$v_i^{t+1}=v_i^t+\frac{\left|\right|v_i^1-v_i^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_i+\frac{\left|\right|v_{i+1}^1-v_{i+1}^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_{i+1}---5$

$v_j^{t+1}=v_j^t+\frac{\left|\right|v_i^1-v_i^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_i+\frac{\left|\right|v_{i+1}^1-v_{i+1}^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_{i+1}---6$

Wherein With

Be i and the coordinate position of j key point in the non-key frame of t frame, t=[1 wherein, 2 ..., (n-m)] and be the index value of all non-key frame;

Be the coordinate displacement absolute value of i key point between t and the non-key frame of t+1 frame, in like manner

Be the coordinate displacement absolute value of j key point between t and the non-key frame of t+1 frame; Thus, obtain the coordinate position of 17 key points in complete n frame sequence;

Adopt the constant deformation method of existing maintenance rigidity, with the drive point of 17 key points as deformation, utilize the drive source of the coordinate position of these key points in complete n frame sequence, thereby carry out the stylized two-dimensional cartoon that the deformation acquisition is decomposed based on non-negative style as deformation.

Embodiment

1) Fig. 1 has showed the system flowchart of the stylized two-dimensional cartoon generation method of decomposing based on non-negative style.At first, from certain two-dimensional cartoon animated character's two-dimensional cartoon video, in the animated video in " Sun Wukong creates a tremendous uproar ", extract the video-frequency band V that comprises the complete action of cartoon figure _CartIn the present embodiment, video-frequency band V _CartTime span be 6 minutes and 22 seconds, per second 20 frames amount to 7640 frames, the resolution of each frame is the 640*488 pixel.To V _CartCarry out Fourier and change noise reduction process, eliminate the influence of background and video noise for later process; From V _CartIn the frame of video playing up out, utilize the Hausdorff distance algorithm to obtain the distance matrix M between the frame and frame in the cartoon video sequence _Cart=R ^{N * n}, wherein n is V _CartThe number of frames of playing up out is 7640 in the present embodiment;

Hausdorff distance in the expression cartoon video frame between i frame and the j frame, matrix M _CartIn each multiply by coefficient respectively

Finish the normalized of Hausdorff distance, d wherein _{Cart_max}Be matrix M _CartIn maximal value, obtaining through normalized M _CartAfterwards, according to preset threshold Come the diagonal values in the filtered matrix, threshold value is found through the overtesting comparison in the present embodiment, is made as

Be optimal value; To obtain respectively

Pairing i frame obtains two-dimensional cartoon key frame sequence thus as key frame Wherein m is quantity of key frames, m=1029 in the present embodiment;

At the two-dimensional cartoon key frame sequence that has obtained Definition A _iBe each frame key frame

The corresponding region area that includes two-dimensional cartoon animated character integrity profile, defconstant C _aAccording to testing for cartoon figure Sun Wukong's size in the present embodiment, find on average in every frame, the shared image magnitude proportion of Sun Wukong to be controlled at 50% and to obtain processing speed and the balance between the precision as a result, so establish constant C here _a=329; For each the frame key frame in the two-dimensional cartoon key frame sequence

Obtain the normalization size to C through following formula _aKey frame:

$I_{\mathrm{cart}}^i=I_{\mathrm{cart}}^i\×\frac{A_i}{C_a}---1$

Obtain through normalized key frame sequence by formula 1 At each the frame key frame in the sequence

Utilize the Sobel contour extraction method to obtain its corresponding A _iIn the complete contour edge of two-dimensional cartoon animated character, at t=200 point of contour edge grab sample, the geometric coordinate center of calculating these 200 points

And the coordinate displacement between this geometric coordinate center and the image center

With A _iAccording to coordinate displacement

Carry out translation, thereby finish

Centralization handle.Key frame successively through handling, is obtained the key frame sequence through centralization $\{I_{\mathrm{cart}}^1,I_{\mathrm{cart}}^2,...,I_{\mathrm{cart}}^m\};$

At the two-dimensional cartoon key frame sequence of finishing normalization and centralization

At each frame key frame

According to two-dimensional cartoon animated character's limbs sign, to get key point at two-dimensional cartoon animated character's head, four limbs, trunk each several part and amount to 17, the two-dimentional skeleton character that obtains this key frame correspondence is expressed vector

Wherein

With

It is key frame

The x of j key point and y coordinate; At two-dimensional cartoon key frame sequence

Obtain the two-dimentional skeleton character expression formula X=[X of its correspondence ₁, X ₂... X _m] ∈ R ^{M * d}, each frame key frame wherein

Two-dimentional skeleton character by correspondence is expressed vectorial X _iRepresent that d is the dimension of feature, in the present embodiment, d=34 is the number that doubles key point; Acquisition belongs to different two-dimensional cartoon animated characters' two-dimentional skeleton character expression formula X, forms two-dimentional bone expression formula set

Wherein variable char represents two-dimensional cartoon animated character's kind, and t represents two-dimensional cartoon animated character's quantity; In the present embodiment, respectively from two-dimensional cartoon cartoon " Sun Wukong creates a tremendous uproar ", extract the pairing two-dimentional bone expression formula set { X of two-dimensional cartoon cartoon role Sun Wukong, Nezha and Er-Lang god in " Triumph of Nezha Against Dragon King ", " precious lotus lamp " _{Char=Sun Wukong}, X _{The char=Nezha}, X _{The char=Er-Lang god}, so t=3; With two-dimentional bone expression formula set Difference according to the two-dimensional cartoon figure action is divided into the r class, thereby sets up two-dimensional cartoon personage's action database; In the present embodiment,, the common action in the two-dimensional cartoon animation is divided for the r=9 kind, wherein comprise " walking ", " race ", " jumping ", elemental motions such as " stretching, extensions " according to the semanteme and the form difference of action.

2) for certain two-dimensional cartoon animated character's two-dimentional skeleton character expression formula X, carry out non-negative style decomposition and ask for the dimensionality reduction matrix W of maintenance based on semantic understanding according to following objective function:

$C(U,V,W)={\min }_{U,V,W}\frac{1}{2}{\left|\right|X-\mathrm{UV}\left|\right|}_F^2+\mathrm{\α}{\mathrm{\Σ}}_{\mathrm{ij}}{V}_{\mathrm{ij}}+\mathrm{\β}{\left|\right|W\tilde{V}-Y\left|\right|}_F^2---2$

Wherein weights α 〉=0 and β 〉=0 is a heterogeneous equilibrium degree coefficient; Matrix U=[u ₁, u ₂..., u _d] ∈ R ^{M * d}With its each column vector as the style base vector, matrix V=[v ₁, v ₂..., v _m] ^T∈ R ^{D * d}Comprise the pairing latent assembly of each style base vector, the constraint condition that will follow in this objective function is U _Ij〉=0 and V _Ij〉=0, and ‖ u _i‖=1; Matrix Y=[y ₁, y ₂..., y _m] ^T∈ 0,1} ^{M * r}Be classified information actual value matrix, r wherein works as X for the quantity of classification _iWhen belonging to the l class, Y so _IlValue be 1, otherwise be 0;

Matrix

Be matrix V=[v ₁, v ₂..., v _m] ^TRestructuring matrix, restructuring procedure is as follows: the definition matrix

Set

Be preceding r proper vector of matrix M, then matrix

Be set

Linear combination according to following definition mode:

Wherein

Be the non-negative weights in the linear combination; In the present embodiment, weights combination

Concrete value obtain by experimental comparison repeatedly, according to the difference of two-dimensional cartoon cartoon role, concrete value has adjusts the effect that reaches optimum.

Finding the solution optimum U and the solution of V need finish by iteration, and algorithm is as follows:

1. initialization: initial U ⁰And V ⁰, according to the constraint condition of front, U here ⁰And V ⁰Must keep non-negative, simultaneously ‖ u _i‖=1.

2. iteration: iteration U according to the following steps ^tAnd V ^tTo restraining:

a)U′＝U ^t-ε(U ^tV ^t-X)(V ^t) ^T

B) if U ' is arranged _Ij＜0, U ' is set so _Ij=0

C) with ‖ u ' _i‖ zooms to 1, and U is set simultaneously ^T+1=U '

d)V ^t+1＝V ^t.*((U ^t+1) ^TX)./((U ^t+1) ^T(U ^t+1)V ^t+ε)

E) increase t certainly

In the algorithm sign of operation .* and ./represent intermolecular multiplication and intermolecular division respectively.Obtaining optimum V and corresponding restructuring matrix by iterative algorithm

After, by the mode of quadratic function differentiate, obtaining the dimensionality reduction matrix W of corresponding maintenance based on semantic understanding, formula is as follows:

$W=Y{\tilde{V}}^T{\left(\tilde{V}{\tilde{V}}^T\right)}^{-1}---4$

3) specify the two-dimensional cartoon animated character char=1 that need generate the two-dimensional cartoon action according to the user, at first according to extracting corresponding two-dimentional skeleton character expression formula X in the described two-dimensional cartoon animated character's of step 1) the action database _Char=1, according to step 2) and the corresponding latent assembly V of described non-negative style decomposition method acquisition _Char=1Should conceal assembly, with other two-dimentional skeleton character expression formula X _Char=2According to step 2) the style base vector U that obtains of described non-negative style decomposition method _Char=2Overlap, form the two-dimentional skeleton character expression formula of isomery

Obtain consequent two-dimentional skeleton character expression formula

After, promptly obtained previously defined 17 key points and existed

Coordinate position in the defined m frame key frame sequence; Utilize following formula to come to obtain the coordinate position of 17 key points in complete n frame sequence based on the coordinate position of 17 key points in the m frame key frame:

$v_i^{t+1}=v_i^t+\frac{\left|\right|v_i^1-v_i^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_i+\frac{\left|\right|v_{i+1}^1-v_{i+1}^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_{i+1}---5$

$v_j^{t+1}=v_j^t+\frac{\left|\right|v_i^1-v_i^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_i+\frac{\left|\right|v_{i+1}^1-v_{i+1}^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_{i+1}---6$

Wherein With

Be i and the coordinate position of j key point in the non-key frame of t frame, t=[1 wherein, 2 ..., (n-m)] and be the index value of all non-key frame;

Be the coordinate displacement absolute value of i key point between t and the non-key frame of t+1 frame, in like manner

Be the coordinate displacement absolute value of j key point between t and the non-key frame of t+1 frame; Thus, obtain the coordinate position of 17 key points in complete n frame sequence;

Adopt the deformation method of existing maintenance rigidity constant (As-rigid-as-possible); In the present embodiment, at first by the two-dimensional cartoon animated character's that will generate in animation Shi Shouhui first frame initial configuration, then this initial configuration is carried out the trigonometric ratio dough sheet and handle, then the position of on the form that completed dough sheet is handled, initially formulating 17 key points; With the drive point of 17 key points, utilize the drive source of the coordinate position of these key points in complete n frame sequence, thereby carry out the stylized two-dimensional cartoon that the deformation acquisition is decomposed based on non-negative style as deformation as deformation.

Claims (4)

1. stylized two-dimensional cartoon generation method of decomposing based on non-negative style is characterized in that its step is as follows:

1) from certain two-dimensional cartoon animated character's two-dimensional cartoon video, extracts the video-frequency band that comprises the complete action of cartoon figure, after the technical finesse of video-frequency band process video image, utilize self-defining extraction method of key frame to extract cartoon figure's two-dimensional cartoon key frame sequence, and two-dimensional cartoon key frame sequence is carried out normalization and centralization processing; Treated two-dimensional cartoon key frame sequence is utilized self-defining feature extracting method, obtain corresponding two-dimentional skeleton character expression formula; Obtain the two-dimentional skeleton character expression formula of several two-dimensional cartoon animated character correspondences, and these two-dimentional skeleton character expression formulas are classified according to self-defining action classification, set up two-dimensional cartoon personage's action database;

2) at certain two-dimensional cartoon animated character's two-dimentional skeleton character expression formula, utilize the objective function and the solution thereof of self-defining non-negative style decomposition, obtain the style base vector and the latent assembly of this two dimension skeleton character expression formula correspondence by the acquiring method of iteration; Utilize acquired latent assembly,, obtain the dimensionality reduction matrix of corresponding maintenance based on semantic understanding by the mode of quadratic function differentiate;

3) specify the two-dimensional cartoon animated character that need generate the two-dimensional cartoon action according to the user, from two-dimensional cartoon animated character's action database, extract corresponding two-dimentional skeleton character expression formula, obtain corresponding latent assembly with non-negative style decomposition method, and overlap with the style base vector that non-negative style decomposition method obtains with other two-dimentional skeleton character expression formula, form the two-dimentional skeleton character expression formula of isomery; With the two-dimentional skeleton character expression formula of isomery according to self-defining unique point driving method, the stylized two-dimensional cartoon that obtains the two-dimensional cartoon key frame of certain two-dimensional cartoon animated character stylization and then obtain to decompose by the key point deformation techniques based on non-negative style.

2. a kind of stylized two-dimensional cartoon generation method of decomposing according to claim 1 based on non-negative style, it is characterized in that: described step 1) comprises:

From certain two-dimensional cartoon animated character's two-dimensional cartoon video, extract the video-frequency band V that comprises the complete action of cartoon figure _Cart, to V _CartCarry out Fourier and change noise reduction process, eliminate the influence of background and video noise for later process; From V _CartIn the frame of video playing up out, utilize the Hausdorff distance algorithm to obtain the distance matrix M between the frame and frame in the cartoon video sequence _Cart=R ^{N * n}, wherein n is V _CartThe number of frames of playing up out, Hausdorff distance in the expression cartoon video frame between i frame and the j frame, matrix M _CartIn each multiply by coefficient respectively Finish the normalized of Hausdorff distance, d wherein _{Cart_max}Be matrix M _CartIn maximal value, obtaining through normalized M _CartAfterwards, according to preset threshold

Come the diagonal values in the filtered matrix, will obtain respectively

Pairing i frame obtains two-dimensional cartoon key frame sequence thus as key frame

Wherein m is a quantity of key frames;

At the two-dimensional cartoon key frame sequence that has obtained

Definition A _iBe each frame key frame

The corresponding region area that includes two-dimensional cartoon animated character integrity profile, defconstant C _a, for each the frame key frame in the two-dimensional cartoon key frame sequence

Obtain the normalization size to C through following formula _aKey frame:

$I_{\mathrm{cart}}^i=I_{\mathrm{cart}}^i\×\frac{A_i}{C_a}---1$

Obtain through normalized key frame sequence by formula 1

At each the frame key frame in the sequence

Utilize the Sobel contour extraction method to obtain its corresponding A _iIn the complete contour edge of two-dimensional cartoon animated character, at t point of contour edge grab sample, calculate this t the point the geometric coordinate center

And the coordinate displacement between this geometric coordinate center and the image center

With A _iAccording to coordinate displacement Carry out translation, thereby finish

Centralization handle, key frame successively through handling, is obtained the key frame sequence through centralization $\{I_{\mathrm{cart}}^1,I_{\mathrm{cart}}^2,...,I_{\mathrm{cart}}^m\};$

At the two-dimensional cartoon key frame sequence of finishing normalization and centralization

At each frame key frame

According to two-dimensional cartoon animated character's limbs sign, to get key point at two-dimensional cartoon animated character's head, four limbs, trunk each several part and amount to 17, the two-dimentional skeleton character that obtains this key frame correspondence is expressed vector

Wherein With

It is key frame

The x of j key point and y coordinate; At two-dimensional cartoon key frame sequence Obtain the two-dimentional skeleton character expression formula X=[X of its correspondence ₁, X ₂... X _m] ∈ R ^{M * d}, each frame key frame wherein

Two-dimentional skeleton character by correspondence is expressed vectorial X _iRepresent that d is the dimension of feature; Acquisition belongs to different two-dimensional cartoon animated characters' two-dimentional skeleton character expression formula X, forms two-dimentional bone expression formula set

Wherein variable char represents two-dimensional cartoon animated character's kind, and t represents two-dimensional cartoon animated character's quantity; With two-dimentional bone expression formula set

Difference according to the two-dimensional cartoon figure action is divided into the r class, thereby sets up two-dimensional cartoon personage's action database.

3. a kind of stylized two-dimensional cartoon generation method of decomposing based on non-negative style according to claim 1 is characterized in that: described step 2) comprising:

For certain two-dimensional cartoon animated character's two-dimentional skeleton character expression formula X, carry out non-negative style decomposition and ask for the dimensionality reduction matrix W of maintenance based on semantic understanding according to following objective function:

$C(U,V,W)={\min }_{U,V,W}\frac{1}{2}{\left|\right|X-\mathrm{UV}\left|\right|}_F^2\mathrm{\α}{\mathrm{\Σ}}_{\mathrm{ij}}{V}_{\mathrm{ij}}+\mathrm{\β}{\left|\right|W\widetilde{V-Y}\left|\right|}_F^2---2$

Wherein weights α 〉=0 and β 〉=0 is a heterogeneous equilibrium degree coefficient; Matrix U=[u ₁, u ₂..., u _d] ∈ R ^{M * d}With its each column vector as the style base vector, matrix V=[v ₁, v ₂..., v _m] ^T∈ R ^{D * d}Comprise the pairing latent assembly of each style base vector, the constraint condition that will follow in this objective function is U _Ij〉=0 and V _Ij〉=0, and ‖ u _i‖=1; Matrix Y=[y ₁, y ₂..., y _m] ^T∈ 0,1} ^{M * r}Be classified information actual value matrix, r is for the quantity of classification, wherein when Xi belongs to the l class, and Y so _IlValue be 1, otherwise be 0;

Matrix

Be matrix V=[v ₁, v ₂..., v _m] ^TRestructuring matrix, restructuring procedure is as follows: the definition matrix

Set

Be preceding r proper vector of matrix M, then matrix

Be set

Linear combination according to following definition mode:

Wherein Be the non-negative weights in the linear combination;

Finding the solution optimum U and the solution of V need finish by iteration, and algorithm is as follows:

1. initialization: initial U ⁰And V ⁰, according to the constraint condition of front, U here ⁰And V ⁰Must keep non-negative, simultaneously ‖ u _i‖=1;

2. iteration: iteration U according to the following steps ^tAnd V ^tTo restraining:

a)U′＝U ^t-ε(U ^tV ^t-X)(V ^t) ^T

B) if U ' is arranged _Ij＜0, U ' is set so _Ij=0

C) with ‖ u ' _i‖ zooms to 1, and U is set simultaneously ^T+1=U '

d)V ^t+1＝V ^t.*((U ^t+1) ^TX)./((U ^t+1) ^T(U ^t+1)V ^t+ε)

E) increase t certainly

In the algorithm sign of operation .* and ./represent intermolecular multiplication and intermolecular division respectively.Obtaining optimum V and corresponding restructuring matrix by iterative algorithm

After, by the mode of quadratic function differentiate, obtaining the dimensionality reduction matrix W of corresponding maintenance based on semantic understanding, formula is as follows:

$W={\tilde{V}}^T{\left(\tilde{V}{\tilde{V}}^T\right)}^{-1}---4.$

4. a kind of stylized two-dimensional cartoon generation method of decomposing according to claim 1 based on non-negative style, it is characterized in that: described step 3) comprises:

Specify the two-dimensional cartoon animated character char=1 that need generate the two-dimensional cartoon action according to the user, at first according to extracting corresponding two-dimentional skeleton character expression formula X in the described two-dimensional cartoon animated character's of step 1) the action database _Char=1, according to step 2) and the corresponding latent assembly V of described non-negative style decomposition method acquisition _Char=1Should conceal assembly, with other two-dimentional skeleton character expression formula X _Char=2According to step 2) the style base vector U that obtains of described non-negative style decomposition method _Char=2Overlap, form the two-dimentional skeleton character expression formula of isomery

${\tilde{X}}_{\mathrm{char}=1}=U_{\mathrm{char}=2}{V}_{\mathrm{char}=1};$

Obtain consequent two-dimentional skeleton character expression formula

After, promptly obtained previously defined 17 key points and existed

Coordinate position in the defined m frame key frame sequence; Utilize following formula to come to obtain the coordinate position of 17 key points in complete n frame sequence based on the coordinate position of 17 key points in the m frame key frame:

$v_i^{t+1}=v_i^t+\frac{\left|\right|v_i^1-v_i^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{V}_i+\frac{\left|\right|v_{i+1}^1-v_{i+1}^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_{i+1}---5$

$v_j^{t+1}=v_j^t+\frac{\left|\right|v_i^1-v_i^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{V}_i+\frac{\left|\right|v_{i+1}^1-v_{i+1}^t\left|\right|}{\left|\right|v_i^1-v_{i+1}^1\left|\right|}\mathrm{\Δ}{v}_{i+1}---6$

Wherein With

Be i and the coordinate position of j key point in the non-key frame of t frame, t=[1 wherein, 2 ..., (n-m)] and be the index value of all non-key frame;

Be the coordinate displacement absolute value of i key point between t and the non-key frame of t+1 frame, in like manner

Be the coordinate displacement absolute value of j key point between t and the non-key frame of t+1 frame; Thus, obtain the coordinate position of 17 key points in complete n frame sequence;

Adopt the constant deformation method of existing maintenance rigidity, with the drive point of 17 key points as deformation, utilize the drive source of the coordinate position of these key points in complete n frame sequence, thereby carry out the stylized two-dimensional cartoon that the deformation acquisition is decomposed based on non-negative style as deformation.

Images