CN116310015A - Computer system, method and medium - Google Patents

Computer system, method and medium Download PDF

Info

Publication number
CN116310015A
CN116310015A CN202310246436.8A CN202310246436A CN116310015A CN 116310015 A CN116310015 A CN 116310015A CN 202310246436 A CN202310246436 A CN 202310246436A CN 116310015 A CN116310015 A CN 116310015A
Authority
CN
China
Prior art keywords
key frame
sequence
person
distance
character
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310246436.8A
Other languages
Chinese (zh)
Inventor
张刘灿
丁佳伟
黄鹏
于秋燕
张翔
戚欠礼
明小凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Ruoxi Enterprise Management Co ltd
Original Assignee
Hangzhou Ruoxi Enterprise Management Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Ruoxi Enterprise Management Co ltd filed Critical Hangzhou Ruoxi Enterprise Management Co ltd
Priority to CN202310246436.8A priority Critical patent/CN116310015A/en
Publication of CN116310015A publication Critical patent/CN116310015A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00Animation
    • G06T13/802D [Two Dimensional] animation, e.g. using sprites
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761Proximity, similarity or dissimilarity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/40Scenes; Scene-specific elements in video content
    • G06V20/46Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/40Scenes; Scene-specific elements in video content
    • G06V20/48Matching video sequences
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention relates to the technical field of cartoon making, and discloses a computer system, which comprises: a frame image extraction module for extracting a frame image from the three-dimensional animated video; a person recognition module for recognizing a person from the frame image and acquiring person pose data; the key frame extraction module is used for screening and obtaining a key frame sequence from the frame image; the gesture sequence correction module firstly extracts a key frame sequence corresponding to the first person and then marks the key frame; a reference sequence correction module for marking key frames of a key frame sequence of a third person; a key frame sequence synthesis module for extracting a reference key frame sequence; synthesizing each reference key frame sequence with the key frame sequence of the first person respectively; the invention can map the reference image for the key frame which needs to be hand painted in the 2D animation 2D process based on image processing and data mining, and provides a reference with higher matching degree for hand painted personnel.

Description

Computer system, method and medium
Technical Field
The invention relates to the technical field of cartoon making, in particular to a computer system.
Background
In the process of 2D animation, edge processing and color processing are performed on the images through deep learning, if the style of the 2D animation is to be reflected, part of frame images need to be replaced by frame images generated based on hand-painted key frames, and therefore the capability of hand-painted staff to draw super-reality character actions among real actions is required, and the requirement on artistic creation capability of the hand-painted staff is high.
Disclosure of Invention
The invention provides a computer system which solves the technical problem of XXXXX in the related technology.
The present invention provides a computer system comprising:
the frame image extraction module is used for extracting a first frame image from the three-dimensional animation video and extracting a second frame image from the two-dimensional animation video; a person recognition module for recognizing a person from the frame image and acquiring person pose data; the information of the person includes the sex of the person, and the person posture data includes person posture parameters; a character matching module for selecting a first character extracted from the first frame image and then matching a second character extracted from the second frame image, the first character having the same information as the second character, the matched second character being marked as a third character; the key frame extraction module is used for screening and obtaining a key frame sequence from the frame image;
the gesture sequence correction module firstly extracts a key frame sequence corresponding to a first person, marks the key frame, wherein the mark value of the key frame to be redrawn is 0, and the mark values of the other key frames are 1;
the reference sequence correction module is used for extracting a key frame sequence corresponding to the third person and then marking key frames of the key frame sequence of the third person;
the key frame sequence synthesis module is used for calculating the reference distances of the key frame sequences of the first person and the second person, and extracting the key frame sequences of the first N second persons with the smallest reference distances of the key frame sequences of the first person as reference key frame sequences; each reference key frame sequence is synthesized with the key frame sequence of the first person.
Further, the method for screening and obtaining the key frame from the frame image comprises the following steps:
step 101, extracting character gesture parameters of a selected character from each frame image to generate a character gesture sequence, wherein each unit of the character gesture sequence comprises the character gesture parameters of the selected character in one frame image;
step 102, traversing backwards from the sequence unit of the character gesture sequence until the first similarity between the traversed sequence unit and the sequence unit from which the traversing starts is smaller than a set first similarity threshold; marking the sequence unit of traversal termination as a first sequence unit;
step 103, iteratively executing step 102 until all sequence units of the character gesture sequence are traversed, wherein the first sequence unit of the character gesture sequence is traversed in the first execution, and the first sequence unit generated in the previous execution is traversed backwards in each subsequent execution;
104, extracting all the generated first sequence units to generate a first human gesture sequence;
step 105, extracting a frame image of the character pose parameter source of the first sequence unit as a key frame, and ordering the key frame according to the time point to generate a key frame sequence.
Further, the calculation formula of the first similarity is as follows:
Figure BDA0004126090470000021
wherein k is human boneNumber of limbs, θ of the stent model i1 An included angle between the ith limb of the character gesture parameter of a sequence unit of a character gesture sequence and the ith limb in the standard gesture model; θ i2 And the included angle between the ith limb of the character gesture parameter of the sequence unit of the other character gesture sequence and the ith limb in the standard gesture model.
Further, the method of keyframe tagging of the keyframe sequence of the third person comprises:
defining a two-dimensional random field, wherein a node set of the two-dimensional random field is denoted as V, and a set of edges between the nodes is denoted as E;
the energy function of the two-dimensional random field is:
Figure BDA0004126090470000031
wherein X is a node set, θ p (x p ) As a potential function of node p, θ pq (x p ,x q ) As a potential function of the edge between nodes p and q, x p The marker value, x, of the image node mapped for node p q A marker value of a key frame sequence mapped for the node q; a mark value of 0 indicates that the character gesture parameter corresponding to the key frame has larger deviation from the real character; a mark value of 1 indicates that the character gesture parameter corresponding to the key frame is close to the real character;
wherein,,
Figure BDA0004126090470000032
simila p,q represents x p And x q A second similarity of character pose parameters mapped by the mapped keyframes;
wherein,,
Figure BDA0004126090470000033
S 1 for a set movement entropy threshold, S (x p ) Is x p The motion entropy of the mapped key frame is calculated as follows:
S(x p )=∑ c∈K cosθ c1,c2 wherein K is the set of limbs of the human skeletal model, wherein θ c1,c2 Represents x p The included angle between the c-th limb of the character gesture parameters of the mapped key frame and the c-th limb in the standard gesture model;
the marker value of each node of the two-dimensional random field is used as the marker value of the key frame sequence when the energy function is minimized.
Further, the calculation formula of the second similarity is as follows:
Figure BDA0004126090470000034
wherein k is the number of limbs, theta of the human skeleton model ip An included angle between the ith limb of a person posture parameter and the ith limb in the standard posture model; θ iq Is the included angle between the ith limb of the gesture parameter of another person and the ith limb in the standard gesture model.
Further, the key frame sequence synthesis module includes:
a distance matrix generation module for establishing an m×n distance matrix, wherein the element of the ith row and the jth column of the distance matrix is denoted as d (i, j);
d (i, j) has a value d ij ,d ij A second distance or a third distance representing an ith keyframe of the keyframe sequence Q and a jth keyframe of the keyframe sequence C;
a first distance calculation module for calculating a second distance filling distance matrix, generating a path K from the 1 st row and 1 st column elements to the mth row and n column elements on the distance matrix, summing values of the distance matrix elements on the path K as path distance values, selecting a path K with the smallest path distance value as a shortest path, and using the path distance value of the shortest path as a first distance dist 1
A fourth distance calculation module for calculating a third distance filling distance matrix, generating a path K from the 1 st row and 1 st column elements to the m th row and n th column elements on the distance matrix, summing values of the distance matrix elements on the path K as path distance values, and selecting a path K with the smallest path distance value as path distance valueShortest path, taking path distance value of the shortest path as fourth distance dist 4
A reference distance calculation module that calculates a reference distance, a reference distance dist, based on the first distance and the fourth distance c The calculation formula of (2) is as follows:
Figure BDA0004126090470000041
wherein t is the maximum value of the number of key frames of the two key frame sequences, and k is the number of limbs of the human skeleton model;
and the synthesis module is used for mapping the key frame sequence of the first person and the key frame marked as 0 of the reference key frame sequence, and the third distance between the two key frames establishing the mapping relation is smaller than a set third distance threshold value.
Further, a second distance dist 2 The calculation formula of (2) is as follows:
dist 2 =|Q i -C j i, wherein Q i The index value C of the ith key frame of the key frame sequence Q j Is the tag value of the j-th key frame of the key frame sequence C.
Further, the calculation formula of the third distance is as follows:
Figure BDA0004126090470000051
wherein k is the number of limbs, theta of the human skeleton model ip An included angle between the ith limb of a person posture parameter and the ith limb in the standard posture model; θ iq Is the included angle between the ith limb of the gesture parameter of another person and the ith limb in the standard gesture model.
The invention provides a method for processing images by a computer, which uses a computer system to execute the following steps:
step 201, extracting a first frame image from a three-dimensional animation video and extracting a second frame image from a two-dimensional animation video;
step 202, identifying a person from a frame image and acquiring person posture data;
step 203, screening and obtaining a key frame sequence from the frame image;
step 204, extracting a key frame sequence corresponding to the first person and the third person, and then marking the key frame;
step 205, calculating the reference distances of the key frame sequences of the first person and the second person, and extracting the key frame sequences of the first N second persons with the smallest reference distances of the key frame sequences of the first person as reference key frame sequences;
step 206, each reference key frame sequence is synthesized with the key frame sequence of the first person.
The invention provides a computer storage medium for storing the above computer system.
The invention has the beneficial effects that:
the invention can map the reference image of the key frame which needs to be hand painted in the 2D animation 2D process based on image processing and data mining, provides a reference with higher matching degree for the hand painted staff, is convenient for imitation, and reduces the requirement on the artistic creation capability of the hand painted staff.
Drawings
FIG. 1 is a block diagram of a computer system of the present invention;
FIG. 2 is a block diagram of a key frame sequence synthesis module of the present invention;
FIG. 3 is a flow chart of a method of screening a frame image for key frames in accordance with the present invention;
fig. 4 is a flow chart of a method of processing an image by a computer of the present invention.
In the figure: the system comprises a frame image extraction module 101, a person identification module 102, a person matching module 103, a key frame extraction module 104, a gesture sequence correction module 105, a reference sequence correction module 106, a key frame sequence synthesis module 107, a distance matrix generation module 1071, a first distance calculation module 1072, a fourth distance calculation module 1073, a reference distance calculation module 1074 and a synthesis module 1075.
Detailed Description
The subject matter described herein will now be discussed with reference to example embodiments. It is to be understood that these embodiments are merely discussed so that those skilled in the art may better understand and implement the subject matter described herein and that changes may be made in the function and arrangement of the elements discussed without departing from the scope of the disclosure herein. Various examples may omit, replace, or add various procedures or components as desired. In addition, features described with respect to some examples may be combined in other examples as well.
Example 1
As shown in fig. 1-3, a computer system, comprising:
a frame image extraction module 101 for extracting a first frame image from a three-dimensional animated video and extracting a second frame image from a two-dimensional animated video;
a person recognition module 102 for recognizing a person from the frame image and acquiring person pose data;
the information of the person includes the sex of the person, and the person posture data includes person posture parameters;
the information of the current persona may also contain more content, such as age, persona type, etc., which facilitates more accurate matching, but the amount of persona data required is large enough, otherwise more content means more filtering items, and insufficient data amount may result in insufficient data amount of the persona available for matching.
A person matching module 103, configured to select a first person extracted from the first frame image, and then match a second person extracted from the second frame image, where the first person is the same as the second person in information, and the matched second person is marked as a third person;
a key frame extraction module 104, configured to screen and obtain a key frame sequence from a frame image;
the method for screening and obtaining the key frames from the frame images comprises the following steps:
step 101, extracting character pose parameters of a selected character (the selected character is a first character or a third character) from each frame image, generating a character pose sequence, wherein each unit of the character pose sequence comprises the character pose parameters of the selected character in one frame image;
step 102, traversing backwards from the sequence unit of the character gesture sequence until the first similarity between the traversed sequence unit and the sequence unit from which the traversing starts is smaller than a set first similarity threshold; marking the sequence unit of traversal termination as a first sequence unit;
step 103, iteratively executing step 102 until all sequence units of the character gesture sequence are traversed, wherein the first sequence unit of the character gesture sequence is traversed in the first execution, and the first sequence unit generated in the previous execution is traversed backwards in each subsequent execution;
104, extracting all the generated first sequence units to generate a first human gesture sequence;
step 105, extracting a frame image of a character attitude parameter source of the first sequence unit as a key frame, and sequencing the key frame according to a time point to generate a key frame sequence;
in this step, the key frame sequence may express a plurality of actions with larger intervals, the key frame sequence is intercepted by setting a time threshold by the time difference of the time points between the key frames of the key frame sequence, and the intercepted key frame sequence enters the gesture sequence correction module 105, the reference sequence correction module 106 and the key frame sequence synthesis module 107 for processing respectively.
The time difference can be set to be 3min, and the number of truncated key frame sequences can be reduced as the time difference increases;
the calculation formula of the first similarity is as follows:
Figure BDA0004126090470000081
wherein k is the number of limbs, theta of the human skeleton model i1 Ith limb and standard gesture of character gesture parameter of sequence unit of character gesture sequenceIncluded angle of ith limb in model; θ i2 An included angle between the ith limb of the character gesture parameter of the sequence unit of the gesture sequence of another character and the ith limb in the standard gesture model;
in one embodiment of the invention, the person pose parameters are generated from the image based on a person pose estimation algorithm such as the openPose algorithm.
The first similarity threshold may be adjusted according to the number of limbs of the skeletal model, and is generally inversely proportional to the number of limbs of the skeletal model.
The gesture sequence correction module 105 firstly extracts a key frame sequence corresponding to a first person, marks the key frame, wherein the mark value of the key frame to be redrawn is 0, and the mark values of the rest key frames are 1;
a reference sequence correction module 106, configured to extract a key frame sequence corresponding to the third person, and then mark key frames of the key frame sequence of the third person;
the method for marking the key frames of the key frame sequence of the third person comprises the following steps:
defining a two-dimensional random field, wherein a node set of the two-dimensional random field is denoted as V, and a set of edges between the nodes is denoted as E;
the energy function of the two-dimensional random field is:
Figure BDA0004126090470000082
wherein X is a node set, θ p (x p ) As a potential function of node p, θ pq (x p ,x q ) As a potential function of the edge between nodes p and q, x p The marker value, x, of the image node mapped for node p q A marker value of a key frame sequence mapped for the node q; a mark value of 0 indicates that the character gesture parameter corresponding to the key frame has larger deviation from the real character; a mark value of 1 indicates that the character gesture parameter corresponding to the key frame is close to the real character;
wherein,,
Figure BDA0004126090470000091
simila p,q represents x p And x q A second similarity of character pose parameters mapped by the mapped keyframes;
the calculation formula of the second similarity is as follows:
Figure BDA0004126090470000092
wherein k is the number of limbs, theta of the human skeleton model ip An included angle between the ith limb of a person posture parameter and the ith limb in the standard posture model; θ iq An included angle between the ith limb of the gesture parameter of another person and the ith limb in the standard gesture model;
wherein,,
Figure BDA0004126090470000093
S 1 for a set movement entropy threshold, S (x p ) Is x p The motion entropy of the mapped key frame is calculated as follows:
Figure BDA0004126090470000094
wherein K is the set of limbs of the human skeletal model, wherein θ c1,c2 Represents x p The included angle between the c-th limb of the character gesture parameters of the mapped key frame and the c-th limb in the standard gesture model;
the standard posture model is a limb angle parameter of the human body skeleton model in a natural standing state that the human body is naturally hung by both hands.
The marker value of each node of the two-dimensional random field is used as the marker value of the key frame sequence when the energy function is minimized.
A keyframe sequence synthesizing module 107, configured to calculate reference distances of keyframe sequences of the first person and the second person, and extract keyframe sequences of the first N second persons having the smallest reference distances from the keyframe sequences of the first person as reference keyframe sequences;
each reference key frame sequence is synthesized with the key frame sequence of the first person.
The key frame sequence synthesis module 107 includes:
a distance matrix generation module 1071 that creates an m×n distance matrix whose i-th row and j-th column elements are denoted as d (i, j);
d (i, j) has a value d ij ,d ij A second distance or a third distance representing an ith keyframe of the keyframe sequence Q and a jth keyframe of the keyframe sequence C;
a first distance calculation module 1072 for calculating a second distance filling distance matrix, generating a path K from the 1 st row and 1 st column element to the m th row and n th column element on the distance matrix, summing values of the elements of the distance matrix on the path K as path distance values, selecting a path K with the smallest path distance value as a shortest path, and selecting the path distance value of the shortest path as a first distance dist 1
A fourth distance calculation module 1073 for calculating a third distance filling distance matrix, generating a path K from the 1 st row and 1 st column element to the m th row and n th column element on the distance matrix, summing values of the elements of the distance matrix on the path K as path distance values, selecting a path K with the smallest path distance value as a shortest path, and selecting the path distance value of the shortest path as a fourth distance dist 4
Second distance dist 2 The calculation formula of (2) is as follows:
dist 2 =|Q i -C j |
wherein Q is i The index value C of the ith key frame of the key frame sequence Q j Is the tag value of the j-th key frame of the key frame sequence C.
The calculation formula of the third distance is as follows:
Figure BDA0004126090470000111
wherein k is the number of limbs, theta of the human skeleton model ip An included angle between the ith limb of a person posture parameter and the ith limb in the standard posture model; θ iq Is the included angle between the ith limb of the gesture parameter of another person and the ith limb in the standard gesture model.
A reference distance calculation module 1074 that calculates a reference distance, reference distance dist, based on the first distance and the fourth distance c The calculation formula of (2) is as follows:
Figure BDA0004126090470000112
wherein t is the maximum value of the number of key frames of the two key frame sequences, and k is the number of limbs of the human skeleton model.
The synthesizing module 1075 is configured to map a key frame sequence of the first person and a key frame labeled 0 of the reference key frame sequence, where a third distance between two key frames establishing the mapping relationship is smaller than a set third distance threshold.
As shown in fig. 4, the present embodiment provides a method for processing an image by using a computer system as described above, which performs the following steps:
step 201, extracting a first frame image from a three-dimensional animation video and extracting a second frame image from a two-dimensional animation video;
step 202, identifying a person from a frame image and acquiring person posture data;
step 203, screening and obtaining a key frame sequence from the frame image;
step 204, extracting a key frame sequence corresponding to the first person and the third person, and then marking the key frame;
step 205, calculating the reference distances of the key frame sequences of the first person and the second person, and extracting the key frame sequences of the first N second persons with the smallest reference distances of the key frame sequences of the first person as reference key frame sequences;
step 206, each reference key frame sequence is synthesized with the key frame sequence of the first person.
The embodiment has been described above with reference to the embodiment, but the embodiment is not limited to the above-described specific implementation, which is only illustrative and not restrictive, and many forms can be made by those of ordinary skill in the art, given the benefit of this disclosure, are within the scope of this embodiment.

Claims (10)

1. A computer system, comprising:
the frame image extraction module is used for extracting a first frame image from the three-dimensional animation video and extracting a second frame image from the two-dimensional animation video; a person recognition module for recognizing a person from the frame image and acquiring person pose data; the information of the person includes the sex of the person, and the person posture data includes person posture parameters; a character matching module for selecting a first character extracted from the first frame image and then matching a second character extracted from the second frame image, the first character having the same information as the second character, the matched second character being marked as a third character; the key frame extraction module is used for screening and obtaining a key frame sequence from the frame image;
the gesture sequence correction module firstly extracts a key frame sequence corresponding to a first person, marks the key frame, wherein the mark value of the key frame to be redrawn is 0, and the mark values of the other key frames are 1;
the reference sequence correction module is used for extracting a key frame sequence corresponding to the third person and then marking key frames of the key frame sequence of the third person;
the key frame sequence synthesis module is used for calculating the reference distances of the key frame sequences of the first person and the second person, and extracting the key frame sequences of the first N second persons with the smallest reference distances of the key frame sequences of the first person as reference key frame sequences; each reference key frame sequence is synthesized with the key frame sequence of the first person.
2. The computer system of claim 1, wherein the means for screening the key frames from the frame images comprises:
step 101, extracting character gesture parameters of a selected character from each frame image to generate a character gesture sequence, wherein each unit of the character gesture sequence comprises the character gesture parameters of the selected character in one frame image;
step 102, traversing backwards from the sequence unit of the character gesture sequence until the first similarity between the traversed sequence unit and the sequence unit from which the traversing starts is smaller than a set first similarity threshold; marking the sequence unit of traversal termination as a first sequence unit;
step 103, iteratively executing step 102 until all sequence units of the character gesture sequence are traversed, wherein the first sequence unit of the character gesture sequence is traversed in the first execution, and the first sequence unit generated in the previous execution is traversed backwards in each subsequent execution;
104, extracting all the generated first sequence units to generate a first human gesture sequence;
step 105, extracting a frame image of the character pose parameter source of the first sequence unit as a key frame, and ordering the key frame according to the time point to generate a key frame sequence.
3. The computer system of claim 2, wherein the first similarity is calculated as:
Figure FDA0004126090460000021
wherein k is the number of limbs, theta of the human skeleton model i1 An included angle between the ith limb of the character gesture parameter of a sequence unit of a character gesture sequence and the ith limb in the standard gesture model; θ i2 And the included angle between the ith limb of the character gesture parameter of the sequence unit of the other character gesture sequence and the ith limb in the standard gesture model.
4. The computer system of claim 1, wherein the method of keyframe tagging of the keyframe sequence of the third person comprises:
defining a two-dimensional random field, wherein a node set of the two-dimensional random field is denoted as V, and a set of edges between the nodes is denoted as E;
the energy function of the two-dimensional random field is:
Figure FDA0004126090460000022
wherein X is a node set, θ p x p As a potential function of node p, θ pq x p ,x q As a potential function of the edge between nodes p and q, x p The marker value, x, of the image node mapped for node p q A marker value of a key frame sequence mapped for the node q; a mark value of 0 indicates that the character gesture parameter corresponding to the key frame has larger deviation from the real character; a mark value of 1 indicates that the character gesture parameter corresponding to the key frame is close to the real character;
wherein,,
Figure FDA0004126090460000023
simila p,q represents x p And x q A second similarity of character pose parameters mapped by the mapped keyframes;
wherein,,
Figure FDA0004126090460000031
S 1 for a set movement entropy threshold, sx p Is x p The motion entropy of the mapped key frame is calculated as follows:
Sx p =∑ c∈K cosθ c1,c2 wherein K is the set of limbs of the human skeletal model, wherein θ c1,c2 Represents x p The included angle between the c-th limb of the character gesture parameters of the mapped key frame and the c-th limb in the standard gesture model;
the marker value of each node of the two-dimensional random field is used as the marker value of the key frame sequence when the energy function is minimized.
5. The computer system of claim 4, wherein the second similarity is calculated as:
Figure FDA0004126090460000032
wherein k is the number of limbs, theta of the human skeleton model ip An included angle between the ith limb of a person posture parameter and the ith limb in the standard posture model; θ iq Is the included angle between the ith limb of the gesture parameter of another person and the ith limb in the standard gesture model.
6. The computer system of claim 1, wherein the key frame sequence synthesis module comprises:
a distance matrix generation module for establishing an m×n distance matrix, wherein the element of the ith row and the jth column of the distance matrix is denoted as d (i, j);
d (i, j) has a value d ij ,d ij A second distance or a third distance representing an ith keyframe of the keyframe sequence Q and a jth keyframe of the keyframe sequence C;
a first distance calculation module for calculating a second distance filling distance matrix, generating a path K from the 1 st row and 1 st column elements to the mth row and n column elements on the distance matrix, summing values of the distance matrix elements on the path K as path distance values, selecting a path K with the smallest path distance value as a shortest path, and using the path distance value of the shortest path as a first distance dist 1
A fourth distance calculation module for calculating a third distance filling distance matrix, generating a path K from the 1 st row and 1 st column elements to the m th row and n th column elements on the distance matrix, summing values of the distance matrix elements on the path K as path distance values, and selecting a path K with the smallest path distance value as the shortest pathThe path, the path distance value of the shortest path is taken as the fourth distance dist 4
A reference distance calculation module that calculates a reference distance, a reference distance dist, based on the first distance and the fourth distance c The calculation formula of (2) is as follows:
Figure FDA0004126090460000041
wherein t is the maximum value of the number of key frames of the two key frame sequences, and k is the number of limbs of the human skeleton model;
and the synthesis module is used for mapping the key frame sequence of the first person and the key frame marked as 0 of the reference key frame sequence, and the third distance between the two key frames establishing the mapping relation is smaller than a set third distance threshold value.
7. The computer system of claim 6, wherein the second distance dist 2 The calculation formula of (2) is as follows:
dist 2 =Q i -C j wherein Q is i The index value C of the ith key frame of the key frame sequence Q j Is the tag value of the j-th key frame of the key frame sequence C.
8. The computer system of claim 6, wherein the third distance is calculated as:
Figure FDA0004126090460000042
wherein k is the number of limbs, theta of the human skeleton model ip An included angle between the ith limb of a person posture parameter and the ith limb in the standard posture model; θ iq Is the included angle between the ith limb of the gesture parameter of another person and the ith limb in the standard gesture model.
9. A method of computer processing an image, characterized in that a computer system according to any of claims 1-8 is applied for performing the steps of:
step 201, extracting a first frame image from a three-dimensional animation video and extracting a second frame image from a two-dimensional animation video;
step 202, identifying a person from a frame image and acquiring person posture data;
step 203, screening and obtaining a key frame sequence from the frame image;
step 204, extracting a key frame sequence corresponding to the first person and the third person, and then marking the key frame;
step 205, calculating the reference distances of the key frame sequences of the first person and the second person, and extracting the key frame sequences of the first N second persons with the smallest reference distances of the key frame sequences of the first person as reference key frame sequences;
step 206, each reference key frame sequence is synthesized with the key frame sequence of the first person.
10. A computer storage medium storing a computer system according to any one of claims 1-8.
CN202310246436.8A 2023-03-15 2023-03-15 Computer system, method and medium Pending CN116310015A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310246436.8A CN116310015A (en) 2023-03-15 2023-03-15 Computer system, method and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310246436.8A CN116310015A (en) 2023-03-15 2023-03-15 Computer system, method and medium

Publications (1)

Publication Number Publication Date
CN116310015A true CN116310015A (en) 2023-06-23

Family

ID=86816305

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310246436.8A Pending CN116310015A (en) 2023-03-15 2023-03-15 Computer system, method and medium

Country Status (1)

Country Link
CN (1) CN116310015A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101958007A (en) * 2010-09-20 2011-01-26 南京大学 Three-dimensional animation posture modeling method by adopting sketch
CN102682302A (en) * 2012-03-12 2012-09-19 浙江工业大学 Human body posture identification method based on multi-characteristic fusion of key frame
CN110047096A (en) * 2019-04-28 2019-07-23 中南民族大学 A kind of multi-object tracking method and system based on depth conditions random field models
CN110602527A (en) * 2019-09-12 2019-12-20 北京小米移动软件有限公司 Video processing method, device and storage medium
CN115797851A (en) * 2023-02-09 2023-03-14 安徽米娱科技有限公司 Animation video processing method and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101958007A (en) * 2010-09-20 2011-01-26 南京大学 Three-dimensional animation posture modeling method by adopting sketch
CN102682302A (en) * 2012-03-12 2012-09-19 浙江工业大学 Human body posture identification method based on multi-characteristic fusion of key frame
CN110047096A (en) * 2019-04-28 2019-07-23 中南民族大学 A kind of multi-object tracking method and system based on depth conditions random field models
CN110602527A (en) * 2019-09-12 2019-12-20 北京小米移动软件有限公司 Video processing method, device and storage medium
CN115797851A (en) * 2023-02-09 2023-03-14 安徽米娱科技有限公司 Animation video processing method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
张晓翔 等: ""基于动态密集条件随机场增量推理计算的多类别视频分割"", 《计算机应用研究》, vol. 37, no. 12, 31 December 2020 (2020-12-31), pages 3781 - 2787 *

Similar Documents

Publication Publication Date Title
CN109408653B (en) Human body hairstyle generation method based on multi-feature retrieval and deformation
US20200320345A1 (en) System and method for visual recognition using synthetic training data
CN108027878B (en) Method for face alignment
Yang et al. Extraction of 2d motion trajectories and its application to hand gesture recognition
CN109086706B (en) Motion recognition method based on segmentation human body model applied to human-computer cooperation
Uddin et al. Human activity recognition using body joint‐angle features and hidden Markov model
EP0901667A2 (en) Principal component analysis of image/control-point location coupling for the automatic location of control points
CN108388882A (en) Based on the gesture identification method that the overall situation-part is multi-modal RGB-D
CN111680550B (en) Emotion information identification method and device, storage medium and computer equipment
CN106682585A (en) Dynamic gesture identifying method based on kinect 2
Seddik et al. Unsupervised facial expressions recognition and avatar reconstruction from Kinect
Auephanwiriyakul et al. Thai sign language translation using scale invariant feature transform and hidden markov models
Ari et al. Facial feature tracking and expression recognition for sign language
Yu et al. A video-based facial motion tracking and expression recognition system
Luqman An efficient two-stream network for isolated sign language recognition using accumulative video motion
Neverova Deep learning for human motion analysis
Agarwal et al. Synthesis of realistic facial expressions using expression map
Baumberger et al. 3d face reconstruction from video using 3d morphable model and silhouette
CN115008454A (en) Robot online hand-eye calibration method based on multi-frame pseudo label data enhancement
Huynh-The et al. Learning action images using deep convolutional neural networks for 3D action recognition
CN115797851B (en) Cartoon video processing method and system
CN110516638B (en) Sign language recognition method based on track and random forest
CN108537855B (en) Ceramic stained paper pattern generation method and device with consistent sketch
CN116310015A (en) Computer system, method and medium
US20240062441A1 (en) System and method for photorealistic image synthesis using unsupervised semantic feature disentanglement

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination