CN110264431A - Video beautification method, device and electronic equipment - Google Patents
Video beautification method, device and electronic equipment Download PDFInfo
- Publication number
- CN110264431A CN110264431A CN201910580715.1A CN201910580715A CN110264431A CN 110264431 A CN110264431 A CN 110264431A CN 201910580715 A CN201910580715 A CN 201910580715A CN 110264431 A CN110264431 A CN 110264431A
- Authority
- CN
- China
- Prior art keywords
- keypoints
- video frame
- video
- target object
- key point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 37
- 238000001514 detection method Methods 0.000 claims abstract description 33
- 238000009499 grossing Methods 0.000 claims abstract description 23
- 238000011282 treatment Methods 0.000 claims abstract description 8
- 210000000746 body region Anatomy 0.000 claims abstract description 6
- 230000003796 beauty Effects 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000001052 transient effect Effects 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 5
- 230000006399 behavior Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 2
- 208000004141 microcephaly Diseases 0.000 claims 1
- 210000003128 head Anatomy 0.000 description 27
- 238000010586 diagram Methods 0.000 description 12
- 238000004590 computer program Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 2
- 210000003423 ankle Anatomy 0.000 description 2
- 230000005611 electricity Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 210000003127 knee Anatomy 0.000 description 2
- 230000005291 magnetic effect Effects 0.000 description 2
- 238000007493 shaping process Methods 0.000 description 2
- 241000406668 Loxodonta cyclotis Species 0.000 description 1
- 241001465754 Metazoa Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011017 operating method Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
A kind of video beautification method, device and electronic equipment are provided in the embodiment of the present disclosure, belong to technical field of image processing, this method comprises: obtaining the first video frame and the second video frame adjacent in target video, first video frame is on the time dimension of target video earlier than second video frame；The critical point detection for being directed to target object is executed to the first video frame and the second video frame respectively, forms the first set of keypoints and the second set of keypoints；Based on the corresponding relationship between key point in second set of keypoints and first set of keypoints, smoothing processing is executed to the key point in second set of keypoints, obtains third set of keypoints；The key point being based respectively in first set of keypoints and the third set of keypoints executes deformation operation to the human region except the head zone of the target object.By the processing scheme of the disclosure, landscaping treatment can be executed to the image in whole body region present in video.
Description
Technical field
This disclosure relates to technical field of image processing more particularly to a kind of video beautification method, device and electronic equipment.
Background technique
Image procossing (image processing) is also known as image processing, is needed for being reached with computer to image
As a result technology.Originating from the 1920s, generally Digital Image Processing.The main contents of image processing techniques include figure
As compression, enhancing restore, matching description identification 3 parts, common processing have image digitazation, image coding, image enhancement,
Image restoration, image segmentation and image analysis etc..Image procossing is to be processed image information to meet people using computer
Visual psychology or application demand behavior, be widely used, be chiefly used in mapping science, atmospheric science, astronomy, U.S. figure, make figure
As improving identification etc..
Universal with smart phone, the video of the shooting of mobile phone is also more and more, but due to the preset lens of smart phone
Head can only be held during taking pictures or using closely taking pictures, the human body proportion taken in the video come is larger, generate photograph
The inaesthetic problem of the people of piece.
Summary of the invention
In view of this, the embodiment of the present disclosure provides a kind of video beautification method, device and electronic equipment, at least partly solve
Problems of the prior art.
In a first aspect, the embodiment of the present disclosure provides a kind of video beautification method, comprising:
The first video frame and the second video frame adjacent in target video are obtained, first video frame is in target video
Earlier than second video frame on time dimension；
The critical point detection for being directed to target object is executed to the first video frame and the second video frame respectively, it is crucial to form first
Point set and the second set of keypoints；
Based on the corresponding relationship between key point in second set of keypoints and first set of keypoints, to institute
The key point stated in the second set of keypoints executes smoothing processing, obtains third set of keypoints；
The key point being based respectively in first set of keypoints and the third set of keypoints, to the target pair
Human region except the head zone of elephant executes deformation operation, is formed crucial with first set of keypoints and the third
Corresponding first beautifying picture of point set and the second beautifying picture, first beautifying picture and the second beautifying picture are to structure
At the video frame of beautification video.
According to a kind of specific implementation of the embodiment of the present disclosure, except the head zone to the target object
Human region executes deformation operation, comprising:
Judge that the human region except the head zone of the target object whether there is close to video frame images edge
Region；
If so, reducing the deformation degree close to the region at video frame images edge.
According to a kind of specific implementation of the embodiment of the present disclosure, except the head zone to the target object
Human region executes deformation operation, comprising:
Based on first set of keypoints and the third set of keypoints, determine the target object first respectively
Video frame and human body contour outline and head protection region in the second video frame；
For the human body contour outline region in addition to the head protection region, centered on human body center line, to the people
Body contour area executes deformation operation, obtains human body contour outline deformation pattern, wherein the head protection region is grasped in the deformation
It is remained unchanged during making.
According to a kind of specific implementation of the embodiment of the present disclosure, the determination target object in the first video frame and
Human body contour outline in second video frame, comprising: sampling processing is carried out to target object on target image using default skeleton pattern,
To obtain the skeleton set of keypoints of the target object；
Obtain the edge set of keypoints that the skeleton set of keypoints matches；
Based on the edge set of keypoints, the human body contour outline of the target object is determined.
It is described that the first video frame and the second video frame are held respectively according to a kind of specific implementation of the embodiment of the present disclosure
Before the hand-manipulating of needle is to the critical point detection of target object, the method also includes:
Based on the human body key point, the first nodal point and the second central point of the target object are determined；
Based on the first nodal point and second central point, the target object human body center line is determined.
According to a kind of specific implementation of the embodiment of the present disclosure, except the head zone to the target object
Human region executes deformation operation, is formed and first set of keypoints and the third set of keypoints corresponding first
After beautifying picture and the second beautifying picture, the method also includes:
Obtain the third video frame adjacent with second video frame；
Critical point detection is carried out to the third video frame, forms the 4th set of keypoints；
Based on the third set of keypoints and the 4th set of keypoints, image beauty is executed to the third video frame
Change operation, forms third beautifying picture；
Using the third beautifying picture as the video frame of the beautification video.
It is described based on second set of keypoints and described the according to a kind of specific implementation of the embodiment of the present disclosure
Corresponding relationship in one set of keypoints between key point executes smooth place to the key point in second set of keypoints
Reason, comprising:
It obtains in the coordinate (x1, y1) and the second set of keypoints of the first key point in the first set of keypoints
The coordinate (x2, y2) of second key point, second key point are corresponding with first key point；
Based on pre-set smoothing factor L, coordinate after calculating the second key point smoothing processing (x2 ', y2 '), wherein
X2 '=x1*L+x2* (1-L)
Y2 '=y1*L+y2* (1-L).
It is described that the first video frame and the second video frame are held respectively according to a kind of specific implementation of the embodiment of the present disclosure
Critical point detection of the hand-manipulating of needle to target object, comprising:
Respectively in the first video frame and the second video frame, the gray level image of the target object is obtained；
It obtains in a local small neighbourhood of any pixel point gc in the gray level image, using gc as center pixel, with r
For p equally distributed pixel collection gp of radius；
Using the difference between each pixel in central pixel point gc and pixel collection gp, around pixel gc
P equally distributed pixels carry out binary conversion treatment；
Based on the target object image after binary conversion treatment, critical point detection is executed to target object.
Second aspect, the embodiment of the present disclosure provide a kind of video beautifying apparatus, comprising:
Module is obtained, for obtaining the first video frame and the second video frame adjacent in target video, first video
Frame is on the time dimension of target video earlier than second video frame；
Detection module, for executing the key point inspection for target object to the first video frame and the second video frame respectively
It surveys, forms the first set of keypoints and the second set of keypoints；
Leveling Block, for based between key point in second set of keypoints and first set of keypoints
Corresponding relationship executes smoothing processing to the key point in second set of keypoints, obtains third set of keypoints；
Execution module, the key for being based respectively in first set of keypoints and the third set of keypoints
Point executes deformation operation to the human region except the head zone of the target object, is formed and the described first crucial point set
Close corresponding with the third set of keypoints the first beautifying picture and the second beautifying picture, first beautifying picture and the
Video frame of two beautifying pictures to constitute beautification video.
The third aspect, the embodiment of the present disclosure additionally provide a kind of electronic equipment, which includes:
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and the instruction is by least one processor
It executes, so that at least one processor is able to carry out the view in any implementation of aforementioned first aspect or first aspect
Frequency beautification method.
Fourth aspect, the embodiment of the present disclosure additionally provide a kind of non-transient computer readable storage medium, the non-transient meter
Calculation machine readable storage medium storing program for executing stores computer instruction, and the computer instruction is for making the computer execute aforementioned first aspect or the
In any implementation of one side based on human body key point image shaping method.
5th aspect, the embodiment of the present disclosure additionally provide a kind of computer program product, which includes
The calculation procedure being stored in non-transient computer readable storage medium, the computer program include program instruction, when the program
Instruction is when being computer-executed, make the computer execute in aforementioned first aspect or any implementation of first aspect based on
Human body key point image shaping method.
In the embodiment of the present disclosure based on human body key point image body beautification scheme, including obtaining adjacent the in target video
One video frame and the second video frame, first video frame is on the time dimension of target video earlier than second video frame；
The critical point detection for being directed to target object is executed to the first video frame and the second video frame respectively, formed the first set of keypoints and
Second set of keypoints；Based on the corresponding pass between second set of keypoints and key point in first set of keypoints
System executes smoothing processing to the key point in second set of keypoints, obtains third set of keypoints；It is based respectively on described
Key point in first set of keypoints and the third set of keypoints, to the people except the head zone of the target object
Body region executes deformation operation, forms the first beauty corresponding with first set of keypoints and the third set of keypoints
Change the video frame of image and the second beautifying picture, first beautifying picture and the second beautifying picture to constitute beautification video.
By the scheme of the disclosure, solves the problems, such as the whole beautification of human body in video.
Detailed description of the invention
It, below will be to needed in the embodiment attached in order to illustrate more clearly of the technical solution of the embodiment of the present disclosure
Figure is briefly described, it should be apparent that, the accompanying drawings in the following description is only some embodiments of the present disclosure, for this field
For those of ordinary skill, without creative efforts, it can also be obtained according to these attached drawings other attached drawings.
Fig. 1 is that a kind of video that the embodiment of the present disclosure provides beautifies flow diagram；
Fig. 2 is that another video that the embodiment of the present disclosure provides beautifies flow diagram；
Fig. 3 is that another video that the embodiment of the present disclosure provides beautifies flow diagram；
Fig. 4 is that another video that the embodiment of the present disclosure provides beautifies flow diagram；
Fig. 5 is a kind of video beautifying apparatus structural schematic diagram that the embodiment of the present disclosure provides；
Fig. 6 is the electronic equipment schematic diagram that the embodiment of the present disclosure provides.
Specific embodiment
The embodiment of the present disclosure is described in detail with reference to the accompanying drawing.
Illustrate embodiment of the present disclosure below by way of specific specific example, those skilled in the art can be by this specification
Disclosed content understands other advantages and effect of the disclosure easily.Obviously, described embodiment is only the disclosure
A part of the embodiment, instead of all the embodiments.The disclosure can also be subject to reality by way of a different and different embodiment
It applies or applies, the various details in this specification can also be based on different viewpoints and application, in the spirit without departing from the disclosure
Lower carry out various modifications or alterations.It should be noted that in the absence of conflict, the feature in following embodiment and embodiment can
To be combined with each other.Based on the embodiment in the disclosure, those of ordinary skill in the art are without creative efforts
Every other embodiment obtained belongs to the range of disclosure protection.
It should be noted that the various aspects of embodiment within the scope of the appended claims are described below.Ying Xian
And be clear to, aspect described herein can be embodied in extensive diversified forms, and any specific structure described herein
And/or function is only illustrative.Based on the disclosure, it will be understood by one of ordinary skill in the art that one described herein
Aspect can be independently implemented with any other aspect, and can combine the two or both in these aspects or more in various ways.
For example, carry out facilities and equipments in terms of any number set forth herein can be used and/or practice method.In addition, can make
With other than one or more of aspect set forth herein other structures and/or it is functional implement this equipment and/or
Practice the method.
It should also be noted that, diagram provided in following embodiment only illustrates the basic structure of the disclosure in a schematic way
Think, component count, shape and the size when only display is with component related in the disclosure rather than according to actual implementation in schema are drawn
System, when actual implementation kenel, quantity and the ratio of each component can arbitrarily change for one kind, and its assembly layout kenel can also
It can be increasingly complex.
In addition, in the following description, specific details are provided for a thorough understanding of the examples.However, fields
The skilled person will understand that the aspect can be practiced without these specific details.
The embodiment of the present disclosure provides a kind of video beautification method.Video beautification method provided in this embodiment can be by a meter
Device is calculated to execute, which can be implemented as software, or be embodied as the combination of software and hardware, which can
To be integrally disposed in server, terminal device etc..
Referring to Fig. 1, a kind of video beautification method of embodiment of the present disclosure offer, comprising:
S101 obtains the first video frame and the second video frame adjacent in target video, and first video frame is in target
Earlier than second video frame on the time dimension of video.
Target video is the video for needing to carry out landscaping treatment, and target video can be to be shot by equipment such as smart phones
The video of acquisition, is also possible to the video obtained by other means, and target video is made of multiple video frames.
It include target object in target video, generally, target object can be people, for example, user passes through mobile phone etc.
Equipment has taken the video comprising personage, and the personage in video just constitutes target object.Target object is in addition to can be
Except people, the other kinds of object such as animal can also be.
Target object is shown as head zone and other human regions in addition to the head on target image.For example, with
Family obtains a photo by mobile phone self-timer, then the organs such as face, ear, nose, hair, mouth, eyes of personage in photo
Together constitute the head zone of target object.The following neck key point of head zone, left shoulder, left hand elbow, left finesse, right shoulder,
Right hand elbow, the right wrist, left stern, right stern, left knee, right knee, left ankle, right ankle constitute the key point of human region.
After obtaining target video, the first adjacent video frame can be obtained by extracting the video frame in target video
With the second video frame, the first video frame and the second video frame can be two video frames of arbitrary neighborhood in target video.
S102 executes the critical point detection for being directed to target object to the first video frame and the second video frame respectively, forms the
One set of keypoints and the second set of keypoints.
After obtaining the first video frame and the second video frame, it can be directed to the mesh of the first video frame and the second video frame respectively
It marks object and executes critical point detection, and then obtain multiple header key points and other human body key points in addition to the head.To mesh
Such as CPM (Convolutional Pose Machine, convolution gesture detector) can be used by marking the detection of object key point,
The modes such as PAF (Part Affinity Fields, part affine position detection method) carry out.Herein for the inspection of key point
Survey method is not construed as limiting.
By carrying out critical point detection to the first video frame, the first set of keypoints can be obtained, by the second video
Frame carries out critical point detection, can obtain the second set of keypoints.
S103, based on the corresponding pass between second set of keypoints and key point in first set of keypoints
System executes smoothing processing to the key point in second set of keypoints, obtains third set of keypoints.
Key point in first video frame and the second video frame will lead to the first video frame and second during detection
Shake is generated between the key point at the same position in video frame, these shakes influence whether smooth between adjacent key frame
Property.
For this reason, it may be necessary to be smoothed to the key point in adjacent the first video frame and the second video frame, to disappear
Except the key point of shake.
Specifically, the coordinate (x1, y1) of the first key point in available first set of keypoints and the second key
The coordinate (x2, y2) of the second key point in point set, the second key point and the first key point are the same portion on target object
The key point of position.Followed by pre-set smoothing factor L, coordinate after calculating the second key point smoothing processing (x2 ',
Y2 '), L is the parameter between 0 and 1, pass through following formula:
X2 '=x1*L+x2* (1-L)
Y2 '=y1*L+y2* (1-L)
Coordinate (x2 ', y2 ') after the smoothing processing of the second key point can be calculated.It in this way, can be to second
Key point all in the second set of keypoints is smoothed in video frame, is formed after smoothing processing in the second video frame
New set of keypoints constitutes third set of keypoints.
S104, the key point being based respectively in first set of keypoints and the third set of keypoints, to described
Human region except the head zone of target object executes deformation operation, is formed and first set of keypoints and described the
Corresponding first beautifying picture of three set of keypoints and the second beautifying picture, first beautifying picture and the second beautifying picture
To constitute the video frame of beautification video.
Human region is during forming video frame on target video, it will usually due to shooting angle etc., cause
The area of human region is bigger compared with the area at other positions of body, to affect the overall beautiful of target object, is
This can adjust display of the target object on target image by way of deforming to human region, to reach beauty
The effect of change.
Before carrying out deformation, the deformation parameter of human region can be further obtained, deformation parameter is illustrated to people
The scaling of body region indicates to execute human region 0.9 times of scaling processing for example, deformation parameter can be 0.9.
Deformation parameter can obtain in several ways, and as a kind of mode, deformation parameter can be according to user corresponding
Interactive interface on the value that inputs determine.Alternatively, alternatively mode, deformation parameter can also be by calculating mesh automatically
The ratio of object body various pieces on targeted graphical is marked to be calculated automatically from.
After obtaining deformation parameter, other human regions (target area) other than head zone can be carried out
Deformation.For example, target area human body centerline direction can be carried out compressive deformation.To obtain the beauty for target object
Change image.
So, deformation operation is executed by being directed to the first video frame and the second video frame respectively, has just obtained first
The video frame of beautifying picture and the second beautifying picture, first beautifying picture and the second beautifying picture to constitute beautification video.
By the above-mentioned means, can beautify to the video frame in target video, to improve target in video frame
The landscaping effect of object.
Referring to fig. 2, described to first video frame and described according to a kind of specific implementation of the embodiment of the present disclosure
After second video frame executes the image beautification operation for target object, the method also includes:
S201 obtains the third video frame adjacent with second video frame.
It is similar with the first video frame of acquisition and the second video frame, in order to further be carried out to video frame all in video
Effect beautification, can further obtain the third video frame adjacent with the second video frame, third video frame is in target video
It is later than the second video frame on time dimension.
S202 carries out critical point detection to the third video frame, forms the 4th set of keypoints.
After obtaining third video frame, critical point detection can be executed to the target object of third video frame, and then obtain
Multiple header key points and multiple head adjacent area key points.Such as CPM can be used to target object critical point detection
(Convolutional Pose Machine, convolution gesture detector), (Part Affinity Fields, part is affine by PAF
Position detection method) etc. modes carry out.The detection method of key point is not construed as limiting herein.
S203 is based on the third set of keypoints and the 4th set of keypoints, executes to the third video frame
Image beautification operation, forms third beautifying picture.
The beautification operating procedure similar with step S103-S104 can be used herein, pass through third set of keypoints pair
After 4th key point combines execution smoothing processing, the set of keypoints after using smoothing processing executes image to third video frame
Beautification operation, forms third beautifying picture.
S204, using the third beautifying picture as the video frame of the beautification video.
By the processing mode in step S201-S204, what can be continued executes image to the video frame in target video
Beautification operation.
It is described to be based on second set of keypoints according to a kind of specific implementation of the embodiment of the present disclosure referring to Fig. 3
With the corresponding relationship in first set of keypoints between key point, the key point in second set of keypoints is executed
Smoothing processing, comprising:
S301 obtains the coordinate (x1, y1) and the second set of keypoints of the first key point in the first set of keypoints
In the second key point coordinate (x2, y2), second key point is corresponding with first key point；
S302 is based on pre-set smoothing factor L, the coordinate (x2 ', y2 ') after calculating the second key point smoothing processing.
Specifically, formula x2 '=x1*L+x2* (1-L) can be used, y2 '=y1*L+y2* (1-L) calculates calculating the
Coordinate (x2 ', y2 ') after two key point smoothing processings.
Referring to fig. 4, described to the first video frame and the second video according to a kind of specific implementation of the embodiment of the present disclosure
The human region of target object and head adjacent area execute critical point detection on frame, comprising:
S401 obtains the gray level image of the target object respectively in the first video frame and the second video frame.
In order to facilitate critical point detection is executed, the first video frame and the second video frame can be converted into gray level image, led to
Gray level image is crossed to calculate the key point of target object.
S402 is obtained in a local small neighbourhood of any pixel point gc in the gray level image, using gc as center pixel
Point, using r as the p of radius equally distributed pixel collection gp.
R can be configured according to the actual needs, for example, r can be several pixel values.
S403, using the difference between each pixel in central pixel point gc and pixel collection gp, to pixel gc
P equally distributed pixels of surrounding carry out binary conversion treatments.
Specifically, when the difference in central pixel point gc and pixel collection gp between any pixel point is greater than 0, it can be with
Pixel value two-value in the pixel collection is turned to 0, conversely, being then 1 by its binaryzation.By carrying out binaryzation to pixel
Processing, can be improved the computational efficiency of key point.
S404 executes critical point detection to target object based on the target object image after binary conversion treatment.
Specifically, ring shift right operation can be executed to the string of binary characters of the target object image after binaryzation, obtain
To the multiple groups invariable rotary feature L of target object image.For every group of invariable rotary feature L, radius is set as r's in target point
Interest range calculates the contribution margin of each target pointEach target point is calculated on multiple and different directions
Feature T determines the key point of target object based on feature T.
For example, the weighted sum on 8 direction Di (i=1,2 ..., 8) can be calculated to each target point, obtain i-th
The feature Ti of target point, wherein Ti=L*w (r).It, can be with by comparing the size between the feature Ti and preset value of target point
Determine whether the target point is key point.
According to a kind of specific implementation of the embodiment of the present disclosure, except the head zone to the target object
Human region executes deformation operation, comprising: judges that the human region except the head zone of the target object whether there is and leans on
The region of nearly video frame image border；If so, reducing the deformation degree close to the region at video frame images edge.
According to a kind of specific implementation of the embodiment of the present disclosure, except the head zone to the target object
Human region executes deformation operation, comprising: is based on first set of keypoints and the third set of keypoints, determines respectively
Human body contour outline and head protection region of the target object in the first video frame and the second video frame；For except the head
Human body contour outline region except protection zone executes deformation operation to the human body contour outline region centered on human body center line,
Obtain human body contour outline deformation pattern, wherein the head protection region remains unchanged during the deformation operation.
It is described that the first video frame and the second video frame are held respectively according to a kind of specific implementation of the embodiment of the present disclosure
Before the hand-manipulating of needle is to the critical point detection of target object, the method also includes: it is based on the human body key point, determines the target
The first nodal point of object and the second central point；Based on the first nodal point and second central point, the target is determined
Object human body center line.
Corresponding with above method embodiment, referring to Fig. 5, the disclosure additionally provides a kind of video beautifying apparatus 50, packet
It includes:
Module 501 is obtained, for obtaining the first video frame and the second video frame adjacent in target video, first view
Frequency frame is on the time dimension of target video earlier than second video frame；
Detection module 502, for executing the key point for being directed to target object to the first video frame and the second video frame respectively
Detection forms the first set of keypoints and the second set of keypoints；
Leveling Block 503, for based on key point in second set of keypoints and first set of keypoints it
Between corresponding relationship, in second set of keypoints key point execute smoothing processing, obtain third set of keypoints；
Execution module 504, the pass for being based respectively in first set of keypoints and the third set of keypoints
Key point executes deformation operation to the human region except the head zone of the target object, is formed and first key point
Gather corresponding with the third set of keypoints the first beautifying picture and the second beautifying picture, first beautifying picture with
Video frame of second beautifying picture to constitute beautification video.
Fig. 5 shown device can it is corresponding execute above method embodiment in content, what the present embodiment was not described in detail
Part, referring to the content recorded in above method embodiment, details are not described herein.
Referring to Fig. 6, the embodiment of the present disclosure additionally provides a kind of electronic equipment 60, which includes:
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and the instruction is by least one processor
It executes, so that at least one processor is able to carry out video beautification method in preceding method embodiment.
The embodiment of the present disclosure additionally provides a kind of non-transient computer readable storage medium, and the non-transient computer is readable to deposit
Storage media stores computer instruction, and the computer instruction is for executing the computer in preceding method embodiment.
The embodiment of the present disclosure additionally provides a kind of computer program product, and the computer program product is non-temporary including being stored in
Calculation procedure on state computer readable storage medium, the computer program include program instruction, when the program instruction is calculated
When machine executes, the computer is made to execute the video beautification method in preceding method embodiment.
Below with reference to Fig. 6, it illustrates the structural schematic diagrams for the electronic equipment 60 for being suitable for being used to realize the embodiment of the present disclosure.
Electronic equipment in the embodiment of the present disclosure can include but is not limited to such as mobile phone, laptop, Digital Broadcasting Receiver
Device, PDA (personal digital assistant), PAD (tablet computer), PMP (portable media player), car-mounted terminal are (such as vehicle-mounted
Navigation terminal) etc. mobile terminal and such as number TV, desktop computer etc. fixed terminal.Electronics shown in Fig. 6
Equipment is only an example, should not function to the embodiment of the present disclosure and use scope bring any restrictions.
As shown in fig. 6, electronic equipment 60 may include processing unit (such as central processing unit, graphics processor etc.) 601,
It can be loaded into random access storage according to the program being stored in read-only memory (ROM) 602 or from storage device 608
Program in device (RAM) 603 and execute various movements appropriate and processing.In RAM 603, it is also stored with the behaviour of electronic equipment 60
Various programs and data needed for making.Processing unit 601, ROM 602 and RAM 603 are connected with each other by bus 604.It is defeated
Enter/export (I/O) interface 605 and is also connected to bus 604.
In general, following device can connect to I/O interface 605: including such as touch screen, touch tablet, keyboard, mouse, figure
As the input unit 606 of sensor, microphone, accelerometer, gyroscope etc.；Including such as liquid crystal display (LCD), loudspeaking
The output device 607 of device, vibrator etc.；Storage device 608 including such as tape, hard disk etc.；And communication device 609.It is logical
T unit 609 can permit electronic equipment 60 and wirelessly or non-wirelessly be communicated with other equipment to exchange data.Although showing in figure
The electronic equipment 60 with various devices is gone out, it should be understood that being not required for implementing or having all devices shown.
It can alternatively implement or have more or fewer devices.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description
Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium
On computer program, which includes the program code for method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communication device 609, or from storage device 608
It is mounted, or is mounted from ROM 602.When the computer program is executed by processing unit 601, the embodiment of the present disclosure is executed
Method in the above-mentioned function that limits.
It should be noted that the above-mentioned computer-readable medium of the disclosure can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the disclosure, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In open, computer-readable signal media may include in a base band or as the data-signal that carrier wave a part is propagated,
In carry computer-readable program code.The data-signal of this propagation can take various forms, including but not limited to
Electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer-readable and deposit
Any computer-readable medium other than storage media, the computer-readable signal media can send, propagate or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: electric wire, optical cable, RF (radio frequency) etc. are above-mentioned
Any appropriate combination.
Above-mentioned computer-readable medium can be included in above-mentioned electronic equipment；It is also possible to individualism, and not
It is fitted into the electronic equipment.
Above-mentioned computer-readable medium carries one or more program, when said one or multiple programs are by the electricity
When sub- equipment executes, so that the electronic equipment: obtaining at least two internet protocol addresses；Send to Node evaluation equipment includes institute
State the Node evaluation request of at least two internet protocol addresses, wherein the Node evaluation equipment is internet from described at least two
In protocol address, chooses internet protocol address and return；Receive the internet protocol address that the Node evaluation equipment returns；Its
In, the fringe node in acquired internet protocol address instruction content distributing network.
Alternatively, above-mentioned computer-readable medium carries one or more program, when said one or multiple programs
When being executed by the electronic equipment, so that the electronic equipment: receiving the Node evaluation including at least two internet protocol addresses and request；
From at least two internet protocol address, internet protocol address is chosen；Return to the internet protocol address selected；Wherein,
The fringe node in internet protocol address instruction content distributing network received.
The calculating of the operation for executing the disclosure can be write with one or more programming languages or combinations thereof
Machine program code, above procedure design language include object oriented program language-such as Java, Smalltalk, C+
+, it further include conventional procedural programming language-such as " C " language or similar programming language.Program code can
Fully to execute, partly execute on the user computer on the user computer, be executed as an independent software package,
Part executes on the remote computer or executes on a remote computer or server completely on the user computer for part.
In situations involving remote computers, remote computer can pass through the network of any kind --- including local area network (LAN)
Or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize Internet service
Provider is connected by internet).
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use
The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box
The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually
It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse
Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding
The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction
Combination realize.
Being described in unit involved in the embodiment of the present disclosure can be realized by way of software, can also be by hard
The mode of part is realized.Wherein, the title of unit does not constitute the restriction to the unit itself under certain conditions, for example, the
One acquiring unit is also described as " obtaining the unit of at least two internet protocol addresses ".
It should be appreciated that each section of the disclosure can be realized with hardware, software, firmware or their combination.
The above, the only specific embodiment of the disclosure, but the protection scope of the disclosure is not limited thereto, it is any
Those familiar with the art is in the technical scope that the disclosure discloses, and any changes or substitutions that can be easily thought of, all answers
Cover within the protection scope of the disclosure.Therefore, the protection scope of the disclosure should be subject to the protection scope in claims.
Claims (11)
1. a kind of video beautification method characterized by comprising
Obtain the first video frame and the second video frame adjacent in target video, time of first video frame in target video
Earlier than second video frame in dimension；
The critical point detection for being directed to target object is executed to the first video frame and the second video frame respectively, forms the first crucial point set
It closes and the second set of keypoints；
Based on the corresponding relationship between key point in second set of keypoints and first set of keypoints, to described
Key point in two set of keypoints executes smoothing processing, obtains third set of keypoints；
The key point being based respectively in first set of keypoints and the third set of keypoints, to the target object
Human region except head zone executes deformation operation, is formed and first set of keypoints and the third key point set
Corresponding first beautifying picture and the second beautifying picture are closed, first beautifying picture and the second beautifying picture are to constitute beauty
Change the video frame of video.
2. the method according to claim 1, wherein the people except the head zone to the target object
Body region executes deformation operation, comprising:
Judge that the human region except the head zone of the target object whether there is close to the region at video frame images edge；
If so, reducing the deformation degree close to the region at video frame images edge.
3. the method according to claim 1, wherein the people except the head zone to the target object
Body region executes deformation operation, comprising:
Based on first set of keypoints and the third set of keypoints, determine the target object in the first video respectively
Frame and human body contour outline and head protection region in the second video frame；
For the human body contour outline region in addition to the head protection region, centered on human body center line, to the human body wheel
Wide region executes deformation operation, obtains human body contour outline deformation pattern, wherein the head protection region is in the deformation operation mistake
It is remained unchanged in journey.
4. the method according to claim 1, wherein the determination target object is in the first video frame and
Human body contour outline in two video frames, comprising:
Sampling processing is carried out to target object on target image using default skeleton pattern, to obtain the skeleton of the target object
Set of keypoints；
Obtain the edge set of keypoints that the skeleton set of keypoints matches；
Based on the edge set of keypoints, the human body contour outline of the target object is determined.
5. the method according to claim 1, wherein described respectively execute the first video frame and the second video frame
Before the critical point detection of target object, the method also includes:
Based on the human body key point, the first nodal point and the second central point of the target object are determined；
Based on the first nodal point and second central point, the target object human body center line is determined.
6. the method according to claim 1, wherein the people except the head zone to the target object
Body region executes deformation operation, forms the first beauty corresponding with first set of keypoints and the third set of keypoints
After changing image and the second beautifying picture, the method also includes:
Obtain the third video frame adjacent with second video frame；
Critical point detection is carried out to the third video frame, forms the 4th set of keypoints；
Based on the third set of keypoints and the 4th set of keypoints, image beautification behaviour is executed to the third video frame
Make, forms third beautifying picture；
Using the third beautifying picture as the video frame of the beautification video.
7. the method according to claim 1, wherein described be based on second set of keypoints and described first
Corresponding relationship in set of keypoints between key point executes smoothing processing to the key point in second set of keypoints,
Include:
Obtain second in the coordinate (x1, y1) and the second set of keypoints of the first key point in the first set of keypoints
The coordinate (x2, y2) of key point, second key point are corresponding with first key point；
Based on pre-set smoothing factor L, coordinate after calculating the second key point smoothing processing (x2 ', y2 '), wherein
X2 '=x1*L+x2* (1-L)
Y2 '=y1*L+y2* (1-L).
8. the method according to claim 1, wherein described respectively execute the first video frame and the second video frame
For the critical point detection of target object, comprising:
Respectively in the first video frame and the second video frame, the gray level image of the target object is obtained；
It obtains in a local small neighbourhood of any pixel point gc in the gray level image, is half with r using gc as center pixel
P equally distributed pixel collection gp of diameter；
Using the difference between each pixel in central pixel point gc and pixel collection gp, to p around pixel gc
Equally distributed pixel carries out binary conversion treatment；
Based on the target object image after binary conversion treatment, critical point detection is executed to target object.
9. a kind of video beautifying apparatus characterized by comprising
Module is obtained, for obtaining the first video frame and the second video frame adjacent in target video, first video frame exists
Earlier than second video frame on the time dimension of target video；
Detection module, for executing the critical point detection for being directed to target object, shape to the first video frame and the second video frame respectively
At the first set of keypoints and the second set of keypoints；
Leveling Block, for based on corresponding between second set of keypoints and key point in first set of keypoints
Relationship executes smoothing processing to the key point in second set of keypoints, obtains third set of keypoints；
Execution module, the key point for being based respectively in first set of keypoints and the third set of keypoints are right
Human region except the head zone of the target object executes deformation operation, is formed and first set of keypoints and institute
State corresponding first beautifying picture of third set of keypoints and the second beautifying picture, first beautifying picture and the second beautification
Video frame of the image to constitute beautification video.
10. a kind of electronic equipment, which is characterized in that the electronic equipment includes:
At least one processor；And
The memory being connect at least one described processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one described processor, and described instruction is by described at least one
It manages device to execute, so that at least one described processor is able to carry out described in aforementioned any claim 1-8 based on human body key
Point image microcephaly's beautification method.
11. a kind of non-transient computer readable storage medium, which stores computer instruction,
The computer instruction is for making the computer execute video beautification method described in aforementioned any claim 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910580715.1A CN110264431A (en) | 2019-06-29 | 2019-06-29 | Video beautification method, device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910580715.1A CN110264431A (en) | 2019-06-29 | 2019-06-29 | Video beautification method, device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110264431A true CN110264431A (en) | 2019-09-20 |
Family
ID=67923193
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910580715.1A Pending CN110264431A (en) | 2019-06-29 | 2019-06-29 | Video beautification method, device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110264431A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110782404A (en) * | 2019-10-11 | 2020-02-11 | 北京达佳互联信息技术有限公司 | Image processing method, device and storage medium |
| CN112184593A (en) * | 2020-10-14 | 2021-01-05 | 北京字跳网络技术有限公司 | Key point determination method, device, equipment and computer readable medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106210448A (en) * | 2016-07-22 | 2016-12-07 | 恒业智能信息技术（深圳）有限公司 | A kind of video image dithering Processing for removing method |
| CN107507155A (en) * | 2017-09-25 | 2017-12-22 | 北京奇虎科技有限公司 | Video segmentation result edge optimization real-time processing method, device and computing device |
| CN107704829A (en) * | 2017-10-09 | 2018-02-16 | 武汉斗鱼网络科技有限公司 | A kind of face key point method for tracing and application and device |
| CN107977940A (en) * | 2017-11-30 | 2018-05-01 | 广东欧珀移动通信有限公司 | background blurring processing method, device and equipment |
| CN108198141A (en) * | 2017-12-28 | 2018-06-22 | 北京奇虎科技有限公司 | Realize image processing method, device and the computing device of thin face special efficacy |
| CN108830784A (en) * | 2018-05-31 | 2018-11-16 | 北京市商汤科技开发有限公司 | A kind of image processing method, device and computer storage medium |
| CN108986023A (en) * | 2018-08-03 | 2018-12-11 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN109191414A (en) * | 2018-08-21 | 2019-01-11 | 北京旷视科技有限公司 | A kind of image processing method, device, electronic equipment and storage medium |
| CN109495688A (en) * | 2018-12-26 | 2019-03-19 | 华为技术有限公司 | Method for previewing of taking pictures, graphic user interface and the electronic equipment of electronic equipment |
| CN109903217A (en) * | 2019-01-25 | 2019-06-18 | 北京百度网讯科技有限公司 | Image distortion method and device |
-
2019
- 2019-06-29 CN CN201910580715.1A patent/CN110264431A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106210448A (en) * | 2016-07-22 | 2016-12-07 | 恒业智能信息技术（深圳）有限公司 | A kind of video image dithering Processing for removing method |
| CN107507155A (en) * | 2017-09-25 | 2017-12-22 | 北京奇虎科技有限公司 | Video segmentation result edge optimization real-time processing method, device and computing device |
| CN107704829A (en) * | 2017-10-09 | 2018-02-16 | 武汉斗鱼网络科技有限公司 | A kind of face key point method for tracing and application and device |
| CN107977940A (en) * | 2017-11-30 | 2018-05-01 | 广东欧珀移动通信有限公司 | background blurring processing method, device and equipment |
| CN108198141A (en) * | 2017-12-28 | 2018-06-22 | 北京奇虎科技有限公司 | Realize image processing method, device and the computing device of thin face special efficacy |
| CN108830784A (en) * | 2018-05-31 | 2018-11-16 | 北京市商汤科技开发有限公司 | A kind of image processing method, device and computer storage medium |
| CN108986023A (en) * | 2018-08-03 | 2018-12-11 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN109191414A (en) * | 2018-08-21 | 2019-01-11 | 北京旷视科技有限公司 | A kind of image processing method, device, electronic equipment and storage medium |
| CN109495688A (en) * | 2018-12-26 | 2019-03-19 | 华为技术有限公司 | Method for previewing of taking pictures, graphic user interface and the electronic equipment of electronic equipment |
| CN109903217A (en) * | 2019-01-25 | 2019-06-18 | 北京百度网讯科技有限公司 | Image distortion method and device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110782404A (en) * | 2019-10-11 | 2020-02-11 | 北京达佳互联信息技术有限公司 | Image processing method, device and storage medium |
| CN110782404B (en) * | 2019-10-11 | 2022-06-10 | 北京达佳互联信息技术有限公司 | Image processing method, device and storage medium |
| CN112184593A (en) * | 2020-10-14 | 2021-01-05 | 北京字跳网络技术有限公司 | Key point determination method, device, equipment and computer readable medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110766777B (en) | Method and device for generating virtual image, electronic equipment and storage medium | |
| WO2021139408A1 (en) | Method and apparatus for displaying special effect, and storage medium and electronic device | |
| CN110298785A (en) | Image beautification method, device and electronic equipment | |
| CN110288551A (en) | Video beautification method, device and electronic equipment | |
| CN110288520A (en) | Image beautification method, device and electronic equipment | |
| CN110288519A (en) | Image beautification method, device and electronic equipment | |
| CN110288553A (en) | Image beautification method, device and electronic equipment | |
| CN110287891A (en) | Gestural control method, device and electronic equipment based on human body key point | |
| CN110222726A (en) | Image processing method, device and electronic equipment | |
| CN110070551A (en) | Rendering method, device and the electronic equipment of video image | |
| CN110070585A (en) | Image generating method, device and computer readable storage medium | |
| CN110378947A (en) | 3D model reconstruction method, device and electronic equipment | |
| CN110288521A (en) | Image beautification method, device and electronic equipment | |
| CN110070063A (en) | Action identification method, device and the electronic equipment of target object | |
| CN109754464A (en) | Method and apparatus for generating information | |
| CN110035236A (en) | Image processing method, device and electronic equipment | |
| CN110069125B (en) | Virtual object control method and device | |
| CN110070555A (en) | Image processing method, device, hardware device | |
| CN110062157A (en) | Render method, apparatus, electronic equipment and the computer readable storage medium of image | |
| CN110070495A (en) | Processing method, device and the electronic equipment of image | |
| CN110264431A (en) | Video beautification method, device and electronic equipment | |
| CN111107278A (en) | Image processing method and device, electronic equipment and readable storage medium | |
| CN110288552A (en) | Video beautification method, device and electronic equipment | |
| CN109981989A (en) | Render method, apparatus, electronic equipment and the computer readable storage medium of image | |
| CN110264430A (en) | Video beautification method, device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190920 |
|
| RJ01 | Rejection of invention patent application after publication |