CN110347877A - A kind of method for processing video frequency, device, electronic equipment and storage medium - Google Patents

A kind of method for processing video frequency, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN110347877A
CN110347877A CN201910569106.6A CN201910569106A CN110347877A CN 110347877 A CN110347877 A CN 110347877A CN 201910569106 A CN201910569106 A CN 201910569106A CN 110347877 A CN110347877 A CN 110347877A
Authority
CN
China
Prior art keywords
frame
processed
video
region
human
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910569106.6A
Other languages
Chinese (zh)
Other versions
CN110347877B (en
Inventor
刘思阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing QIYI Century Science and Technology Co Ltd
Original Assignee
Beijing QIYI Century Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing QIYI Century Science and Technology Co Ltd filed Critical Beijing QIYI Century Science and Technology Co Ltd
Priority to CN201910569106.6A priority Critical patent/CN110347877B/en
Publication of CN110347877A publication Critical patent/CN110347877A/en
Application granted granted Critical
Publication of CN110347877B publication Critical patent/CN110347877B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7837Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
    • G06F16/784Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content the detected or recognised objects being people
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20132Image cropping

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Library & Information Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

This application involves a kind of method for processing video frequency, device, electronic equipment and storage mediums, this method comprises: screening includes the frame to be processed of target person from video to be processed;The human region of the target person is identified from the frame to be processed;From the frame to be processed cut obtain include the human region clipping region image;It is generated according to the clipping region image and cuts video.The technical solution comes out each frame image zooming-out that video includes target person, identifies the human region in each frame image, and carry out into cutting based on the people's body region, the image synthesis after cutting is cut video.In this way, due to cut be based on human region, compared with the existing technology for, cut the human body image in video than more complete, also, human body image is in intermediate position in cutting out video, display is obvious.In addition, the content being not concerned with due to having eliminated other users, so that cutting out the content of the target person in video only including user's concern.

Description

A kind of method for processing video frequency, device, electronic equipment and storage medium
Technical field
This application involves data processing fields more particularly to a kind of method for processing video frequency, device, electronic equipment and storage to be situated between Matter.
Background technique
With the development of internet, more and more users watch video content in each network video platform.
Currently, existing network video platform have " only see that TA sees function, can be the segment in video about designated person It picks out.The function is to play the time slice containing designated person, is to carry out editing to video on a timeline.
Although the prior art can come out the time slice editing comprising designated person according to user's needs, depending on The position of the designated person may be in Video Edge position in frequency content.
Summary of the invention
In order to solve the above-mentioned technical problem or it at least is partially solved above-mentioned technical problem, this application provides a kind of views Frequency processing method, device, electronic equipment and storage medium.
In a first aspect, this application provides a kind of method for processing video frequency, comprising:
Screening includes the frame to be processed of target person from video to be processed;
The human region of the target person is identified from the frame to be processed;
From the frame to be processed cut obtain include the human region clipping region image;
It is generated according to the clipping region image and cuts video.
Optionally, the screening from video to be processed includes the frame to be processed of target person, comprising:
The video to be processed is split as video frame;
When in the video frame including the facial image of the target person, determine that the video frame is described to be processed Frame.
Optionally, the screening from video to be processed includes the frame to be processed of target person, further includes:
When not including the facial image in the video frame, the people in the preset time period of the video frame front and back is counted The frequency of occurrence of face image;
When the frequency of occurrence is more than preset times, determine that the video frame is the frame to be processed.
Optionally, the human region that the target person is identified from the frame to be processed, comprising:
The human body key point of the target person is extracted from the frame to be processed;
The human region of the target person is calculated according to the human body key point.
Optionally, the human region that the target person is identified from the frame to be processed, further includes:
Judge whether the human body key point meets preset condition;
When the human body key point meets preset condition, the operation for calculating the human region is executed;The human body closes Key point meets preset condition, comprising:
The human body key point includes designated key point;
And/or
The number of the human body key point is greater than or equal to predetermined number.
Optionally, from the frame to be processed cut obtain include the human region clipping region image, comprising:
Human body center point coordinate is calculated according to the human body key point coordinate;
It is calculated according to the human body center point coordinate, the resolution ratio of video to be processed and default resolution ratio to described default point The reduction ratio of resolution;
The first candidate Crop Area is calculated according to the human body center point coordinate, the default resolution ratio and the reduction ratio Domain;
Judge whether the human region exceeds the range of the described first candidate clipping region;
When range of the human region without departing from the described first candidate clipping region, according to the described first candidate cutting Region cuts the frame to be processed, obtains the clipping region image.
Optionally, from the frame to be processed cut obtain include the human region clipping region image, further includes:
When range of the human region beyond the described first candidate clipping region, according to the seat of the human region Mark, the reduction ratio and the default resolution ratio determine the second candidate clipping region;
The frame to be processed is cut according to the described second candidate clipping region, obtains the clipping region image.
Optionally, described generated according to the clipping region image cuts video, comprising:
When the true resolution of the cut out areas image is less than default resolution ratio, the cut out areas image is carried out Super-resolution processing obtains the target image for meeting the default resolution ratio;
The target image is synthesized, obtains described cutting out video.
Second aspect, this application provides a kind of video process apparatus, comprising:
Screening module, for be processed frame of the screening including target person from video to be processed;
Identification module, for identifying the human region of the target person from the frame to be processed;
Cut module, for from the frame to be processed cut obtain include the human region clipping region image;
Generation module cuts video for generating according to the clipping region image.
The third aspect, this application provides a kind of electronic equipment, comprising: processor, communication interface, memory and communication are total Line, wherein processor, communication interface, memory complete mutual communication by communication bus;
The memory, for storing computer program;
The processor when for executing the program stored on memory, realizes above method step.
Fourth aspect, this application provides a kind of computer readable storage mediums, are stored thereon with computer program, the meter Above method step is realized when calculation machine program is executed by processor.
Above-mentioned technical proposal provided by the embodiments of the present application has the advantages that by video include mesh compared with prior art Each frame image zooming-out of mark personage comes out, and identifies the human region in each frame image, and based on the people's body region carry out into It cuts, the image synthesis after cutting is cut into video.In this way, being based on human region, compared with the existing technology due to cutting For, the human body image in video is cut than more complete, also, human body image is in intermediate position in cutting out video, Display is obvious.In addition, the content being not concerned with due to having eliminated other users, so that cutting out only includes user's concern in video The content of target person.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention Example, and be used to explain the principle of the present invention together with specification.
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, for those of ordinary skill in the art Speech, without any creative labor, is also possible to obtain other drawings based on these drawings.
Fig. 1 is a kind of flow chart of method for processing video frequency provided by the embodiments of the present application;
Fig. 2 is a kind of flow chart for method for processing video frequency that another embodiment of the application provides;
Fig. 3 is a kind of flow chart for method for processing video frequency that another embodiment of the application provides;
Fig. 4 is a kind of flow chart for method for processing video frequency that another embodiment of the application provides;
Fig. 5 is the schematic diagram of human body key point provided by the embodiments of the present application;
Fig. 6 is the schematic diagram of human region provided by the embodiments of the present application;
Fig. 7 is a kind of flow chart of method for processing video frequency provided by the embodiments of the present application;
Fig. 8 is the schematic diagram of human region provided by the embodiments of the present application and clipping region;
Fig. 9 is the schematic diagram of human region and clipping region that another embodiment of the application provides;
Figure 10 is a kind of block diagram of video process apparatus provided by the embodiments of the present application;
Figure 11 is the block diagram of screening module 91 provided by the embodiments of the present application;
Figure 12 is the block diagram of identification module 92 provided by the embodiments of the present application;
Figure 13 is the block diagram for the identification module 92 that another embodiment of the application provides;
Figure 14 is the block diagram provided by the embodiments of the present application for cutting module 93;
Figure 15 is the block diagram of generation module 94 provided by the embodiments of the present application;
Figure 16 is the structural schematic diagram of a kind of electronic equipment provided by the embodiments of the present application.
Specific embodiment
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is A part of the embodiment of the application, instead of all the embodiments.Based on the embodiment in the application, ordinary skill people Member's every other embodiment obtained without making creative work, shall fall in the protection scope of this application.
A kind of method for processing video frequency is provided for the embodiments of the invention first below to be introduced.
Method provided by the embodiment of the present invention can be applied to any electronic equipment for needing video to handle, for example, can Think the electronic equipments such as server, terminal, is not specifically limited herein, for convenience of description, subsequent referred to as electronic equipment.
Fig. 1 is a kind of flow chart of method for processing video frequency provided by the embodiments of the present application.As shown in Figure 1, this method includes Following steps:
Step S11, screening includes the frame to be processed of target person from video to be processed.
Step S12 identifies the human region of target person from frame to be processed.
Wherein, human region refers to region completely including target person human body image.
Step S13, from frame to be processed cut obtain include human region clipping region image.
Step S14 is generated according to clipping region image and is cut video.
Each frame image zooming-out that video includes target person is come out, is identified in each frame image by the embodiment of the present application Human region, and carried out based on the people's body region into cutting, the image synthesis after cutting cut into video.In this way, due to cutting out Cut be based on human region, compared with the existing technology for, cut the human body image in video than more complete, also, human body Image is in intermediate position in cutting out video, and display is obvious.In addition, due to having eliminated in other users are not concerned with Hold, so that cutting out the content of the target person in video only including user's concern.
Fig. 2 is a kind of flow chart for method for processing video frequency that another embodiment of the application provides.As shown in Fig. 2, in step In S11, comprising:
Video to be processed is split as video frame by step S21.
Step S22 determines that video frame is frame to be processed when in video frame including the facial image of target person.
Further, when not including facial image in video frame, interior face of preset time period before and after the video frame is counted The frequency of occurrence of image;When frequency of occurrence is more than preset times, determine that video frame is frame to be processed.
The process of the frame to be processed with a specific example to screening including target person is illustrated below.
Fig. 3 is a kind of flow chart for method for processing video frequency that another embodiment of the application provides.As shown in figure 3, step S11 Can with specifically includes the following steps:
Video to be processed is split as video frame by step S31, which is 50 frames/second.
Step S32, in video frame whether include target person A facial image, if it is executing step S34, if not, Execute step S33.
There is the number of facial image whether more than 200 times, if it is execution in 2.5 seconds in step S33, video frame front and back Step S34, if not, executing step S35.
Step S34, the video frame include target person A, are video frame to be processed.
Step S35, the video frame do not include target person A, are not video frames to be processed.
In the present embodiment, 2.5 are shared and is applied in example in 2.5 seconds before and after the video frame, the video frame previous frame, if there is 200 Frame all recognizes the facial image, i.e., 80% frame all has the target person, it is contemplated that the continuity of video content can then recognize It also include the target person for the video frame.
In the present embodiment, identification includes the video frame of target person in several ways, so that the identification to target person More accurate reduction is omitted comprehensively, is as much as possible all screened all video frames including target person, is improved subsequent sanction Cut the smoothness of rear video.
Fig. 4 is a kind of flow chart for method for processing video frequency that another embodiment of the application provides.As shown in figure 4, step S12 Include:
Step S41 extracts the human body key point of target person from frame to be processed;
Step S42 calculates the human region of target person according to human body key point.
The quantity of human body key point can be chosen according to actual needs, can choose 14,17 etc..
Fig. 5 is the schematic diagram of human body key point provided by the embodiments of the present application.As shown in figure 5,14 key points of human body and Its coordinate is as follows:
Header key point p0=(x0, y0)
Neck key point p1=(x1, y1)
Right shoulder key point p2=(x2, y2)
Left shoulder key point p3=(x3, y3)
Right hand elbow key point p4=(x4, y4)
Left hand elbow key point p5=(x5, y5)
Right finesse key point p6=(x6, y6)
Left finesse key point p7=(x7, y7)
Right hipbone key point p8=(x8, y8)
Left hipbone key point p9=(x9, y9)
Right knee key point p10=(x10, y10)
Left knee key point p11=(x11, y11)
Right ankle key point p12=(x12, y12)
Left ankle key point p13=(x13, y13)
Wherein it is possible to the frame lower left corner to be processed is set as coordinate origin O, from coordinate origin O along frame left margin to be processed to It is upper positive for y-axis, be to upper right from coordinate origin O along frame lower boundary to be processedxAxis is positive.
Further, step S12 further include: when calculating human region, need whether to meet human body key point default Condition is judged, when human body key point meets preset condition, executes the operation of above-mentioned steps S42.Human body key point meets Preset condition comprises at least one of the following situation:
(1) human body key point includes designated key point.
Detect that human body key point must include header key point, neck key point and left and right shoulder for example, can set Portion's key point.
(2) number of human body key point is greater than or equal to predetermined number.
For example, can set need to detect at least 11 key points, it is just eligible.
In another example above two situation can be combined, and must detect header key point, neck key point and Left and right shoulder key point, and the key point detected at least 11, just determine that human body key point meets preset condition.
If not extracting qualified human body key point, subsequent processing is not carried out to the frame to be processed, can abandoned The frame.
Human region can be calculated according to the resolution ratio of human body key point coordinate, default flare factor and video to be processed It obtains.
Fig. 6 is the schematic diagram of human region provided by the embodiments of the present application.As shown in fig. 6, for example, set human region as Rectangular area 61, then can determine human region by calculating top left co-ordinate and the bottom right angular coordinate of the rectangular area Position and size.Wherein, the resolution ratio of frame to be processed is w × h, and w, h are the width and height of frame to be processed, the default expansion system of x, y-axis Number is respectively α, β.
Firstly, determine that the maximum value of all human body key point x coordinates is,
The minimum value of all human body key point x coordinates is,All human bodies are crucial Point y-coordinate maximum value be,The minimum value of all human body key point y-coordinates is,
Calculate the width and height of human region:
Top left co-ordinateWith bottom right angular coordinateIn,Point Not are as follows:
Due to general using selected frame in the prior art, frame selects the part of required cutting in image, then is cut, can The situation that human region cannot completely be cut out can occur.And in the present embodiment, by according to human body key point to target person Entire body position is estimated, the relatively complete human region of target person is obtained, so that after the subsequent cutting based on human region In each frame video, for target person image than more complete and be in intermediate position, display is obvious.
Fig. 7 is a kind of flow chart of method for processing video frequency provided by the embodiments of the present application.As shown in fig. 7, step S13 is cut out The process of frame to be processed is as follows:
Step S71 calculates human body center point coordinate according to human body key point coordinate.
Step S72 is calculated according to human body center point coordinate, the resolution ratio of video to be processed and default resolution ratio to default point The reduction ratio of resolution.
Step S73 calculates the first candidate clipping region according to human body center point coordinate, default resolution ratio and reduction ratio.
Step S74, judges whether human region exceeds the range of the first candidate clipping region, if so, executing step S76, if not, executing step S75.
Step S75, when range of the human region without departing from the first candidate clipping region, according to the first candidate clipping region Frame to be processed is cut, clipping region image is obtained.
Step S76, when range of the human region beyond the first candidate clipping region, according to the coordinate of human region, contracting Subtract ratio and default resolution ratio determines the second candidate clipping region.
Step S77 cuts frame to be processed according to the second candidate clipping region, obtains clipping region image.
The process of above-mentioned cutting frame to be processed is described in detail below.
The first step, if being extracted n human body key point, human body center point coordinate are as follows:
Second step, according to pc, video to be processed resolution ratio w wait for place and default resolution ratio w ' × h ' calculating to default resolution The reduction ratio of rate.
Clipping region is set as rectangular area, then can sit and lower right corner seat by calculating the upper left corner of the rectangular area Mark position and the size to determine clipping region.Wherein, w ', h ' are respectively the width and height of clipping region.
Fig. 8 and Fig. 9 is respectively the schematic diagram of human region provided by the embodiments of the present application and clipping region.
According to position of the human region 61 in frame to be processed, if the clipping region upper left corner being calculated and the lower right corner Coordinate is in the frame regional scope to be processed, as shown in figure 8, that can be cut according to default resolution ratio.It is also possible to The case where appearance can not be cut according to default resolution ratio, is cut as shown in figure 9, needing to reduce resolution ratio at this time.
Firstly, determining that coordinate range of the clipping region in x-axis is as follows based on human body center point coordinate:
The coordinate reduction ratio calculated according to the clipping region left border are as follows:
Value range be (0,0.5].
The coordinate reduction ratio calculated according to the clipping region right side boundary are as follows:
Value range be (0,0.5].
The clipping region actual coordinate reduction ratio can select the minimum value of above-mentioned two coordinate reduction ratio:
Value range be (0,0.5].
2 times of the clipping region actual coordinate reduction ratio, i.e.,For the reduction ratio for presetting resolution ratio.
Third step, according to above-mentioned reduction ratio, can calculate the first candidate clipping region in x-axis the maximum value of coordinate and Minimum value are as follows:
For example, situation as shown in Figure 8, w '=10, w=50, when human body center point coordinate is (18,7),Then It, can be according to as it can be seen that do not reduce to clipping region 81 actually Default resolution ratio is cut.
In another example situation as shown in Figure 9, w '=10, w=50, when human body center point coordinate is (3,7), xc=3, ThenIn fact, It can be seen that actually The width of clipping region 82 is 6, is actually 0.6 to the reduction ratio of default resolution ratio.
It can determine that the coordinate range of the first candidate clipping region on the y axis is as follows according to above-mentioned coordinate reduction ratio:
4th step, judges whether human region exceeds the range of the first candidate clipping region.
It can determine whether human region exceeds the first time by comparing the height of the first candidate clipping region and human region Select the range of clipping region.
The height of first candidate clipping regionHuman region it is a height of
5th step, whenWhen, as shown in figure 8, range of the human region without departing from the first candidate clipping region, nothing Human region need to be cut.
It calculates the first candidate clipping region y-coordinate maximum value and minimum value is respectively as follows:
For example, situation as shown in Figure 8, h '=6, when human body center point coordinate is (18,7), the y-axis of clipping region 81 Coordinate range are as follows:
The top left co-ordinate of first candidate clipping regionWith bottom right angular coordinateSuch as Under:
According toWithCoordinate, frame to be processed is cut, clipping region image is obtained.
6th step, whenWhen, i.e. the human region range that has exceeded the first candidate clipping region is then needed to people Body region is cut.As shown in figure 9, clipping region 82 cannot include completely human region, need human region dismissing one Part, can be according to human region top left co-ordinateIt is real to calculate the range of the second candidate clipping region on the y axis Border are as follows:
The top left co-ordinate bottom right angular coordinate of second candidate clipping region is as follows:
For example, situation as shown in Figure 9, h '=6, human body center point coordinate are (3,7), human region upper left corner y-axis is sat MarkHuman region lower right corner y-axis coordinatePass through the above-mentioned reduction ratio being calculatedThen A height of the 3.6 of clipping region after reduction, and a height of the 6 of human region, it needs to cut human region.According toIt can determine The y-axis coordinate range of second candidate clipping region 82 are as follows:
As it can be seen that should guarantee the top half of human region as far as possible when clipping region cannot include human region completely, that is, wrap Being partially contained in clipping region for neck key point is included, the part of the leg of human region can be cut.
In the embodiment of the present application, the position of coordinate origin O may be set to be other positions, for example, frame to be processed is left Upper angle is set as coordinate origin O, and above-mentioned 6th step calculates the second candidate clipping region y-axis coordinate range are as follows:
In the above-described embodiments, the human region in each frame is intercepted as much as possible completely and is come out, when due to human body area Domain is in the marginal position in video, when so that can not normally intercept according to default resolution ratio, can be scaled down default point Resolution is cut;When cannot human region be intercepted completely, then retain the key component of human region.In this way, not only improving The accuracy of the cutting of target person human body image also improves the completeness and efficiency of human body image.In addition, by above-mentioned The true resolution that embodiment can be seen that final clipping region image is likely less than default resolution ratio, at this time, it may be necessary to cutting It cuts out area image and carries out super-resolution processing, the target image for meeting default resolution ratio is obtained, in this way, user can watch more Add continuous clearly video.
Adoptable super-resolution algorithms include, such as super-resolution convolutional neural networks (Super- ResolutionConvolutional Neural Network, SRCNN) algorithm, the enhancing depth for single image super-resolution Rest network (Enhanced Deep Residual Networks for Single Image Super-Resolution, EDSR the above-mentioned resolution processes by clipping region image can be achieved to default resolution ratio, herein not one by one in) algorithm, etc. It enumerates.
Following is embodiment of the present disclosure, can be used for executing embodiments of the present disclosure.
Figure 10 is a kind of block diagram of video process apparatus provided by the embodiments of the present application, which can be by software, hard Part or both is implemented in combination with as some or all of of electronic equipment.As shown in Figure 10, which includes:
Screening module 91, for be processed frame of the screening including target person from video to be processed;
Identification module 92, for identifying the human region of target person from frame to be processed;
Cut module 93, for from frame to be processed cut obtain include human region clipping region image;
Generation module 94 cuts video for generating according to clipping region image.
Figure 11 is the block diagram of screening module 91 provided by the embodiments of the present application, and as shown in figure 11, which includes:
Submodule 101 is split, for video to be processed to be split as video frame;
Frame determines submodule 102, for when in video frame include target person facial image when, determine video frame be to Handle frame.
Further, frame determines submodule 102, and being also used to work as in video frame does not include facial image, before counting video frame Afterwards in preset time period facial image frequency of occurrence;When frequency of occurrence is more than preset times, determine that video frame is to be processed Frame.
Figure 12 is the block diagram of identification module 92 provided by the embodiments of the present application, and as shown in figure 12, which includes:
Key point extracting sub-module 111, for extracting the human body key point of target person from frame to be processed;
Human region computational submodule 112, for calculating the human region of target person according to human body key point.
Figure 13 is the block diagram of identification module 92 provided by the embodiments of the present application, as shown in figure 13, further, identification module 92 further include:
Judging submodule 113, for judging whether human body key point meets preset condition;
Human region computational submodule 112, for executing and calculating human region when human body key point meets preset condition Operation.
Wherein, human body key point meets preset condition, comprising:
Human body key point includes designated key point;
And/or
The number of human body key point is greater than or equal to predetermined number.
Figure 14 is the block diagram provided by the embodiments of the present application for cutting module 93, and as shown in figure 14, which includes:
Central point computational submodule 121, for calculating human body center point coordinate according to human body key point coordinate;
Reduction ratio computational submodule 122, for according to the resolution ratio of human body center point coordinate, video to be processed and default Resolution ratio calculates the reduction ratio to default resolution ratio;
Coordinate computational submodule 123, for calculating first according to human body center point coordinate, default resolution ratio and reduction ratio Candidate clipping region;
Judging submodule 124, for judging whether human region exceeds the range of the first candidate clipping region:
Submodule 125 is cut, for being waited according to first when range of the human region without departing from the first candidate clipping region It selects clipping region to cut frame to be processed, obtains clipping region image.
As shown in figure 14, the cutting module 93 further include:
Coordinate computational submodule 123, when for exceeding the range of the first candidate clipping region when human region, according to human body Coordinate, reduction ratio and the default resolution ratio in region determine the second candidate clipping region;
Submodule 125 is cut, for cutting according to the second candidate clipping region to frame to be processed, obtains clipping region Image.
Figure 15 is the block diagram of generation module 94 provided by the embodiments of the present application, and as shown in figure 15, which includes:
Super-resolution processing submodule 131, for when the true resolution of cut out areas image is less than default resolution ratio, Super-resolution processing is carried out to cut out areas image, obtains the target image for meeting default resolution ratio;
Synthesis submodule 132 obtains cutting out video for synthesizing target image.
The embodiment of the present application also provides a kind of electronic equipment, and as shown in figure 16, electronic equipment may include: processor 1501, communication interface 1502, memory 1503 and communication bus 1504, wherein processor 1501, communication interface 1502, storage Device 1503 completes mutual communication by communication bus 1504.
Memory 1503, for storing computer program;
Processor 1501 when for executing the program stored on memory 1503, realizes the step of above method embodiment Suddenly.The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral ComponentInterconnect, P C I) bus or expanding the industrial standard structure (Extended Industry StandardArchitecture, EISA) bus etc..The communication bus can be divided into address bus, data/address bus, control bus Deng.Only to be indicated with a thick line in figure, it is not intended that an only bus or a type of bus convenient for indicating.
Communication interface is for the communication between above-mentioned electronic equipment and other equipment.
Memory may include random access memory (Random Access Memory, RAM), also may include non-easy The property lost memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory may be used also To be storage device that at least one is located remotely from aforementioned processor.
Above-mentioned processor can be general processor, including central processing unit (CentralProcessing Unit, CPU), network processing unit (Network Processor, NP) etc.;It can also be digital signal processor (Digital SignalProcessing, DSP), specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic Device, discrete gate or transistor logic, discrete hardware components.
The application also provides a kind of computer readable storage medium, is stored thereon with computer program, the computer program The step of above method embodiment is realized when being executed by processor.
It should be noted that for above-mentioned apparatus, electronic equipment and computer readable storage medium embodiment, due to It is substantially similar to embodiment of the method, so being described relatively simple, related place is referring to the part explanation of embodiment of the method It can.
Explanation is needed further exist for, herein, the relational terms of such as " first " and " second " or the like are only used Distinguish one entity or operation from another entity or operation, without necessarily requiring or implying these entities or There are any actual relationship or orders between operation.Moreover, the terms "include", "comprise" or its any other change Body is intended to non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wrapped Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article Or the element that equipment is intrinsic.In the absence of more restrictions, the element limited by sentence " including one ... ", Be not precluded include element process, method, article or equipment in there is also other identical elements.
The above is only a specific embodiment of the invention, make skilled artisans appreciate that or realizing of the invention.It is right A variety of modifications of these embodiments will be apparent to one skilled in the art, general original as defined herein Reason can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, the present invention will not It is intended to be limited to the embodiments shown herein, and is to fit to consistent most with applied principle and features of novelty herein Wide range.

Claims (11)

1. a kind of method for processing video frequency characterized by comprising
Screening includes the frame to be processed of target person from video to be processed;
The human region of the target person is identified from the frame to be processed;
From the frame to be processed cut obtain include the human region clipping region image;
It is generated according to the clipping region image and cuts video.
2. the method according to claim 1, wherein the screening from video to be processed includes target person Frame to be processed, comprising:
The video to be processed is split as video frame;
When in the video frame including the facial image of the target person, determine that the video frame is the frame to be processed.
3. according to the method described in claim 2, it is characterized in that, the screening from video to be processed includes target person Frame to be processed, further includes:
When not including the facial image in the video frame, the face figure in the preset time period of the video frame front and back is counted The frequency of occurrence of picture;
When the frequency of occurrence is more than preset times, determine that the video frame is the frame to be processed.
4. the method according to claim 1, wherein described identify the target person from the frame to be processed Human region, comprising:
The human body key point of the target person is extracted from the frame to be processed;
The human region of the target person is calculated according to the human body key point.
5. according to the method described in claim 4, it is characterized in that, described identify the target person from the frame to be processed Human region, further includes:
Judge whether the human body key point meets preset condition;
When the human body key point meets preset condition, the operation for calculating the human region is executed;
The human body key point meets preset condition, comprising:
The human body key point includes designated key point;
And/or
The number of the human body key point is greater than or equal to predetermined number.
6. according to the method described in claim 4, obtaining including the human body it is characterized in that, cutting from the frame to be processed The clipping region image in region, comprising:
Human body center point coordinate is calculated according to the human body key point coordinate;
It is calculated according to the human body center point coordinate, the resolution ratio of video to be processed and default resolution ratio to the default resolution ratio Reduction ratio;
The first candidate clipping region is calculated according to the human body center point coordinate, the default resolution ratio and the reduction ratio;
Judge whether the human region exceeds the range of the described first candidate clipping region;
When range of the human region without departing from the described first candidate clipping region, according to the described first candidate clipping region The frame to be processed is cut, the clipping region image is obtained.
7. according to the method described in claim 6, obtaining including the human body it is characterized in that, cutting from the frame to be processed The clipping region image in region, further includes:
When range of the human region beyond the described first candidate clipping region, according to the coordinate of the human region, institute It states reduction ratio and the default resolution ratio determines the second candidate clipping region;
The frame to be processed is cut according to the described second candidate clipping region, obtains the clipping region image.
8. the method according to claim 1, wherein described generated according to the clipping region image cuts view Frequently, comprising:
When the true resolution of the cut out areas image is less than default resolution ratio, oversubscription is carried out to the cut out areas image Resolution processing, obtains the target image for meeting the default resolution ratio;
The target image is synthesized, obtains described cutting out video.
9. a kind of video process apparatus characterized by comprising
Screening module, for be processed frame of the screening including target person from video to be processed;
Identification module, for identifying the human region of the target person from the frame to be processed;
Cut module, for from the frame to be processed cut obtain include the human region clipping region image;
Generation module cuts video for generating according to the clipping region image.
10. a kind of electronic equipment characterized by comprising processor, communication interface, memory and communication bus, wherein place Device, communication interface are managed, memory completes mutual communication by communication bus;
The memory, for storing computer program;
The processor when for executing the program stored on memory, realizes the described in any item methods of claim 1-8 Step.
11. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program quilt Claim 1-8 described in any item method and steps are realized when processor executes.
CN201910569106.6A 2019-06-27 2019-06-27 Video processing method and device, electronic equipment and storage medium Active CN110347877B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910569106.6A CN110347877B (en) 2019-06-27 2019-06-27 Video processing method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910569106.6A CN110347877B (en) 2019-06-27 2019-06-27 Video processing method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN110347877A true CN110347877A (en) 2019-10-18
CN110347877B CN110347877B (en) 2022-02-11

Family

ID=68176788

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910569106.6A Active CN110347877B (en) 2019-06-27 2019-06-27 Video processing method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN110347877B (en)

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110856014A (en) * 2019-11-05 2020-02-28 北京奇艺世纪科技有限公司 Moving image generation method, moving image generation device, electronic device, and storage medium
CN111145093A (en) * 2019-12-20 2020-05-12 北京五八信息技术有限公司 Image display method, image display device, electronic device, and storage medium
CN111179281A (en) * 2019-12-24 2020-05-19 广东省智能制造研究所 Human body image extraction method and human body action video extraction method
CN111222493A (en) * 2020-01-20 2020-06-02 北京捷通华声科技股份有限公司 Video processing method and device
CN111311617A (en) * 2020-03-26 2020-06-19 北京奇艺世纪科技有限公司 Method, device and equipment for cutting dynamic graph and storage medium
CN111356016A (en) * 2020-03-11 2020-06-30 北京松果电子有限公司 Video processing method, video processing apparatus, and storage medium
CN111460219A (en) * 2020-04-01 2020-07-28 百度在线网络技术(北京)有限公司 Video processing method and device and short video platform
CN111881755A (en) * 2020-06-28 2020-11-03 腾讯科技(深圳)有限公司 Method and device for cutting video frame sequence
CN112131984A (en) * 2020-09-11 2020-12-25 咪咕文化科技有限公司 Video clipping method, electronic device and computer-readable storage medium
CN112132836A (en) * 2020-08-14 2020-12-25 咪咕文化科技有限公司 Video image clipping method and device, electronic equipment and storage medium
CN112164108A (en) * 2020-11-27 2021-01-01 大汉软件股份有限公司 Method for ensuring correct display of position of character in thumbnail
CN112612434A (en) * 2020-12-16 2021-04-06 杭州当虹科技股份有限公司 Video vertical screen solution method based on AI technology
CN112765399A (en) * 2020-12-25 2021-05-07 联想(北京)有限公司 Video data processing method and electronic equipment
CN112800805A (en) * 2019-10-28 2021-05-14 上海哔哩哔哩科技有限公司 Video editing method, system, computer device and computer storage medium
CN112860633A (en) * 2019-11-28 2021-05-28 上海宇季文化传播有限公司 Animation file searching system for graphic features
CN112948627A (en) * 2019-12-11 2021-06-11 杭州海康威视数字技术股份有限公司 Alarm video generation method, display method and device
CN113014793A (en) * 2019-12-19 2021-06-22 华为技术有限公司 Video processing method and electronic equipment
CN113269790A (en) * 2021-03-26 2021-08-17 北京达佳互联信息技术有限公司 Video clipping method and device, electronic equipment, server and storage medium
CN113469113A (en) * 2021-07-19 2021-10-01 浙江大华技术股份有限公司 Action counting method and device, electronic equipment and storage medium
CN113591644A (en) * 2021-07-21 2021-11-02 此刻启动(北京)智能科技有限公司 Mirror-moving video processing method and system, storage medium and electronic equipment
WO2021217927A1 (en) * 2020-04-29 2021-11-04 平安国际智慧城市科技股份有限公司 Video-based exercise evaluation method and apparatus, and computer device and storage medium
CN113613059A (en) * 2021-07-30 2021-11-05 杭州时趣信息技术有限公司 Short-cast video processing method, device and equipment
CN114448952A (en) * 2020-10-19 2022-05-06 腾讯科技(深圳)有限公司 Streaming media data transmission method and device, storage medium and electronic equipment
WO2022100162A1 (en) * 2020-11-13 2022-05-19 深圳市前海手绘科技文化有限公司 Method and apparatus for producing dynamic shots in short video
CN114885210A (en) * 2022-04-22 2022-08-09 海信集团控股股份有限公司 Course video processing method, server and display equipment
CN116074582A (en) * 2023-01-31 2023-05-05 北京奇艺世纪科技有限公司 Implant position determining method and device, electronic equipment and storage medium
CN116074581A (en) * 2023-01-31 2023-05-05 北京奇艺世纪科技有限公司 Implant position determining method and device, electronic equipment and storage medium
CN117082207A (en) * 2023-06-16 2023-11-17 广东视安通智慧显控股份有限公司 Doorbell system based on image motion shearing

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070126884A1 (en) * 2005-12-05 2007-06-07 Samsung Electronics, Co., Ltd. Personal settings, parental control, and energy saving control of television with digital video camera
CN103428537A (en) * 2013-07-30 2013-12-04 北京小米科技有限责任公司 Video processing method and video processing device
CN103442252A (en) * 2013-08-21 2013-12-11 宇龙计算机通信科技(深圳)有限公司 Method and device for processing video
CN106548148A (en) * 2016-10-26 2017-03-29 北京意泰物联软件科技有限公司 Method and system for identifying unknown face in video
CN107809670A (en) * 2017-10-31 2018-03-16 长光卫星技术有限公司 Suitable for the video clipping system and method for large area array meter level high-resolution satellite
CN108710829A (en) * 2018-04-19 2018-10-26 北京红云智胜科技有限公司 A method of the expression classification based on deep learning and the detection of micro- expression
CN109040780A (en) * 2018-08-07 2018-12-18 北京优酷科技有限公司 A kind of method for processing video frequency and server
CN109190454A (en) * 2018-07-17 2019-01-11 北京新唐思创教育科技有限公司 The method, apparatus, equipment and medium of target person in video for identification
CN109447072A (en) * 2018-11-08 2019-03-08 北京金山安全软件有限公司 Thumbnail clipping method and device, electronic equipment and readable storage medium
CN109670474A (en) * 2018-12-28 2019-04-23 广东工业大学 A kind of estimation method of human posture based on video, device and equipment
CN113423001A (en) * 2021-06-16 2021-09-21 深圳市海雀科技有限公司 Method, system and device for playing video

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070126884A1 (en) * 2005-12-05 2007-06-07 Samsung Electronics, Co., Ltd. Personal settings, parental control, and energy saving control of television with digital video camera
CN103428537A (en) * 2013-07-30 2013-12-04 北京小米科技有限责任公司 Video processing method and video processing device
CN103442252A (en) * 2013-08-21 2013-12-11 宇龙计算机通信科技(深圳)有限公司 Method and device for processing video
CN106548148A (en) * 2016-10-26 2017-03-29 北京意泰物联软件科技有限公司 Method and system for identifying unknown face in video
CN107809670A (en) * 2017-10-31 2018-03-16 长光卫星技术有限公司 Suitable for the video clipping system and method for large area array meter level high-resolution satellite
CN108710829A (en) * 2018-04-19 2018-10-26 北京红云智胜科技有限公司 A method of the expression classification based on deep learning and the detection of micro- expression
CN109190454A (en) * 2018-07-17 2019-01-11 北京新唐思创教育科技有限公司 The method, apparatus, equipment and medium of target person in video for identification
CN109040780A (en) * 2018-08-07 2018-12-18 北京优酷科技有限公司 A kind of method for processing video frequency and server
CN109447072A (en) * 2018-11-08 2019-03-08 北京金山安全软件有限公司 Thumbnail clipping method and device, electronic equipment and readable storage medium
CN109670474A (en) * 2018-12-28 2019-04-23 广东工业大学 A kind of estimation method of human posture based on video, device and equipment
CN113423001A (en) * 2021-06-16 2021-09-21 深圳市海雀科技有限公司 Method, system and device for playing video

Cited By (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112800805A (en) * 2019-10-28 2021-05-14 上海哔哩哔哩科技有限公司 Video editing method, system, computer device and computer storage medium
CN110856014B (en) * 2019-11-05 2023-03-07 北京奇艺世纪科技有限公司 Moving image generation method, moving image generation device, electronic device, and storage medium
CN110856014A (en) * 2019-11-05 2020-02-28 北京奇艺世纪科技有限公司 Moving image generation method, moving image generation device, electronic device, and storage medium
CN112860633B (en) * 2019-11-28 2022-11-29 上海宇季文化传播有限公司 Animation file searching system for graphic features
CN112860633A (en) * 2019-11-28 2021-05-28 上海宇季文化传播有限公司 Animation file searching system for graphic features
CN112948627A (en) * 2019-12-11 2021-06-11 杭州海康威视数字技术股份有限公司 Alarm video generation method, display method and device
CN113014793A (en) * 2019-12-19 2021-06-22 华为技术有限公司 Video processing method and electronic equipment
WO2021121374A1 (en) * 2019-12-19 2021-06-24 华为技术有限公司 Video processing method and electronic device
CN111145093A (en) * 2019-12-20 2020-05-12 北京五八信息技术有限公司 Image display method, image display device, electronic device, and storage medium
CN111179281A (en) * 2019-12-24 2020-05-19 广东省智能制造研究所 Human body image extraction method and human body action video extraction method
CN111222493A (en) * 2020-01-20 2020-06-02 北京捷通华声科技股份有限公司 Video processing method and device
CN111356016A (en) * 2020-03-11 2020-06-30 北京松果电子有限公司 Video processing method, video processing apparatus, and storage medium
US11488383B2 (en) 2020-03-11 2022-11-01 Beijing Xiaomi Pinecone Electronics Co., Ltd. Video processing method, video processing device, and storage medium
CN111311617A (en) * 2020-03-26 2020-06-19 北京奇艺世纪科技有限公司 Method, device and equipment for cutting dynamic graph and storage medium
CN111311617B (en) * 2020-03-26 2024-07-19 北京奇艺世纪科技有限公司 Method, device, equipment and storage medium for cutting out dynamic diagram
CN111460219A (en) * 2020-04-01 2020-07-28 百度在线网络技术(北京)有限公司 Video processing method and device and short video platform
WO2021217927A1 (en) * 2020-04-29 2021-11-04 平安国际智慧城市科技股份有限公司 Video-based exercise evaluation method and apparatus, and computer device and storage medium
CN111881755A (en) * 2020-06-28 2020-11-03 腾讯科技(深圳)有限公司 Method and device for cutting video frame sequence
CN111881755B (en) * 2020-06-28 2022-08-23 腾讯科技(深圳)有限公司 Method and device for cutting video frame sequence
CN112132836A (en) * 2020-08-14 2020-12-25 咪咕文化科技有限公司 Video image clipping method and device, electronic equipment and storage medium
CN112131984A (en) * 2020-09-11 2020-12-25 咪咕文化科技有限公司 Video clipping method, electronic device and computer-readable storage medium
CN114448952A (en) * 2020-10-19 2022-05-06 腾讯科技(深圳)有限公司 Streaming media data transmission method and device, storage medium and electronic equipment
WO2022100162A1 (en) * 2020-11-13 2022-05-19 深圳市前海手绘科技文化有限公司 Method and apparatus for producing dynamic shots in short video
CN112164108A (en) * 2020-11-27 2021-01-01 大汉软件股份有限公司 Method for ensuring correct display of position of character in thumbnail
CN112612434A (en) * 2020-12-16 2021-04-06 杭州当虹科技股份有限公司 Video vertical screen solution method based on AI technology
CN112765399A (en) * 2020-12-25 2021-05-07 联想(北京)有限公司 Video data processing method and electronic equipment
CN113269790A (en) * 2021-03-26 2021-08-17 北京达佳互联信息技术有限公司 Video clipping method and device, electronic equipment, server and storage medium
CN113469113A (en) * 2021-07-19 2021-10-01 浙江大华技术股份有限公司 Action counting method and device, electronic equipment and storage medium
CN113591644A (en) * 2021-07-21 2021-11-02 此刻启动(北京)智能科技有限公司 Mirror-moving video processing method and system, storage medium and electronic equipment
CN113613059A (en) * 2021-07-30 2021-11-05 杭州时趣信息技术有限公司 Short-cast video processing method, device and equipment
CN113613059B (en) * 2021-07-30 2024-01-26 杭州时趣信息技术有限公司 Short-cast video processing method, device and equipment
CN114885210A (en) * 2022-04-22 2022-08-09 海信集团控股股份有限公司 Course video processing method, server and display equipment
CN114885210B (en) * 2022-04-22 2023-11-28 海信集团控股股份有限公司 Tutorial video processing method, server and display device
CN116074582A (en) * 2023-01-31 2023-05-05 北京奇艺世纪科技有限公司 Implant position determining method and device, electronic equipment and storage medium
CN116074581A (en) * 2023-01-31 2023-05-05 北京奇艺世纪科技有限公司 Implant position determining method and device, electronic equipment and storage medium
CN117082207A (en) * 2023-06-16 2023-11-17 广东视安通智慧显控股份有限公司 Doorbell system based on image motion shearing

Also Published As

Publication number Publication date
CN110347877B (en) 2022-02-11

Similar Documents

Publication Publication Date Title
CN110347877A (en) A kind of method for processing video frequency, device, electronic equipment and storage medium
JP4824411B2 (en) Face extraction device, semiconductor integrated circuit
CN112288665B (en) Image fusion method and device, storage medium and electronic equipment
JP4473754B2 (en) Virtual fitting device
US8706663B2 (en) Detection of people in real world videos and images
CN104134435B (en) Image processing equipment and image processing method
CN108875533B (en) Face recognition method, device, system and computer storage medium
WO2019237745A1 (en) Facial image processing method and apparatus, electronic device and computer readable storage medium
CN104794462A (en) Figure image processing method and device
CN112102340B (en) Image processing method, apparatus, electronic device, and computer-readable storage medium
TWI525555B (en) Image processing apparatus and processing method thereof
KR101786754B1 (en) Device and method for human age estimation
US20190370537A1 (en) Keypoint detection to highlight subjects of interest
TW202044194A (en) Image processing method, apparatus and computer storage medium
CN111489290A (en) Face image super-resolution reconstruction method and device and terminal equipment
CN110458790A (en) A kind of image detecting method, device and computer storage medium
CN108765317A (en) A kind of combined optimization method that space-time consistency is stablized with eigencenter EMD adaptive videos
CN110505398B (en) Image processing method and device, electronic equipment and storage medium
CN107146197A (en) A kind of reduced graph generating method and device
JP2017045441A (en) Image generation method and image generation system
CN111008935A (en) Face image enhancement method, device, system and storage medium
CN106997580B (en) Picture processing method and device
CN113706373A (en) Model reconstruction method and related device, electronic equipment and storage medium
CN107529071A (en) A kind of video data handling procedure and device
JP6403207B2 (en) Information terminal equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant