CN109511019A - A kind of video summarization method, terminal and computer readable storage medium - Google Patents

A kind of video summarization method, terminal and computer readable storage medium Download PDF

Info

Publication number
CN109511019A
CN109511019A CN201710827915.3A CN201710827915A CN109511019A CN 109511019 A CN109511019 A CN 109511019A CN 201710827915 A CN201710827915 A CN 201710827915A CN 109511019 A CN109511019 A CN 109511019A
Authority
CN
China
Prior art keywords
moving target
motion profile
frame
video
original
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN201710827915.3A
Other languages
Chinese (zh)
Inventor
范贤友
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN201710827915.3A priority Critical patent/CN109511019A/en
Publication of CN109511019A publication Critical patent/CN109511019A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/8549Creating video summaries, e.g. movie trailer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Image Analysis (AREA)

Abstract

The present invention provides a kind of video summarization method, terminal and computer readable storage mediums, determine the motion profile of moving target each in original video, according to relative positional relationship of each motion profile in original video, relative position of each motion profile in summarized radio is determined;Then, according to the relative position between determining first motion track and each motion profile, each motion profile is synthesized, forms summarized radio.Implementation through the invention carries out shearing to original video for reference with moving target and plucks choosing, thereby may be ensured that compression ratio, and remain video main body, i.e. moving target again, ensure that the rich of content to greatest extent, improve the effect of video frequency abstract.

Description

A kind of video summarization method, terminal and computer readable storage medium
Technical field
The present invention relates to multimedia technology field more particularly to a kind of video summarization method, terminal and calculate readable storage Medium.
Background technique
Video summarization technique is either video compress or Video summary technology, is a kind of video matchmaker emerging in recent years Body processing technique, development prospect are very wide.It can be very long by one end video, keeping original information content and critical event While be concentrated within even several seconds a few minutes, because of referred to herein as video " abstract ".
The video frequency abstract algorithm of mainstream at present, can conclude are as follows: key frame (key frame) method, video are skipped (video Skim) method and dynamic video abstract (dynamic video synopsis).
Key frame method is a kind of video summarization method of static state, determines to use often by the importance for comparing different frame Which frame is as key frame, as a result, being presented by the static images of one or more key frame.Key frame approach meeting The multidate information for losing target, cannot retain complete dynamic process, have biggish limitation.
Video is skipped, and is selectively to browse video clip, and the letter of entire video is represented with the combination of critical segment Breath, a typical way are exactly then quickly to play, when there is critical event generation, then just when without motion target in video It often plays, this method remains the partial dynamic information of behavior, but compresses relatively low.
However, in current video frequency abstract algorithm or the degree of compression is big, but the content lost is more, it is such as crucial Frame method or the content of reservation are more, but the degree compressed is small, if video is skipped method, be exactly difficult to combine compression ratio and Content.
Summary of the invention
The embodiment of the invention provides a kind of video summarization method, terminal and computer readable storage mediums, it is intended to solve Video frequency abstract can not combine compression ratio and content in the prior art, the problem for effect difference of making a summary.
In order to solve the above-mentioned technical problem, the embodiment of the invention provides a kind of video summarization methods, comprising:
Determine the motion profile of moving target each in original video;The motion profile includes: the moving target in original The set of primitive frame in video;
According to relative positional relationship of each motion profile in original video, phase of each motion profile in summarized radio is determined To position;
According to the phase of first motion fixed in summarized radio track and each motion profile in summarized radio To position, each motion profile is synthesized, the summarized radio is formed.
In addition, the embodiment of the present invention also provides a kind of video frequency abstract device, comprising:
Module of target detection, for determining the motion profile of moving target each in original video;The motion profile includes: The set of primitive frame of the moving target in original video;
Track determining module determines each movement rail for the relative positional relationship according to each motion profile in original video Relative position of the mark in summarized radio;
Synthesis module is being made a summary according to first motion fixed in summarized radio track and each motion profile Each motion profile is synthesized, forms the summarized radio by the relative position in video.
In addition, the embodiment of the present invention also provides a kind of terminal, including processor, memory and communication bus;The communication Bus is for realizing the connection communication between the processor and memory;The processor is deposited in the memory for executing The video frequency abstract program of storage, the step of to realize video summarization method above-mentioned.
In addition, the embodiment of the present invention also provides a kind of computer readable storage medium, the computer readable storage medium It is stored with one or more computer program, before the computer program can be executed by one or more processor to realize The step of video summarization method stated.
The beneficial effects of the present invention are:
The present invention provides a kind of video summarization method, terminal and computer readable storage mediums, determine in original video The motion profile of each moving target determines that each motion profile exists according to relative positional relationship of each motion profile in original video Relative position in summarized radio;It then, will according to the relative position between determining first motion track and each motion profile Each motion profile is synthesized, and summarized radio is formed.Implementation through the invention, with moving target be referring to original video into Choosing is plucked in row shearing, thereby may be ensured that compression ratio, and remain video main body, i.e. moving target again, is guaranteed to greatest extent Content it is rich, improve the effect of video frequency abstract.
Detailed description of the invention
Fig. 1 is a kind of video summarization method flow chart that first embodiment of the invention provides;
Fig. 2 is a kind of motion profile figure for moving target that second embodiment of the invention provides;
Fig. 3 is that a kind of summarized radio that third embodiment of the invention provides synthesizes schematic diagram;
Fig. 4 is that a kind of summarized radio that second embodiment of the invention provides synthesizes schematic diagram;
Fig. 5 is a kind of video summarization method flow chart that third embodiment of the invention provides;
Fig. 6 is a kind of video frequency abstract device composition schematic diagram that third embodiment of the invention provides;
Fig. 7 is a kind of terminal composition schematic diagram that fourth embodiment of the invention provides.
Specific embodiment
First embodiment
Referring to FIG. 1, Fig. 1 is a kind of video summarization method flow chart that first embodiment of the invention provides, comprising:
S101, the motion profile for determining moving target each in original video;Motion profile includes: moving target in original video In primitive frame set;
S102, the relative positional relationship according to each motion profile in original video determine each motion profile in summarized radio In relative position;
S103, according to first motion fixed in summarized radio track and each motion profile in summarized radio Relative position synthesizes each motion profile, forms summarized radio.
In S101, the motion profile of moving target each in original video is determined.Wherein, moving target as its name suggests, refers to It is in video for dynamic target.It is to occur to a certain extent certainly among entire video for moving target Variation, the specific manifestation of variation are then usually that the displacement of pixel occurs, or from scratch, from the process having to nothing. Video file is as composed by multiple frames, these frame pictures are played out according to certain time interval can form finally Video, in other words, each moving target, the actually component part of video file, that is, frame tableaux come It presents.Accordingly, it is determined that the motion profile of each moving target may include: determining moving target in original video in original video Location information;The motion profile of moving target is determined according to the location information of moving target.The wherein position letter of moving target Breath includes the primitive frame where moving target and the position in corresponding primitive frame.Primitive frame indicates the frame in original video, Original frame number, then it represents that the serial number of primitive frame, the totalframes of a video file are that indefinite, few possibility has several hundred frames, and More, video content is longer, may include frames up to ten thousand.One video file can find smallest partition as unit of frame, And moving target persistently changes position, to form the image of movement just in these continuous frames.Moving target is really It is fixed, other than movement, according to actual needs, under the premise of movement, then certain screening can also be carried out, select and more accord with Desired moving target is closed, the moving target of these identifications is also based primarily upon for the tracking of video to carry out.
Dynamic part in one video file be it is very much, therefore, in order to accurately judge whether the target belongs to Same moving target determines that location information of the moving target in original video may include: according to feature phase in the present embodiment Judge whether the moving target in each frame belongs to same moving target like degree;Based on judging result, the movement of moving target is determined Track.Specifically, according to characteristic similarity, including LBP (Local Binary Patterns, local binary patterns), Haar- Like (Haar-like is a kind of description method of graphic feature, is gained the name because it is similar to Haar wavelet transformation), histogram etc. Etc. modes, can determine whether the object inside different frame is the image for belonging to same target, can also be further determined that It whether is moving target.Further, it is also possible to according to other informations such as overlapping areas, by the moving target of present frame and former frame Moving target is associated.Associated direct result is exactly to have obtained the motion profile of moving target, rising including moving target Initial point and terminating point.Wherein, the specific method of determination of start frame and abort frame may include presetting when detecting that similarity meets It is required that object movement has occurred, then while regarded as moving target, recall it and mobile frame do not occur initially, or Its frame not occurred is traced back to, as start frame;And abort frame, then it is for a moving target, alreading exceed N frame does not have It updates, that is, there is no when movement, being then regarded as the target stop motion, can not send out the target for position Raw mobile first frame is as abort frame.In this case, if in subsequent moving object detection, discovery should be with once again Regard as stop moving target movement has occurred again, then can also by the moving target again with moving target before into Row combination, is still considered as a motion profile.Correspondingly, when a moving target is in the moving target tracked, according to similar Degree or the information such as overlapping area can not find and match, then this moving target is considered as new moving target carry out with Track.
The motion profile of moving target is determined according to the location information of moving target.The motion profile of moving target is exactly The set of the primitive frame of original video including the moving target, for the same moving target, the set of this primitive frame It is usually continuous, the track of the movement of the moving target in video is reflected on the whole, from start frame to abort frame.And In one video file, often there are multiple moving targets, and at the beginning of the movement of these moving targets and the end time It is all not quite similar, that is, the start frame and abort frame of each moving target are not necessarily identical, referring to FIG. 2, Fig. 2 shows three The integration of the motion profile of a moving target A, B, C, D shows, and wherein moving target A is since the 1st frame of original video to the 9th frame Terminate, moving target B is to 28 frame ends since 20 frames of original video, and moving target C is since 46 frames of original video to 49 frames Terminate;Moving target D is then since the 47th frame to the 50th frame end.It can be seen that the start-stop frame of four moving targets is not Identical, therefore, the set for the primitive frame that corresponding motion profile is included also is not quite similar.
In S102, according to relative positional relationship of each motion profile in original video, determine that each motion profile is regarded in abstract Relative position in frequency.Relative position of the motion profile of each moving target in original video is determining, and in summarized radio In, in order to shorten the duration of summarized radio as far as possible, the interval of the corresponding primitive frame of each motion profile is not more than in original video In interval.So, in order to determine relative position of the motion profile of moving target in summarized radio, in the present embodiment, May include:
In the motion profile for judging different moving targets, if there are identical original frame numbers;When there are identical originals When beginning frame number, according to identical original frame number, the relative position between corresponding motion profile is determined;Wherein, have identical Relative position of the motion profile of original frame number in summarized radio is consistent with the relative position in original video.Wherein, different Motion profile there are identical original frame number, expression, for the moving target different for two, motion profile has Common primitive frame, two moving targets occurred simultaneously at least in original video.So, in this context, a kind of optional Embodiment be exactly retain the positional relationship between the two moving targets in summarized radio, that is, original frame number and Position in corresponding primitive frame, the interaction that can retain as far as possible between different motion target that may be present in this way are closed System.The specific practice for retaining the positional relationship between two moving targets is, according to identical frame number, to determine the original of start frame Frame number is how many, it is, the start frame of the motion profile of one of moving target is determined, in the fortune of another moving target Position in dynamic rail mark.Usually determine the movement of start frame of the start frame of shorter motion profile in longer motion profile Position in track, this guarantees that determining result still falls within the movement of the longer moving target of motion profile to a certain extent In footprint.In addition, it is noted that the synthesis process of summarized radio, can also be in the summarized radio synthesized, It is inserted into the motion profile of new moving target, in this case, the summarized radio synthesized also can be considered the fortune of moving target Dynamic rail mark, track here may be the combination of the motion profile of multiple moving targets certainly.It in this case, then can be true Surely the start frame for the new moving target being inserted into corresponding position in summarized radio, then inserts the motion profile of new moving target Enter in summarized radio.Specifically, assuming to find in the motion profile of new moving target, movement of the xth frame in other moving targets Exist in track, and is y frame, then, the movement rail of the start frame of the motion profile of new moving target in other moving targets Position in mark is then y-x frame.Position of the start frame in the motion profile of other moving targets, the i.e. fortune of the moving target Initial position of the dynamic rail mark in summarized radio.Referring to FIG. 2, moving target C and D have same number of frames in original video in Fig. 2, It is exactly that C and D has identical frame number;Referring to FIG. 3, Fig. 3, which shows moving target C, D in Fig. 2, is collectively referred to as a summarized radio Schematic diagram.Wherein, the primitive frame that xth frame and y frame all refer to.
Further, when there are identical frame number, determine that initial position of the moving target in summarized radio can also wrap It includes: when the motion profile of moving target at least has two identical frame numbers with the motion profile of other moving targets, according to phase With the frame number and phase of the initial frame of position and moving target of any one in frame number in the motion profile of moving target With the spacing value between frame number, recalls initial frame position corresponding in summarized radio, determine moving target in summarized radio In initial position.Identical primitive frame between the motion profile of multiple moving targets often more than one, it is likely that have more A identical primitive frame;When there are identical primitive frame, under normal circumstances since the movement of moving target is continuous, then These frame numbers are also continuous, so selecting any one same number of frames to determine the start frame of the motion profile of moving target i.e. It can.For convenience's sake, first identical primitive frame can be directly selected to determine the motion profile of moving target.
In addition, determining whether the track degree of overlapping of the motion profile of each moving target is less than when identical frame number is not present Equal to preset threshold, and the most short summarized radio duration according to track degree of overlapping less than or equal to preset threshold, determine moving target Initial frame position corresponding in summarized radio, obtain initial position of the moving target in summarized radio.There is no phases Same frame number, then it represents that the motion profile of each moving target did not occur simultaneously in original video, was all respectively to appear in difference Frame in, then, need not just limit position of the motion profile of each moving target in summarized radio in this case, move It can further be compressed between the motion profile of target.Specifically, being then the motion profile according to each moving target Track degree of overlapping determines.Wherein, what track degree of overlapping referred to is exactly the degree of overlapping between motion profile, due to each moving target There is no same number of frames between motion profile, then under certain condition, the motion profile of the moving target in original video rearward moves The position forward in summarized radio is moved, to reduce the duration of summarized radio, increases the compression ratio of summarized radio.So, when It when taking such measure, is then likely to occur, juxtaposition occurs in the motion profile of each moving target, in view of this, calculating each Track degree of overlapping between the motion profile of moving target, under the premise of meeting the condition of degree of overlapping, resultant motion target Motion profile, so that the duration of summarized radio is most short.
Specifically, if overlapping area between the image of different moving targets, with the lesser moving target of wherein area The area ratio of image be denoted as collision ratio, then track degree of overlapping includes the motion profile and other moving targets of moving target The sum of collision ratio between motion profile.Particularly, the synthesis process of summarized radio can also be in the summarized radio synthesized In, it is inserted into the motion profile of new moving target, in this case, the summarized radio synthesized also can be considered moving target Motion profile, track here may be the combination of the motion profile of multiple moving targets certainly.It in this case, then can be with The motion profile of each moving target in the motion profile and summarized radio of new moving target is determined into track degree of overlapping one by one, and Summation, as the motion profile of new moving target and the whole track degree of overlapping of summarized radio.In track, degree of overlapping is little When preset threshold, for example preset threshold can be set to differ from 0%~10%, and resulting track degree of overlapping is in this range It inside can be considered the track degree of overlapping met the requirements, then, under the premise of this, by the start frame of the motion profile of moving target Position as early as possible is set, i.e., shortens the duration of summarized radio as far as possible.Referring to FIG. 2, moving target A, B in Fig. 2, Identical frame number is not present between C;Referring to FIG. 4, Fig. 4 shows the abstract view in Fig. 2 after moving target A, B, C three synthesis The schematic diagram of frequency.
In the present embodiment, before determining initial position of the motion profile of each moving target in summarized radio, It can also include: that the quantity of the moving target for from the motion profile of each moving target, being included is dropped according to each frame number Sequence sequence, and sequentially determine each moving target motion profile and subsequent processing.Due to the movement mesh in an original video Mark be have it is multiple, when carrying out video frequency abstract, may be related to for each moving target processing successive problem;? In the present embodiment, a kind of optional mode is then with the sequence containing the frame number more than moving target first, to carry out video frequency abstract Processing, moving target is more, and the difficulty of processing is then bigger, and other subsequent moving targets it is less frame processing will more along reason It comes out as an article;For example, there are 10 moving targets in frame x, there is 1 moving target in frame y, the moving target in frame x is more, first to frame x In the corresponding track of moving target handled, can make the corresponding moving target of other subsequent frames processing Shi Gengrong Identical frame number is easily found, furthermore the moving target of frame x is more, and more crowded for other opposite frames, wherein reference locus is overlapped The factor of degree can moving target fewer than other more complicated, and only one moving target in frame y, in contrast track is overlapped Degree is just more preferable to be determined.
In the present embodiment, in the motion profile for determining moving target, motion profile includes the image of moving target The set of primitive frame;Specifically, the set of primitive frame may include: the complete frame image in original video, where moving target;Or In original video, the partial frame image including moving target, partial frame image is that the figure of moving target is only presented in complete frame image Wherein complete frame includes partial frame and background frames for the part of picture.Complete frame and partial frame, what is referred to is the frame for constituting original video respectively Ontology, i.e. complete frame, and the part of the frame of the image of moving target, i.e. partial frame is only presented.Complete frame compared with partial frame, The biggest problems are that it includes contain much information, it further includes other movements that he, which includes except the image containing moving target, Image and background video of target etc., however, other images are all not necessarily required to other than the image of moving target It is obtained together in the motion profile of this moving target obtains, wherein the image of other moving targets can be by other The monitoring of moving target obtains, background video due to its be substantially it is constant, can only obtain primary, institute With complete frame image and partial frame image are all the component part of optional motion profile in the present embodiment, and complete frame is whole Property it is stronger, contain more information, but its capacity occupied is big;The content of partial frame is few and concentrates, but it is in synthesis Preceding quantity of documents may be more.Particularly, when acquisition be partial frame image when, can also according to certain period, obtain The background frames of original video, in order to the subsequent synthetic operation to each motion profile.
A kind of video summarization method is present embodiments provided, determines the motion profile of moving target each in original video, root According to relative positional relationship of each motion profile in original video, relative position of each motion profile in summarized radio is determined;So Afterwards, according to the relative position between determining first motion track and each motion profile, each motion profile is synthesized, shape At summarized radio.By the implementation of the present embodiment, shearing is carried out to original video for reference with moving target and plucks choosing, so as to protect Compression ratio is demonstrate,proved, and remains video main body, i.e. moving target again, the rich of content is ensure that greatest extent, improves The effect of video frequency abstract.
Second embodiment
Referring to FIG. 5, Fig. 5 is the video summarization method refined flow chart that second embodiment of the invention provides.
Video summarization method in the present embodiment, initial initialization section need to model original video frame by frame Operation, specifically can be used the algorithm of comparative maturity, such as frame difference method, GMM gauss hybrid models, Vibe (Visual Background Extractor, visual background are extracted etc.), by, with operation, obtaining the movement in present frame between present frame and background frames The moving target of moving target and former frame is associated by target, such as according to overlapping area, characteristic similarity etc. come into Row association.When a moving target can not find association matching in the moving target tracked, it is regarded as new moving target;When When the moving target tracked is more than that N frame does not update, is regarded as the moving target and has disappeared, stop to the moving target with Track.If there is moving target in present frame, the complete frame or part are saved in, the frame number is recorded and image is deposited Store up the mapping relations in path.According to above-mentioned tracking result, the motion profile of each moving target in the available video trajectories.An one motion profile i.e. moving target is in video from the process occurred to disappearance, motion profile Track element can be described as:<targetid, frameid, x, y, the targetid of width, height>wherein are moving target Number, while being also the number of motion profile, the same targetid can be used in all elements in identical strip path curve; Frameid is the moving target corresponding frame number in original video;X is the horizontal seat in the upper left corner of the moving target in video Mark;Y is the upper left corner ordinate of the moving target in video, and width is width, and height is height, is denoted as traj_ target.According to transverse and longitudinal coordinate and Width x Height information, that is, according to positions and dimensions information, movement mesh can be uniquely determined The shared location and range size being marked in present frame.
Obtained all motion profiles are determined, according to the frame number in motion profile, it may be determined that the corresponding movement mesh of frame number Mark, indicates that these moving targets occur in the corresponding frame of the frame number;When carrying out the synthesis of summarized radio, by each motion profile It is ranked up according to certain sequence, for example, being sorted in descending order with the number of corresponding moving target in different frame numbers, for example sorts Data afterwards are that correspond to target be 1,2,3,4,5, Frame3 to correspond to 2,3,6,7 etc. to Frame1;It reads in the order again each The motion profile for the moving target that frame includes is numbered, and is saved after repeating according to the appearance of motion profile ID sequence removal, is obtained Result be (1,2,3,4,5,6,7 ...) the queue of Candidate Motion track.
The following are specific process flows:
S501, initialization:
Array bool traj_state [tracking quantity -1] is used to mark the processing status of each motion profile, and false is indicated Untreated, true indicates processed, and false is all when initial;
It is sky that the queue of Candidate Motion track, which is arranged, is denoted as vector<int>candidataId;
The component part of summarized radio, i.e. abstract frame: abstract frame is as motion profile, all by track element traj_ Target is constituted, the difference is that the traj_target in abstract frame is by belonging different moving target, i.e. targetid It is inconsistent.It can be denoted as vector<traj_target>absFrame;
S502, take not yet processed motion profile as motion profile to be processed from the queue of Candidate Motion track todo_traj;
If S503, not finding motion profile to be processed, process is ended processing.
S504, judge the corresponding all frame numbers of motion profile todo_traj to be processed, if there is frame number to be present in abstract In the abstract frame absFrame of video;
If S505, had existed, the initial position of the motion profile is calculated according to its position in summarized radio. Specific calculation method are as follows: if in todo_traj motion profile including N number of position, respectively T1, T2 ... Tx ... Tn, respectively The corresponding original frame number in position be F1, F2 ... Fx ... Fn;If Fx exists in the absFrame [y], Tx should reply to the topic to AbsFrame (y), similarly T (x-1) replies to the topic to absFrame (y-1), T (x+1) and replies to the topic to absFrame (y+1), i.e. todo_ The initial position of replying to the topic of traj motion profile is at absFrame [y-x].If there is multiple presence, can be with any one It is quasi-;For simplicity can be subject to first frame number.
If all frame numbers of S506, the motion profile are not present in the frame number of established summarized radio, root Initial position is found according to track degree of overlapping.A kind of calculation method of optional track degree of overlapping are as follows: by the friendship of two moving targets The ratio of the lesser area of area is denoted as collision ratio in folded area and moving target.The primitive frame Fi of certain moving target to be synthesized With all motion profile elements included in track degree of overlapping=Fi of frame absFrame [j] and absFrame [j] of having made a summary In included motion profile element collision than the sum of.Such as comprising being R1, R2 to motion profile in Fi;The frame for including in Aj Number be Fx, Fy, the position that wherein Fx includes is R3, the position for including in R4, Fy be R5, R6, Fi and Aj track degree of overlapping= (R1, R3)+(R1, R4)+(R1, R5)+(R1, R6)+(R2, R3)+(R2, R4)+(R2, R5)+(R2, R6).Assuming that current kinetic There is N frame in track, if there is absFrame [s], meets F0 and absFrame [s], F1 and absFrame [s+1] ... ..Fn with The sum of collision ratio of absFrame [s+n] is less than the threshold value of setting, then it is assumed that s can be initial position.Initial position can have very It is more, but for bigger compression ratio, it should which that the duration for setting summarized radio is most short, and a kind of feasible method is then, since the 1st frame Search, the position until finding the condition of satisfaction.
S507, according to initial position, all motion profile elements of current kinetic target are sequentially added in each abstract frame, And traj_state [todo_traj] is set to true, labeled as processed.
S508, other the still untreated motion profiles for including by the corresponding primitive frame of current trajectory todo_traj The queue of Candidate Motion track is added in ID;Then, S502 is continued back at.
After the abstract frame of summarized radio determines, so that it may carry out the synthetic operation of video;According to the moving target of selection Frame type it is different, synthesis carries out in different ways: when the set of the primitive frame of the motion profile of moving target is complete When frame, then duplicate removal synthesis is directly carried out;When the set of the primitive frame of the motion profile of moving target is partial frame, then will Partial frame and background frames are synthesized according to the synthesis order of setting.
3rd embodiment
Referring to FIG. 6, Fig. 6 is a kind of video frequency abstract device composition schematic diagram that third embodiment of the invention provides, comprising:
Module of target detection 601, for determining the motion profile of moving target each in original video;Motion profile includes: The set of primitive frame of the moving target in original video;
Track determining module 602 determines each movement for the relative positional relationship according to each motion profile in original video Relative position of the track in summarized radio;
Synthesis module 603, for being plucked according to first motion fixed in summarized radio track and each motion profile The relative position in video is wanted, each motion profile is synthesized, forms summarized radio.
Module of target detection 601 is for determining the motion profile of moving target each in original video.Wherein, moving target cares for Name Si Yi is referred in video as dynamic target.It is one certainly among entire video for moving target Determine to be changed in degree, the specific manifestation of variation is then usually the displacement that pixel occurs, or from scratch, from having To the process of nothing.Video file is as composed by multiple frames, these frame pictures play out just according to certain time interval Final video can be formed, in other words, each moving target, the actually component part of video file, that is, frame Tableaux present.Accordingly, it is determined that the motion profile of each moving target may include: to determine movement mesh in original video The location information being marked in original video;The motion profile of moving target is determined according to the location information of moving target.Wherein move The location information of target includes the primitive frame where moving target and the position in corresponding primitive frame.Primitive frame indicates Frame in original video, original frame number, then it represents that the serial number of primitive frame, the totalframes of a video file be it is indefinite, few can There can be several hundred frames, and more, video content is longer, may include frames up to ten thousand.One video file can find smallest partition As unit of frame, and moving target persistently changes position, to form the image of movement just in these continuous frames. The determination of moving target according to actual needs, under the premise of movement, then can also carry out certain sieve other than movement Choosing, select more satisfactory moving target, for the tracking of video be also based primarily upon these assert moving target come into Row.
Dynamic part in one video file be it is very much, therefore, in order to accurately judge whether the target belongs to Same moving target, in the present embodiment, module of target detection 601 can be also used for: be judged in each frame according to characteristic similarity Moving target whether belong to same moving target;Based on judging result, the motion profile of moving target is determined.Specifically, root According to characteristic similarity, including LBP, Haar-like, histogram etc. mode can determine whether is object inside different frame It is the image for belonging to same target, can also further determines whether to be moving target.Further, it is also possible to according to overlapping area Etc. other informations, the moving target of the moving target of present frame and former frame is associated.Associated direct result is exactly to obtain The motion profile for having arrived moving target, the starting point and ending point including moving target.Wherein, start frame and abort frame is specific Method of determination may include, and when detecting that similarity meets the object of preset requirement and movement has occurred, then be regarded as moving While target, recall it and mobile frame does not occur initially, or trace back to its frame not occurred, as start frame;And it terminates Frame, then be N frame is alreadyd exceed there is no updating for a moving target, that is, position there is no it is mobile when, then It is regarded as the target stop motion, it can be using the target there is no mobile first frames as abort frame.In such case Under, if finding this in subsequent moving object detection once again with the moving target for regarding as stopping and movement having occurred again, The moving target can also be so combined with moving target before again, still be considered as a motion profile.Correspondingly, working as One moving target is in the moving target tracked, according to information such as similarity or overlapping areas, can not find and matches, This moving target is then considered as new moving target to be tracked.
Module of target detection 601 is also used to determine the motion profile of moving target.The motion profile of moving target, is exactly wrapped The set for including the primitive frame of the original video of the moving target, for the same moving target, the collection of this primitive frame is unified As be it is continuous, the track of the movement of the moving target in video is reflected on the whole, from start frame to abort frame.And one In a video file, often have multiple moving targets, and at the beginning of the movement of these moving targets and the end time all It is not quite similar, that is, the start frame and abort frame of each moving target are not necessarily identical, referring to FIG. 2, Fig. 2 shows three The integration of the motion profile of moving target A, B, C, D shows, and wherein moving target A is since the 1st frame of original video to the 9th frame knot Beam, moving target B is to 28 frame ends since 20 frames of original video, and moving target C is since 46 frames of original video to 49 frame knots Beam;Moving target D is then since the 47th frame to the 50th frame end.It can be seen that the start-stop frame of four moving targets not phases Together, therefore, the set for the primitive frame that corresponding motion profile is included also is not quite similar.
Track determining module 602 determines each movement for the relative positional relationship according to each motion profile in original video Relative position of the track in summarized radio.Relative position of the motion profile of each moving target in original video be it is determining, And in summarized radio, in order to shorten the duration of summarized radio as far as possible, the interval of the corresponding primitive frame of each motion profile will not Greater than the interval in original video.So, in order to determine relative position of the motion profile of moving target in summarized radio, In the present embodiment, track determining module 602 be can be also used for:
In the motion profile for judging different moving targets, if there are identical original frame numbers;When there are identical originals When beginning frame number, according to identical original frame number, the relative position between corresponding motion profile is determined;Wherein, have identical Relative position of the motion profile of original frame number in summarized radio is consistent with the relative position in original video.Wherein, different Motion profile there are identical original frame number, expression, for the moving target different for two, motion profile has Common primitive frame, two moving targets occurred simultaneously at least in original video.So, in this context, a kind of optional Embodiment be exactly retain the positional relationship between the two moving targets in summarized radio, that is, original frame number and Position in corresponding primitive frame, the interaction that can retain as far as possible between different motion target that may be present in this way are closed System.The specific practice for retaining the positional relationship between two moving targets is, according to identical frame number, to determine the original of start frame Frame number is how many, it is, the start frame of the motion profile of one of moving target is determined, in the fortune of another moving target Position in dynamic rail mark.Usually determine the movement of start frame of the start frame of shorter motion profile in longer motion profile Position in track, this guarantees that determining result still falls within the movement of the longer moving target of motion profile to a certain extent In footprint.In addition, it is noted that the synthesis process of summarized radio, can also be in the summarized radio synthesized, It is inserted into the motion profile of new moving target, in this case, the summarized radio synthesized also can be considered the fortune of moving target Dynamic rail mark, track here may be the combination of the motion profile of multiple moving targets certainly.It in this case, then can be true Surely the start frame for the new moving target being inserted into corresponding position in summarized radio, then inserts the motion profile of new moving target Enter in summarized radio.Specifically, assuming to find in the motion profile of new moving target, movement of the xth frame in other moving targets Exist in track, and is y frame, then, the movement rail of the start frame of the motion profile of new moving target in other moving targets Position in mark is then y-x frame.Position of the start frame in the motion profile of other moving targets, the i.e. fortune of the moving target Initial position of the dynamic rail mark in summarized radio.Referring to FIG. 2, moving target C and D have same number of frames in original video in Fig. 2, It is exactly that C and D has identical frame number;Referring to FIG. 3, Fig. 3, which shows moving target C, D in Fig. 2, is collectively referred to as a summarized radio Schematic diagram.Wherein, the primitive frame that xth frame and y frame all refer to.
Further, when there are identical frame number, determine that initial position of the moving target in summarized radio can also wrap It includes: when the motion profile of moving target at least has two identical frame numbers with the motion profile of other moving targets, according to phase With the frame number and phase of the initial frame of position and moving target of any one in frame number in the motion profile of moving target With the spacing value between frame number, recalls initial frame position corresponding in summarized radio, determine moving target in summarized radio In initial position.Identical primitive frame between the motion profile of multiple moving targets often more than one, it is likely that have more A identical primitive frame;When there are identical primitive frame, under normal circumstances since the movement of moving target is continuous, then These frame numbers are also continuous, so selecting any one same number of frames to determine the start frame of the motion profile of moving target i.e. It can.For convenience's sake, first identical primitive frame can be directly selected to determine the motion profile of moving target.
In addition, determining whether the track degree of overlapping of the motion profile of each moving target is less than when identical frame number is not present Equal to preset threshold, and the most short summarized radio duration according to track degree of overlapping less than or equal to preset threshold, determine moving target Initial frame position corresponding in summarized radio, obtain initial position of the moving target in summarized radio.There is no phases Same frame number, then it represents that the motion profile of each moving target did not occur simultaneously in original video, was all respectively to appear in difference Frame in, then, need not just limit position of the motion profile of each moving target in summarized radio in this case, move It can further be compressed between the motion profile of target.Specifically, being then the motion profile according to each moving target Track degree of overlapping determines.Wherein, what track degree of overlapping referred to is exactly the degree of overlapping between motion profile, due to each moving target There is no same number of frames between motion profile, then under certain condition, the motion profile of the moving target in original video rearward moves The position forward in summarized radio is moved, to reduce the duration of summarized radio, increases the compression ratio of summarized radio.So, when It when taking such measure, is then likely to occur, juxtaposition occurs in the motion profile of each moving target, in view of this, calculating each Track degree of overlapping between the motion profile of moving target, under the premise of meeting the condition of degree of overlapping, resultant motion target Motion profile, so that the duration of summarized radio is most short.
Specifically, if overlapping area between the image of different moving targets, with the lesser moving target of wherein area The area ratio of image be denoted as collision ratio, then track degree of overlapping includes the motion profile and other moving targets of moving target The sum of collision ratio between motion profile.Particularly, the synthesis process of summarized radio can also be in the summarized radio synthesized In, it is inserted into the motion profile of new moving target, in this case, the summarized radio synthesized also can be considered moving target Motion profile, track here may be the combination of the motion profile of multiple moving targets certainly.It in this case, then can be with The motion profile of each moving target in the motion profile and summarized radio of new moving target is determined into track degree of overlapping one by one, and Summation, as the motion profile of new moving target and the whole track degree of overlapping of summarized radio.In track, degree of overlapping is little When preset threshold, for example preset threshold can be set to differ from 0%~10%, and resulting track degree of overlapping is in this range It inside can be considered the track degree of overlapping met the requirements, then, under the premise of this, by the start frame of the motion profile of moving target Position as early as possible is set, i.e., shortens the duration of summarized radio as far as possible.Referring to FIG. 2, moving target A, B in Fig. 2, Identical frame number is not present between C;Referring to FIG. 4, Fig. 4 shows the abstract view in Fig. 2 after moving target A, B, C three synthesis The schematic diagram of frequency.
In the present embodiment, before determining initial position of the motion profile of each moving target in summarized radio, It can also include: that the quantity of the moving target for from the motion profile of each moving target, being included is dropped according to each frame number Sequence sequence, and sequentially determine each moving target motion profile and subsequent processing.Due to the movement mesh in an original video Mark be have it is multiple, when carrying out video frequency abstract, may be related to for each moving target processing successive problem;? In the present embodiment, a kind of optional mode is then with the sequence containing the frame number more than moving target first, to carry out video frequency abstract Processing, moving target is more, and the difficulty of processing is then bigger, and other subsequent moving targets it is less frame processing will more along reason It comes out as an article;For example, there are 10 moving targets in frame x, there is 1 moving target in frame y, the moving target in frame x is more, first to frame x In the corresponding track of moving target handled, can make the corresponding moving target of other subsequent frames processing Shi Gengrong Identical frame number is easily found, furthermore the moving target of frame x is more, and more crowded for other opposite frames, wherein reference locus is overlapped The factor of degree can moving target fewer than other more complicated, and only one moving target in frame y, in contrast track is overlapped Degree is just more preferable to be determined.
In the present embodiment, in the motion profile for determining moving target, motion profile includes the image of moving target The set of primitive frame;Specifically, the set of primitive frame may include: the complete frame image in original video, where moving target;Or In original video, the partial frame image including moving target.
A kind of video frequency abstract device is present embodiments provided, determines the motion profile of moving target each in original video, root According to relative positional relationship of each motion profile in original video, relative position of each motion profile in summarized radio is determined;So Afterwards, according to the relative position between determining first motion track and each motion profile, each motion profile is synthesized, shape At summarized radio.By the implementation of the present embodiment, shearing is carried out to original video for reference with moving target and plucks choosing, so as to protect Compression ratio is demonstrate,proved, and remains video main body, i.e. moving target again, the rich of content is ensure that greatest extent, improves The effect of video frequency abstract.
Fourth embodiment
Referring to FIG. 7, Fig. 7 is a kind of terminal composition schematic diagram that fourth embodiment of the invention provides, comprising: processor 701, memory 702 and communication bus 703;Communication bus 703 is for realizing the connection between processor 701 and memory 702 Communication;Processor 701 is for executing the video frequency abstract program stored in memory 702, to realize that the video of previous embodiment is plucked Method is wanted, which is not described herein again.
In addition, the present embodiment additionally provides a kind of computer readable storage medium, deposited in the computer readable storage medium One or more computer program is contained, computer program can be executed by one or more processor, to realize aforementioned reality The video summarization method of example is applied, which is not described herein again.
Obviously, those skilled in the art should be understood that each module of aforementioned present invention or each step can be with general Computing device realizes that they can be concentrated on a single computing device, or be distributed in constituted by multiple computing devices On network, optionally, they can be realized with the program code that computing device can perform, it is thus possible to be stored in It is performed by computing device in storage medium (ROM/RAM, magnetic disk, CD), and in some cases, it can be to be different from this The sequence at place executes shown or described step, perhaps they are fabricated to each integrated circuit modules or by it In multiple modules or step be fabricated to single integrated circuit module to realize.So the present invention is not limited to any specific Hardware and software combine.
The above content is specific embodiment is combined, further detailed description of the invention, and it cannot be said that this hair Bright specific implementation is only limited to these instructions.For those of ordinary skill in the art to which the present invention belongs, it is not taking off Under the premise of from present inventive concept, a number of simple deductions or replacements can also be made, all shall be regarded as belonging to protection of the invention Range.

Claims (10)

1. a kind of video summarization method, comprising:
Determine the motion profile of moving target each in original video;The motion profile includes: the moving target in original video In primitive frame set;
According to relative positional relationship of each motion profile in original video, opposite position of each motion profile in summarized radio is determined It sets;
According to the opposite position of first motion fixed in summarized radio track and each motion profile in summarized radio It sets, each motion profile is synthesized, the summarized radio is formed.
2. video summarization method as described in claim 1, which is characterized in that each motion profile of determination is in summarized radio Relative position include:
In the motion profile for judging different moving targets, if there are identical original frame numbers;
When there are identical original frame number, according to the identical original frame number, the phase between corresponding motion profile is determined To position;Wherein, have relative position of the motion profile of identical original frame number in the summarized radio and in the original Relative position in video is consistent;
When identical original frame number is not present, the track overlapping between the motion profile there is no identical original frame number is determined Degree, and determine when the track degree of overlapping is less than preset threshold, phase of the motion profile of each moving target in summarized radio To position;Wherein, when the track degree of overlapping is less than preset threshold, the relative position between each motion profile makes described pluck Want the duration of video most short.
3. video summarization method as claimed in claim 2, which is characterized in that described true when there are identical original frame number Fixed relative position of each motion profile in summarized radio further include: identical when at least there is two between two motion profiles When original frame number, according to position of any one in the identical original frame number in a wherein motion profile, and Spacing value between the initial frame of the motion profile and the identical primitive frame determines the motion profile and another fortune Relative position between dynamic rail mark.
4. video summarization method as claimed in claim 2, which is characterized in that if the friendship between the image of different moving targets Folded area, is denoted as collision ratio with the area ratio of the image of the lesser moving target of wherein area, then the track degree of overlapping packet Include the sum of the collision ratio between the motion profile of moving target and the motion profile of other moving targets.
5. video summarization method according to any one of claims 1-4, which is characterized in that the determination is respectively transported in original video The motion profile of moving-target includes:
Determine that location information of the moving target in original video, the location information include the original where the moving target Beginning frame and the position in corresponding primitive frame;
The motion profile of the moving target is determined according to the location information of the moving target.
6. video summarization method as claimed in claim 5, which is characterized in that position of the determining moving target in original video Confidence breath includes: to judge whether the moving target in each frame belongs to same moving target according to characteristic similarity;It is tied based on judgement Fruit determines the location information of the moving target.
7. video summarization method according to any one of claims 1-4, which is characterized in that the set of the primitive frame includes institute It states in original video, the complete frame image where the moving target;Or in the original video, the partial frame figure including moving target Picture;The complete frame includes the partial frame and background frames.
8. the video summarization method of any one of 1-4 as claimed in claim, which is characterized in that described according to the summarized radio In before fixed first motion track, further includes: from the motion profile of each moving target, according to each frame number institute The quantity for the moving target for including carries out descending sort, and sequentially determines the first motion track and each motion profile The sequence of processing.
9. a kind of terminal, which is characterized in that including processor, memory and communication bus;The communication bus is for realizing institute State the connection communication between processor and memory;The processor is for executing the video frequency abstract journey stored in the memory The step of sequence, video summarization methods described in any item with realization such as claim 1-8.
10. a kind of computer readable storage medium, which is characterized in that be stored in the computer readable storage medium one or The multiple computer programs of person, the computer program can be executed by one or more processor, to realize such as claim 1-8 The step of described in any item video summarization methods.
CN201710827915.3A 2017-09-14 2017-09-14 A kind of video summarization method, terminal and computer readable storage medium Withdrawn CN109511019A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710827915.3A CN109511019A (en) 2017-09-14 2017-09-14 A kind of video summarization method, terminal and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710827915.3A CN109511019A (en) 2017-09-14 2017-09-14 A kind of video summarization method, terminal and computer readable storage medium

Publications (1)

Publication Number Publication Date
CN109511019A true CN109511019A (en) 2019-03-22

Family

ID=65744473

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710827915.3A Withdrawn CN109511019A (en) 2017-09-14 2017-09-14 A kind of video summarization method, terminal and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN109511019A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111047622A (en) * 2019-11-20 2020-04-21 腾讯科技(深圳)有限公司 Method and device for matching objects in video, storage medium and electronic device
CN113724281A (en) * 2020-05-25 2021-11-30 艾阳科技股份有限公司 Image compression and identification method and system thereof
CN113949823A (en) * 2021-09-30 2022-01-18 广西中科曙光云计算有限公司 Video concentration method and device
CN116647690A (en) * 2023-05-30 2023-08-25 石家庄铁道大学 Video concentration method based on space-time rotation

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103778237A (en) * 2014-01-27 2014-05-07 北京邮电大学 Video abstraction generation method based on space-time recombination of active events
CN103929685A (en) * 2014-04-15 2014-07-16 中国华戎控股有限公司 Video abstract generating and indexing method
CN104717457A (en) * 2013-12-13 2015-06-17 华为技术有限公司 Video condensing method and device
CN104717573A (en) * 2015-03-05 2015-06-17 广州市维安电子技术有限公司 Video abstract generation method
CN104883628A (en) * 2014-02-28 2015-09-02 华为软件技术有限公司 Method, device and equipment for generating video abstract, device and equipment
CN106856577A (en) * 2015-12-07 2017-06-16 北京航天长峰科技工业集团有限公司 The video abstraction generating method of multiple target collision and occlusion issue can be solved

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104717457A (en) * 2013-12-13 2015-06-17 华为技术有限公司 Video condensing method and device
CN103778237A (en) * 2014-01-27 2014-05-07 北京邮电大学 Video abstraction generation method based on space-time recombination of active events
CN104883628A (en) * 2014-02-28 2015-09-02 华为软件技术有限公司 Method, device and equipment for generating video abstract, device and equipment
CN103929685A (en) * 2014-04-15 2014-07-16 中国华戎控股有限公司 Video abstract generating and indexing method
CN104717573A (en) * 2015-03-05 2015-06-17 广州市维安电子技术有限公司 Video abstract generation method
CN106856577A (en) * 2015-12-07 2017-06-16 北京航天长峰科技工业集团有限公司 The video abstraction generating method of multiple target collision and occlusion issue can be solved

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111047622A (en) * 2019-11-20 2020-04-21 腾讯科技(深圳)有限公司 Method and device for matching objects in video, storage medium and electronic device
CN111047622B (en) * 2019-11-20 2023-05-30 腾讯科技(深圳)有限公司 Method and device for matching objects in video, storage medium and electronic device
CN113724281A (en) * 2020-05-25 2021-11-30 艾阳科技股份有限公司 Image compression and identification method and system thereof
CN113949823A (en) * 2021-09-30 2022-01-18 广西中科曙光云计算有限公司 Video concentration method and device
CN116647690A (en) * 2023-05-30 2023-08-25 石家庄铁道大学 Video concentration method based on space-time rotation
CN116647690B (en) * 2023-05-30 2024-03-01 石家庄铁道大学 Video concentration method based on space-time rotation

Similar Documents

Publication Publication Date Title
CN109511019A (en) A kind of video summarization method, terminal and computer readable storage medium
CN110602554B (en) Cover image determining method, device and equipment
CN108090497B (en) Video classification method and device, storage medium and electronic equipment
CN111988638B (en) Method and device for acquiring spliced video, electronic equipment and storage medium
CN108399380A (en) A kind of video actions detection method based on Three dimensional convolution and Faster RCNN
CN103310475B (en) animation playing method and device
CN105451029B (en) A kind of processing method and processing device of video image
CN110147469B (en) Data processing method, device and storage medium
CN112541867B (en) Image processing method, device, electronic equipment and computer readable storage medium
CN113515998B (en) Video data processing method, device and readable storage medium
CN113518256A (en) Video processing method and device, electronic equipment and computer readable storage medium
CN107563357B (en) Live-broadcast clothing dressing recommendation method and device based on scene segmentation and computing equipment
CN110427806A (en) Video frequency identifying method, device and computer readable storage medium
CN110245609A (en) Pedestrian track generation method, device and readable storage medium storing program for executing
CN111586466A (en) Video data processing method and device and storage medium
CN108037830A (en) A kind of implementation method of augmented reality
CN112613508A (en) Object identification method, device and equipment
US20170076153A1 (en) Systems and Methods for Contextual Video Shot Aggregation
CN111488847A (en) System, method and terminal for acquiring sports game video goal segment
Bez et al. Multimodal soccer highlight identification using a sparse subset of frames integrating long-term sliding windows
CN116580054B (en) Video data processing method, device, equipment and medium
CN110223219A (en) The generation method and device of 3D rendering
CN113905188B (en) Video stitching dynamic adjustment method, system, electronic device and storage medium
CN115272057A (en) Training of cartoon sketch image reconstruction network and reconstruction method and equipment thereof
CN114245031A (en) Image display method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20190322

WW01 Invention patent application withdrawn after publication