CN104463864A - Multistage parallel key frame cloud extraction method and system - Google Patents
Multistage parallel key frame cloud extraction method and system Download PDFInfo
- Publication number
- CN104463864A CN104463864A CN201410731007.0A CN201410731007A CN104463864A CN 104463864 A CN104463864 A CN 104463864A CN 201410731007 A CN201410731007 A CN 201410731007A CN 104463864 A CN104463864 A CN 104463864A
- Authority
- CN
- China
- Prior art keywords
- key frame
- video
- parallel
- multistage
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
- G06V20/47—Detecting features for summarising video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
Abstract
The invention discloses a multistage parallel key frame cloud extraction method and system. The key frame cloud extraction system comprises a video input module, a video shot segmentation module, a multistage parallel key frame cloud processing module and a key frame output module. The video input module is arranged to be a video input interface for extracting key frames. The video shot segmentation module is arranged to carry out shot segmentation on input video, so that key frame extraction parallel processing is carried out conveniently. The multistage parallel key frame cloud processing module is arranged to carry out shot key frame group generating on video shots through a multistage parallel strategy via parallel frame feature vector extraction and parallel clustering operation. The key frame output module is arranged to carry out combination on the key frame groups generated by the shots, and the whole video key frame groups are output finally. According to the multistage parallel key frame cloud extraction method and system, efficiency is high during large-scale key frame extracting processing, meanwhile, great expandability and stability are achieved, and the requirement for large-scale key frame extraction can be met.
Description
Technical field
The present invention relates to key frame cloud extractive technique, more particularly, relate to a kind of multistage parallel key frame cloud extracting method and multistage parallel key frame cloud extraction system.
Background technology
Key frame of video refers to most important, the representational one or more image in energy representative shot.The extraction of key frame can greatly reduce the treatment capacity of video data, is subject to the extensive concern of researcher.
Cloud computing seamlessly can expand to large-scale cluster, and can tolerate the error code of part of nodes, and even very most of node occurs to lose efficacy also can not affect the true(-)running of program, and therefore cloud computing has good scalability and reliability.
The main method of existing key-frame extraction comprises: based on the method for shot boundary, based on motion analysis extraction key frame, based on image information extraction key frame, based on cluster extraction key frame etc.Major part research mainly concentrates on the extraction accuracy aspect of key frame of video, but the extraction of key frame of video relates to many digital operations, tool frame of video quantity is many, serial video Key Frame Extraction can length consuming time, and unit processing power is limited, therefore be necessary to study a kind of multistage parallel key frame cloud extracting method and cloud extraction system.
Summary of the invention
In order to solve above-mentioned prior art Problems existing, the object of the present invention is to provide a kind of multistage parallel key frame cloud extraction system, wherein, described multistage parallel key frame cloud extraction system comprises: video input module, is constructed to the video input interface extracting key frame; Video lens segmentation module, is constructed to carry out shot segmentation, to carry out key-frame extraction parallel processing to the video of input; Multistage parallel key frame cloud processing module, is constructed to video lens by multistage parallel strategy, by parallel extraction frame proper vector, parallel clustering operation, generates each camera lens key frame group; Key frame output module, the key frame group be constructed to each camera lens generates combines, and finally exports the key frame group of whole video.
Another object of the present invention is also to provide a kind of multistage parallel key frame cloud extracting method, and wherein, described multistage parallel key frame cloud extracting method comprises: accept the video that need extract key frame; Shot segmentation is carried out to described video; Multistage parallel key frame cloud is carried out to described camera lens and extracts operation; Extracted camera lens key frame group is combined, exports final video key frame group.
Further, described multistage parallel key frame cloud extraction operation comprises parallel extraction frame of video proper vector, operates according to frame of video proper vector parallel clustering.
Further, described camera lens key frame group combination, comprises final cluster and produces the final key frame of video.
Further, described video feature vector cluster operation can be the characteristics of image clustering algorithm of k-means cluster, fuzzy C-means clustering or other applicable parallel processings.
Multistage parallel key frame cloud extracting method of the present invention and cloud extraction system can improve key-frame extraction efficiency greatly, have good scalability and reliability simultaneously.
Accompanying drawing explanation
Fig. 1 is multistage parallel key frame cloud extraction system schematic diagram according to an embodiment of the invention.
Fig. 2 is that multistage parallel key frame cloud extracts the process flow diagram of dispatching method according to an embodiment of the invention.
Fig. 3 is that multistage parallel key frame cloud extracts certain example operation figure according to an embodiment of the invention.
Embodiment
Be described in detail embodiments of the invention now, its sample table shows in the accompanying drawings, and wherein, identical label represents same parts all the time.Be described to explain the present invention to embodiment below with reference to the accompanying drawings.In the accompanying drawings, for clarity, the thickness in layer and region can be exaggerated.In the following description, obscuring of the present invention's design that the unnecessary detailed description in order to avoid known features and/or function causes, can omit the unnecessary detailed description of known features and/or function.
Fig. 1 is multistage parallel key frame cloud extraction system schematic diagram according to an embodiment of the invention.
With reference to Fig. 1, multistage parallel key frame cloud extraction system comprises according to an embodiment of the invention: video input module 10, is constructed to the video input interface extracting key frame; Video lens segmentation module 20, is constructed to carry out shot segmentation, to carry out key-frame extraction parallel processing to the video of input; Multistage parallel key frame cloud processing module 30, is constructed to video lens by multistage parallel strategy, by parallel extraction frame proper vector, parallel clustering operation, generates each camera lens key frame group; Key frame output module 40, the key frame group be constructed to each camera lens generates combines, and finally exports the key frame group of whole video.
In addition, multistage parallel key frame cloud extraction operation comprises parallel extraction frame of video proper vector, operates according to frame of video proper vector parallel clustering.
Described video feature vector cluster operation can be the characteristics of image clustering algorithm of k-means cluster, fuzzy C-means clustering or other applicable parallel processings.
Accordingly, present invention also offers a kind of multistage parallel key frame cloud extracting method, specifically please refer to Fig. 2, it is the process flow diagram of multistage parallel key frame cloud extracting method according to an embodiment of the invention.
With reference to Fig. 2, multistage parallel key frame cloud extracting method comprises according to an embodiment of the invention: S1, accept the video that need carry out Key Frame Extraction; S2, video lens segmentation is carried out to video; S3, to camera lens multistage parallel extract key frame of video; S4, array output is carried out to camera lens key frame group.
In the present embodiment, after video lens segmentation, can walk abreast and extract frame of video proper vector, can walk abreast afterwards and carry out video feature vector cluster operation, cluster can be the characteristics of image clustering algorithm of k-means cluster, fuzzy C-means clustering or other applicable parallel processings.Key frame will be extracted come that the present invention will be described for parallel extraction frame of video Vector Processing and parallel k-means cluster of carrying out below.Wherein, K-means algorithm is the very typical clustering algorithm based on distance, and adopt distance as the evaluation index of similarity, namely think that the distance of two objects is nearer, its similarity is larger.And, FuzzycMeans Clustering algorithm fuzzy c-means algorithm (FCMA) is as one of major technique without supervision machine learning, by the method for fuzzy theory to important data analysis and modeling, establish the uncertainty description of sample generic, can reflect reality the world more objectively.
Specifically, with reference to Fig. 3, if after S1, S2 step, video has been divided into k camera lens (k >=1, k ∈ Z).S3 is constructed to following steps:
S31: one-level camera lens map operates, this operation realizes the camera lens of segmentation to be distributed in one-level cloud computing platform node, node described herein is dual role, be specially: be Datanode and TaskTasker role in first order cloud platform, be Namenode and Jobtracker role in the cloud computing platform node of the second level.In S301 operation, the <key of map construction of function input data record, value> is < camera lens ID, video lens position >, function operation is: be copied to by camera lens in first order cloud computing platform node, to carry out the map operation of the second level.
S32: second level map is done to the camera lens be distributed on node and operates pre-service.Shot segmentation is become one group of video frame images.Camera lens 1 comprises video frame number N
1, camera lens 2 comprises video frame number N
2, camera lens K comprises video frame number N
k.N
1, N
2... N
krefer to each camera lens N
1, N
2..., N
kthe actual video frame number (N comprised
1, N
2... N
k>=1, N
1, N
2... N
k∈ Z).
S33: carry out second level map operation to each camera lens is parallel.Described second level map operates, and is each frame to be distributed in next stage cloud computing platform Tasktracker node, parallel extraction frame of video proper vector.In described S33 operation, the <key of map construction of function input data record, value> is < frame ID, two field picture position >, function operation is for carry out characteristic vector pickup to frame of video, the form that Output rusults <key, value> are right is < frame ID, frame proper vector >.
S34: the result obtained is operated to S33 and carries out third level map operation.Described S34 operation, is the < frame ID generated S33, and frame proper vector > record group is parallel carries out cloud cluster operation.Specifically can be described as: complete each Frame and calculate to the distance of initial frame cluster centre, and again mark its new cluster classification belonged to, it is input as the < frame ID that S33 generates, the cluster centre of all records of frame proper vector > and last round of iteration (or initial clustering).Each map function reads in cluster centre description document, and each measuring point of map function to input calculates its nearest class center, and does the mark of new classification.The <key of Map function input data record, value> is < frame ID, frame proper vector >; The form exporting intermediate result <key, value> is < frame generic, frame proper vector >.
Above-mentioned camera lens 1, camera lens 2 ..., camera lens k initial cluster center number be described to m
1, m
2..., m
k(m
1, m
2..., m
k>=1, m
1, m
2..., m
k∈ Z) its value determines according to each lens data frame sum and certain rule, to a certain extent, m
1, m
2..., m
kalso the crucial frame number that each camera lens will produce is represented.
Above-mentioned initial cluster center is constructed to: according to m
1, m
2..., m
kvalue, respectively from camera lens 1, camera lens 2 ..., randomly draw m in camera lens k
1, m
2..., m
kindividual levy proper vector as camera lens 1, camera lens 2 ..., camera lens k initial cluster center.
Above-mentioned Frame calculates to the distance of cluster centre, can be described as Euclidean distance, mahalanobis distance etc.
The map function that S34 is constructed can be described as:
void map(Object key, Text value, Context context)
{
Calculate the distance of frame to each cluster centre;
More above-mentioned distance;
Frame is summed up in the point that the nearest class of that distance belonging to class center;
By < frame generic, frame proper vector > writes intermediate file;
}
The above-mentioned map stage can carry out shuffle operation, completes the packet sequencing of results of intermediate calculations.
S35: according to the output of S34, upgrades cluster centre, for next round map-reduce.Described S35 operation, the form that input data <key, value> is right is < cluster category IDs, { record attribute vector set } >; A reduce task will be given in the record (i.e. the record of identical cluster centre category IDs) that all key are identical.S35 operation is described to: the number of the point that cumulative key is identical and each record component and, ask the average of each component, obtain new cluster centre.S35 operates Output rusults <key, and the form that value> is right is < cluster category IDs, mean vector >.S35 operating process can be described as:
Void reduce(Text key，Iterable<Text> values, Context context)
{
For (for all records that key is identical)
{
Ask the average of each attribute;
}
By < cluster category IDs, mean vector > writes destination file;
}
S36: iterative process.S36 step can be described as: to the Output rusults of S34, judges whether this cluster restrains.Specifically can be described as: the distance of the cluster centre that more last round of map-reduce obtains and epicycle map-reduce cluster centre.If distance is less than given threshold values, then algorithm terminates.No it, then the cluster centre of epicycle is replaced last round of cluster centre, and start a new round map-reduce operation.
S4: array output is carried out to camera lens key frame group.The final each camera lens key frame of video group poly-to S3 step carries out array output key frame of video.
In sum, multistage parallel key frame cloud extracting method and cloud extraction system according to an embodiment of the invention, has the feature of multistage parallel operation, can greatly improve key-frame extraction efficiency.Meanwhile, system based on cloud computing platform, therefore has good extensibility and stability.
Although specifically show with reference to its exemplary embodiment and describe the present invention, but it should be appreciated by those skilled in the art, when not departing from the spirit and scope of the present invention that claim limits, the various changes in form and details can be carried out to it.
Claims (6)
1. a multistage parallel key frame cloud extraction system, it is characterized in that, it comprises:
Video input module, is constructed to the video input interface extracting key frame;
Video lens segmentation module, is constructed to carry out shot segmentation, to carry out key-frame extraction parallel processing to the video of input;
Multistage parallel key frame cloud processing module, is constructed to video lens by multistage parallel strategy, by parallel extraction frame proper vector, parallel clustering operation, generates camera lens key frame group;
Key frame output module, the key frame group be constructed to camera lens generates combines, and finally exports the key frame group of whole video.
2. multistage parallel key frame cloud extraction system according to claim 1, is characterized in that, multistage parallel key frame cloud processing module is carried out parallel extraction frame of video proper vector, operated according to frame of video proper vector parallel clustering.
3. multistage parallel key frame cloud extraction system according to claim 2, it is characterized in that, the video feature vector cluster operation in described multistage parallel key frame cloud processing module can be the characteristics of image clustering algorithm of k-means cluster, fuzzy C-means clustering or other applicable parallel processings.
4. a multistage parallel key frame cloud extracting method, is characterized in that, described multistage parallel key frame cloud extracting method comprises the steps:
S1. the video that need carry out Key Frame Extraction is accepted;
S2. video lens segmentation is carried out to video;
S3. key frame of video is extracted to camera lens multistage parallel;
S4. array output is carried out to camera lens key frame group.
5. multistage parallel key frame cloud extracting method according to claim 4, is characterized in that, described step s3 comprises parallel frame of video proper vector of extracting and operates, and operates according to frame of video proper vector parallel clustering.
6. multistage parallel key frame cloud extracting method according to claim 5, it is characterized in that, described video feature vector cluster operation is one or more in the characteristics of image clustering algorithm of k-means cluster or fuzzy C-means clustering or other applicable parallel processings.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410731007.0A CN104463864B (en) | 2014-12-05 | 2014-12-05 | Multistage parallel key frame cloud extracting method and system |
| PCT/CN2015/092349 WO2016086731A1 (en) | 2014-12-05 | 2015-10-21 | Multi-level parallel key frame cloud extraction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410731007.0A CN104463864B (en) | 2014-12-05 | 2014-12-05 | Multistage parallel key frame cloud extracting method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104463864A true CN104463864A (en) | 2015-03-25 |
| CN104463864B CN104463864B (en) | 2018-08-14 |
Family
ID=52909846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410731007.0A Active CN104463864B (en) | 2014-12-05 | 2014-12-05 | Multistage parallel key frame cloud extracting method and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN104463864B (en) |
| WO (1) | WO2016086731A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016086731A1 (en) * | 2014-12-05 | 2016-06-09 | 华南师范大学 | Multi-level parallel key frame cloud extraction method and system |
| CN108921773A (en) * | 2018-07-04 | 2018-11-30 | 百度在线网络技术（北京）有限公司 | Human body tracking processing method, device, equipment and system |
| CN110889857A (en) * | 2019-11-15 | 2020-03-17 | 北京邮电大学 | Mobile Web real-time video frame segmentation method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1719909A (en) * | 2005-07-15 | 2006-01-11 | 复旦大学 | Method for measuring audio-video frequency content change |
| US20120027295A1 (en) * | 2009-04-14 | 2012-02-02 | Koninklijke Philips Electronics N.V. | Key frames extraction for video content analysis |
| CN103064935A (en) * | 2012-12-24 | 2013-04-24 | 深圳先进技术研究院 | System and method for multimedia data parallel processing |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7016540B1 (en) * | 1999-11-24 | 2006-03-21 | Nec Corporation | Method and system for segmentation, classification, and summarization of video images |
| CN101296373B (en) * | 2007-04-27 | 2011-11-23 | 北京信心晟通科技发展有限公司 | Multimedia data processing system and method based on material exchange format |
| CN102693299B (en) * | 2012-05-17 | 2015-01-07 | 西安交通大学 | System and method for parallel video copy detection |
| CN104463864B (en) * | 2014-12-05 | 2018-08-14 | 华南师范大学 | Multistage parallel key frame cloud extracting method and system |
-
2014
- 2014-12-05 CN CN201410731007.0A patent/CN104463864B/en active Active
-
2015
- 2015-10-21 WO PCT/CN2015/092349 patent/WO2016086731A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1719909A (en) * | 2005-07-15 | 2006-01-11 | 复旦大学 | Method for measuring audio-video frequency content change |
| US20120027295A1 (en) * | 2009-04-14 | 2012-02-02 | Koninklijke Philips Electronics N.V. | Key frames extraction for video content analysis |
| CN103064935A (en) * | 2012-12-24 | 2013-04-24 | 深圳先进技术研究院 | System and method for multimedia data parallel processing |
Non-Patent Citations (1)
| Title |
|---|
| 刘守群: "海量网络视频快速检索关键技术研究", 《中国博士学位论文全文数据库 信息科技辑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016086731A1 (en) * | 2014-12-05 | 2016-06-09 | 华南师范大学 | Multi-level parallel key frame cloud extraction method and system |
| CN108921773A (en) * | 2018-07-04 | 2018-11-30 | 百度在线网络技术（北京）有限公司 | Human body tracking processing method, device, equipment and system |
| CN110889857A (en) * | 2019-11-15 | 2020-03-17 | 北京邮电大学 | Mobile Web real-time video frame segmentation method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104463864B (en) | 2018-08-14 |
| WO2016086731A1 (en) | 2016-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112199375B (en) | Cross-modal data processing method and device, storage medium and electronic device | |
| Moreira et al. | Image provenance analysis at scale | |
| CN205721777U (en) | Visual search equipment and system | |
| Du et al. | Geometric hypergraph learning for visual tracking | |
| US9098741B1 (en) | Discriminitive learning for object detection | |
| CN111325245B (en) | Repeated image recognition method, device, electronic equipment and computer readable storage medium | |
| Hii et al. | Multigap: Multi-pooled inception network with text augmentation for aesthetic prediction of photographs | |
| CN104112005B (en) | Distributed mass fingerprint identification method | |
| Zhong et al. | Maskgroup: Hierarchical point grouping and masking for 3d instance segmentation | |
| CN113255714A (en) | Image clustering method and device, electronic equipment and computer readable storage medium | |
| JP2018045302A (en) | Information processing device, information processing method and program | |
| Li et al. | Co-saliency detection based on hierarchical consistency | |
| Nie et al. | SHREC’18 track: 2D scene sketch-based 3D scene retrieval | |
| CN104463864A (en) | Multistage parallel key frame cloud extraction method and system | |
| CN105913423B (en) | A kind of deterministic models approximating method based on super-pixel | |
| Cao et al. | GMN: generative multi-modal network for practical document information extraction | |
| CN104778272B (en) | A kind of picture position method of estimation excavated based on region with space encoding | |
| Cuffaro et al. | Segmentation free object discovery in video | |
| CN106844338B (en) | method for detecting entity column of network table based on dependency relationship between attributes | |
| Liao et al. | Depthwise grouped convolution for object detection | |
| JP5890340B2 (en) | Image classification device and image classification program | |
| CN112257689A (en) | Training and recognition method of face recognition model, storage medium and related equipment | |
| CN114913330B (en) | Point cloud component segmentation method and device, electronic equipment and storage medium | |
| JP4957924B2 (en) | Document image feature value generation apparatus, document image feature value generation method, and document image feature value generation program | |
| Cao et al. | Stable image matching for 3D reconstruction in outdoor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |