CN113411666A - Automatic clipping method, apparatus, camera, and computer-readable storage medium - Google Patents
Automatic clipping method, apparatus, camera, and computer-readable storage medium Download PDFInfo
- Publication number
- CN113411666A CN113411666A CN202110679454.6A CN202110679454A CN113411666A CN 113411666 A CN113411666 A CN 113411666A CN 202110679454 A CN202110679454 A CN 202110679454A CN 113411666 A CN113411666 A CN 113411666A
- Authority
- CN
- China
- Prior art keywords
- segment
- value
- extremum
- processed
- extreme
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
Abstract
The invention discloses an automatic clipping method, which comprises the following steps: s1: carrying out smoothing treatment on a grading curve of a video to be edited; s2: calculating a maximum value point and a minimum value point in the smoothed score curve; s3: acquiring at least one preliminary extreme value segment according to two adjacent maximum value points or two adjacent minimum value points; s4: processing the preliminary extreme value fragments according to a preset grading threshold value to obtain processed extreme value fragments; s5: and editing the video according to the processed extremum segment. Compared with the prior art, the method and the device for determining the extremum segment determine the extremum segment through the scoring curve and the scoring threshold of the video frame, and solve the problem that only a single video frame can be screened but the video segment cannot be screened in the prior art.
Description
Technical Field
The present application relates to the field of video processing technologies, and in particular, to a method and an apparatus for automatically editing a video, a camera, and a computer-readable storage medium.
Background
The automatic clipping function helps users to clip highlights in videos quickly, and is therefore popular with a large number of photographers. Therefore, most existing shooting devices (such as a motion camera and a panoramic camera) have the function.
The current automatic clipping algorithm mainly comprises two parts, wherein the first part is used for evaluating a video frame, different indexes of the video frame are obtained through various algorithms, and then the indexes are combined together through methods such as setting a scoring rule and the like to obtain the score of the video frame; the second part is to clip the video based on the scores, i.e. the score curve of the video frames obtained from the first part, and then clip the video.
However, the current solution of directly applying the extremum searching strategy to the second part of the automatic clipping has the following disadvantages: 1. the method can be only used for searching extreme points, namely, only a certain frame meeting the conditions can be screened from the video frame scoring curve, but not used for screening the video segments; 2. susceptible to noise, slight perturbations in the scoring curve may result in large differences in the search results, thereby affecting video clip quality.
Therefore, there is a need for improvements to existing video clipping methods.
Disclosure of Invention
The invention aims to overcome the defects of the existing video clipping method and provides an automatic video clipping method, an automatic video clipping device, a camera and a computer-readable storage medium, so as to overcome part of the defects of the existing video clipping method.
In a first aspect, a preferred embodiment of the present invention discloses an automatic clipping method, which includes: s1: carrying out smoothing treatment on a grading curve of a video to be edited; s2: calculating a maximum value point and a minimum value point in the smoothed score curve; s3: acquiring at least one preliminary extreme value segment according to two adjacent maximum value points or two adjacent minimum value points; s4: processing the preliminary extreme value fragments according to a preset grading threshold value to obtain processed extreme value fragments; s5: and editing the video according to the processed extremum segment.
In a specific aspect of this embodiment, the smoothing in step S1 may be implemented by median filtering or low-pass filtering.
In a specific aspect of this embodiment, the calculation of the maximum value point and the minimum value point in the score curve after the calculation and smoothing in step S2 may be implemented by: firstly, carrying out first-order derivation on the score curve after smoothing treatment, selecting a point with a derivative value of 0 as an extreme point of the score curve, taking the extreme point with a left derivative being negative and a right derivative being positive as an extreme point, and taking the extreme point with the left derivative being positive and the right derivative being negative as an extreme point.
In a specific aspect of this embodiment, step S4 is: and contracting each preliminary extreme value segment until the average score of the segment meets the requirement of a grading threshold value to obtain the processed extreme value segment.
In an optimized solution of this embodiment, step S4 further includes: and judging whether the length of the processed extremum segment is greater than a set value, if so, retaining, and otherwise, deleting.
In an optimized solution of this embodiment, step S4 further includes: and judging whether the interval between two adjacent processed extremum segments is larger than a set value, if so, simultaneously keeping the two processed extremum segments, and if not, deleting one of the processed extremum segments.
Further, the deleted extremum segment is a later time or shorter length extremum segment.
In a second aspect, the present invention also discloses an automatic editing apparatus, comprising: the smoothing module is used for smoothing the grading curve of the video to be clipped; the extreme point calculating module is used for calculating a maximum point and a minimum point in the score curve after smoothing treatment; the initial extreme value segment acquisition module is used for acquiring at least one initial extreme value segment according to two adjacent maximum value points or two adjacent minimum value points; the initial extreme value fragment processing module is used for processing the initial extreme value fragment according to a preset grading threshold value to obtain a processed extreme value fragment; and the video clipping module is used for clipping the video according to the processed extremum segment.
In a third aspect, the present invention also discloses a camera, which includes a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor to implement the automatic clipping method.
In a fourth aspect, the present invention also discloses a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the automatic clipping method described above.
Compared with the prior art, the extreme value segment is determined through the scoring curve and the scoring threshold of the video frame, the problem that only a single video frame can be screened but the video segment cannot be screened in the prior art is solved, the robustness of the scheme is high, and the video extreme value segment can be edited.
Drawings
Fig. 1 is a flowchart of an automatic clipping method in embodiment 1 of the present invention.
Fig. 2 is a schematic view of the score curve before smoothing in embodiment 1 of the present invention.
Fig. 3 is a schematic diagram of the score curve after being smoothed in embodiment 1 of the present invention.
Fig. 4 is a schematic diagram of obtaining an extremum segment in embodiment 1 of the present invention.
Fig. 5 is a block diagram showing the structure of an automatic clipping device according to embodiment 2 of the present invention.
Fig. 6 is a block diagram of a camera in embodiment 3 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clearly apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
Example 1
As shown in fig. 1 to 4, the automatic clipping method in the present embodiment includes the following steps.
S1: and performing smoothing treatment on the grading curve of the video to be clipped.
Firstly, scoring each frame of image of the video to be clipped according to a preset scoring rule or model, wherein the scoring mode can include: A. the score of the video frame is obtained through calculation of a preset score index or formula, for example, the score of the video frame is obtained according to the score indexes such as the definition of the video frame and the integrity of a figure in an image; B. and (4) grading each video frame by constructing a picture grading model based on a neural network. Next, as shown in fig. 2, after the score of each video frame of the video to be edited is obtained, a score curve is constructed by taking the ranking number of each video frame as the X axis and the score value of the video frame as the Y axis. Finally, as shown in fig. 3, after the score curve is obtained, the score curve is smoothed. In this embodiment, this can be achieved by performing median filtering or low-pass filtering on the score curve in fig. 2. Comparing fig. 2 and fig. 3, it can be seen that the score curve in fig. 3 is smoother.
S2: and calculating a maximum value point and a minimum value point in the smoothed scoring curve.
In the present embodiment, the calculation of the maximum value point and the minimum value point in the score curve after the calculation smoothing process in step S2 can be realized by: first-order derivation is performed on the smoothed score curve, and since the score curve in this embodiment is a discrete curve, the approximate calculation method of the derivative value at f (x) is as follows: and f (x)' = f (x +1) -f (x), selecting a point with a derivative value of 0 as an extreme point of the scoring curve, taking an extreme point with a negative left derivative and a positive right derivative as minimum points, and taking an extreme point with a positive left derivative and a negative right derivative as maximum points. As shown in fig. 4, a1 to a6 at the peak position of the score curve are maximum points, and B1 to B5 at the valley position of the score curve are minimum points.
S3: and acquiring at least one preliminary extremum segment according to the two adjacent maximum value points or the two adjacent minimum value points.
In the present embodiment, taking an example of finding a relatively stable segment of a shot as an example, the larger the shot shake, the higher the score, the smaller the shake, and the lower the score, so that the score curve score is lower and more suitable for viewing, and the clip at this time is to obtain a video frame with a lower average score, so that preliminary extremum segments are obtained before two adjacent maximum points, as shown in fig. 4, the present embodiment includes 5 preliminary extremum segments, specifically, A1B1A2, A2A3, A3B3A4, A4B4A5, and A5B5a 6. In other aspects of this embodiment, if a high definition video frame is to be obtained, if the quality of the video frame is higher and the score is higher, a preliminary extremum segment is obtained between two adjacent minimum value points. Therefore, according to different scoring criteria, the preliminary extremum segment can be obtained at two adjacent maximum value points or at two adjacent minimum value points.
S4: and processing the preliminary extreme value fragments according to a preset grading threshold value to obtain processed extreme value fragments.
In this embodiment, each obtained preliminary extreme value segment is shrunk until the average score of the segment meets the requirement of the scoring threshold, so as to obtain a processed extreme value segment, specifically, taking the preliminary extreme value segment A1B1a2 as an example, and taking A1 and a2 as starting points, respectively, so as to gradually shrink toward the minimum value point B1, so as to obtain a processed extreme value segment D1. Similarly, the preliminary extremum segments A2B2A3, A3B3A4, A4B4A5 and A5B5a6 may be processed in the same manner, as shown in fig. 4, to obtain corresponding processed extremum segments D2, D3, D4 and D5. It should be noted that, if a certain preliminary extremum segment shrinks to a length of zero and cannot be satisfied, the preliminary extremum segment is removed, that is, the preliminary extremum segment does not have a corresponding processed extremum segment.
In view of the reasons that the content of the extreme value segment that is too short is less, the value is not high, and the like, in a specific scheme of this embodiment, the extreme value segment that is processed with too short length is further subjected to deletion processing, specifically: firstly, judging whether the length of the processed extremum segment is greater than a set value (such as 30 frames), if so, retaining, and if not, deleting. As shown in fig. 4, since the processed segment D1 of the preliminary extremum segment A1B1a2 is too short, the remaining extremum segments processed by this step are D2, D3, D4, and D5.
In order to exclude the excessively concentrated extremum segments and make the distribution of the processed extremum segments more uniform, in a specific solution of this embodiment, it is determined whether an interval between two adjacent processed extremum segments is greater than a predetermined value (e.g., 5 frames), if so, the two processed extremum segments are retained at the same time, and if not, one of the processed extremum segments is deleted. Specifically, in the present embodiment, the extremum segment with the later deletion time is selected, as shown in fig. 4, since the interval between the processed extremum segments D2 and D3 is small, the extremum segment D3 is selected to be deleted after the processing in this step, and therefore, the remaining extremum segments are D2, D4, and D5. In other embodiments of this embodiment, the deleted extremum segment may also be a processed extremum segment with a shorter length.
S5: and editing the video according to the processed extremum segment.
In the present embodiment, the remaining extremum segments D2, D4, and D5 are subjected to merging processing, thereby completing clipping of the video. It will be appreciated that the extremum segments may also be deleted and the remaining video segments may then be merged to complete the clipping of the video.
Example 2
As shown in fig. 5, the present embodiment discloses an automatic clipping apparatus including: the smoothing module is used for smoothing the score curve of the video to be clipped, for example, the score curve can be processed by a filter with a median filtering function or a low-pass filtering function; the extreme point calculating module is used for calculating a maximum point and a minimum point in the score curve after smoothing treatment; the initial extreme value segment acquisition module is used for acquiring at least one initial extreme value segment according to two adjacent maximum value points or two adjacent minimum value points; the initial extreme value fragment processing module is used for processing the initial extreme value fragment according to a preset grading threshold value to obtain a processed extreme value fragment; and the video clipping module is used for clipping the video according to the processed extremum segment.
Example 3
As shown in fig. 6, the present embodiment discloses a camera, which includes a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor to implement the automatic clipping method in embodiment 1.
Example 4
Disclosed in this embodiment is a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the automatic clipping method in embodiment 1.
It should be noted that the executable instructions in the above-described embodiments may, but need not, correspond to files in a file system, may be stored in a portion of a file that holds other programs or data, such as in one or more scripts in a hypertext markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). By way of example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing associated hardware, and the storage medium may be a computer-readable storage medium, such as a ferroelectric Memory (FRAM), a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), an Erasable Programmable Read Only Memory (EEPROM), a flash Memory, a magnetic surface Memory, an optical disc, or a Compact disc Read Only Memory (CD-ROM), etc.; or may be various devices including one or any combination of the above memories.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. An automatic clipping method, comprising:
s1: carrying out smoothing treatment on a grading curve of a video to be edited;
s2: calculating a maximum value point and a minimum value point in the smoothed score curve;
s3: acquiring at least one preliminary extreme value segment according to two adjacent maximum value points or two adjacent minimum value points;
s4: processing the preliminary extreme value fragments according to a preset grading threshold value to obtain processed extreme value fragments;
s5: and editing the video according to the processed extremum segment.
2. The automatic clipping method according to claim 1, wherein the smoothing process in step S1 is a median filter or a low-pass filter.
3. The automatic clipping method according to claim 1, wherein the step S2 is: and performing first-order derivation on the smoothed score curve, selecting a point with a derivative value of 0 as an extreme point of the score curve, taking the extreme point with a negative left derivative and a positive right derivative as an extreme point, and taking the extreme point with a positive left derivative and a negative right derivative as an extreme point.
4. The automatic clipping method according to claim 1, wherein the step S4 is: and contracting each preliminary extreme value segment until the average score of the segment meets the requirement of a grading threshold value to obtain the processed extreme value segment.
5. The automatic clipping method according to claim 4, wherein the step S4 further comprises: and judging whether the length of the processed extremum segment is greater than a set value, if so, retaining, and otherwise, deleting.
6. The automatic clipping method according to claim 4, wherein the step S4 further comprises: and judging whether the interval between two adjacent processed extremum segments is larger than a set value, if so, simultaneously keeping the two processed extremum segments, and if not, deleting one of the processed extremum segments.
7. The automatic clipping method of claim 6, wherein processed extremum segments that are later in time or shorter in length are deleted.
8. An automatic clipping apparatus, characterized by comprising:
the smoothing module is used for smoothing the grading curve of the video to be clipped;
the extreme point calculating module is used for calculating a maximum point and a minimum point in the score curve after smoothing treatment;
the initial extreme value segment acquisition module is used for acquiring at least one initial extreme value segment according to two adjacent maximum value points or two adjacent minimum value points;
the initial extreme value fragment processing module is used for processing the initial extreme value fragment according to a preset grading threshold value to obtain a processed extreme value fragment;
and the video clipping module is used for clipping the video according to the processed extremum segment.
9. A camera comprising a processor and a memory, the memory having stored thereon a computer program which, when executed by the processor, implements the automatic clipping method of any one of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements the automatic clipping method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110679454.6A CN113411666A (en) | 2021-06-18 | 2021-06-18 | Automatic clipping method, apparatus, camera, and computer-readable storage medium |
| PCT/CN2022/098912 WO2022262766A1 (en) | 2021-06-18 | 2022-06-15 | Automatic clipping method and device, camera, and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110679454.6A CN113411666A (en) | 2021-06-18 | 2021-06-18 | Automatic clipping method, apparatus, camera, and computer-readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113411666A true CN113411666A (en) | 2021-09-17 |
Family
ID=77681462
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110679454.6A Pending CN113411666A (en) | 2021-06-18 | 2021-06-18 | Automatic clipping method, apparatus, camera, and computer-readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113411666A (en) |
| WO (1) | WO2022262766A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113992975A (en) * | 2021-10-13 | 2022-01-28 | 咪咕视讯科技有限公司 | Video playing method, device, equipment and computer storage medium |
| WO2022262766A1 (en) * | 2021-06-18 | 2022-12-22 | 影石创新科技股份有限公司 | Automatic clipping method and device, camera, and computer readable storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2224357A1 (en) * | 2009-02-27 | 2010-09-01 | BRITISH TELECOMMUNICATIONS public limited company | Video segmentation |
| US10289916B2 (en) * | 2015-07-21 | 2019-05-14 | Shred Video, Inc. | System and method for editing video and audio clips |
| CN110996169B (en) * | 2019-07-12 | 2022-03-01 | 北京达佳互联信息技术有限公司 | Method, device, electronic equipment and computer-readable storage medium for clipping video |
| CN112770061A (en) * | 2020-12-16 | 2021-05-07 | 影石创新科技股份有限公司 | Video editing method, system, electronic device and storage medium |
| CN113411666A (en) * | 2021-06-18 | 2021-09-17 | 影石创新科技股份有限公司 | Automatic clipping method, apparatus, camera, and computer-readable storage medium |
-
2021
- 2021-06-18 CN CN202110679454.6A patent/CN113411666A/en active Pending
-
2022
- 2022-06-15 WO PCT/CN2022/098912 patent/WO2022262766A1/en unknown
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022262766A1 (en) * | 2021-06-18 | 2022-12-22 | 影石创新科技股份有限公司 | Automatic clipping method and device, camera, and computer readable storage medium |
| CN113992975A (en) * | 2021-10-13 | 2022-01-28 | 咪咕视讯科技有限公司 | Video playing method, device, equipment and computer storage medium |
| CN113992975B (en) * | 2021-10-13 | 2023-10-17 | 咪咕视讯科技有限公司 | Video playing method, device, equipment and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022262766A1 (en) | 2022-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113411666A (en) | Automatic clipping method, apparatus, camera, and computer-readable storage medium | |
| CN108664920B (en) | Real-time large-scale cascading face clustering method and device | |
| CN111522996B (en) | Video clip retrieval method and device | |
| CN110807757B (en) | Image quality evaluation method and device based on artificial intelligence and computer equipment | |
| CN106651755A (en) | Panoramic image processing method and device for terminal and terminal | |
| CN109685144B (en) | Method and device for evaluating video model and electronic equipment | |
| KR101843516B1 (en) | Method for determination of focal length for a zoom lens | |
| CN110263628A (en) | Obstacle detection method, device, electronic equipment and storage medium | |
| CN114297432A (en) | Video retrieval method, device and equipment and computer readable storage medium | |
| CN112785572A (en) | Image quality evaluation method, device and computer readable storage medium | |
| CN115858861A (en) | Video compression method, electronic device and computer-readable storage medium | |
| CN113420871B (en) | Image quality evaluation method, image quality evaluation device, storage medium, and electronic device | |
| CN113825012B (en) | Video data processing method and computer device | |
| CN111539206B (en) | Method, device, equipment and storage medium for determining sensitive information | |
| CN111314736B (en) | Video copyright analysis method and device, electronic equipment and storage medium | |
| CN103257954B (en) | The proofreading method of word, system and check and correction server in ancient books | |
| CN109886963B (en) | Image processing method and system | |
| CN115019138A (en) | Video subtitle erasing, model training and interaction method, device and storage medium | |
| CN114140822A (en) | Pedestrian re-identification method and device | |
| CN114419018A (en) | Image sampling method, system, device and medium | |
| CN111143589A (en) | Image processing method and device and storage medium | |
| CN112533024A (en) | Face video processing method and device and storage medium | |
| CN112258401B (en) | Image enhancement method and device | |
| CN116883275B (en) | Image rain removing method, system, device and medium based on boundary guidance | |
| CN114677444B (en) | Optimized visual SLAM method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |