CN113099128B - Video processing method and video processing system - Google Patents
Video processing method and video processing system Download PDFInfo
- Publication number
- CN113099128B CN113099128B CN202110378777.1A CN202110378777A CN113099128B CN 113099128 B CN113099128 B CN 113099128B CN 202110378777 A CN202110378777 A CN 202110378777A CN 113099128 B CN113099128 B CN 113099128B
- Authority
- CN
- China
- Prior art keywords
- image
- image frames
- video
- key
- image frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
Abstract
The application relates to a video processing method and a video processing system, which can automatically screen image frames with poor quality from a plurality of key image frames in a video to be edited, can specifically distinguish image frames generating lens pollution from image frames generating fuzzy distortion, automatically classify and collect the image frames generating lens pollution and the image frames generating fuzzy distortion as materials and send the materials to a video editing terminal, so that the image frames generating lens pollution and the image frames generating fuzzy distortion can be directly deleted or reserved by the video editing terminal according to user requirements subsequently, the working efficiency of the video editing terminal is improved, and the video quality of the video after editing is improved.
Description
Technical Field
The present application relates to the field of video processing technologies, and in particular, to a video processing method and a video processing system.
Background
The video editing refers to a process of remixing added materials such as pictures, background music, special effects and scenes with a video, cutting and combining video sources, and generating new videos with different expressive power through secondary coding. With the vigorous development of the self-media industry, a plurality of video editing software with rich functions appears.
However, the existing video editing software has a great problem that some image frames with poor quality in the video to be edited cannot be automatically identified according to the user requirements, and the types of the image frames with poor quality cannot be distinguished, so that a large number of image frames with poor quality exist in the edited video, and the video quality is affected.
Disclosure of Invention
Based on this, it is necessary to provide a video processing method for solving the problem that the conventional video editing method cannot automatically identify some image frames with poor quality in the video to be edited according to the user requirement.
The application provides a video processing method, which comprises the following steps:
extracting a plurality of key image frames in a video to be edited, and creating a first image frame set and a second image frame set;
screening image frames generating lens pollution from a plurality of key image frames, placing all the image frames generating lens pollution into the first image frame set, and placing the rest key image frames into the second image frame set;
screening out image frames with fuzzy distortion from all key image frames in the second image frame set;
creating a third image frame collection, and placing all image frames with blurring distortion into the third image frame collection;
and sending the first image frame set, the second image frame set and the third image frame set to a video clip terminal.
The present application further provides a video processing system, the video processing system comprising:
a video screening terminal for executing the video processing method as mentioned in the foregoing;
the video editing terminal is in communication connection with the video screening terminal and is used for receiving the first image frame set, the second image frame set and the third image frame set screened by the video screening terminal, and performing video editing based on the first image frame set, the second image frame set and the third image frame set to generate an edited video;
and the server is respectively in communication connection with the video screening terminal and the video clip terminal and is used for sending the video to be clipped to the video screening terminal and receiving the clipped video sent by the video clip terminal.
The application relates to a video processing method and a video processing system, which can automatically screen image frames with poor quality from a plurality of key image frames in a video to be edited, can specifically distinguish image frames generating lens pollution from image frames generating fuzzy distortion, automatically classify and collect the image frames generating lens pollution and the image frames generating fuzzy distortion as materials and send the materials to a video editing terminal, so that the image frames generating lens pollution and the image frames generating fuzzy distortion can be directly deleted or reserved by the video editing terminal according to user requirements subsequently, the working efficiency of the video editing terminal is improved, and the video quality of the video after editing is improved.
Drawings
Fig. 1 is a schematic flowchart of a video processing method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a video processing system according to an embodiment of the present application;
fig. 3 is a schematic diagram illustrating a contaminated image area appearing in Z consecutive key image frames in a video processing method according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The application provides a video processing method. It should be noted that the video processing method provided by the present application can be used for processing video in any data format, including but not limited to one of MP4 format, MPEG format, AVI format, ASF format, MOV format, WMV format, 3GP format, FLV format, and RMVB format.
In addition, the video processing method provided by the application is not limited to the execution subject. Optionally, an execution subject of the video processing method provided by the present application may be a video screening terminal. Specifically, the executing subject of the video processing method provided by the present application may be one or more processors in the video screening terminal.
As shown in fig. 1, in an embodiment of the present application, the video processing method includes the following steps S100 to S500:
s100, extracting a plurality of key image frames in a video to be edited, and creating a first image frame set and a second image frame set.
Specifically, the image frame in the video is a single image frame of the minimum unit in the motion picture, and is equivalent to each frame of the shot on the motion picture film. On the time axis of the clipping software, the image frame appears as a grid or a marker. The key image frame is equivalent to an original image in the two-dimensional animation, and refers to a frame where a key action in the movement or change of the character or the object is located. The image frame between two key image frames is called a normal frame, a transition frame or an intermediate frame.
The key image frame, also called I-frame, is the most important frame for inter-frame compression coding. Video is encoded in "groups", each Group being called a GOP (Group of Picture). Each GOP is started from a key image frame, the key image frame is a complete picture, the image frames in the middle of the GOP are incomplete, and the key image frame, the front image frame, the rear image frame and the like are required to be calculated together. The adjustment of the key image frame interval affects the length of the GOP, and thus the speed of reading the GOP. If the interval of the key picture frame is set to be excessively large (GOP length is excessively large), it may be forced to use B/P frames instead where the key picture frame must be used, which may degrade picture quality. The key image frame is the basis of inter-frame compression, and a typical GOP structure is generally: IBBPBBPBBPBBPBBPBB. The former and latter reference frames are called B frames, that is, the data of the frame is obtained by referring to the data of the former and latter frames and the change of the frame. The forward reference frame is called a P frame. Assuming that the key image frame, i.e., the I-frame, is corrupted, the entire GOP structure is corrupted, i.e., ibbpbbpbbbpbbpbb, such multiple frames are corrupted together.
Therefore, automatically extracting a plurality of key image frames in the video to be edited can extract the core content in the video to be edited. The core contents are screened, and whether image frames with poor quality exist in the core contents is analyzed, so that the core contents can be screened out.
S200, screening image frames generating lens pollution from the plurality of key image frames. And putting all image frames generating lens pollution into the first image frame set. And putting the rest key image frames into the second image frame set.
In particular, the image frame that generates the lens blur is a type of low-quality key image frame.
When a user uses a camera to capture a video, or uses a mobile terminal (e.g., a mobile phone) to capture a video, the lens of the camera or the mobile terminal is prone to lens contamination due to the capture environment. For example, a sudden rain, a lens contaminated with rain, or a vehicle passing by, a lens contaminated with mud. This may degrade some key image frames in the captured video.
Of course, there may be a case where all of the plurality of key image frames extracted in S100 are image frames causing lens contamination. And a user can also find that the lens is polluted in the shooting process, the shooting is interrupted, the lens is cleaned, and the shooting is recovered to be normal after the cleaning. In this case, some image frames that cause lens contamination inevitably exist in the video. Of course, it is also possible that no image frame causing lens contamination exists in the plurality of key image frames extracted in S100.
The step can realize that the image frame generating the lens pollution is automatically screened out from a plurality of key image frames.
S300, screening out the image frame with the fuzzy distortion from all the key image frames in the second image frame aggregate.
Specifically, the image frame with blur distortion is another type of low-quality key image frame, unlike the image frame that causes lens contamination. The blurred image frames appear for a number of reasons, which may be due to an inappropriate aperture size, too high sensitivity ISO settings, camera or phone shake during shooting, lack of or imperfect focus, and so on.
The image frames which generate the lens pollution do not exist in the second image frame set, and the step can realize that the image frames which generate the lens pollution are automatically screened from all the key image frames in the second image frame set.
S400, creating a third image frame set, and placing all image frames with fuzzy distortion into the third image frame set.
Specifically, the image frames generating the lens pollution and the image frames generating the fuzzy distortion are automatically classified and collected to be used as materials to be sent to the video editing terminal, and the subsequent video editing terminal can conveniently and orderly manage the materials.
S500, sending the first image frame set, the second image frame set and the third image frame set to a video clip terminal.
Specifically, the video clipping terminal may subsequently select any image frame set to perform video clipping, for example, delete all key image frames in the first image frame set and all key image frames in the third image frame set, that is, delete all image frames causing lens contamination and image frames with blur distortion, and select only normal key image frames in the second image frame set to perform video clipping.
Of course, it is also possible to select to keep the key image frames in the first image frame set and the second image frame set, delete all the key image frames in the third image frame set, that is, delete only the image frames with blur distortion, and keep all the image frames causing lens contamination. Some rainy lens pictures are aesthetic, can be reserved, do not influence the quality of the video, and even can improve the visual sense of the video. The advantages of the classified storage and uploading video clip terminal are embodied, and the degree of freedom is high.
The embodiment can realize that the image frames with poor quality are automatically screened from the plurality of key image frames in the video to be clipped, the image frames generating the lens pollution and the image frames generating the fuzzy distortion can be specifically distinguished, the image frames generating the lens pollution and the image frames generating the fuzzy distortion are automatically classified and collected to be used as materials to be sent to the video clipping terminal, the image frames generating the lens pollution and the image frames generating the fuzzy distortion can be conveniently and directly deleted or reserved by the video clipping terminal according to the user requirements subsequently, the working efficiency of the video clipping terminal is improved, and the video quality of the clipped video is improved.
In an embodiment of the present application, the S200 includes the following S210 to S240:
and S210, acquiring the time stamp of each key image frame. And sequencing the plurality of key image frames according to the time nodes displayed by the time stamps in time sequence.
Specifically, each key image frame has a timestamp. The time stamp displays the time node corresponding to the key image frame. We can order multiple key image frames according to chronological order.
S220, dividing Z key image frames adjacent to the time node into a time window, and further dividing all the key image frames into a plurality of time windows. Z is a positive integer greater than 10.
Specifically, for example, if there are 100 key image frames, and Z is 10, the 100 key image frames are arranged in chronological order, then the 100 key image frames are divided into 10 time windows.
S230, dividing each key image frame into K in an array form 1 ×K 2 An image area. Image plane covered by each image areaThe products are equal. Each image region R (i, j). i is the line number of the image area. j is the column number of the image area.
Specifically, since the video to be edited is captured using one capturing device, the sizes of all key image frames are the same. Therefore, the number of image areas divided by each key image frame is equal.
Each image region is denoted as R (i, j), and for example, as shown in FIG. 3, the key image frame shown in FIG. 3 has 16 image regions (K) in total of 4 × 4 1 ＝4，K 2 The image area marked black in fig. 3 is R (1, 2), and represents the image area in the first row and the second column in the key image frame (i is 1, j is 2).
S240, selecting a time window, analyzing the gray level change condition of each image area in the continuous Z key image frames, and judging whether the polluted image area appears in the continuous Z key image frames.
Specifically, the gray level change condition of Z consecutive key image frames in each image area is analyzed, and whether the image area has a condition with a small gray level change amplitude can be judged.
When the lens is contaminated, the contaminated area cannot be imaged normally. Therefore, the gray scale change of a polluted image area in a plurality of consecutive key image frames is small, for example, a position of the lens is covered by a mud dot, and the image area corresponding to the position in a plurality of consecutive key image frames is always a black area, and the gray scale change is small or almost unchanged.
Therefore, whether the polluted image area appears in the continuous Z key image frames or not can be judged by analyzing the gray scale change condition of each image area in the continuous Z key image frames.
As for why the setting judges whether or not the contaminated image area appears in the consecutive Z key image frames, rather than the setting judges whether or not the contaminated image area appears in all the key image frames, because there is a possibility that the user wipes the lens after finding that the lens is contaminated during the photographing process, and the contaminated image area is restored to a clean state again in the key image frames appearing subsequently. Therefore, the time window and the judgment logic for judging whether the contaminated image area appears in the continuous Z key image frames in the time window are set.
In this embodiment, by setting a time window, at least one time window is set for each adjacent Z key image frames, and whether an image area is contaminated in consecutive Z key image frames in each time window is determined, so that an image frame causing lens contamination can be accurately screened out from a plurality of key image frames.
In an embodiment of the present application, the S230 includes the following S231 to S232:
s231, the pixel size of the key image frame is acquired. The expression of the pixel size of the key image frame is formula 1.
I-mxn formula 1
Wherein I is the pixel size of the key image frame. M is the horizontal pixel size. N is the vertical pixel size.
Specifically, for example, the pixel size of the key image frame is 1080P, that is, 1920 × 1080, then M equals 1920 and N equals 1080.
S232, determining K according to the pixel size of the key image frame 1 And K 2 A value of (a) such that K 1 And K 2 Satisfies formula 2.
Where M is the horizontal pixel size in equation 1 and N is the vertical pixel size in equation 1.
Specifically, set K 1 And K 2 The ratio of (a) to (b) is equal to the ratio of (M) to (N), which may make the segmentation of the plurality of image regions in the key image frame more reasonable. For example, M equals 1920, N equals 1080, then K 1 And K 2 The ratio of (a) is equal to the ratio of 1920 to 1080, achieving equal-scale segmentation of the plurality of image areas.
In this embodiment, by setting K 1 And K 2 The ratio of the image area to the key image frame is equal to the ratio of the M to the N, so that the segmentation of the image area in the key image frame is more reasonable, the covered area of each image area can be ensured to be the same, and the phenomenon that the covered area of a certain image area is too large to cause the unbalance of the follow-up gray level change analysis can be avoided.
In one embodiment of the present application, K 1 And K 2 The value of (c) also needs to satisfy the condition of equation 3.
In particular, K 1 And K 2 The numerical values of the image areas are all set to be larger than 10, so that the situation that the number of the image areas in one row or one column is too small can be avoided, and the situation that the judgment of the polluted image areas is not accurate can be avoided.
Set up K 1 And K 2 The purpose of the product of (2) is to limit the total number of image regions and avoid excessive image regions, which results in excessive calculation pressure of a subsequent judgment algorithm of polluted image regions. In other words, the total number of image areas cannot be too small or too large.
In the present embodiment, by limiting K 1 And K 2 Are set to all be greater than 10, and set to K 1 And K 2 The product of (2) and (2) is less than or equal to 500, the number of the image areas contained in each row and the number of the image areas contained in each column can be limited to be in an optimal numerical range, the accuracy of judging the polluted image areas can be ensured, and the cost increase caused by overlarge calculation pressure of a judging algorithm can be avoided.
In an embodiment of the present application, the step S240 includes the following steps S241 to S247:
s241, selecting a time window.
S242, an image region R (i, j) is selected.
And S243, calculating the average gray value change rate of each pixel point in the image area R (i, j) in the continuous Z key image frames according to a formula 4.
Wherein the content of the first and second substances,
is the average rate of change of gray level.
And the gray value of the Lth pixel point in the R (i, j) image area in the nth key image frame is obtained. R (i, j) is the number of image regions. And L is the serial number of the pixel point. n is the number of key image frames. Z is the total number of consecutive key image frames within a time window.
Specifically, the measurement adopted by the present embodiment is the average gray value change rate of one pixel point in the consecutive Z key image frames, so that the measurement and calculation are more accurate. It can be understood that if the average gray value change rate of a pixel point in Z consecutive key image frames is smaller, it indicates that the pixel point is closer to a state without change in the Z consecutive key image frames. It can be understood that, if the more the pixels in an image region are unchanged, the more the image region is shielded by the foreign object, the more the image region is polluted.
S244, selecting the pixel points with the average gray value change rate larger than the preset gray value change rate as the pollution pixel points, and counting the number of the pollution pixel points in the image area R (i, j).
Specifically, the preset gray value change rate may be set to 10%. It is understood that an image area is not necessarily a contaminated image area if too few contaminated pixels are present in the image area, but is considered to be a contaminated area if too many contaminated pixels are present in the image area. However, in this embodiment, whether the image region R (i, j) is a contaminated image region is not directly determined according to the number of the contaminated pixel points, but is determined according to the percentage of the number of the contaminated pixel points to the total number of the pixel points, which is specifically referred to in subsequent steps S245 to S247.
S245, calculating the percentage of the number of the polluted pixel points in the image region R (i, j) to the total number of the pixel points in the image region R (i, j), and recording the calculation result as the pollution percentage.
Specifically, in the step, the pollution percentage is calculated, and the pollution degree of one image area is measured by using the pollution percentage, so that whether the image area is the polluted image area is judged, and the judgment result is more accurate by comparing the number of the polluted pixel points with the number threshold.
And S246, judging whether the pollution percentage is larger than a preset pollution percentage.
Specifically, the preset percentage may be set to 90%.
And S247, if the contamination percentage is greater than the preset contamination percentage, determining an image area which is contaminated in the continuous Z key image frames, and taking the image area R (i, j) as the contaminated image area.
Specifically, in this step, if the contamination percentage is greater than the preset contamination percentage, it is determined that the image region R (i, j) is a contaminated image region. The following steps are S251 to S252 or S261 to S265, and two embodiments will be described later.
Taking fig. 3 as an example, the contaminated pixel points in the image region R (1, 2) are displayed as black dots, and the number is 12. The number of the non-pollution pixel points, namely the normal pixel points, is 4, and the white dots are displayed. If the preset contamination percentage is set to 70%. Then, the percentage of the number 12 of the contaminated pixels in the image region R (1, 2) to the total number 16 of the pixels in the image region R (i, j) is calculated to be 75% and greater than the preset contamination percentage of 70%, then it can be determined that a contaminated image region occurs in Z consecutive key image frames (Z is 5 in fig. 3, and fig. 3 analyzes the gray level change of the 11 th frame to the 15 th frame), and the contaminated image region is the image region R (1, 2).
In the embodiment, the percentage of the number of the polluted pixel points in one image area to the total number of the pixel points in the image area is calculated and compared with the preset percentage value, so that whether the image area is the polluted image area is determined, the judgment result is more accurate, and the possible errors are avoided.
For example, if it is determined whether the image area is the contaminated image area without determining whether the number of the contaminated pixel points is greater than a threshold value based on the percentage of the total number of the pixel points in the image area, then the following situations may occur:
in an image area, the number of pixels with small average gray value change rate is large, but actually, the pixels are an environmental background, such as grassland and sand, and they also appear as a plurality of pixels with small average gray value change rate in a certain image area, and although the number is large, the average gray value change rate is not particularly high, because the area covered by one image area is large. Therefore, whether the image area is the polluted image area is judged according to the percentage of the number of the polluted pixel points in the image area to the total number of the pixel points in the image area, and the judgment is more reasonable.
In an embodiment of the present application, after the S246, the S240 further includes the following S248 to S249:
s248, if the contamination percentage is less than or equal to the preset contamination percentage, determining that the image region R (i, j) is not a contaminated image region.
S249, the process returns to S242 to select the next image region R (i, j).
Specifically, if it is determined that the image region R (i, j) is not a contaminated image region, the process returns to S242 and continues to execute S243 to S246 on the next image region until a contaminated image region is found.
If all the image areas are subjected to the calculation of the pollution percentage and are compared with the preset pollution percentage, and no polluted image area is found, the time window is considered to have no polluted image area, the Z key image frames in the time window are not the image frames generating the lens pollution, and the Z key image frames in the time window can be placed into a second image frame set subsequently.
However, it is only that all image areas of this time window have undergone gray scale change analysis, and it is also required to return to S241 and perform S242 to S249 once again for all time windows.
In an embodiment of the present application, after the S240, the S200 further includes the following S251 to S252:
and S251, if the polluted image area appears in the continuous Z key image frames, taking the continuous Z key image frames as the image frames generating the lens pollution.
And S252, repeatedly executing the steps from S241 to S251 until the gray level change conditions in the continuous Z key image frames in all the time windows are analyzed, and placing all the image frames generating the lens pollution into a first image frame set. And placing the rest key image frames into the second image frame set.
In the present embodiment, S247 is executed, and S251 is executed after S247 is executed, which is a processing manner after S247 is executed. In this embodiment, as long as a contaminated image area is found, it is not necessary to return to S242 to perform gray scale change analysis on the next image area, and the consecutive Z key image frames are directly placed into the first image frame set as image frames generating lens contamination.
And executing S252 to analyze the gray level change conditions of the Z continuous key image frames in all the time windows.
In this embodiment, when a contaminated image area appears in the consecutive Z key image frames, the consecutive Z key image frames are used as image frames generating lens contamination, the determination speed of this determination method is fast, the calculation pressure is small,
in an embodiment of the present application, after the S240, the S200 further includes the following S261 to S265:
s261, if the contaminated image areas appear in the consecutive Z keyframe frames, further counting the number of the contaminated image areas appearing in the consecutive Z keyframe frames.
S262, according to the number of the polluted image areas in the continuous Z key image frames, a public is adopted
Equation 5 calculates the image area contamination level.
Where δ is the image area contamination level. P is the sequence number of the time window. Q P The number of image regions that appear contaminated in successive Z keyframe frames in the time window P. K 1 ×K 2 Is the total number of image areas.
S263, determine whether the image area is more than the predetermined area.
And S264, if the image area pollution degree is greater than the preset area pollution degree, taking the continuous Z key image frames as image frames generating lens pollution.
And S265, repeatedly executing the steps from S240 to S264 until the gray level change conditions in the continuous Z key image frames in all the time windows are analyzed, and placing all the image frames generating the lens pollution into a first image frame set. And placing the rest key image frames into the second image frame set.
In this embodiment, S247 is executed, and S261 is executed after S247 is completed, which is another processing manner after S247, and is different from S251 to S252.
In this embodiment, instead of having a contaminated image area in the consecutive Z keyframe frames, the Z keyframe frames are regarded as the image frames causing the lens contamination, and the number of contaminated image areas in the consecutive Z keyframe frames is counted.
The counting of the number of image areas with contamination in the consecutive Z key image frames specifically includes:
s261a returns to S242 to select the next image region R (i, j), until all image regions are judged to be contaminated, and count the number of contaminated image regions in the Z consecutive key image frames.
And then calculating the percentage of the image area with the pollution to the total number of the image areas to obtain the pollution degree of the image areas. And comparing the image area pollution degree with a preset area pollution degree, and if the image area pollution degree is greater than the preset area pollution degree, taking the Z key image frames as image frames generating lens pollution.
The image area pollution degree calculation and comparison process is added to the judgment of the image frame generating the lens pollution, so that the screening of the image frame generating the lens pollution is more accurate, the image frame generating the lens pollution more conforms to the actual situation, and the error rate of the image frame which is determined as generating the lens pollution by the normal key image frame is effectively reduced.
In an embodiment of the present application, the S300 includes the following S310 to S320:
and S310, performing edge detection on each key image frame in the second image frame set, and calculating the image definition of each key image frame in the second image frame set.
S320, screening the key image frames with the image definition smaller than the image definition threshold value to serve as blurred and distorted image frames.
Specifically, the present embodiment screens out blurred and distorted image frames by calculating the image sharpness of each key image frame. The method for calculating the image definition of the key image frame is to perform edge detection on the key image frame.
Edge detection may use different operators. The operator that can be used includes, but is not limited to, one of the sobel operator, canny operator, and laplacian operator (laplacian operator).
A method for calculating the image definition of a key image frame by using one of operators, namely a sobel operator is introduced. The Sobel operator is a discrete differentiation operator. The edge detection is carried out by adopting a Sobel operator, and the edge detection is combined with a Tenengrad gradient function, so that the method is very effective for calculating the image definition.
First, the key image frame is convolved using the sobel operator. The sobel operator comprises two groups of 3x3 matrixes which are horizontal operators respectively
And vertical direction operator
The key image frame and the image frame are convoluted in a plane, and then a horizontal brightness difference approximate value and a longitudinal brightness difference approximate value can be respectively obtained, as shown in a formula 6.
Where Gx (x, y) is a lateral luminance difference approximation. Gy (x, y) is a longitudinal luminance difference approximation. And V is a key image frame.
The horizontal gradient approximation and the vertical gradient approximation of each pixel in the keyframe may be combined using equation 7 below to calculate the gradient magnitude of the pixel.
Wherein, G (x, y) is the gradient size of the pixel point.
The gradient direction can be calculated using equation 8.
The image sharpness based on the Tenengrad gradient function is defined as the following equation 9:
d (f) ═ y Σ x | G (x, y) |, G (x, y) > T equation 9
Where T is a given edge detection threshold.
The application also provides a video processing system.
As shown in fig. 2, in an embodiment of the present application, the video processing system includes a video screening terminal 100, a video clip terminal 200, and a server 300. The video screening terminal 100 is communicatively connected to the video clip terminal 200. The server 300 is in communication connection with the video screening terminal 100. The server 300 is also communicatively connected to the video clip terminal 200.
The video screening terminal 100 is configured to execute the video processing method provided in any of the foregoing embodiments. The video clip terminal 200 is configured to receive the first image frame set, the second image frame set, and the third image frame set screened by the video screening terminal 100. The video clipping terminal 200 further clips a video based on the first image frame set, the second image frame set, and the third image frame set, and generates a clipped video. The server 300 is configured to send a video to be clipped to the video screening terminal 100, and receive the clipped video sent by the video clipping terminal 200.
It should be noted that, the video processing system provided in this embodiment applies the aforementioned video processing method, and therefore, for brevity of description, the same devices or components appearing in the aforementioned video processing method and the video processing system of this embodiment are uniformly labeled in this embodiment, and the same devices or components of the video processing method part are not labeled. The same devices or components that may occur: a video screening terminal 100, a video clip terminal 200, and a server 300.
The technical features of the embodiments described above may be arbitrarily combined, the order of execution of the method steps is not limited, and for simplicity of description, all possible combinations of the technical features in the embodiments are not described, however, as long as there is no contradiction between the combinations of the technical features, the combinations of the technical features should be considered as the scope of the present description.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (8)
1. A method of video processing, the method comprising:
s100, extracting a plurality of key image frames in a video to be edited, and creating a first image frame set and a second image frame set;
s200, screening image frames generating lens pollution from a plurality of key image frames, placing all the image frames generating lens pollution into the first image frame set, and placing the rest key image frames into the second image frame set;
s300, screening out image frames with fuzzy distortion from all key image frames in the second image frame set;
s400, creating a third image frame set, and placing all image frames with blurring distortion into the third image frame set;
s500, sending the first image frame set, the second image frame set and the third image frame set to a video clip terminal;
the S200 includes:
s210, acquiring a time stamp of each key image frame, and sequencing a plurality of key image frames according to a time sequence according to time nodes displayed by the time stamps;
s220, dividing Z key image frames adjacent to the time node into a time window, and further dividing all the key image frames into a plurality of time windows; z is a positive integer greater than 10;
s230, dividing each key image frame into K in an array form 1 ×K 2 Each image area is equal in image area, and each image area is marked as R (i, j), wherein i is the row number of the image area, and j is the column number of the image area;
s240, selecting a time window, analyzing the gray level change condition of each image area in Z continuous key image frames, and judging whether the polluted image area appears in the Z continuous key image frames;
s251, if polluted image areas appear in the continuous Z key image frames, taking the continuous Z key image frames as image frames generating lens pollution;
s252, repeatedly executing S240 to S251 until the gray level change in the Z consecutive key image frames in all time windows is analyzed, placing all image frames generating lens contamination into the first image frame set, and placing the rest key image frames into the second image frame set;
the S240 includes:
s241, selecting a time window;
s242, selecting an image region R (i, j);
s243, calculating the average gray value change rate of each pixel point in the image region R (i, j) in continuous Z key image frames;
s244, selecting pixel points with the average gray value change rate larger than the preset gray value change rate as pollution pixel points, and counting the number of the pollution pixel points in the image area R (i, j);
s245, calculating the percentage of the number of polluted pixel points in the image region R (i, j) to the total number of pixel points in the image region R (i, j), and recording the calculation result as the pollution percentage;
s246, judging whether the pollution percentage is larger than a preset pollution percentage;
and S247, if the contamination percentage is greater than the preset contamination percentage, determining an image area which is contaminated in the continuous Z key image frames, and taking the image area R (i, j) as the contaminated image area.
2. The video processing method according to claim 1, wherein the S230 comprises:
s231, acquiring the pixel size of a key image frame, wherein the expression of the pixel size of the key image frame is formula 1;
i ═ mxn formula 1;
wherein, I is the pixel size of the key image frame, M is the horizontal pixel size, and N is the vertical pixel size;
s232, determining K according to the pixel size of the key image frame 1 And K 2 A value of such that K 1 And K 2 Satisfies formula 2;
where M is the horizontal pixel size in equation 1 and N is the vertical pixel size in equation 1.
3. The video processing method of claim 2, wherein K is 1 And K 2 The value of (c) also needs to satisfy the condition of formula 3;
4. the video processing method according to claim 3, wherein said S243 comprises:
calculating the average gray value change rate of each pixel point in the image region R (i, j) in continuous Z key image frames according to a formula 4;
wherein, the first and the second end of the pipe are connected with each other,
as the average rate of change of the gray level value,
the gray value of the L-th pixel point in the n-th key image frame in the R (i, j) image area is represented, R (i, j) is the serial number of the image area, L is the serial number of the pixel point, n is the serial number of the key image frame, and Z is the total number of a plurality of key image frames which are continuously arranged in a time window.
5. The video processing method according to claim 4, wherein after the S246, the S240 further comprises:
s248, if the pollution percentage is less than or equal to the preset pollution percentage, determining that the image area R (i, j) is not a polluted image area;
s249, the process returns to S242 to select the next image region R (i, j).
6. The video processing method according to claim 5, wherein said S200, before said taking said consecutive Z key image frames as image frames generating lens pollution, further comprises:
s261, if polluted image areas appear in the continuous Z key image frames, further counting the number of the polluted image areas appearing in the continuous Z key image frames;
s262, calculating the pollution degree of the image area by adopting a formula 5 according to the number of the polluted image areas in the continuous Z key image frames;
wherein, delta is the image area pollution degree, P is the sequence number of the time window, Q P The number of image regions in which contamination occurs in Z consecutive key image frames in the time window P, K 1 ×K 2 Is the total number of image areas;
s263, judging whether the pollution degree of the image area is larger than the preset area pollution degree;
and S264, if the pollution degree of the image area is greater than the preset area pollution degree, taking the continuous Z key image frames as image frames generating lens pollution.
7. The video processing method according to claim 6, wherein the S300 comprises:
s310, performing edge detection on each key image frame in the second image frame set, and calculating the image definition of each key image frame in the second image frame set;
s320, screening the key image frames with the image definition smaller than the image definition threshold value to serve as blurred and distorted image frames.
8. A video processing system, comprising:
a video screening terminal for performing the video processing method according to any one of claims 1 to 7;
the video editing terminal is in communication connection with the video screening terminal and is used for receiving the first image frame set, the second image frame set and the third image frame set screened by the video screening terminal, and performing video editing based on the first image frame set, the second image frame set and the third image frame set to generate an edited video;
and the server is respectively in communication connection with the video screening terminal and the video clipping terminal and is used for sending the video to be clipped to the video screening terminal and receiving the clipped video sent by the video clipping terminal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110378777.1A CN113099128B (en) | 2021-04-08 | 2021-04-08 | Video processing method and video processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110378777.1A CN113099128B (en) | 2021-04-08 | 2021-04-08 | Video processing method and video processing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113099128A CN113099128A (en) | 2021-07-09 |
| CN113099128B true CN113099128B (en) | 2022-09-13 |
Family
ID=76675143
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110378777.1A Active CN113099128B (en) | 2021-04-08 | 2021-04-08 | Video processing method and video processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113099128B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101807198A (en) * | 2010-01-08 | 2010-08-18 | 中国科学院软件研究所 | Video abstraction generating method based on sketch |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100708337B1 (en) * | 2003-06-27 | 2007-04-17 | 주식회사 케이티 | Apparatus and method for automatic video summarization using fuzzy one-class support vector machines |
| US7889794B2 (en) * | 2006-02-03 | 2011-02-15 | Eastman Kodak Company | Extracting key frame candidates from video clip |
| US20120148149A1 (en) * | 2010-12-10 | 2012-06-14 | Mrityunjay Kumar | Video key frame extraction using sparse representation |
| CN111583169A (en) * | 2019-01-30 | 2020-08-25 | 杭州海康威视数字技术股份有限公司 | Pollution treatment method and system for vehicle-mounted camera lens |
| CN110121103A (en) * | 2019-05-06 | 2019-08-13 | 郭凌含 | The automatic editing synthetic method of video and device |
| CN110996183B (en) * | 2019-07-12 | 2022-01-21 | 北京达佳互联信息技术有限公司 | Video abstract generation method, device, terminal and storage medium |
| CN110611846A (en) * | 2019-09-18 | 2019-12-24 | 安徽石轩文化科技有限公司 | Automatic short video editing method |
-
2021
- 2021-04-08 CN CN202110378777.1A patent/CN113099128B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101807198A (en) * | 2010-01-08 | 2010-08-18 | 中国科学院软件研究所 | Video abstraction generating method based on sketch |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113099128A (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1483909B1 (en) | Systems and methods for digitally re-mastering or otherwise modifying motion pictures or other image sequences data | |
| JP5478047B2 (en) | Video data compression pre-processing method, video data compression method and video data compression system using the same | |
| US20120019728A1 (en) | Dynamic Illumination Compensation For Background Subtraction | |
| US20160205291A1 (en) | System and Method for Minimizing Motion Artifacts During the Fusion of an Image Bracket Based On Preview Frame Analysis | |
| CN112367474B (en) | Self-adaptive light field imaging method, device and equipment | |
| CN112104869B (en) | Video big data storage and transcoding optimization system | |
| CN108364282B (en) | Image mosaic detection method and image mosaic detection system | |
| CN110443783B (en) | Image quality evaluation method and device | |
| US9165352B2 (en) | Precipitation removal for vision-based parking management systems | |
| US20110085026A1 (en) | Detection method and detection system of moving object | |
| US9094617B2 (en) | Methods and systems for real-time image-capture feedback | |
| US10628681B2 (en) | Method, device, and non-transitory computer readable medium for searching video event | |
| CN113099128B (en) | Video processing method and video processing system | |
| JP5002575B2 (en) | Unsteady degree estimation device, unsteady degree estimation method, unsteady degree estimation program | |
| CN111726543B (en) | Method and camera for improving dynamic range of image | |
| CN112884805A (en) | Cross-scale self-adaptive mapping light field imaging method | |
| Chen et al. | A universal reference-free blurriness measure | |
| Landge et al. | Blur detection methods for digital images-a survey | |
| CN113538297B (en) | Image rain removing method based on gradient priori knowledge and N-S equation | |
| KR101267279B1 (en) | Video processing apparatus and method for removing rain from video | |
| CN114155201A (en) | Method, device and equipment for detecting lens attached with target object and readable storage medium | |
| JP4939860B2 (en) | Method and apparatus for analyzing image data | |
| JP6721022B2 (en) | Image compression device and program | |
| JP5599228B2 (en) | Busy detection system and busy detection program | |
| CN111314611A (en) | Shooting method and device for multiple moving objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |