CN110517215B - Video compression processing method, electronic equipment and storage medium - Google Patents
Video compression processing method, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN110517215B CN110517215B CN201910804189.2A CN201910804189A CN110517215B CN 110517215 B CN110517215 B CN 110517215B CN 201910804189 A CN201910804189 A CN 201910804189A CN 110517215 B CN110517215 B CN 110517215B
- Authority
- CN
- China
- Prior art keywords
- frame image
- frame
- restored
- video
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000006835 compression Effects 0.000 title claims abstract description 30
- 238000007906 compression Methods 0.000 title claims abstract description 30
- 238000003672 processing method Methods 0.000 title claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 77
- 238000000034 method Methods 0.000 claims abstract description 40
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 17
- 238000006748 scratching Methods 0.000 claims description 27
- 230000002393 scratching effect Effects 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 10
- 230000003287 optical effect Effects 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 25
- 230000005540 biological transmission Effects 0.000 abstract description 8
- 244000025254 Cannabis sativa Species 0.000 description 14
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 2
- 238000012015 optical character recognition Methods 0.000 description 2
- 241000722921 Tulipa gesneriana Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000000887 face Anatomy 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention provides a video compression processing method, electronic equipment and a storage medium, wherein the method comprises the following steps: determining frame images contained in the video; identifying background elements and primary elements in the frame image; the main element is related to the content to be expressed by the scene of the video; carrying out cutout processing on background elements in the frame image to obtain a cutout processed frame image; wherein, the frame image after the matting processing contains the main elements; and synthesizing the frame images subjected to the image matting processing to obtain a compressed video. The electronic device and the storage medium perform the above method. The video compression processing method, the electronic device and the storage medium provided by the embodiment of the invention can obviously compress the video, thereby obviously improving the video storage effect and transmission effect.
Description
Technical Field
The present invention relates to the field of video processing technologies, and in particular, to a video compression processing method, an electronic device, and a storage medium.
Background
The video contains huge information amount, and is difficult to store and transmit, so the video needs to be compressed so as to be convenient for storing and transmitting the video.
The main video compression standards in the prior art are h.264 and h.265 standards, and video is compressed integrally through h.264 and h.265 standards, so that the video is convenient to store and transmit. However, in the above processing method for compressing the video as a whole, the compressed video capacity is still large, and the enhancement effect on the aspects of storage and transmission is not obvious enough.
Therefore, how to avoid the above-mentioned defects can significantly compress the video, and further significantly improve the video storage effect and transmission effect, becomes a problem that needs to be solved urgently.
Disclosure of Invention
To solve the problems in the prior art, embodiments of the present invention provide a video compression processing method, an electronic device, and a storage medium.
The embodiment of the invention provides a video compression processing method, which comprises the following steps:
determining frame images contained in the video;
identifying background elements and primary elements in the frame image; the main element is related to the content to be expressed by the scene of the video;
carrying out cutout processing on background elements in the frame image to obtain a cutout processed frame image; wherein, the frame image after the matting processing contains the main elements;
and synthesizing the frame images subjected to the image matting processing to obtain a compressed video.
Wherein, to carry out the cutout processing to the background element in the frame image, obtain the frame image after the cutout processing, specifically include:
determining a sectional shape;
continuously increasing the size ratio from the initial size ratio to scratch out a scratching area corresponding to the size ratio from the background elements, identifying the background elements after each scratching until the background element identification result is lower than a preset threshold value, and taking the frame image subjected to the previous scratching as a frame image subjected to scratching processing;
wherein, the center of the matting region is the center of the background element, and the shape of the matting region is the matting shape.
Wherein, the determining the shape of the sectional drawing specifically comprises:
determining a shape of the background element as a matte shape; alternatively, the matte shapes are randomly generated.
Synthesizing the frame images subjected to the matting processing to obtain a compressed video, wherein the synthesizing comprises the following steps:
determining a fill color value, the fill color value not occupied by the frame image and having a lowest color value;
filling the matting part in the frame image after the matting processing by using the filling color value;
and synthesizing the filled frame images to obtain a compressed video.
Wherein after obtaining the compressed video, the method further comprises:
determining continuous N frames of images corresponding to the same shooting lens, wherein the continuous N frames of images are filled with processed frame images; the same shooting lens is a shooting lens corresponding to the same shooting scene; n is an integer greater than 1;
and restoring each frame of image in the continuous N frames of images, and determining that the difference value between the restored non-first frame of image and the restored previous frame of image is less than a preset difference value.
Wherein, determining that the difference value between the restored non-first frame image and the restored previous frame image is smaller than a preset difference value comprises:
for the second frame of the video stream, the first frame,
if the difference value between the restored second frame image and the restored first frame image is greater than or equal to the preset difference value, the current size proportion of the second frame is continuously reduced so as to scratch the corresponding scratching area of the size proportion from the background element, and the restored second frame image and the restored first frame image are restored after scratching each time until the difference value between the scratched restored second frame image and the restored first frame image is less than the preset difference value.
Wherein, determining that the difference value between the restored non-first frame image and the restored previous frame image is smaller than a preset difference value comprises:
for the ith frame, i is an integer which is greater than or equal to 3 and less than or equal to N;
if the difference value between the restored ith frame image and the restored ith-1 frame image is larger than or equal to a preset difference value, carrying out matting processing on a background element in the ith frame image according to an optical flow algorithm and the ith-1 frame image.
After obtaining the compressed video, the method further includes:
recording a playing time interval corresponding to the same shooting lens and an algorithm used for restoring the image;
and carrying the information for recording the playing time interval and the algorithm in the compressed video.
An embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein,
the processor, when executing the computer program, implements the method steps of:
determining frame images contained in the video;
identifying background elements and primary elements in the frame image; the main element is related to the content to be expressed by the scene of the video;
carrying out cutout processing on background elements in the frame image to obtain a cutout processed frame image; wherein, the frame image after the matting processing contains the main elements;
and synthesizing the frame images subjected to the image matting processing to obtain a compressed video.
An embodiment of the invention provides a non-transitory computer readable storage medium having a computer program stored thereon, which when executed by a processor implements the following method steps:
determining frame images contained in the video;
identifying background elements and primary elements in the frame image; the main element is related to the content to be expressed by the scene of the video;
carrying out cutout processing on background elements in the frame image to obtain a cutout processed frame image; wherein, the frame image after the matting processing contains the main elements;
and synthesizing the frame images subjected to the image matting processing to obtain a compressed video.
According to the video compression processing method, the electronic device and the storage medium provided by the embodiment of the invention, the frame images after the cutout processing are obtained by performing the cutout processing on the background elements in the frame images of the video, and the frame images after the cutout processing are synthesized to obtain the compressed video, so that the video can be remarkably compressed, and the video storage effect and the video transmission effect are further remarkably improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a flow chart of a video compression processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating background elements of an embodiment of the present invention;
FIG. 3 is a schematic diagram of a background element of a grass matting process according to an embodiment of the present invention;
4(a) -4 (f) are schematic diagrams illustrating the process flow of the object matting according to the shape of the background element according to the embodiment of the present invention;
FIGS. 5(a) -5 (c) are schematic diagrams illustrating a process flow of random shape matting according to background elements according to an embodiment of the present invention, respectively;
FIGS. 6(a) -6 (b) are diagrams illustrating the contrast effect between the restored image and the original image according to the embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart of a video compression processing method according to an embodiment of the present invention, and as shown in fig. 1, a video compression processing method according to an embodiment of the present invention includes the following steps:
s101: and determining frame images contained in the video.
Specifically, frame images contained in the video are determined. The method steps can be executed by computer equipment, and for the embodiment of the invention, the method steps can be executed by a video compression server, which is referred to as a server for short hereinafter.
S102: identifying background elements and primary elements in the frame image; the primary element is related to the content to be expressed by the scene of the video.
Specifically, a background element and a main element in the frame image are identified; the primary element is related to the content to be expressed by the scene of the video. It should be noted that: different scenes may correspond to different primary elements, for example a sports game scene corresponds to a primary element that is an athlete; the flower is the main element corresponding to the flower growing learning video scene. Scenes can be determined by video topic names, for example, a video topic name named "how to grow tulip", and a video scene can be determined for flower growing learning. For videos in movie and television shows, main elements can be determined through the existence of people, and people are taken as the main elements in the scenes where people exist generally; for scenes where no human is present, objects in the center region of the image may be determined as the primary elements. The identification of the primary elements of the video in the scene may include:
identifying people appearing in the video:
the method specifically comprises the following steps: and identifying the persons appearing in the video by using AI technology such as a CNN neural network model.
The main people in the video are identified through the specific information of the people, such as lines, faces and the like:
the method specifically comprises the following steps:
a. the lip features are located using an Active Shape Model (ASM) or an Active Appearance Model (AAM). And judging whether the person in the video speaks or not through the change of the lip characteristic points, what the person speaks, and recording the starting/ending time points of the lip action.
b. The content of the station words is recognized by speech Recognition, or Optical Character Recognition ("OCR"). The person who speaks the speech is matched through the speech and the change of the lip characteristic points (lip language recognition). And determining the character with the matched speech lines and lip language recognition results as the main character.
The method can also comprise the following steps:
and (4) establishing a star face recognition feature library, and determining all the recognized stars as main figures.
Display content information can be acquired, when the area of the identified figure is larger than the preset percentage of the picture display area, the figure is determined to be a main figure, and the preset percentage can be set independently according to actual conditions and can be selected to be 30%.
The background elements in the embodiment of the invention are all elements capable of reflecting the background of the whole image. FIG. 2 is a schematic diagram illustrating background elements of an embodiment of the present invention, wherein the background elements include sky, grass, and roads for the type of predetermined scene mainly including characters as shown in FIG. 2; background elements do not include flowers, grass. Background elements in the video may be identified by neural network algorithms.
S103: carrying out cutout processing on background elements in the frame image to obtain a cutout processed frame image; wherein the frame image after the matting processing contains the main elements.
Specifically, performing cutout processing on a background element in the frame image to obtain a cutout-processed frame image; wherein the frame image after the matting processing contains the main elements. In order to obtain a frame image of a video, a frame splitting process may be performed on the video, specifically as follows:
performing frame splitting processing on a video to be processed by using the ffmpeg, and recording a frame rate of the video before frame splitting;
and extracting the audio in the video by using ffmpeg, and recording the code rate of the audio before the extraction of the video. ffmpeg is a set of open source computer programs that can be used to record, convert digital audio, video, and convert them into streams.
The background elements may be classified according to the recognition result of the background elements, and referring to fig. 2, may be classified as sky, grassland, and road. Taking a grassland as an example to explain the matting process, fig. 3 is a schematic diagram of matting process performed on a background element as a grassland according to an embodiment of the present invention, as shown in fig. 3, a black part in fig. 3 is a partial image to be scratched out, and a complete main element in the scene is reserved, that is, a frame image containing the main element after the matting process is obtained, where the main element in fig. 3 is a person. For the partial images which are removed by scratching and correspond to the sky and the road respectively, the description is not repeated.
It should be noted that, in this step, matting processing is performed on the background elements by performing matting attempt on the background elements, and then, on the premise that the recognition result of the background elements reaches a certain accuracy rate, matting processing is performed on each type of background elements in each frame of image, so as to ensure the rationality of matting partial image corresponding to each type of background elements in each frame of image.
S104: and synthesizing the frame images subjected to the image matting processing to obtain a compressed video.
Specifically, the frame images after the matting processing are synthesized to obtain a compressed video. Referring to fig. 3, the synthesizing of the frame image may specifically include: using ffmpeg to synthesize the frame images subjected to image matting processing into a video according to the original frame rate and sequence; and (5) using ffmpeg to synthesize the frame images subjected to the image matting processing into audio according to the original code rate and sequence. After the composition is completed, the compressed video may be transmitted to the terminal for subsequent processing by the terminal.
The method of the embodiment of the invention has higher compression ratio than the prior compression technology, and can be used in an overlapping way on the basis of the prior compression technology.
According to the video compression processing method provided by the embodiment of the invention, the frame images after the cutout processing are obtained by performing the cutout processing on the background elements in the frame images of the video, and the frame images after the cutout processing are synthesized to obtain the compressed video, so that the video can be obviously compressed, and the video storage effect and the video transmission effect are obviously improved.
On the basis of the above embodiment, the performing matting processing on the background element in the frame image to obtain the frame image after the matting processing specifically includes:
determining the shape of the sectional drawing.
Specifically, the shape of the cutout is determined. The shape of the cutout can be determined according to the shape of the background element in the frame image; alternatively, the matte shapes are randomly generated. Fig. 4(a) -4 (f) are schematic diagrams of the process flow of the matting according to the shape of the background element object according to the embodiment of the present invention, respectively, and the matting shape determined according to the shape of the grass is as shown in fig. 4 (a).
Continuously increasing the size ratio from the initial size ratio to scratch out a scratching area corresponding to the size ratio from the background elements, identifying the background elements after each scratching until the background element identification result is lower than a preset threshold value, and taking the frame image subjected to the previous scratching as a frame image subjected to scratching processing; wherein, the center of the matting region is the center of the background element, and the shape of the matting region is the matting shape.
Specifically, the size ratio is continuously increased from the initial size ratio, so that a matting region corresponding to the size ratio is scratched from the background elements, the background elements are identified after each scratching, and when the identification result of the background elements is lower than a preset threshold value, the frame image subjected to the scratching at the previous time is used as the frame image subjected to the scratching processing; wherein, the center of the matting region is the center of the background element, and the shape of the matting region is the matting shape. The center point of the grass was taken as the center of the matting area, as shown by the black dot in fig. 4 (b). For images that have not been matted, the recognition of grass is 99.7%, as shown in FIG. 4 (c). The initial size proportion can be set independently according to actual conditions, and can be selected to be 5%, namely the ratio of the initially set sectional area to the total area of the background elements.
The preset threshold value can be set independently according to actual conditions, and can be selected to be 65%. As shown in fig. 4(d), the matting area corresponding to the first matting size ratio is shown as a black part in fig. 4(d), after the first matting, the grass is identified, and at this time, the identification result of the grass is 90% and is higher than 65%; as shown in fig. 4(e), increase the size proportion again, can increase the size proportion according to increasing gradually the predetermined proportion numerical value in proper order, predetermined proportion numerical value can independently set up according to actual conditions, for example, the second time size proportion is 35% (compare first time size proportion 5%, it has increased 30%), the area of scratching that the size proportion corresponds for the second time is shown as the black part in fig. 4(e), after scratching for the second time, the discernment meadow, be 70% to the meadow discernment result this moment, be higher than 65%; as shown in fig. 4(f), the size ratio is continuously increased, for example, the third size ratio is 65% (compared with the second size ratio which is 35%, the size ratio is increased by 30%), the matting area corresponding to the third matting size ratio is as shown by the black part in fig. 4(f), after the third matting, the grass is identified, and at this time, the identification result of the grass is 45% and is lower than 65%, as shown in fig. 4 (f). Therefore, the grass is subjected to matting according to the secondary size proportion of 35%.
The video compression processing method provided by the embodiment of the invention can effectively and reasonably obtain the frame image after the cutout processing, thereby obviously compressing the video and further obviously improving the video storage effect and the video transmission effect.
On the basis of the above embodiment, the determining the shape of the cutout specifically includes:
determining a shape of the background element as a matte shape; alternatively, the matte shapes are randomly generated.
Specifically, the shape of the background element is determined as the scratch shape, which can be referred to the above description, and for the randomly generated scratch shape, a grass field is taken as an example of a certain type of background element a, and the randomly generated scratch shape has a certain area and can be selected as an ellipse. Fig. 5(a) -5 (c) are schematic diagrams respectively illustrating a process flow of random shape matting according to background elements in an embodiment of the present invention.
Referring to the above example, the recognition result of the grass after the first matting is 99.7%, which is higher than 65%, the size ratio is increased, the size ratio can be increased according to the sequentially increasing preset ratio value, the preset ratio value can be set independently according to the actual situation, for example, the second size ratio is 35%, the recognition result is 90%, which is higher than 65%, as shown in fig. 5(b), and the size ratio is continuously increased, for example, the third size ratio is 65%, the recognition result is 50%, which is lower than 65%, as shown in fig. 5 (c). Therefore, the grass is subjected to matting processing according to the secondary size proportion of 35%, and for specific description, reference may be made to the above description of determining the matting shape according to the shape of the background element, which is not described again.
The video compression processing method provided by the embodiment of the invention can effectively and reasonably obtain the frame image after the cutout processing, thereby obviously compressing the video and further obviously improving the video storage effect and the video transmission effect.
On the basis of the above embodiment, the synthesizing the frame images after the matting processing to obtain a compressed video includes:
determining a fill color value that is not occupied by the frame image and has a lowest color value.
Specifically, a fill color value is determined, which is not occupied by the frame image and has the lowest color value. The color value with the lowest color value can be a color value corresponding to black, if the partial image except the partial image to be scratched (corresponding to the scratching part) in the current frame image contains a corresponding color value of black, the color value of the partial image to be scratched is determined to be a first target color value, and the first target color value is a non-black color value within a preset range corresponding to the color value of black; the preset range can be set autonomously according to the actual situation, the non-black color value is not a non-black color value which can be recognized by human eyes, and can be a non-black color value which can be recognized by a server, for example, the black corresponding color value is #000000, and for the color value #000010, the non-black color value belongs to the preset range, although the color value #000010 looks black for human eyes. That is, if the partial image other than the scratched-out partial image contains #000000, the color value of the scratched-out partial image is determined to be a color value close to black that can be recognized by the server.
If the partial image except the partial image to be scratched out in the current frame image does not contain the corresponding color value of black, determining the color value of the scratched-out partial image as a second target color value; the second target color value is a black corresponding color value. That is, if the partial image other than the scratched-out partial image does not contain #000000, the color value of the scratched-out partial image is determined to be # 000000.
And filling the matting part in the frame image after the matting processing by using the filling color value.
Specifically, the matting part in the frame image after the matting processing is filled with the filling color value. The color value processing strategy is adopted to fill the scratched partial image, so that the optimization of the image volume can be ensured, and further, the video compression is realized to a greater extent.
And synthesizing the filled frame images to obtain a compressed video.
Specifically, the filled frame images are synthesized to obtain a compressed video. The synthesis process can refer to the above description, and is not repeated. It should be noted that: to further compress the image volume, images are not saved in png, et al, transparent channel-bearing image formats.
According to the video compression processing method provided by the embodiment of the invention, the color values are filled in the matting part, so that the video can be further obviously compressed, and the video storage effect and the video transmission effect are further obviously improved.
On the basis of the above embodiment, after obtaining the compressed video, the method further includes:
determining continuous N frames of images corresponding to the same shooting lens, wherein the continuous N frames of images are filled with processed frame images; the same shooting lens is a shooting lens corresponding to the same shooting scene; n is an integer greater than 1.
Specifically, determining continuous N frames of images corresponding to the same shooting lens, wherein the continuous N frames of images are filled with processed frame images; the same shooting lens is a shooting lens corresponding to the same shooting scene; n is an integer greater than 1. For example, the live video clip of the hectometer running includes the athlete entry introduction, the competition stage and the after-competition interview in sequence, the athlete entry introduction, the competition stage and the after-competition interview respectively correspond to three same shooting shots, the number of the image frames corresponding to the athlete entry introduction can be 1-N frames, all background elements in the 1-N frames of images are subjected to image matting processing, and the image frames are filled with the processed frame images.
And restoring each frame of image in the continuous N frames of images, and determining that the difference value between the restored non-first frame of image and the restored previous frame of image is less than a preset difference value.
Specifically, each frame of image in the continuous N frames of images is restored, and it is determined that a difference value between the restored non-first frame of image and the restored previous frame of image is smaller than a preset difference value. The concrete description is as follows:
the method comprises the steps of restoring a first frame by using a restored image algorithm (such as a GAN network), restoring a second frame by using the GAN network, carrying out difference comparison on a restored second frame image and a restored first frame image, if the difference value between the restored second frame image and the restored first frame image is less than M% (M% corresponds to a preset difference value, and can be set autonomously according to actual conditions), continuing to restore a third frame by using the GAN network, carrying out difference comparison on a restored third frame image and a restored second frame image corresponding to the difference value of less than M%, if the difference value between the restored third frame image and the restored second frame image is less than M%, continuing to restore a fourth frame by using the GAN network, carrying out difference comparison on a restored fourth frame image and a restored third frame image corresponding to the difference value of less than M%, and so on, if all difference values of the times are less than M%, determining to verify continuous N frame images corresponding to the same shooting lens, and after the continuous N frames of images corresponding to the same shooting lens are confirmed to pass the verification, sending the continuous N frames of images corresponding to the same shooting lens passing the verification to the terminal so that the terminal can restore the continuous N frames of images corresponding to the same shooting lens. The server verifies that the continuous N frames of images corresponding to the same shooting lens pass, so that the terminal can restore the original image more vividly when restoring the continuous N frames of images corresponding to the same shooting lens.
The video compression processing method provided by the embodiment of the invention verifies the continuous N frames of images corresponding to the same shooting lens through corresponding steps, thereby effectively ensuring the verification effect.
On the basis of the foregoing embodiment, the determining that a difference value between the restored non-first-frame image and the restored previous-frame image is smaller than a preset difference value includes:
for the second frame of the video stream, the first frame,
if the difference value between the restored second frame image and the restored first frame image is greater than or equal to the preset difference value, the current size proportion of the second frame is continuously reduced so as to scratch the corresponding scratching area of the size proportion from the background element, and the restored second frame image and the restored first frame image are restored after scratching each time until the difference value between the scratched restored second frame image and the restored first frame image is less than the preset difference value.
Specifically, if the difference value between the restored second frame image and the restored first frame image is greater than or equal to the preset difference value, the current size ratio of the second frame is continuously reduced, so that the corresponding scratching region of the size ratio in the background element is scratched, and the restored second frame image and the restored first frame image are restored after scratching each time until the difference value between the scratched restored second frame image and the restored first frame image is less than the preset difference value. The restored second frame image and the restored first frame image are subjected to difference comparison, if the difference value between the restored second frame image and the restored first frame image is larger than or equal to M%, the corresponding size proportion of the grassland, the sky and the road can be reduced in the matting processing of the second frame image, and the method can be realized by reducing the preset proportion value or the initial size proportion. If the preset proportion value or the initial size proportion is reduced to perform matting processing on each type of background elements in the second frame image, the situation that the difference value between the restored second frame image and the restored first frame image is larger than or equal to the preset difference value still exists, and the execution can be repeated until the difference value between the restored second frame image and the restored first frame image is smaller than the preset difference value.
The video compression processing method provided by the embodiment of the invention ensures the verification effect by adopting a reasonable processing strategy for processing, not only can obviously compress the video, but also can ensure the image effect of the terminal on the restoration of the synthesized video.
On the basis of the above embodiment, determining that a difference value between the restored non-first frame image and the restored previous frame image is smaller than a preset difference value includes:
for the ith frame, i is an integer which is greater than or equal to 3 and less than or equal to N;
if the difference value between the restored ith frame image and the restored ith-1 frame image is larger than or equal to a preset difference value, carrying out matting processing on a background element in the ith frame image according to an optical flow algorithm and the ith-1 frame image.
Specifically, if the difference value between the restored ith frame image and the restored ith-1 frame image is greater than or equal to a preset difference value, matting processing is performed on background elements in the ith frame image according to an optical flow algorithm and the ith-1 frame image. Taking i ═ 3 as an example, referring to the above description, if the difference value between the restored third frame image and the restored second frame image is greater than or equal to the preset difference value, performing matting processing on each type of background elements in the restored third frame image according to the optical flow algorithm and the restored second frame image, and referring to the description of the case where i is 3 for the case where the value of i is other than 3, which is not described again. The method of performing matting processing on each type of background elements by adopting an optical flow algorithm is a mature technology in the field and is not described in detail.
The video compression processing method provided by the embodiment of the invention ensures the verification effect by adopting a reasonable processing strategy for processing, not only can obviously compress the video, but also can ensure the image effect of the terminal on the restoration of the synthesized video.
On the basis of the above embodiment, after obtaining the compressed video, the method further includes:
and recording the playing time interval corresponding to the same shooting lens and an algorithm used for restoring the image.
Specifically, the playing time interval corresponding to the same shooting lens and the algorithm used for restoring the image are recorded and recorded. Referring to the above example, the algorithm used to restore the image may be a GAN network.
And carrying the information for recording the playing time interval and the algorithm in the compressed video.
Specifically, the information recording the playing time interval and the algorithm is carried in the compressed video. The compressed video carrying the information can be sent to the terminal, so that the terminal can restore the frame image of the playing time period by adopting the algorithm in the information. Fig. 6(a) to 6(b) are respectively a comparison effect graph of the image restored by the embodiment of the present invention and the original image, fig. 6(a) is the image effect graph after restoration, fig. 6(b) is the original image effect graph, and comparing fig. 6(a) and 6(b), it can be seen that: the restored image is very close to the original image, and the overall effect of the image is not affected except that the details of the selected part are lost as shown in fig. 6 (b).
According to the video compression processing method provided by the embodiment of the invention, the compressed video can be conveniently restored by the terminal by carrying the corresponding information in the compressed video.
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 7, the electronic device includes: a processor (processor)701, a memory (memory)702, and a bus 703;
the processor 701 and the memory 702 complete mutual communication through a bus 703;
the processor 701 is configured to call the program instructions in the memory 702 to execute the methods provided by the above-mentioned method embodiments, for example, including: determining frame images contained in the video; identifying background elements and primary elements in the frame image; the main element is related to the content to be expressed by the scene of the video; carrying out cutout processing on background elements in the frame image to obtain a cutout processed frame image; wherein, the frame image after the matting processing contains the main elements; and synthesizing the frame images subjected to the image matting processing to obtain a compressed video.
The present embodiment discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method provided by the above-mentioned method embodiments, for example, comprising: determining frame images contained in the video; identifying background elements and primary elements in the frame image; the main element is related to the content to be expressed by the scene of the video; carrying out cutout processing on background elements in the frame image to obtain a cutout processed frame image; wherein, the frame image after the matting processing contains the main elements; and synthesizing the frame images subjected to the image matting processing to obtain a compressed video.
The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the methods provided by the above method embodiments, for example, including: determining frame images contained in the video; identifying background elements and primary elements in the frame image; the main element is related to the content to be expressed by the scene of the video; carrying out cutout processing on background elements in the frame image to obtain a cutout processed frame image; wherein, the frame image after the matting processing contains the main elements; and synthesizing the frame images subjected to the image matting processing to obtain a compressed video.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
The above-described embodiments are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
1. A video compression processing method, comprising:
determining frame images contained in the video;
identifying background elements and primary elements in the frame image; the main element is related to the content to be expressed by the scene of the video;
carrying out cutout processing on background elements in the frame image to obtain a cutout processed frame image; wherein, the frame image after the matting processing contains the main elements;
synthesizing the frame images subjected to the matting processing to obtain a compressed video;
after obtaining the compressed video, the method further comprises:
determining continuous N frames of images corresponding to the same shooting lens, wherein the continuous N frames of images are filled with processed frame images; the same shooting lens is a shooting lens corresponding to the same shooting scene; n is an integer greater than 1; restoring each frame of image in the continuous N frames of images, and determining that the difference value between the restored non-first frame of image and the restored previous frame of image is smaller than a preset difference value;
the determining that the difference value between the restored non-first frame image and the restored previous frame image is smaller than the preset difference value includes: for the second frame, if the difference value between the restored second frame image and the restored first frame image is greater than or equal to the preset difference value, the current size proportion of the second frame is continuously reduced, so that the scratching area corresponding to the size proportion is scratched from the background element, and the restored second frame image and the restored first frame image are restored after scratching each time until the difference value between the scratched restored second frame image and the restored first frame image is less than the preset difference value.
2. The video compression processing method according to claim 1, wherein the matting processing is performed on the background element in the frame image to obtain the frame image after the matting processing, specifically including:
determining a sectional shape;
continuously increasing the size ratio from the initial size ratio to scratch out a scratching area corresponding to the size ratio from the background elements, identifying the background elements after each scratching until the background element identification result is lower than a preset threshold value, and taking the frame image subjected to the previous scratching as a frame image subjected to scratching processing;
wherein, the center of the matting region is the center of the background element, and the shape of the matting region is the matting shape.
3. The video compression processing method according to claim 2, wherein the determining the shape of the scratch specifically includes:
determining a shape of the background element as a matte shape; alternatively, the matte shapes are randomly generated.
4. The video compression processing method according to any one of claims 1 to 3, wherein the synthesizing the frame images after the matting processing to obtain the compressed video includes:
determining a fill color value, the fill color value not occupied by the frame image and having a lowest color value;
filling the matting part in the frame image after the matting processing by using the filling color value;
and synthesizing the filled frame images to obtain a compressed video.
5. The method of claim 1, wherein determining that a difference between the restored non-first frame image and the restored previous frame image is smaller than a preset difference comprises:
for the ith frame, i is an integer which is greater than or equal to 3 and less than or equal to N;
if the difference value between the restored ith frame image and the restored ith-1 frame image is larger than or equal to a preset difference value, carrying out matting processing on a background element in the ith frame image according to an optical flow algorithm and the ith-1 frame image.
6. The video compression processing method according to claim 1, further comprising, after obtaining the compressed video:
recording a playing time interval corresponding to the same shooting lens and an algorithm used for restoring the image;
and carrying the information for recording the playing time interval and the algorithm in the compressed video.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method according to any of claims 1 to 6 are implemented when the computer program is executed by the processor.
8. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910804189.2A CN110517215B (en) | 2019-08-28 | 2019-08-28 | Video compression processing method, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910804189.2A CN110517215B (en) | 2019-08-28 | 2019-08-28 | Video compression processing method, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110517215A CN110517215A (en) | 2019-11-29 |
| CN110517215B true CN110517215B (en) | 2022-03-25 |
Family
ID=68627701
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910804189.2A Active CN110517215B (en) | 2019-08-28 | 2019-08-28 | Video compression processing method, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110517215B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111294600A (en) * | 2020-03-17 | 2020-06-16 | 广州致远电子有限公司 | Compression method, decompression method and device for video sequence frame |
| CN113610865B (en) * | 2021-07-27 | 2024-03-29 | Oppo广东移动通信有限公司 | Image processing method, device, electronic equipment and computer readable storage medium |
| CN117576247B (en) * | 2024-01-17 | 2024-03-29 | 江西拓世智能科技股份有限公司 | Picture generation method and system based on artificial intelligence |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101324963A (en) * | 2008-07-24 | 2008-12-17 | 上海交通大学 | Fluid video synthetic method based on static image |
| CN106210612A (en) * | 2015-04-30 | 2016-12-07 | 杭州海康威视数字技术股份有限公司 | Method for video coding, coding/decoding method and device thereof |
| CN106204567A (en) * | 2016-07-05 | 2016-12-07 | 华南理工大学 | A kind of natural background video matting method |
| CN108965746A (en) * | 2018-07-26 | 2018-12-07 | 北京竞业达数码科技股份有限公司 | Image synthesizing method and system |
| CN109544587A (en) * | 2018-10-31 | 2019-03-29 | 深圳变设龙信息科技有限公司 | A kind of FIG pull handle method, apparatus and terminal device |
-
2019
- 2019-08-28 CN CN201910804189.2A patent/CN110517215B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101324963A (en) * | 2008-07-24 | 2008-12-17 | 上海交通大学 | Fluid video synthetic method based on static image |
| CN106210612A (en) * | 2015-04-30 | 2016-12-07 | 杭州海康威视数字技术股份有限公司 | Method for video coding, coding/decoding method and device thereof |
| CN106204567A (en) * | 2016-07-05 | 2016-12-07 | 华南理工大学 | A kind of natural background video matting method |
| CN108965746A (en) * | 2018-07-26 | 2018-12-07 | 北京竞业达数码科技股份有限公司 | Image synthesizing method and system |
| CN109544587A (en) * | 2018-10-31 | 2019-03-29 | 深圳变设龙信息科技有限公司 | A kind of FIG pull handle method, apparatus and terminal device |
Non-Patent Citations (1)
| Title |
|---|
| 保持时间连贯性的视频抠图与合成;李亚星;《中国优秀硕士学位论文全文数据库》;20170531;I138-1115 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110517215A (en) | 2019-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110517215B (en) | Video compression processing method, electronic equipment and storage medium | |
| CN109145784B (en) | Method and apparatus for processing video | |
| US20170163953A1 (en) | Method and electronic device for processing image containing human face | |
| CN112312231B (en) | Video image coding method and device, electronic equipment and medium | |
| CN110460899B (en) | Bullet screen content display method, terminal equipment and computer readable storage medium | |
| CN111985281B (en) | Image generation model generation method and device and image generation method and device | |
| CN110751649A (en) | Video quality evaluation method and device, electronic equipment and storage medium | |
| WO2022252372A1 (en) | Image processing method, apparatus and device, and computer-readable storage medium | |
| CN113628100B (en) | Video enhancement method, device, terminal and storage medium | |
| CN113068034B (en) | Video encoding method and device, encoder, equipment and storage medium | |
| CN112818737A (en) | Video identification method and device, storage medium and terminal | |
| CN115171199B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| KR20240089729A (en) | Image processing methods, devices, storage media and electronic devices | |
| CN115297288A (en) | Monitoring data storage method for driving simulator | |
| CN117440114A (en) | Virtual image video generation method, device, equipment and medium | |
| CN113949899B (en) | Video quality assessment method and device | |
| CN114630139A (en) | Quality evaluation method of live video and related equipment thereof | |
| US10154292B2 (en) | Information pushing method and system, cloud server and information server | |
| CN112533024A (en) | Face video processing method and device and storage medium | |
| CN112860941A (en) | Cover recommendation method, device, equipment and medium | |
| CN117714712B (en) | Data steganography method, equipment and storage medium for video conference | |
| CN115942046B (en) | Method for intelligently cutting video and storage medium | |
| CN117676136B (en) | Method and system for processing group-connected data | |
| US20240087185A1 (en) | Virtual view drawing method, rendering method, and decoding method | |
| CN113256501B (en) | Image processing method, storage medium and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |