CN114841868A - Video high-quality restoration method and system for live webcast big data - Google Patents
Video high-quality restoration method and system for live webcast big data Download PDFInfo
- Publication number
- CN114841868A CN114841868A CN202210311319.0A CN202210311319A CN114841868A CN 114841868 A CN114841868 A CN 114841868A CN 202210311319 A CN202210311319 A CN 202210311319A CN 114841868 A CN114841868 A CN 114841868A
- Authority
- CN
- China
- Prior art keywords
- image
- restoration
- key
- repairing
- key frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 129
- 238000001514 detection method Methods 0.000 claims abstract description 75
- 238000013135 deep learning Methods 0.000 claims abstract description 32
- 238000004590 computer program Methods 0.000 claims description 5
- 238000011410 subtraction method Methods 0.000 claims description 4
- 230000007547 defect Effects 0.000 claims description 3
- 230000008439 repair process Effects 0.000 description 28
- 238000005516 engineering process Methods 0.000 description 11
- 238000013527 convolutional neural network Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a video high-quality restoration method and system for live webcast big data, and relates to the technical field of data restoration. The method comprises the following steps: acquiring and adopting a key frame detection method to detect and extract a key frame image in target live video data; repairing the key frame image by adopting an image repairing method based on deep learning; calculating and determining an image to be restored according to the difference between each frame image in the non-key frame images and the images of the frames before and after the non-key frame images; detecting an image to be repaired; repairing the image to be repaired by adopting an image repairing method or a basic image repairing method based on deep learning according to the detection result; and obtaining a complete target restoration image based on the key restoration image and the non-key restoration image to finish video restoration. The invention combines the common restoration method and the restoration method based on deep learning to carry out targeted restoration on different images, thereby greatly improving the quality of video image restoration and saving the resource consumption.
Description
Technical Field
The invention relates to the technical field of data restoration, in particular to a video high-quality restoration method and system for live broadcast big data.
Background
With the vigorous development of the internet, the network video live broadcast is more and more generalized and becomes an important part in the life of people. The network video live broadcast not only can provide more entertainment options for the people, but also can provide more consumption options for the people. However, in the process of live video, a large amount of live data often has a situation of partial deletion, which greatly affects the watching effect of live video.
In view of the above problems, although some video repair schemes are proposed in the prior art, the existing video repair method has certain limitations, and particularly, a high-quality repair result cannot be obtained in a high proportion for massive live video data. Therefore, how to perform high-quality repair on massive live video data is a problem to be solved urgently.
Disclosure of Invention
In order to overcome the above problems or at least partially solve the above problems, embodiments of the present invention provide a method and a system for high-quality video repair for webcast big data, which combine a common repair method and a repair method based on deep learning to perform targeted repair on different images, thereby greatly improving the quality of video image repair and saving resource consumption.
The embodiment of the invention is realized by the following steps:
in a first aspect, an embodiment of the present invention provides a network live broadcast big data-oriented video high-quality repair method, including the following steps:
acquiring target live video data, and detecting and extracting a key frame image in the target live video data by adopting a key frame detection method;
repairing the key frame image by adopting an image repairing method based on deep learning to obtain a key repairing image;
calculating and determining an image to be restored according to the difference between each frame image in non-key frame images in target live broadcast video data and the previous and next frame images;
detecting an image to be repaired to generate a detection result;
repairing the image to be repaired by adopting an image repairing method or a basic image repairing method based on deep learning according to the detection result to obtain a non-key repaired image;
and obtaining a complete target restoration image based on the key restoration image and the non-key restoration image to finish video restoration.
In order to solve the technical problem that precise, efficient and high-quality restoration cannot be performed on massive live video data in the prior art, the method combines a common restoration method and a restoration method based on deep learning to perform targeted restoration on different images, utilizes a complex method for large-area missing images and a common method for non-large-area missing images, avoids resource consumption caused by the complex method for all images, and greatly saves resource consumption; and the video image restoration quality is greatly improved by using a key frame detection technology and an image detection technology to be restored.
Based on the first aspect, in some embodiments of the present invention, the method for calculating the difference between each frame image in the non-key frame images in the target live video data and the previous and next frame images includes the following steps:
and calculating the difference between each frame image of the non-key frame images in the target live broadcast video data and the frame images before and after the non-key frame images by adopting a difference subtraction method so as to determine the difference between each frame image in the non-key frame images and the frame images before and after the non-key frame images.
Based on the first aspect, in some embodiments of the present invention, the method for determining an image to be restored according to a difference between each frame image in non-key frame images in target live video data and a frame image before and after the frame image includes the following steps:
judging whether the difference value between each frame image in the non-key frame images and the previous and next frame images is greater than a preset reference threshold value or not, if so, determining that the frame image has a defect, and identifying the frame image as an image to be repaired; if not, the frame image is determined to be absent.
Based on the first aspect, in some embodiments of the present invention, the method for detecting an image to be repaired and generating a detection result includes the following steps:
detecting and judging whether the local area pixel value in the image to be repaired is larger than a preset area pixel threshold value according to the local area pixel value of the image to be repaired, and if so, generating an area large-area missing detection result; and if not, generating a basic missing detection result.
Based on the first aspect, in some embodiments of the present invention, the method for repairing an image to be repaired by using an image repairing method based on deep learning or a basic image repairing method according to a detection result includes the following steps:
repairing the image to be repaired by adopting an image repairing method based on deep learning according to the large-area missing detection result of the region;
and repairing the image to be repaired by adopting a basic image repairing method according to the basic missing detection result.
Based on the first aspect, in some embodiments of the present invention, the Image restoration method based on deep learning includes a Pluralistic Image restoration method and an Image restoration method based on a CNN network structure.
Based on the first aspect, in some embodiments of the present invention, the basic image inpainting method includes a patch match algorithm and a Context Decoder algorithm.
In a second aspect, an embodiment of the present invention provides a network live broadcast big data oriented video high quality repair system, including a key frame detection module, a key frame repair module, a difference detection module, a repair detection module, a non-key repair module, and a complete repair module, where:
the key frame detection module is used for acquiring target live video data, and detecting and extracting key frame images in the target live video data by adopting a key frame detection method;
the key frame restoration module is used for restoring the key frame image by adopting an image restoration method based on deep learning to obtain a key restoration image;
the difference detection module is used for calculating and determining an image to be restored according to the difference between each frame image in the non-key frame images in the target live broadcast video data and the previous and next frame images;
the restoration detection module is used for detecting the image to be restored to generate a detection result;
the non-key restoration module is used for restoring the image to be restored by adopting an image restoration method or a basic image restoration method based on deep learning according to the detection result so as to obtain a non-key restoration image;
and the completion restoration module is used for obtaining a complete target restoration image based on the key restoration image and the non-key restoration image and completing video restoration.
In order to solve the technical problem that accurate, efficient and high-quality restoration cannot be performed on massive live video data in the prior art, the system combines a common restoration method and a restoration method based on deep learning based on the mutual matching of a key frame detection module, a key frame restoration module, a difference detection module, a restoration detection module, a non-key restoration module and a completion restoration module and the like, performs targeted restoration on different images, utilizes a complex method for large-area missing images and a common method for non-large-area missing images, avoids resource consumption caused by the complex method for all images, and greatly saves resource consumption; and the video image restoration quality is greatly improved by using a key frame detection technology and an image detection technology to be restored.
In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory for storing one or more programs; a processor. The program or programs, when executed by a processor, implement the method of any of the first aspects as described above.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the method according to any one of the first aspect described above.
The embodiment of the invention at least has the following advantages or beneficial effects:
the embodiment of the invention provides a video high-quality restoration method and system for network live broadcast big data, which solve the technical problem that precise, efficient and high-quality restoration cannot be performed on massive live broadcast video data in the prior art, and the method combines a common restoration method and a restoration method based on deep learning to perform targeted restoration on different images, and avoids resource consumption caused by a complex method for large-area missing images and a common method for non-large-area missing images, thereby greatly saving the resource consumption; and the video image restoration quality is greatly improved by using a key frame detection technology and an image detection technology to be restored.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a flowchart of a video high-quality repair method for webcast big data according to an embodiment of the present invention;
fig. 2 is a flowchart of calculating differences of images of frames in a video high-quality restoration method for live webcast big data according to an embodiment of the present invention;
fig. 3 is a flowchart of determining an image to be restored in a live webcast big data-oriented video high-quality restoration method according to an embodiment of the present invention;
fig. 4 is a schematic block diagram of a video high-quality repair system for webcast big data according to an embodiment of the present invention;
fig. 5 is a block diagram of an electronic device according to an embodiment of the present invention.
Icon: 100. a key frame detection module; 200. a key frame repair module; 300. a difference detection module; 400. a repair detection module; 500. a non-critical repair module; 600. completing the repair module; 101. a memory; 102. a processor; 103. a communication interface.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Examples
As shown in fig. 1 to fig. 3, in a first aspect, an embodiment of the present invention provides a method for repairing a video with high quality facing to live webcast big data, including the following steps:
s1, acquiring target live video data, and detecting and extracting a key frame image in the target live video data by adopting a key frame detection method;
in some embodiments of the present invention, the method for detecting and extracting a key frame image in target live video data by using the key frame detection method may use LDA as a label, then use a dual-flow convolutional network to fit LDA, and use LDA to distinguish key frames, so as to extract and obtain corresponding key frame images. And detecting the key frames in the video by using a key frame detection technology for any section of video in the mass video data.
S2, repairing the key frame image by adopting an image repairing method based on deep learning to obtain a key repairing image; the Image restoration method based on deep learning comprises a Pluralistic Image restoration method and an Image restoration method based on a CNN network structure. Other image restoration methods based on deep learning, such as image restoration based on RNN, may also be used in the present invention, which is not described herein again. And performing a corresponding image restoration method based on deep learning on all the key frame images. Since the key frames are usually more important images, all key frame images are repaired, and even if no missing image exists, the key frame images are optimized by using a repair algorithm to obtain a better video image. Convolutional Neural Networks (CNN) are the earliest two-dimensional image recognition sensors, and a supervised learning manner is adopted, each layer is a two-dimensional plane composed of a plurality of independent neurons, such as a classical LeNet-5 structure, data sequentially passes through an input layer, a Convolutional layer 1, a pooling layer 1, a Convolutional layer 2, a pooling layer 2, a full-link layer 1, an activation function layer, a full-link layer 2 and an output layer, and the structure can extract features layer by layer, and finally, the full-link layer is used for completing a plurality of classification tasks. The weights from part of neurons in the same layer of the convolutional neural network to the next layer are set to be the same, namely the weights are shared, the complexity of a model can be reduced when data are trained, and the huge image restoration problem is subjected to dimension reduction processing. The image restoration is carried out based on the CNN, the image processing of big data can be met, and the restoration efficiency is improved. The RNN-based network structure can correspond the time sequence to the pixel distribution of the image, the pixel information can be predicted according to two spatial dimensions when the missing region is completed, and the prediction of the next pixel point is based on the previously generated pixel point, so that the image repairing effect is achieved.
S3, calculating and determining an image to be repaired according to the difference between each frame image in the non-key frame images in the target live broadcast video data and the previous and next frame images;
further, S31, calculating the difference between each frame image of the non-key frame image in the target live broadcast video data and the previous and next frame images thereof by using a difference subtraction method, so as to determine the difference between each frame image in the non-key frame image and the previous and next frame images thereof. S32, judging whether the difference value between each frame image in the non-key frame images and the previous and next frame images is larger than a preset reference threshold value, if so, determining that the frame image has a defect, and regarding the frame image as an image to be repaired; if not, the frame image is determined to be absent.
In some embodiments of the present invention, for non-key frame images, the difference between each frame and its previous and subsequent frames (if the previous and subsequent frames are not considered for key frame images) is detected by using a difference subtraction method. If the difference value between a certain frame image in the non-key frame images and the previous and next frame images is larger than a preset reference threshold value, the occurrence of very obvious difference is proved, namely the frame image is proved to have deficiency, and the frame image is regarded as an image to be repaired. And performing difference judgment and comparison on each frame image in the non-key frame images to determine a final complete image to be repaired.
S4, detecting the image to be repaired to generate a detection result;
further, detecting and judging whether the pixel value of the local area in the image to be repaired is larger than a preset area pixel threshold value according to the pixel value of the local area in the image to be repaired, and if so, generating an area large-area missing detection result; and if not, generating a basic missing detection result.
S5, repairing the image to be repaired by adopting an image repairing method or a basic image repairing method based on deep learning according to the detection result to obtain a non-key repaired image;
further, repairing the image to be repaired by adopting an image repairing method based on deep learning according to the large-area missing detection result of the region; and repairing the image to be repaired by adopting a basic image repairing method according to the basic missing detection result. The basic image restoration method comprises a PatchMatch algorithm and a Context Decoder algorithm.
In some embodiments of the present invention, an image to be repaired is detected, and if a situation that pixel values of local regions are highly similar occurs in the image to be repaired, the image to be repaired is directly determined as a large-area region missing, and is repaired by using a deep learning-based repair method. If the situation that the pixel values of the local areas are highly similar does not occur in the image to be repaired, the image is determined not to have large-area loss, and the image is repaired by using a common repairing method. Different repairing methods are adopted according to different deletion degrees, so that repairing resources are greatly saved, and repairing efficiency is improved.
And S6, obtaining a complete target restoration image based on the key restoration image and the non-key restoration image, and completing video restoration.
In order to solve the technical problem that precise, efficient and high-quality restoration cannot be performed on massive live video data in the prior art, the method combines a common restoration method and a restoration method based on deep learning to perform targeted restoration on different images, utilizes a complex method for large-area missing images and a common method for non-large-area missing images, avoids resource consumption caused by the complex method for all images, and greatly saves resource consumption; and the video image restoration quality is greatly improved by using a key frame detection technology and an image detection technology to be restored. The method and the device for repairing all videos in the mass live video data improve the watching effect of the videos by utilizing the method and the device for repairing all videos.
As shown in fig. 4, in a second aspect, an embodiment of the present invention provides a network-live-broadcast-oriented video high-quality repair system, including a key frame detection module 100, a key frame repair module 200, a difference detection module 300, a repair detection module 400, a non-key repair module 500, and a complete repair module 600, where:
a key frame detection module 100, configured to acquire target live video data, and detect and extract a key frame image in the target live video data by using a key frame detection method;
a key frame restoration module 200, configured to restore a key frame image by using an image restoration method based on deep learning to obtain a key restored image;
the difference detection module 300 is configured to calculate and determine an image to be restored according to differences between each frame image in non-key frame images in target live broadcast video data and previous and next frame images thereof;
the restoration detection module 400 is configured to detect an image to be restored and generate a detection result;
the non-key restoration module 500 is configured to restore the image to be restored by using an image restoration method based on deep learning or a basic image restoration method according to the detection result to obtain a non-key restoration image;
and a completion restoration module 600 configured to obtain a complete target restoration image based on the key restoration image and the non-key restoration image, and complete video restoration.
In order to solve the technical problem that accurate, efficient and high-quality restoration cannot be performed on massive live video data in the prior art, the system combines a common restoration method and a restoration method based on deep learning based on the mutual matching of various modules such as a key frame detection module 100, a key frame restoration module 200, a difference detection module 300, a restoration detection module 400, a non-key restoration module 500, a completion restoration module 600 and the like, performs targeted restoration on different images, utilizes a complex method for large-area missing images and a common method for non-large-area missing images, avoids resource consumption caused by the complex method for all images, and greatly saves resource consumption; and the video image restoration quality is greatly improved by using a key frame detection technology and an image detection technology to be restored.
As shown in fig. 5, in a third aspect, an embodiment of the present application provides an electronic device, which includes a memory 101 for storing one or more programs; a processor 102. The one or more programs, when executed by the processor 102, implement the method of any of the first aspects as described above.
Also included is a communication interface 103, and the memory 101, processor 102 and communication interface 103 are electrically connected to each other, directly or indirectly, to enable transfer or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 101 may be used to store software programs and modules, and the processor 102 executes the software programs and modules stored in the memory 101 to thereby execute various functional applications and data processing. The communication interface 103 may be used for communicating signaling or data with other node devices.
The Memory 101 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
The processor 102 may be an integrated circuit chip having signal processing capabilities. The Processor 102 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In the embodiments provided in the present application, it should be understood that the disclosed method and system and method can be implemented in other ways. The method and system embodiments described above are merely illustrative, for example, the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by the processor 102, implements the method as in any one of the first aspect described above. The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (10)
1. A video high-quality restoration method facing network live broadcast big data is characterized by comprising the following steps:
acquiring target live video data, and detecting and extracting a key frame image in the target live video data by adopting a key frame detection method;
repairing the key frame image by adopting an image repairing method based on deep learning to obtain a key repairing image;
calculating and determining an image to be restored according to the difference between each frame image in non-key frame images in target live broadcast video data and the previous and next frame images;
detecting an image to be repaired to generate a detection result;
repairing the image to be repaired by adopting an image repairing method or a basic image repairing method based on deep learning according to the detection result to obtain a non-key repaired image;
and obtaining a complete target restoration image based on the key restoration image and the non-key restoration image to finish video restoration.
2. The method as claimed in claim 1, wherein the method for calculating the difference between each frame image and its previous and subsequent frame images in the non-key frame images in the target live video data comprises the following steps:
and calculating the difference value between each frame image of the non-key frame images in the target live broadcast video data and the previous and next frame images by adopting a difference value subtraction method so as to determine the difference between each frame image in the non-key frame images and the previous and next frame images.
3. The method as claimed in claim 2, wherein the method for determining the image to be restored according to the difference between each frame image in the non-key frame images in the target live broadcast video data and the previous and subsequent frame images comprises the following steps:
judging whether the difference value between each frame image in the non-key frame images and the previous and next frame images is greater than a preset reference threshold value or not, if so, determining that the frame image has a defect, and identifying the frame image as an image to be repaired; if not, the frame image is determined to be absent.
4. The method for high-quality video restoration oriented to the live webcast big data as claimed in claim 1, wherein the method for detecting the image to be restored and generating the detection result comprises the following steps:
detecting and judging whether the local area pixel value in the image to be repaired is larger than a preset area pixel threshold value according to the local area pixel value of the image to be repaired, and if so, generating an area large-area missing detection result; and if not, generating a basic missing detection result.
5. The method for repairing the video of the live webcast big data in high quality as claimed in claim 4, wherein the method for repairing the image to be repaired by using the image repairing method based on the deep learning or the basic image repairing method according to the detection result comprises the following steps:
repairing the image to be repaired by adopting an image repairing method based on deep learning according to the large-area missing detection result of the region;
and repairing the image to be repaired by adopting a basic image repairing method according to the basic missing detection result.
6. The method for repairing video of network live broadcast large data in high quality as claimed in any one of claims 1 to 5, wherein the Image repairing method based on deep learning includes Pluralistic Image Completion multivariate Image restoration method and Image repairing method based on CNN network structure.
7. The method for repairing video of webcast big data with high quality as claimed in any one of claims 1 to 5, wherein the basic image repairing method comprises a PatchMatch algorithm and a Context Decoder algorithm.
8. The utility model provides a video high quality restoration system towards live big data of network, which comprises a key frame detection module, a key frame restoration module, a difference detection module, a restoration detection module, a non-key restoration module and a completion restoration module, wherein:
the key frame detection module is used for acquiring target live video data, and detecting and extracting key frame images in the target live video data by adopting a key frame detection method;
the key frame restoration module is used for restoring the key frame image by adopting an image restoration method based on deep learning to obtain a key restoration image;
the difference detection module is used for calculating and determining an image to be restored according to the difference between each frame image in the non-key frame images in the target live broadcast video data and the previous and next frame images;
the restoration detection module is used for detecting the image to be restored to generate a detection result;
the non-key restoration module is used for restoring the image to be restored by adopting an image restoration method or a basic image restoration method based on deep learning according to the detection result so as to obtain a non-key restoration image;
and the completion restoration module is used for obtaining a complete target restoration image based on the key restoration image and the non-key restoration image and completing video restoration.
9. An electronic device, comprising:
a memory for storing one or more programs;
a processor;
the one or more programs, when executed by the processor, implement the method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210311319.0A CN114841868A (en) | 2022-03-28 | 2022-03-28 | Video high-quality restoration method and system for live webcast big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210311319.0A CN114841868A (en) | 2022-03-28 | 2022-03-28 | Video high-quality restoration method and system for live webcast big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114841868A true CN114841868A (en) | 2022-08-02 |
Family
ID=82563589
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210311319.0A Withdrawn CN114841868A (en) | 2022-03-28 | 2022-03-28 | Video high-quality restoration method and system for live webcast big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114841868A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116647710A (en) * | 2023-06-05 | 2023-08-25 | 美琦佳成网络科技有限公司 | Live broadcast method, system and storage medium based on social group chat |
-
2022
- 2022-03-28 CN CN202210311319.0A patent/CN114841868A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116647710A (en) * | 2023-06-05 | 2023-08-25 | 美琦佳成网络科技有限公司 | Live broadcast method, system and storage medium based on social group chat |
| CN116647710B (en) * | 2023-06-05 | 2024-01-26 | 美琦佳成网络科技有限公司 | Live broadcast method, system and storage medium based on social group chat |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sajid et al. | Universal multimode background subtraction | |
| CN108256562B (en) | Salient target detection method and system based on weak supervision time-space cascade neural network | |
| Lin et al. | Image manipulation detection by multiple tampering traces and edge artifact enhancement | |
| Wang et al. | Cycle-snspgan: Towards real-world image dehazing via cycle spectral normalized soft likelihood estimation patch gan | |
| CN111860414A (en) | Method for detecting Deepfake video based on multi-feature fusion | |
| JP2021068056A (en) | On-road obstacle detecting device, on-road obstacle detecting method, and on-road obstacle detecting program | |
| CN112785157A (en) | Risk identification system updating method and device and risk identification method and device | |
| CN109787958A (en) | Network flow real-time detection method and detection terminal, computer readable storage medium | |
| CN111680680B (en) | Target code positioning method and device, electronic equipment and storage medium | |
| Li et al. | [Retracted] A Pavement Crack Detection Method Based on Multiscale Attention and HFS | |
| CN114841868A (en) | Video high-quality restoration method and system for live webcast big data | |
| CN114596441A (en) | Massive image denoising method and system for live webcast social big data | |
| Wang et al. | Togethernet: Bridging image restoration and object detection together via dynamic enhancement learning | |
| Li et al. | A motion blur QR code identification algorithm based on feature extracting and improved adaptive thresholding | |
| Zhang et al. | PRNU-based image forgery localization with deep multi-scale fusion | |
| Liu et al. | Dual-branch self-attention network for pedestrian attribute recognition | |
| CN111753729B (en) | False face detection method and device, electronic equipment and storage medium | |
| Esfahani et al. | ISAIR: Deep inpainted semantic aware image representation for background subtraction | |
| Liang et al. | Rain wiper: An incremental randomly wired network for single image deraining | |
| Bao et al. | Quality-aware Selective Fusion Network for VDT Salient Object Detection | |
| CN113936135A (en) | Detection image generation method and system | |
| Dharmalingam et al. | Adaptive frost filtered quantile regressive artificial deep structure learning framework for image quality assessment | |
| Kumar et al. | Realtime dehazing using colour uniformity principle | |
| CN114611565A (en) | Data processing method, device, equipment and storage medium | |
| CN112861874A (en) | Expert field denoising method and system based on multi-filter denoising result |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220802 |
|
| WW01 | Invention patent application withdrawn after publication |