WO2020042826A1 - 视频流降噪方法和装置、电子设备及存储介质 - Google Patents

视频流降噪方法和装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2020042826A1
WO2020042826A1 PCT/CN2019/096884 CN2019096884W WO2020042826A1 WO 2020042826 A1 WO2020042826 A1 WO 2020042826A1 CN 2019096884 W CN2019096884 W CN 2019096884W WO 2020042826 A1 WO2020042826 A1 WO 2020042826A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
blocks
image frame
image blocks
frequency domain
Prior art date
Application number
PCT/CN2019/096884
Other languages
English (en)
French (fr)
Inventor
章佳杰
Original Assignee
北京达佳互联信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京达佳互联信息技术有限公司 filed Critical 北京达佳互联信息技术有限公司
Publication of WO2020042826A1 publication Critical patent/WO2020042826A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing

Definitions

  • the present application relates to the technical field of video processing, and in particular, to a method and an apparatus for reducing noise in a video stream, an electronic device, and a storage medium.
  • a video stream will be generated during the live broadcast.
  • the inventor found that when performing noise reduction processing on a video stream, it is only performing noise reduction on each independent image frame in the video stream. This method of reducing noise on the video stream is suitable for pictures. Still video. The video image corresponding to the video stream generated by the live broadcast will inevitably have a large number of images of small objects with strong motion. After the current video stream noise reduction method is used to perform noise reduction processing on the live video stream, the video image corresponding to the video stream will appear blurred. The problem.
  • embodiments of the present application provide a method and device for reducing noise in a video stream, an electronic device, and a storage medium.
  • a method for reducing noise in a video stream includes: receiving a video stream, wherein the video stream includes a plurality of image frames; and determining a current waiting state in the video stream.
  • a reference image frame for noise reduction processing searching each candidate image frame of the reference image frame from the video stream, wherein each of the candidate image frames is located before and / or before the reference image frame in a time domain; A preset number of image frames following the reference image frame; dividing the reference image frame into a plurality of image blocks; and merging each of the image blocks and similar image blocks in each of the candidate image frames for splicing, Get the target image frame after noise reduction.
  • a video stream noise reduction device configured to include: a receiving module configured to receive a video stream, wherein the video stream includes a plurality of image frames; a determining module Is configured to determine a reference image frame in the video stream that is currently to be processed for noise reduction; a search module is configured to search each candidate image frame of the reference image frame from the video stream, wherein each of the candidate The image frame is a preset number of image frames in front of the reference image frame and / or after the reference image frame in a time domain; a dividing module is configured to divide the reference image frame into a plurality of image blocks; The noise reduction module is configured to fuse each of the image blocks with similar image blocks in each of the candidate image frames and perform splicing to obtain a target image frame after the noise reduction process.
  • an electronic device including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform any of the foregoing video stream noise reduction method.
  • a non-transitory computer-readable storage medium is provided, and when an instruction in the storage medium is executed by a processor of an electronic device, the electronic device is caused to perform any one of the foregoing video streaming drops. Noise method.
  • a computer program product that, when instructions in the computer program product are executed by a processor of an electronic device, causes the electronic device to perform any of the foregoing video stream noise reduction methods.
  • the video stream noise reduction solution receives a video stream and determines a reference image frame to be currently processed for noise reduction; searches for each candidate image frame of the reference image frame from the video stream; and compares the reference image according to each candidate image frame The frame is subjected to noise reduction processing to obtain a target image frame after the noise reduction processing.
  • This video stream noise reduction scheme performs noise reduction processing on reference image frames based on each candidate image frame.
  • the information of each candidate image frame can be used to increase the signal-to-noise ratio of the reference image frame, which can effectively reduce the noise of the reference image frame. Effectively reduce the noise of each reference image frame in the video stream, so it can effectively reduce the noise of the video stream, so that the video stream displays a clear video picture.
  • Fig. 1 is a flowchart of steps in a method for reducing noise in a video stream according to an exemplary embodiment
  • Fig. 2 is a flowchart of steps in a method for reducing noise in a video stream according to an exemplary embodiment
  • Fig. 3 is a block diagram of a video stream noise reduction device according to an exemplary embodiment
  • Fig. 4 is a structural block diagram of an electronic device according to an exemplary embodiment
  • Fig. 5 is a structural block diagram of an electronic device according to an exemplary embodiment.
  • Fig. 1 is a flowchart illustrating a method for reducing noise in a video stream according to an exemplary embodiment.
  • the method for reducing noise in a video stream shown in Fig. 1 is used in an electronic device, and includes the following steps:
  • Step 101 Receive a video stream.
  • the method for noise reduction of a video stream in the embodiments of the present application can be applied to the processing of a video stream in a video chat, and can also be applied to the processing of a video stream in a live broadcast process.
  • the electronic device performs noise reduction processing on the video stream collected from the camera.
  • the video stream includes multiple image frames.
  • Step 102 Determine a reference image frame in the video stream to be processed for noise reduction.
  • Each image frame in the video stream needs to be processed for noise reduction, and the image frame currently to be processed for noise reduction is a reference image frame.
  • steps 102 to 105 are performed. Therefore, looping steps 102 to 105 can perform noise reduction processing on each image frame in the video stream to obtain a noise-reduced video. flow.
  • Step 103 Search each candidate image frame of the reference image frame from the video stream.
  • Each candidate image frame is a preset number of image frames that are located before the reference image frame and / or after the reference image frame in the time domain.
  • a preset number of image frames in the time domain before the reference image frame and after the reference image frame can be searched as candidate image frames.
  • the preset number can be set by a person skilled in the art according to actual needs. For example, the preset number can be set to 4, and two image frames are selected from the reference image frame before and after the candidate image frame.
  • the preset number can also be set to 6 or 8, etc. This is not specifically limited in the embodiments of the present application.
  • Step 104 Divide the reference image frame into a plurality of image blocks.
  • the specific number of image blocks into which the reference image frame is divided can be set by a person skilled in the art according to actual needs, which is not specifically limited in the embodiments of the present application, for example, it is set to 2, 4, 6, or 8 Blocks etc.
  • Step 105 Each image block is merged with similar image blocks in each candidate image frame and then spliced to obtain a target image frame after the noise reduction process.
  • the image block is fused with similar image blocks in each candidate image frame to obtain a fused image block, and all the obtained fused image blocks are stitched together. Then get the target image frame.
  • This type of noise reduction processing is performed on the reference image frame based on each candidate image frame.
  • the information of each candidate image frame can be used to increase the signal-to-noise ratio of the reference image frame, thereby achieving a good noise reduction effect.
  • the similar image block in the candidate image frame is the image block with the highest similarity to the image block and the similarity greater than a certain threshold after the image block is traversed in the candidate image frame.
  • the size of the similar image block is the same as the image block size in the reference image frame, and the similarity between the image block in the reference image frame and the image block in the subsequent image frame is obtained by the number of the same pixel point and the position of the pixel point, etc. determine.
  • the video stream noise reduction method shown in this exemplary embodiment receives a video stream and determines a reference image frame to be currently processed for noise reduction; searches each candidate image frame of the reference image frame from the video stream; and compares the reference image according to each candidate image frame The frame is subjected to noise reduction processing to obtain a target image frame after the noise reduction processing.
  • This video stream noise reduction scheme performs noise reduction processing on reference image frames based on each candidate image frame.
  • the information of each candidate image frame can be used to increase the signal-to-noise ratio of the reference image frame, which can effectively reduce the noise of the reference image frame. Effectively reduce the noise of each reference image frame in the video stream, so it can effectively reduce the noise of the video stream.
  • Fig. 2 is a flowchart illustrating a method for reducing noise in a video stream according to an exemplary embodiment.
  • the method for reducing noise in a video stream shown in Fig. 2 is used in an electronic device, and includes the following steps.
  • Step 201 Receive a video stream.
  • the video stream includes multiple image frames, and each image frame is continuous in the time domain.
  • the image frame to be noise-reduced may be used as the reference image frame.
  • steps 202 to 208 are a process of performing noise reduction processing on an image frame.
  • the process may be repeatedly performed to perform noise reduction processing on each image frame in the video stream.
  • Step 202 Determine a reference image frame to be processed for noise reduction.
  • Step 203 Search each candidate image frame of the reference image frame from the video stream.
  • Each candidate image frame is a preset number of image frames that are located before the reference image frame and / or after the reference image frame in the time domain.
  • An optional implementation manner is to search for a preset number of image frames in the time domain before the reference image frame and after the reference image frame as candidate image frames.
  • the reference image frame After determining each candidate image frame of the reference image frame, the reference image frame may be subjected to noise reduction processing according to each candidate image frame of the reference image frame to obtain the target image frame after the noise reduction process.
  • noise reduction processing For a specific noise reduction processing method, refer to steps 204 to 208.
  • Step 204 Divide the reference image frame into a plurality of image blocks.
  • the number of image blocks into which a reference image frame is divided can be set by a person skilled in the art according to actual requirements, and there is no specific limitation on this in the embodiment of the present application, for example, the number of image blocks is set to 2, 4, 6, or 8 .
  • Step 205 For each image block in the reference image frame, in each candidate image frame, determine each similar image block of the image block.
  • the reference image frame is divided into four image blocks, and the candidate image frames of the reference image frame are A, B, C, and D, then each image block has a similar image block in A, B, C, and D , That is, one image block corresponds to four similar image blocks.
  • Step 206 align the image block with each of the similar image blocks.
  • This step is repeatedly performed to align each image block included in the reference image frame to be processed for noise reduction with a similar image block of each image block.
  • An optional method for aligning the image block with each of the similar image blocks is: determining a relative offset between each of the similar image blocks and the image block in an image frame; The similar image block is reverse-shifted according to the relative offset corresponding to the similar image block to align the similar image block with the image block.
  • each similar image block of the image block is determined in each candidate image frame; the way to align the image block with each similar image block is as follows:
  • the image Gaussian pyramid of the reference image frame is constructed, where different levels of the Gaussian pyramid correspond to different levels of detail of the reference image frame; for each image block of each level, the image block and each candidate The image regions in the image frame are searched and aligned; the search and alignment results corresponding to different levels of the Gaussian pyramid are accumulated step by step to align the image blocks with similar image blocks in the candidate image frames.
  • the relative displacement of each image block is characterized by color and saturation, where the color represents the direction of the displacement vector and the saturation value represents the size of the displacement vector.
  • the image block is 21x21 in size.
  • the maximum matching offset between adjacent images can reach 64 pixels, and the amount of calculation is only slightly larger than the range of 5 pixels to search. It can be seen that this method can not only ensure the selection accuracy of similar image blocks, but also reduce the amount of calculation.
  • Step 207 Fusion the aligned image blocks and each of the similar image blocks to obtain a fused image block.
  • This step is repeatedly performed, and each image block included in the reference image frame to be noise-reduced is fused with similar image blocks aligned with each image block to obtain each fused image block.
  • An optional method of fusing the aligned image blocks and each of the similar image blocks to obtain a fused image block is to calculate the same pixels in the aligned image blocks and each of the similar image blocks.
  • the pixel average value of the points; a fused image block is generated according to the pixel average value of each of the pixel points.
  • This method of merging image blocks by calculating the average pixel value consumes a small amount of calculation.
  • Another optional method is to fuse the aligned image blocks and each of the similar image blocks to obtain a fused image block as follows:
  • the image block may be subjected to discrete cosine transform to obtain a first frequency domain distribution, and each similar image block may be subjected to discrete cosine transform to obtain each second frequency domain distribution.
  • the weights corresponding to each of the second frequency domain distribution and the first frequency domain distribution are determined according to each degree of difference;
  • the target frequency domain distribution is determined according to the weight corresponding to each second frequency domain distribution, each second frequency domain distribution, the weight corresponding to the first frequency domain distribution, and the first frequency domain distribution; the target frequency domain distribution is converted into a fused image Piece.
  • This method of optionally merging the aligned image blocks and each of the similar image blocks to obtain a fused image block can effectively avoid the problems of blurring and smearing of the fused image block.
  • Step 208 splicing each fused image block corresponding to the reference image frame to obtain a target image frame after noise reduction processing.
  • the target image frame When stitching each fused image block corresponding to a reference image frame, the target image frame can be obtained by splicing along the edges of each fused image block.
  • This kind of robust fusion image block stitching method the resulting target image frame has a block effect, that is, stitching. There is a problem of missing images at the edges.
  • an optional method of splicing each of the fused image blocks corresponding to a reference image frame to obtain a target image frame is as follows:
  • each fused image block corresponding to the image frame is overlapped with an adjacent fused image block by a predetermined area to obtain a second stitched image
  • the preset area can be set to the area of half an image block.
  • the stitching edge of the fused image block in the first stitched image is corrected to obtain the target image frame after the noise reduction process.
  • each image block is not without overlap. Instead, each fused image block is overlapped with the adjacent fused image block by half.
  • the two-dimensional cosine window can be used for stitching during the stitching process.
  • the two adjacent fused image blocks are weighted.
  • the video stream noise reduction method shown in this exemplary embodiment receives a video stream and determines a reference image frame to be currently processed for noise reduction; searches each candidate image frame of the reference image frame from the video stream; and compares the reference image according to each candidate image frame The frame is subjected to noise reduction processing to obtain a target image frame after the noise reduction processing.
  • This video stream noise reduction scheme performs noise reduction processing on reference image frames based on each candidate image frame.
  • the information of each candidate image frame can be used to increase the signal-to-noise ratio of the reference image frame, which can effectively reduce the noise of the reference image frame. Effectively reduce the noise of each image frame in the video stream, so it can effectively reduce the noise of the video stream.
  • Fig. 3 is a block diagram of a video stream noise reduction device according to an exemplary embodiment.
  • the device includes a receiving module 301, a determination module 302, a search module 303, a division module 304, and a noise reduction module 305.
  • the receiving module 301 is configured to receive a video stream, where the video stream includes multiple image frames; the determining module 302 is configured to determine a reference image frame in the video stream that is currently to be noise-reduced; the search module 303, Configured to search each candidate image frame of the reference image frame from the video stream, wherein each of the candidate image frames is located before the reference image frame and / or after the reference image frame in a time domain A preset number of image frames; a division module 304 configured to divide the reference image frame into a plurality of image blocks; a noise reduction module 305 configured to separate each of the image blocks and each of the candidate image frames The similar image blocks in the image are merged and spliced to obtain the target image frame after the noise reduction process.
  • the noise reduction module 305 may include: an alignment sub-module 3051 configured to determine, for each of the image blocks in the reference image frame, the image in each of the candidate image frames, respectively. Each similar image block of the block; aligning the image block with each of the similar image blocks; a fusion submodule 3052 configured to fuse the aligned image block with each of the similar image blocks to obtain a fusion An image block; a stitching submodule 3053, configured to stitch each of the fused image blocks corresponding to the image frame to obtain a target image frame after noise reduction processing.
  • an alignment sub-module 3051 configured to determine, for each of the image blocks in the reference image frame, the image in each of the candidate image frames, respectively. Each similar image block of the block; aligning the image block with each of the similar image blocks; a fusion submodule 3052 configured to fuse the aligned image block with each of the similar image blocks to obtain a fusion An image block; a stitching submodule 3053, configured to stitch each of the fused image blocks
  • the alignment sub-module 3051 may include: an offset determination unit configured to determine a relative offset between each of the similar image blocks and the image blocks; a first alignment unit configured to For each of the similar image blocks, reversely shift the similar image blocks according to the relative offset corresponding to the similar image blocks to align the similar image blocks with the image blocks.
  • the alignment submodule 3051 may include a scaling unit configured to construct an image Gaussian pyramid of the reference image frame by reducing the reference image frame by different multiples, wherein different levels of the Gaussian pyramid Corresponding to different levels of detail of the reference image frame; the accumulation unit is configured to search and align the image block with the image region in each of the candidate image frames for each image block of each level; align the Gaussian pyramid The search alignment results corresponding to the different levels of the image are accumulated stepwise to align the image blocks with similar image blocks in the candidate image frames.
  • the fusion sub-module 3052 may include: an average value calculation unit configured to calculate an average pixel value of the same pixel point in the image block and each of the similar image blocks after being aligned;
  • the fusion unit is configured to generate a fusion image block according to a pixel average value of each of the pixel points.
  • the fusion submodule 3052 may include a frequency domain distribution determining unit configured to determine a first frequency domain distribution of the aligned image blocks and a second frequency domain distribution of each of the similar image blocks, respectively.
  • a difference determination unit configured to determine a degree of difference between each of the second frequency domain distribution and the first frequency domain distribution;
  • a weight determination unit configured to determine each of the first Weights corresponding to the two frequency domain distributions and the first frequency domain distribution;
  • the target frequency domain distribution determining unit is configured to, according to the weights corresponding to each of the second frequency domain distributions, each of the second frequency domain distributions, the A weight corresponding to the first frequency domain distribution and the first frequency domain distribution determine a target frequency domain distribution;
  • a transformation unit is configured to transform the target frequency domain distribution into a fused image block.
  • the stitching sub-module 3053 may include: a first stitching unit configured to stitch each of the fused image blocks corresponding to the reference image frame along the edge of the fused image block to obtain a first stitched image; A two stitching unit is configured to overlap each of the fused image blocks corresponding to the reference image frame with an adjacent fused image block by a predetermined area to stitch a second stitched image; a correction unit is configured to The second stitching image performs a correction process on a stitching edge of a fused image block in the first stitching image to obtain a target image frame after noise reduction processing.
  • Fig. 4 is a block diagram of an electronic device 400 according to an exemplary embodiment.
  • the electronic device may be a mobile terminal or a server.
  • the electronic device is used as an example for description.
  • the electronic device 400 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.
  • the electronic device 400 may include one or more of the following components: a processing component 402, a memory 404, a power component 406, a multimedia component 408, an audio component 410, an input / output (I / O) interface 412, and a sensor component 414 , And communication component 416.
  • the processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations.
  • the processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the method described above.
  • the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components.
  • the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.
  • the memory 404 is configured to store various types of data to support operations at the electronic device 400. Examples of such data include instructions for any application or method for operating on the electronic device 400, contact data, phone book data, messages, pictures, videos, and the like.
  • the memory 404 may be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), Programming read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM Programming read-only memory
  • PROM programmable read-only memory
  • ROM read-only memory
  • magnetic memory flash memory
  • flash memory magnetic disk or optical disk.
  • the power supply component 406 provides power to various components of the electronic device 400.
  • the power component 406 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 400.
  • the multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and a user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user.
  • the touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or slide action, but also detect duration and pressure related to the touch or slide operation.
  • the multimedia component 408 includes a front camera and / or a rear camera. When the electronic device 400 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.
  • the audio component 410 is configured to output and / or input audio signals.
  • the audio component 410 includes a microphone (MIC) that is configured to receive an external audio signal when the electronic device 400 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode.
  • the received audio signal may be further stored in the memory 404 or transmitted via the communication component 416.
  • the audio component 410 further includes a speaker for outputting audio signals.
  • the I / O interface 412 provides an interface between the processing component 402 and a peripheral interface module.
  • the peripheral interface module may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
  • the sensor component 414 includes one or more sensors for providing various aspects of the state evaluation of the electronic device 400.
  • the sensor component 414 can detect the on / off state of the electronic device 400, and the relative positioning of the components.
  • the component is the display and keypad of the electronic device 400.
  • the sensor component 414 can also detect the electronic device 400 or an electronic device 400.
  • the position of the component changes, the presence or absence of the user's contact with the mobile terminal 400, the orientation or acceleration / deceleration of the electronic device 400, and the temperature change of the electronic device 400.
  • the sensor component 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact.
  • the sensor component 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.
  • the sensor component 414 may further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • the communication component 416 is configured to facilitate wired or wireless communication between the electronic device 400 and other devices.
  • the electronic device 400 may access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof.
  • the communication section 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel.
  • the communication component 416 further includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra wideband
  • Bluetooth Bluetooth
  • the electronic device 400 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), It is implemented by programming a gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and is used to perform the video stream noise reduction method shown in FIG. 1 and FIG. 2 described above.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA gate array
  • controller a controller
  • microcontroller a microcontroller
  • microprocessor or other electronic components
  • a non-transitory computer-readable storage medium including instructions may be executed by the processor 420 of the electronic device 400 to complete the foregoing FIG. 1 and FIG. 2 Video stream noise reduction method shown in.
  • the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
  • a computer program product is also provided.
  • the instructions in the computer program product are executed by the processor 420 of the electronic device 400, the electronic device 400 is caused to execute the video shown in FIG. 1 and FIG. 2 described above. Stream noise reduction method.
  • Fig. 5 is a block diagram of an electronic device according to an exemplary embodiment.
  • the electronic device may be a mobile terminal or a server.
  • the electronic device is used as an example for description.
  • the electronic device 500 includes a processing component 501, which further includes one or more processors, and a memory resource represented by the memory 502, for storing instructions executable by the processing component 501, such as an application program.
  • the application program stored in the memory 502 may include one or more modules each corresponding to a set of instructions.
  • the processing component 501 is configured to execute instructions to execute the video stream noise reduction method shown in FIG. 1 and FIG. 2 described above.
  • the electronic device 500 may further include a power supply component 503 configured to perform power management of the electronic device 500, a wired or wireless network interface 504 configured to connect the electronic device 500 to a network, and an input / output (I / O) interface 505 .
  • the electronic device 500 can operate based on an operating system stored in the memory 502, such as Windows ServerTM, Mac OSXTM, UnixTM, LinuxTM, FreeBSDTM, or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)
  • Picture Signal Circuits (AREA)

Abstract

本申请是关于一种视频流降噪方法和装置、电子设备及存储介质，其中所述方法包括：接收视频流，其中所述视频流中包含多个图像帧；确定所述视频流中当前待降噪处理的参考图像帧；从所述视频流中搜索所述参考图像帧的各候选图像帧；将所述参考图像帧划分为多个图像块；分别将各所述图像块与各所述候选图像帧中的相似图像块融合后进行拼接，得到降噪处理后的目标图像帧。通过本申请提供的视频流降噪方法，能够对视频流进行有效降噪，使视频流展示出清晰的视频画面。

Description

视频流降噪方法和装置、电子设备及存储介质
相关申请的交叉引用
本申请要求于2018年08月31日提交中国专利局、申请号为201811014197.9，发明名称为“视频流降噪方法和装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频处理技术领域，尤其涉及一种视频流降噪方法和装置、电子设备及存储介质。
背景技术
目前随着电子设备功能的不断强大，各种类型的应用程序应运而生，例如：社交类应用程序、直播类应用程序以及支付类应用程序等。用户可以通过电子设备上安装的直播类应用程序直播或者观看直播。
在直播过程中会产生视频流，发明人发现目前对视频流进行降噪处理时，仅是分别对视频流中各个独立的图像帧进行降噪，该种对视频流降噪的方式适用于画面静止的视频。而直播产生的视频流对应的视频画面中难免有大量运动剧烈的小物体影像，采用现有的视频流降噪方法对直播视频流进行降噪处理后，视频流对应的视频画面会出现画面模糊的问题。
可见，目前迫切需要本领域技术人员提供一种对视频流进行有效降噪的方法。
发明内容
为克服相关技术中存在的问题，本申请实施例提供了一种视频流降噪方法和装置、电子设备及存储介质。
根据本申请实施例的第一方面，提供一种视频流降噪方法，其中，所述 方法包括：接收视频流，其中所述视频流中包含多个图像帧；确定所述视频流中当前待降噪处理的参考图像帧；从所述视频流中搜索所述参考图像帧的各候选图像帧，其中，各所述候选图像帧为在时域上位于所述参考图像帧前和/或所述参考图像帧后的预设数量的图像帧；将所述参考图像帧划分为多个图像块；分别将各所述图像块与各所述候选图像帧中的相似图像块融合后进行拼接，得到降噪处理后的目标图像帧。
根据本申请实施例的第二方面，提供一种视频流降噪装置，其中，所述装置包括：接收模块，被配置为接收视频流，其中所述视频流中包含多个图像帧；确定模块，被配置为确定所述视频流中当前待降噪处理的参考图像帧；搜索模块，被配置为从所述视频流中搜索所述参考图像帧的各候选图像帧，其中，各所述候选图像帧为在时域上位于所述参考图像帧前和/或所述参考图像帧后的预设数量的图像帧；划分模块，被配置为将所述参考图像帧划分为多个图像块；降噪模块，被配置为分别将各所述图像块与各所述候选图像帧中的相似图像块融合后进行拼接，得到降噪处理后的目标图像帧。
根据本申请实施例的第三方面，提供一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为执行上述任一种视频流降噪方法。
根据本申请实施例的第四方面，提供一种非临时性计算机可读存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得电子设备执行上述任一种视频流降噪方法。
根据本申请实施例的第五方面，提供根据一种计算机程序产品，当所述计算机程序产品中的指令由电子设备的处理器执行时，使得电子设备执行上述任一种视频流降噪方法。
本申请的实施例提供的视频流降噪方案，接收视频流并确定当前待降噪处理的参考图像帧；从视频流中搜索参考图像帧的各候选图像帧；根据各候选图像帧对参考图像帧进行降噪处理，得到降噪处理后的目标图像帧。该视频流降噪方案，基于各候选图像帧对参考图像帧进行降噪处理，可以利用各 候选图像帧的信息增加参考图像帧的信噪比，能够对参考图像帧进行有效降噪，由于能够对视频流中各参考图像帧进行有效降噪，因此可实现对视频流的有效降噪，从而使视频流展示出清晰的视频画面。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本申请。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本申请的原理。
图1是根据一示例性实施例示出的一种视频流降噪方法的步骤流程图；
图2是根据一示例性实施例示出的一种视频流降噪方法的步骤流程图；
图3是根据一示例性实施例示出的一种视频流降噪装置的框图；
图4是根据一示例性实施例示出的一种电子设备的结构框图；
图5是根据一示例性实施例示出的一种电子设备的结构框图。
具体实施方式
图1是根据一示例性实施例示出的一种视频流降噪方法的流程图，如图1所示的视频流降噪方法用于电子设备中，包括以下步骤：
步骤101：接收视频流。
本申请实施例中的视频流降噪方法可适用于视频聊天中视频流的处理，也可适用于直播过程中视频流的处理。电子设备从摄像头采集视频流后对其进行降噪处理。
其中，视频流中包含多个图像帧。
步骤102：确定视频流中当前待降噪处理的参考图像帧。
视频流中的每个图像帧均需进行降噪处理，当前待降噪处理的图像帧则为参考图像帧。对每个图像帧进行降噪处理时，均执行步骤102至步骤105，因此，循环执行步骤102至步骤105即可对视频流中的各图像帧进行降噪处 理，得到降噪处理后的视频流。
步骤103：从视频流中搜索参考图像帧的各候选图像帧。
其中，各候选图像帧为在时域上位于参考图像帧前和/或参考图像帧后的预设数量的图像帧。一种可选的实现方式中，可搜索时域上位于参考图像帧前和参考图像帧后预设数量的各图像帧作为候选图像帧。其中，预设数量可以由本领域技术人员根据实际需求进行设置，例如预设数量可以设置为4，则从参考图像帧前后分别选取两个图像帧作为候选图像帧。预设数量还可以设置为6或者8个等，本申请实施例中对此不做具体限制。
步骤104：将参考图像帧划分为多个图像块。
将参考图像帧划分成的图像块的具体个数可以由本领域技术人员根据实际需求进行设置，本申请实施例中对此不做具体限制，例如：设置为2块、4块、6块或者8块等。
步骤105：分别将各图像块与各候选图像帧中的相似图像块融合后进行拼接，得到降噪处理后的目标图像帧。
参考图像帧被划分为多个图像块后，针对每个图像块，将该图像块分别与各候选图像帧中的相似图像块进行融合后得到一个融合图像块，将得到的所有融合图像块拼接后得到目标图像帧。该种基于各候选图像帧对参考图像帧进行降噪处理，可以利用各候选图像帧的信息增加参考图像帧的信噪比，从而达到良好的降噪效果。
对于参考图像帧中的每个图像块，其在候选图像帧的相似图像块，为将该图像块在候选图像帧中遍历后，与该图像块相似度最高且相似度大于一定阈值的图像块，相似图像块的大小与参考图像帧中的图像块大小一致，其中在参考图像帧中的图像块与后续图像帧中的图像块的相似度通过相同像素点的个数及像素点位置等来确定。
本示例性实施例示出的视频流降噪方法，接收视频流并确定当前待降噪处理的参考图像帧；从视频流中搜索参考图像帧的各候选图像帧；根据各候选图像帧对参考图像帧进行降噪处理，得到降噪处理后的目标图像帧。该视 频流降噪方案，基于各候选图像帧对参考图像帧进行降噪处理，可以利用各候选图像帧的信息增加参考图像帧的信噪比，能够对参考图像帧进行有效降噪，由于能够对视频流中各参考图像帧进行有效降噪，因此可实现对视频流的有效降噪。
图2是根据一示例性实施例示出的一种视频流降噪方法的流程图，如图2所示的视频流降噪方法用于电子设备中，包括以下步骤。
步骤201：接收视频流。
其中，视频流中包含多个图像帧，各图像帧在时域上连续。在具体实现过程中，需要对视频流中的各图像帧均进行降噪处理，从而达到对视频流的降噪。在对视频流中的图像帧进行降噪处理时，可将当前待降噪处理的图像帧作为参考图像帧。本申请实施例中步骤202至步骤208为对一个图像帧进行降噪处理的流程，在具体实现过程中，可以重复执行该流程对视频流中的各图像帧进行降噪处理。
步骤202：确定当前待降噪处理的参考图像帧。
步骤203：从视频流中搜索参考图像帧的各候选图像帧。
其中，各候选图像帧为在时域上位于参考图像帧前和/或参考图像帧后的预设数量的图像帧。一种可选地实现方式为，搜索时域上位于参考图像帧前和参考图像帧后预设数量的各图像帧作为候选图像帧。
确定参考图像帧的各候选图像帧后，可根据参考图像帧的各候选图像帧，对参考图像帧进行降噪处理，得到降噪处理后的目标图像帧。具体降噪处理方式参照步骤204至步骤208中所示。
步骤204：将参考图像帧划分为多个图像块。
将参考图像帧划分成的图像块个数可以由本领域技术人员根据实际需求进行设置，本申请实施例中对此不做具体限制，例如：设置为2块、4块、6块或者8块等。
步骤205：对于参考图像帧中的每个图像块，分别在各候选图像帧中，确定该图像块的各相似图像块。
将参考图像帧划分成多个图像块后，需要分别从各候选帧中确定各图像块中的相似图像块。
例如：将参考图像帧划分为四个图像块，参考图像帧的候选图像帧分别为A、B、C以及D，则每个图像块在A、B、C以及D中分别有一个相似图像块，也即一个图像块对应四个相似图像块。
步骤206：将该图像块与各所述相似图像块进行对齐。
重复执行该步骤对当前待降噪处理的参考图像帧中包含的各图像块分别与各图像块的相似图像块进行对齐。
一种可选地将该图像块与各所述相似图像块进行对齐的方式为：分别确定各所述相似图像块与所述图像块在图像帧中位置的相对偏移量；针对每个所述相似图像块，依据所述相似图像块对应的所述相对偏移量对所述相似图像块进行反向偏移，以将所述相似图像块与所述图像块对齐。
该种对齐方式存在一个矛盾点：在进行相似图像块搜索时，若搜索范围大则计算量相应增加，若搜索范围小，很可能搜不到相似度高的图像块。为解决这个问题，可以通过建立图像高斯金字塔，在图像的不同细节层级进行搜索对齐的工作。在最粗糙的层级，搜索可以跨越较大距离，相应地匹配精度较低，然后逐步增加精度层级，逐步提高匹配精度。在每一层级搜索范围都比较小，而经过逐级累加可以在最后的层级上得到比较精确的匹配结果。
该种对于参考图像帧中的每个图像块，分别在各候选图像帧中，确定图像块的各相似图像块；将图像块与各相似图像块进行对齐的方式具体如下：
通过对参考图像帧缩小不同倍数，构建参考图像帧的图像高斯金字塔，其中，高斯金字塔的不同层级对应参考图像帧的不同细节层级；针对每个层级的每个图像块，将图像块与各候选图像帧中的图像区域进行搜索对齐；将高斯金字塔的不同层级对应的搜索对齐结果进行逐级累加，以对齐图像块与候选图像帧中的相似图像块。
采用上述对齐方式对图像块与相似图像块进行对齐时，对每一个图像块的相对位移的情况用颜色和饱和度表征，其中，颜色代表位移向量方向，饱 和度值代表位移向量的大小。在最细的层级上图像块为21x21大小，在这种策略下，相邻图像之间最大匹配偏移可达64像素，而计算量仅比搜索5像素范围略大。可见，该种方式既能够保证相似图像块的选择精度，又能够降低计算量。
步骤207：将对齐后的所述图像块与各所述相似图像块进行融合，得到融合图像块。
重复执行该步骤，对当前待降噪处理的参考图像帧中包含的各图像块分别与各图像块对齐后的相似图像块融合，得到各融合图像块。
一种可选地将对齐后的所述图像块与各所述相似图像块进行融合，得到融合图像块的方式为：计算对齐后的所述图像块与各所述相似图像块中，相同像素点的像素平均值；依据各所述像素点的像素平均值，生成融合图像块。
该种通过计算像素平均值来融合图像块的方式所消耗地计算量小。
另一种可选地将对齐后的所述图像块与各所述相似图像块进行融合，得到融合图像块的方式如下：
首先，分别确定对齐后的图像块的第一频域分布和各相似图像块的第二频域分布；
具体地可以对图像块进行离散余弦变换，得到第一频域分布，分别将各相似图像块进行离散余弦变换，得到各第二频域分布。
其次，分别确定各第二频域分布与第一频域分布的差异度；
再次，分别根据各差异度，确定各第二频域分布和第一频域分布对应的权重；
针对某一第二频域分布，该第二频域分布与第一频域分布的差异度越大则其对应的权重越小，反之，该第二频域分布与第一频域分布的差异度越小则其对应的权重越大。确定出各第二频域分布和第一频域分布的权重后，将各权重进行归一化处理，最终使得各频域分布的权重之和为1。
最后，根据各第二频域分布对应的权重、各第二频域分布、第一频域分布对应的权重以及第一频域分布，确定目标频域分布；将目标频域分布转化 为融合图像块。
该种可选地将对齐后的所述图像块与各所述相似图像块进行融合，得到融合图像块的方式，能够有效避免融合图像块模糊、拖影的问题。
步骤208：将参考图像帧对应的各融合图像块进行拼接，得到降噪处理后的目标图像帧。
在对参考图像帧对应的各融合图像块进行拼接时，可沿各融合图像块边缘进行拼接得到目标图像帧，该种鲁棒的融合图像块拼接方式，得到的目标图像帧具有块效应即拼接边缘处会存在影像缺失的问题。
为解决该种问题，一种可选地将参考图像帧对应的各所述融合图像块进行拼接，得到目标图像帧的方式如下：
首先，将图像帧对应的各所述融合图像块，沿融合图像块边缘进行拼接得到第一拼接图像；
其次，将图像帧对应的每个融合图像块，均与相邻融合图像块重叠预设面积，拼接得到第二拼接图像；
预设面积可以设置为半个图像块的面积。
最后，基于第二拼接图像对第一拼接图像中的融合图像块拼接边缘处进行修正处理，得到降噪处理后的目标图像帧。
该种拼接方式中每一个图像块之间并非没有重叠，而是将每一个融合图像块均与相邻融合图像块重叠一半的形式进行拼合，在拼合过程中可使用二维余弦窗对需要拼接的两个相邻融合图像块进行加权处理。
本示例性实施例示出的视频流降噪方法，接收视频流并确定当前待降噪处理的参考图像帧；从视频流中搜索参考图像帧的各候选图像帧；根据各候选图像帧对参考图像帧进行降噪处理，得到降噪处理后的目标图像帧。该视频流降噪方案，基于各候选图像帧对参考图像帧进行降噪处理，可以利用各候选图像帧的信息增加参考图像帧的信噪比，能够对参考图像帧进行有效降噪，由于能够对视频流中各图像帧进行有效降噪，因此可实现对视频流的有效降噪。
图3是根据一示例性实施例示出的一种视频流降噪装置的框图，参照图3该装置包括：接收模块301、确定模块302、搜索模块303、划分模块304以及降噪模块305。
接收模块301，被配置为接收视频流，其中所述视频流中包含多个图像帧；确定模块302，被配置为确定所述视频流中当前待降噪处理的参考图像帧；搜索模块303，被配置为从所述视频流中搜索所述参考图像帧的各候选图像帧，其中，各所述候选图像帧为在时域上位于所述参考图像帧前和/或所述参考图像帧后的预设数量的图像帧；划分模块304，被配置为将所述参考图像帧划分为多个图像块；降噪模块305，被配置为分别将各所述图像块与各所述候选图像帧中的相似图像块融合后进行拼接，得到降噪处理后的目标图像帧。
可选地，所述降噪模块305可以包括：对齐子模块3051，被配置为对于所述参考图像帧中的每个所述图像块，分别在各所述候选图像帧中，确定所述图像块的各相似图像块；将所述图像块与各所述相似图像块进行对齐；融合子模块3052，被配置为将对齐后的所述图像块与各所述相似图像块进行融合，得到融合图像块；拼接子模块3053，被配置为将所述图像帧对应的各所述融合图像块进行拼接，得到降噪处理后的目标图像帧。
可选地，所述对齐子模块3051可以包括：偏移量确定单元，被配置为分别确定各所述相似图像块与所述图像块的相对偏移量；第一对齐单元，被配置为对于每个所述相似图像块，依据所述相似图像块对应的所述相对偏移量对所述相似图像块进行反向偏移，以将所述相似图像块与所述图像块对齐。
可选地，所述对齐子模块3051可以包括：缩放单元，被配置为通过对所述参考图像帧缩小不同倍数，构建所述参考图像帧的图像高斯金字塔，其中，所述高斯金字塔的不同层级对应参考图像帧的不同细节层级；累加单元，被配置为对于每个层级的每个图像块，将所述图像块与各所述候选图像帧中的图像区域进行搜索对齐；将所述高斯金字塔的不同层级对应的搜索对齐结果进行逐级累加，以对齐所述图像块与所述候选图像帧中的相似图像块。
可选地，所述融合子模块3052可以包括：均值计算单元，被配置为计算 对齐后的所述图像块与各所述相似图像块中，相同像素点的像素平均值；
融合单元，被配置为依据各所述像素点的像素平均值，生成融合图像块。
可选地，所述融合子模块3052可以包括：频域分布确定单元，被配置为分别确定对齐后的所述图像块的第一频域分布和各所述相似图像块的第二频域分布；差异确定单元，被配置为分别确定各所述第二频域分布与所述第一频域分布的差异度；权重确定单元，被配置为分别根据各所述差异度，确定各所述第二频域分布和所述第一频域分布对应的权重；目标频域分布确定单元，被配置为根据各所述第二频域分布对应的权重、各所述第二频域分布、所述第一频域分布对应的权重以及所述第一频域分布，确定目标频域分布；转化单元，被配置为将所述目标频域分布转化为融合图像块。
可选地，所述拼接子模块3053可以包括：第一拼接单元，被配置为将所述参考图像帧对应的各所述融合图像块，沿融合图像块边缘进行拼接得到第一拼接图像；第二拼接单元，被配置为将所述参考图像帧对应的每个所述融合图像块，均与相邻融合图像块重叠预设面积，拼接得到第二拼接图像；修正单元，被配置为基于所述第二拼接图像对所述第一拼接图像中的融合图像块拼接边缘处进行修正处理，得到降噪处理后的目标图像帧。
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
图4是根据一示例性实施例示出的一种电子设备400的框图。电子设备可以为移动终端也可以为服务器，本申请实施例中以电子设备为移动终端为例进行说明。例如，电子设备400可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等。
参照图4，电子设备400可以包括以下一个或多个组件：处理组件402，存储器404，电源组件406，多媒体组件408，音频组件410，输入/输出(I/O)的接口412，传感器组件414，以及通信组件416。
处理组件402通常控制电子设备400的整体操作，诸如与显示，电话呼 叫，数据通信，相机操作和记录操作相关联的操作。处理组件402可以包括一个或多个处理器420来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件402可以包括一个或多个模块，便于处理组件402和其他组件之间的交互。例如，处理部件402可以包括多媒体模块，以方便多媒体组件408和处理组件402之间的交互。
存储器404被配置为存储各种类型的数据以支持在电子设备400的操作。这些数据的示例包括用于在电子设备400上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器404可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件406为电子设备400的各种组件提供电力。电源组件406可以包括电源管理系统，一个或多个电源，及其他与为电子设备400生成、管理和分配电力相关联的组件。
多媒体组件408包括在电子设备400和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件408包括一个前置摄像头和/或后置摄像头。当电子设备400处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件410被配置为输出和/或输入音频信号。例如，音频组件410包括一个麦克风(MIC)，当电子设备400处于操作模式，如呼叫模式、记录模 式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器404或经由通信组件416发送。在一些实施例中，音频组件410还包括一个扬声器，用于输出音频信号。
I/O接口412为处理组件402和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件414包括一个或多个传感器，用于为电子设备400提供各个方面的状态评估。例如，传感器组件414可以检测到电子设备400的打开/关闭状态，组件的相对定位，例如所述组件为电子设备400的显示器和小键盘，传感器组件414还可以检测电子设备400或电子设备400一个组件的位置改变，用户与移动终端400接触的存在或不存在，电子设备400方位或加速/减速和电子设备400的温度变化。传感器组件414可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件414还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件414还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件416被配置为便于电子设备400和其他设备之间有线或无线方式的通信。电子设备400可以接入基于通信标准的无线网络，如WiFi，2G或3G，或它们的组合。在一个示例性实施例中，通信部件416经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信部件416还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，电子设备400可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述图1、图2中所示的视频流降噪方法。
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器404，上述指令可由电子设备400的处理器420执行以完成上述图1、图2中所示的视频流降噪方法。例如，所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
在示例性实施例中，还提供了一种计算机程序产品，当计算机程序产品中的指令由电子设备400的处理器420执行时，使得电子设备400执行上述图1、图2中所示的视频流降噪方法。
图5是根据一示例性实施例示出的一种电子设备的框图。电子设备可以为移动终端也可以为服务器，本申请实施例中以电子设备为服务器为例进行说明。参照图5，电子设备500包括处理组件501，其进一步包括一个或多个处理器，以及由存储器502所代表的存储器资源，用于存储可由处理组件501的执行的指令，例如应用程序。存储器502中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件501被配置为执行指令，以执行上述图1、图2中所示的视频流降噪方法。
电子设备500还可以包括一个电源组件503被配置为执行电子设备500的电源管理，一个有线或无线网络接口504被配置为将电子设备500连接到网络，和一个输入输出(I/O)接口505。电子设备500可以操作基于存储在存储器502的操作系统，例如Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTM或类似。
本领域技术人员在考虑说明书及实践这里申请的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未申请的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。

Claims (16)

  1. 一种视频流降噪方法，所述方法包括：
    接收视频流，其中所述视频流中包含多个图像帧；
    确定所述视频流中当前待降噪处理的参考图像帧；
    从所述视频流中搜索所述参考图像帧的各候选图像帧，其中，各所述候选图像帧为在时域上位于所述参考图像帧前和/或所述参考图像帧后的预设数量的图像帧；
    将所述参考图像帧划分为多个图像块；
    分别将各所述图像块与各所述候选图像帧中的相似图像块融合后进行拼接，得到降噪处理后的目标图像帧。
  2. 根据权利要求1所述的方法，所述分别将各所述图像块与各所述候选图像帧中的相似图像块融合后进行拼接，得到降噪处理后的目标图像帧，包括：
    对于所述参考图像帧中的每个所述图像块，分别在各所述候选图像帧中，确定所述图像块的各相似图像块；
    将所述图像块与各所述相似图像块进行对齐；
    将对齐后的所述图像块与各所述相似图像块进行融合，得到融合图像块；
    将所述图像帧对应的各所述融合图像块进行拼接，得到降噪处理后的目标图像帧。
  3. 根据权利要求2所述的方法，所述将所述图像块与各所述相似图像块进行对齐，包括：
    分别确定各所述相似图像块与所述图像块的相对偏移量；
    对于每个所述相似图像块，依据所述相似图像块对应的所述相对偏移量对所述相似图像块进行反向偏移，以将所述相似图像块与所述图像块对齐。
  4. 根据权利要求2所述的方法，所述对于所述参考图像帧中的每个所述图像块，分别在各所述候选图像帧中，确定所述图像块的各相似图像块；将 所述图像块与各所述相似图像块进行对齐，包括：
    通过对所述参考图像帧缩小不同倍数，构建所述参考图像帧的图像高斯金字塔，其中，所述高斯金字塔的不同层级对应所述参考图像帧的不同细节层级；
    对于每个层级的每个图像块，将所述图像块与各所述候选图像帧中的图像区域进行搜索对齐；
    将所述高斯金字塔的不同层级对应的搜索对齐结果进行逐级累加，以对齐所述图像块与所述候选图像帧中的相似图像块。
  5. 根据权利要求2所述的方法，所述将对齐后的所述图像块与各所述相似图像块进行融合，得到融合图像块，包括：
    计算对齐后的所述图像块与各所述相似图像块中，相同像素点的像素平均值；
    依据各所述像素点的像素平均值，生成融合图像块。
  6. 根据权利要求2所述的方法，所述将对齐后的所述图像块与各所述相似图像块进行融合，得到融合图像块，包括：
    分别确定对齐后的所述图像块的第一频域分布和各所述相似图像块的第二频域分布；
    分别确定各所述第二频域分布与所述第一频域分布的差异度；
    分别根据各所述差异度，确定各所述第二频域分布和所述第一频域分布对应的权重；
    根据各所述第二频域分布对应的权重、各所述第二频域分布、所述第一频域分布对应的权重以及所述第一频域分布，确定目标频域分布；
    将所述目标频域分布转化为融合图像块。
  7. 根据权利要求2所述的方法，所述将所述参考图像帧对应的各所述融合图像块进行拼接，得到降噪处理后的目标图像帧，包括：
    将所述参考图像帧对应的各所述融合图像块，沿融合图像块边缘进行拼接得到第一拼接图像；
    将所述参考图像帧对应的每个所述融合图像块，均与相邻融合图像块重叠预设面积，拼接得到第二拼接图像；
    基于所述第二拼接图像对所述第一拼接图像中的融合图像块拼接边缘处进行修正处理，得到降噪处理后的目标图像帧。
  8. 一种视频流降噪装置，所述装置包括：
    接收模块，被配置为接收视频流，其中所述视频流中包含多个图像帧；
    确定模块，被配置为确定所述视频流中当前待降噪处理的参考图像帧；
    搜索模块，被配置为从所述视频流中搜索所述参考图像帧的各候选图像帧，其中，各所述候选图像帧为在时域上位于所述参考图像帧前和/或所述参考图像帧后的预设数量的图像帧；
    划分模块，被配置为将所述参考图像帧划分为多个图像块；
    降噪模块，被配置为分别将各所述图像块与各所述候选图像帧中的相似图像块融合后进行拼接，得到降噪处理后的目标图像帧。
  9. 根据权利要求8所述的装置，所述降噪模块包括：
    对齐子模块，被配置为对于所述参考图像帧中的每个所述图像块，分别在各所述候选图像帧中，确定所述图像块的各相似图像块；将所述图像块与所述各相似图像块进行对齐；
    融合子模块，被配置为将对齐后的所述图像块与所述各相似图像块进行融合，得到融合图像块；
    拼接子模块，被配置为将所述图像帧对应的各所述融合图像块进行拼接，得到降噪处理后的目标图像帧。
  10. 根据权利要求9所述的装置，所述对齐子模块包括：
    偏移量确定单元，被配置为分别确定各所述相似图像块与所述图像块的相对偏移量；
    第一对齐单元，被配置为针对每个所述相似图像块，依据所述相似图像块对应的所述相对偏移量对所述相似图像块进行反向偏移，以将所述相似图像块与所述图像块对齐。
  11. 根据权利要求9所述的装置，所述对齐子模块包括：
    缩放单元，被配置为通过对所述参考图像帧缩小不同倍数，构建所述参考图像帧的图像高斯金字塔，其中，所述高斯金字塔的不同层级对应所述参考图像帧的不同细节层级；
    累加单元，被配置为对于每个层级的每个图像块，将所述图像块与各所述候选图像帧中的图像区域进行搜索对齐；将所述高斯金字塔的不同层级对应的搜索对齐结果进行逐级累加，以对齐所述图像块与所述候选图像帧中的相似图像块。
  12. 根据权利要求9所述的装置，所述融合子模块包括：
    均值计算单元，被配置为计算对齐后的所述图像块与各所述相似图像块中，相同像素点的像素平均值；
    融合单元，被配置为依据各所述像素点的像素平均值，生成融合图像块。
  13. 根据权利要求9所述的装置，所述融合子模块包括：
    频域分布确定单元，被配置为分别确定对齐后的所述图像块的第一频域分布和各所述相似图像块的第二频域分布；
    差异确定单元，被配置为分别确定各所述第二频域分布与所述第一频域分布的差异度；
    权重确定单元，被配置为分别根据各所述差异度，确定各所述第二频域分布和所述第一频域分布对应的权重；
    目标频域分布确定单元，被配置为根据各所述第二频域分布对应的权重、各所述第二频域分布、所述第一频域分布对应的权重以及所述第一频域分布，确定目标频域分布；
    转化单元，被配置为将所述目标频域分布转化为融合图像块。
  14. 根据权利要求9所述的装置，所述拼接子模块包括：
    第一拼接单元，被配置为将所述参考图像帧对应的各所述融合图像块，沿融合图像块边缘进行拼接得到第一拼接图像；
    第二拼接单元，被配置为将所述参考图像帧对应的每个所述融合图像块， 均与相邻融合图像块重叠预设面积，拼接得到第二拼接图像；
    修正单元，被配置为基于所述第二拼接图像对所述第一拼接图像中的融合图像块拼接边缘处进行修正处理，得到降噪处理后的目标图像帧。
  15. 一种电子设备，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为执行权利要求1-7中任一项所述的视频流降噪方法。
  16. 一种非临时性计算机可读存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行权利要求1-7中任一项所述的视频流降噪方法。
PCT/CN2019/096884 2018-08-31 2019-07-19 视频流降噪方法和装置、电子设备及存储介质 WO2020042826A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811014197.9A CN109246332A (zh) 2018-08-31 2018-08-31 视频流降噪方法和装置、电子设备及存储介质
CN201811014197.9 2018-08-31

Publications (1)

Publication Number Publication Date
WO2020042826A1 true WO2020042826A1 (zh) 2020-03-05

Family

ID=65059910

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/096884 WO2020042826A1 (zh) 2018-08-31 2019-07-19 视频流降噪方法和装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN109246332A (zh)
WO (1) WO2020042826A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113608511A (zh) * 2021-08-04 2021-11-05 大唐海南能源开发有限公司 一种基于物联网的智慧电厂管控系统及其方法

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109246332A (zh) * 2018-08-31 2019-01-18 北京达佳互联信息技术有限公司 视频流降噪方法和装置、电子设备及存储介质
CN109840491A (zh) * 2019-01-25 2019-06-04 平安科技（深圳）有限公司 视频流播放方法、系统、计算机装置及可读存储介质
CN110070511B (zh) * 2019-04-30 2022-01-28 北京市商汤科技开发有限公司 图像处理方法和装置、电子设备及存储介质
CN110570356B (zh) * 2019-09-18 2023-10-20 北京市商汤科技开发有限公司 图像处理方法和装置、电子设备及存储介质
CN110971786B (zh) * 2019-11-21 2022-04-19 维沃移动通信有限公司 一种拍摄方法及电子设备
CN111047516B (zh) * 2020-03-12 2020-07-03 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机设备和存储介质
CN111629262B (zh) * 2020-05-08 2022-04-12 Oppo广东移动通信有限公司 视频图像处理方法和装置、电子设备及存储介质
CN113766150B (zh) * 2021-08-31 2024-03-26 北京安酷智芯科技有限公司 一种降噪方法、电路系统、电子设备及计算机可读存储介质
CN116402695A (zh) * 2022-06-28 2023-07-07 上海玄戒技术有限公司 视频数据的处理方法、装置及存储介质
CN115841425B (zh) * 2022-07-21 2023-11-17 爱芯元智半导体(宁波)有限公司 视频降噪方法、装置、电子设备及计算机可读取存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105611181A (zh) * 2016-03-30 2016-05-25 努比亚技术有限公司 多帧拍摄图像合成装置和方法
CN107172322A (zh) * 2017-06-16 2017-09-15 北京飞识科技有限公司 一种视频降噪方法和装置
US9883083B2 (en) * 2009-06-05 2018-01-30 Cisco Technology, Inc. Processing prior temporally-matched frames in 3D-based video denoising
CN109246332A (zh) * 2018-08-31 2019-01-18 北京达佳互联信息技术有限公司 视频流降噪方法和装置、电子设备及存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI323605B (en) * 2006-05-12 2010-04-11 Realtek Semiconductor Corp Apparatus and method for reducing temporal noise
JP2009290827A (ja) * 2008-06-02 2009-12-10 Sony Corp 画像処理装置および画像処理方法
CN103945214B (zh) * 2013-01-23 2016-03-30 中兴通讯股份有限公司 终端侧时间域视频质量评价方法及装置
US9197947B2 (en) * 2013-03-14 2015-11-24 Arris Technology, Inc. Devices and methods for dynamic video processing
CN108122206A (zh) * 2016-11-29 2018-06-05 深圳市中兴微电子技术有限公司 一种低照度图像降噪方法及装置
CN107169939A (zh) * 2017-05-31 2017-09-15 广东欧珀移动通信有限公司 图像处理方法及相关产品
CN108447029A (zh) * 2018-02-12 2018-08-24 深圳创维－Rgb电子有限公司 一种视频序列的去噪方法、装置、服务器和存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9883083B2 (en) * 2009-06-05 2018-01-30 Cisco Technology, Inc. Processing prior temporally-matched frames in 3D-based video denoising
CN105611181A (zh) * 2016-03-30 2016-05-25 努比亚技术有限公司 多帧拍摄图像合成装置和方法
CN107172322A (zh) * 2017-06-16 2017-09-15 北京飞识科技有限公司 一种视频降噪方法和装置
CN109246332A (zh) * 2018-08-31 2019-01-18 北京达佳互联信息技术有限公司 视频流降噪方法和装置、电子设备及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113608511A (zh) * 2021-08-04 2021-11-05 大唐海南能源开发有限公司 一种基于物联网的智慧电厂管控系统及其方法

Also Published As

Publication number Publication date
CN109246332A (zh) 2019-01-18

Similar Documents

Publication Publication Date Title
WO2020042826A1 (zh) 视频流降噪方法和装置、电子设备及存储介质
US9674395B2 (en) Methods and apparatuses for generating photograph
CN106651955B (zh) 图片中目标物的定位方法及装置
WO2020135127A1 (zh) 一种行人识别方法及装置
JP6335289B2 (ja) 画像フィルタを生成する方法及び装置
WO2016192325A1 (zh) 视频文件的标识处理方法及装置
CN110557547B (zh) 镜头位置调整方法及装置
WO2017031901A1 (zh) 人脸识别方法、装置及终端
CN110619350B (zh) 图像检测方法、装置及存储介质
US20170064245A1 (en) Method, device, terminal device, and storage medium for video effect processing
CN109784164B (zh) 前景识别方法、装置、电子设备及存储介质
CN107944367B (zh) 人脸关键点检测方法及装置
CN108900903B (zh) 视频处理方法及装置、电子设备和存储介质
US11310443B2 (en) Video processing method, apparatus and storage medium
CN109509195B (zh) 前景处理方法、装置、电子设备及存储介质
CN112634160A (zh) 拍照方法及装置、终端、存储介质
KR102366995B1 (ko) 이미지 처리 모델의 훈련 방법, 장치 및 매체
CN110796012B (zh) 图像处理方法、装置、电子设备及可读存储介质
CN110674932A (zh) 一种二阶段卷积神经网络目标检测网络训练方法及装置
US9665925B2 (en) Method and terminal device for retargeting images
CN110728180B (zh) 图像处理方法、装置及存储介质
CN106469446B (zh) 深度图像的分割方法和分割装置
CN108596957B (zh) 物体跟踪方法及装置
CN114666490B (zh) 对焦方法、装置、电子设备和存储介质
CN113315903B (zh) 图像获取方法和装置、电子设备、存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19856401

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19856401

Country of ref document: EP

Kind code of ref document: A1