CN111127347A - Noise reduction method, terminal and storage medium - Google Patents

Noise reduction method, terminal and storage medium Download PDF

Info

Publication number
CN111127347A
CN111127347A CN201911253681.1A CN201911253681A CN111127347A CN 111127347 A CN111127347 A CN 111127347A CN 201911253681 A CN201911253681 A CN 201911253681A CN 111127347 A CN111127347 A CN 111127347A
Authority
CN
China
Prior art keywords
image
noise
value
noise reduction
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911253681.1A
Other languages
Chinese (zh)
Inventor
权威
马元蛟
罗俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN201911253681.1A priority Critical patent/CN111127347A/en
Publication of CN111127347A publication Critical patent/CN111127347A/en
Priority to PCT/CN2020/121645 priority patent/WO2021114868A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

The embodiment of the application discloses a noise reduction method, a terminal and a storage medium, wherein the noise reduction method comprises the following steps: performing registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; the previous frame of image is a previous image which is continuous with the current image in the video to be denoised; calculating a noise estimation value of a current image, and determining a fusion weight according to a first noise value and the noise estimation value of a previous frame of image; obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight; continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is the next image which is continuous with the current image in the video to be denoised.

Description

Noise reduction method, terminal and storage medium
Technical Field
The embodiment of the application relates to the technical field of image processing, in particular to a noise reduction method, a terminal and a storage medium.
Background
The multi-frame video denoising is realized based on a multi-frame denoising technology, namely after collecting multi-frame photos or images, different pixel points with noise point properties are found under different frame numbers, and a relatively clean and pure image is obtained through weighting synthesis.
However, for a scene with a local moving object, ghost images are easily generated by adopting the α fusion method for noise reduction processing, and the effect of video noise reduction is reduced.
Disclosure of Invention
The embodiment of the application provides a noise reduction method, a terminal and a storage medium, which can effectively improve the fusion noise reduction effect of multi-frame video images and improve the video quality when noise reduction processing is performed.
The technical scheme of the embodiment of the application is realized as follows:
in a first aspect, an embodiment of the present application provides a noise reduction method, where the method includes:
performing registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; the previous frame image is a previous image which is continuous with the current image in the video to be denoised;
calculating a noise estimation value of the current image, and determining a fusion weight according to a first noise value of the previous frame of image and the noise estimation value;
obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight;
continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is a subsequent image which is continuous with the current image in the video to be denoised.
In a second aspect, an embodiment of the present application provides a terminal, where the terminal includes: an acquisition unit, a calculation unit, a determination unit, a noise reduction unit,
the acquiring unit is used for carrying out registration processing on the current image by utilizing the first noise reduction image of the previous frame of image to obtain a registered image; the previous frame image is a previous image which is continuous with the current image in the video to be denoised;
the computing unit is used for computing a noise estimation value of the current image;
the determining unit is used for determining a fusion weight according to a first noise value of the previous frame image and the noise estimation value;
the obtaining unit is further configured to obtain a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight;
the denoising unit is used for continuing to perform denoising processing on the next frame of image according to the second denoising image and the second noise value until each frame of image in the video to be denoised is traversed; and the next frame image is a subsequent image which is continuous with the current image in the video to be denoised.
In a third aspect, an embodiment of the present application provides a terminal, where the terminal includes a processor and a memory storing instructions executable by the processor, and when the instructions are executed by the processor, the noise reduction method is implemented.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program is stored, and the program is applied to a terminal, and when being executed by a processor, the program implements the noise reduction method as described above.
The embodiment of the application provides a noise reduction method, a terminal and a storage medium, wherein the terminal carries out registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; the previous frame of image is a previous image which is continuous with the current image in the video to be denoised; calculating a noise estimation value of a current image, and determining a fusion weight according to a first noise value and the noise estimation value of a previous frame of image; obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight; continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is the next image which is continuous with the current image in the video to be denoised. That is to say, when the terminal performs noise reduction processing on the video to be noise reduced, the fusion weight can be determined by using the noise reduction image and the noise value after the noise reduction of the previous frame of image and combining with the noise estimation value of the current image, so that the fusion weight can be used to complete the fusion noise reduction processing on the current image, and the noise reduction image and the noise value after the noise reduction of the current image can be continuously used to continue the noise reduction on the next frame of image until each frame of image in the video to be noise reduced is traversed. That is to say, in the denoising method provided by the present application, the fusion weight is determined based on the noise value corresponding to the previous frame image and the noise value corresponding to the current image, so that when denoising is performed, the fusion denoising effect of the multi-frame video image can be effectively improved, and the video quality is improved.
Drawings
FIG. 1 is a first schematic flow chart of a noise reduction method;
FIG. 2 is a schematic diagram of a video to be denoised;
FIG. 3 is a schematic diagram of a pixel point;
FIG. 4 is a schematic diagram of a second implementation flow of the noise reduction method;
FIG. 5 is a third schematic flow chart of the implementation of the noise reduction method;
FIG. 6 is a block diagram of the structure of the fusion noise reduction process;
FIG. 7 is a schematic diagram of a fusion noise reduction process;
FIG. 8 is a first schematic diagram of the structure of the terminal;
fig. 9 is a schematic diagram of a terminal structure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant application and are not limiting of the application. It should be noted that, for the convenience of description, only the parts related to the related applications are shown in the drawings.
In the video field, when the ambient brightness is poor, the acquired image is accompanied by more noise, and if the brightness enhancement processing is performed on the original video through brightness enhancement, the noise in the image is correspondingly enhanced, so that the visual effect of the video is seriously affected. The general image noise reduction mode only aims at a single-frame image, has higher calculation complexity and has poorer noise reduction effect when being applied to a video image. The multi-frame video denoising is a common method for video denoising, and at present, the multi-frame video denoising specifically includes the following processes: the image registration technology is utilized to align the image content, then the pixels of the video are fused, and the noise intensity can be reduced in a multi-frame fusion mode because the noise signal of the image is mostly a random signal.
It can be understood that the multi-frame video denoising is realized based on a multi-frame denoising technology, that is, after acquiring a plurality of frames of pictures or images, different pixel points with noise properties are found at different frame numbers, and a cleaner and cleaner image is obtained through weighting and synthesizing. In general, when a terminal shoots, the number and the positions of noise points with a plurality of frames are calculated and screened, the positions of the noise points are replaced by the frames without the noise points, and a clean image is obtained through repeated weighting and replacement, wherein the finally imaged image is synthesized by images with a plurality of frames, so that the ghost of part of objects can be seen in some occasions, and the ghost can be ignored as long as the ghost of the image is not influenced.
However, in order to ensure that the overall brightness of the fused image does not change, an α fusion mode is mostly adopted, the method is a weighted summation mode, the selection of the fusion weight usually adopts fixed parameters or is selected according to the difference value of the fusion pixels, and specifically, the selection of the weight can directly determine the effect after noise reduction.
In order to overcome the defects, the fusion weight can be more accurately determined by utilizing the noise estimation value estimated by the preset noise model, so that fusion noise reduction can be performed by adopting a higher weight principle based on the pixel with lower noise, and the video noise reduction effect and the picture purity are improved. That is to say, when the terminal performs noise reduction processing on the video to be noise reduced, the fusion weight can be determined by using the noise reduction image and the noise value after the noise reduction of the previous frame of image and combining with the noise estimation value of the current image, so that the fusion weight can be used to complete the fusion noise reduction processing on the current image, and the noise reduction image and the noise value after the noise reduction of the current image can be continuously used to continue the noise reduction on the next frame of image until each frame of image in the video to be noise reduced is traversed. That is to say, in the denoising method provided by the present application, the fusion weight is determined based on the noise value corresponding to the previous frame image and the noise value corresponding to the current image, so that when denoising is performed, the fusion denoising effect of the multi-frame video image can be effectively improved, and the video quality is improved.
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
An embodiment of the present application provides a noise reduction method, where the noise reduction method is applied in a terminal, fig. 1 is a schematic view of an implementation flow of the noise reduction method, as shown in fig. 1, in an embodiment of the present application, a method for performing noise reduction by a terminal may include the following steps:
step 101, performing registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; and the previous frame image is a previous image which is continuous with the current image in the video to be denoised.
In the embodiment of the application, when the terminal performs noise reduction processing on the current image in the video to be denoised, the terminal may perform registration processing on the current image by using the first denoising image of the previous frame of image, so as to obtain the registered image.
It should be noted that, in the embodiments of the present application, the terminal may be any device having communication and storage functions, for example: tablet computers, mobile phones, electronic readers, remote controllers, Personal Computers (PCs), notebook computers, vehicle-mounted devices, network televisions, wearable devices, and the like.
It can be understood that, in the embodiment of the present application, the terminal may be configured with a shooting device, so that the video to be denoised may be acquired by using the shooting device. The photographing device may be an image sensor, among others. For example, the camera may be a Charge-coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS) configured as a terminal.
Further, in the embodiment of the application, the video to be noise-reduced may also be received or acquired by the terminal, and specifically, the terminal may receive the video to be noise-reduced from other devices, may download the video to be noise-reduced from the server, and may also read the video to be noise-reduced from the local memory.
It should be noted that, in the embodiment of the present application, the video to be denoised may include a plurality of consecutive frames of images. The multi-frame image may be a sequence of images arranged along a time axis.
Specifically, in the embodiment of the present application, fig. 2 is a schematic composition diagram of a video to be denoised, and as shown in fig. 2, the video to be denoised may include an image sequence of an image 1, an image 2, an image 3, an image 4 … … image (n-1), and an image n, which are respectively arranged along times t1, t2, t3, t4 … … t (n-1), and tn, where n is an integer greater than 5, the previous frame of image may be a previous image in the video to be denoised, which is consecutive to the current image, for example, the current image is an image 3, the previous frame of image is an image 2, and the next frame of image is an image 4.
Further, in the embodiment of the present application, when performing noise reduction on a video to be noise-reduced, the terminal may perform noise reduction on the image sequence in sequence according to a time axis sequence in the video to be noise-reduced, that is, the terminal performs noise reduction on each frame of image in sequence according to the sequence of image 1, image 2, image 3, image 4, image 5, and image 6. Therefore, when the terminal performs noise reduction on the current image of the video to be subjected to noise reduction, all the preamble images before the current image are subjected to noise reduction processing.
It can be understood that, in the embodiment of the present application, since the terminal performs noise reduction on the image sequence in sequence according to the time axis sequence in the video to be noise reduced, before performing noise reduction on the current image, the terminal may first obtain a noise-reduced image corresponding to the previous frame of image, that is, a first noise-reduced image, and may also obtain a noise value corresponding to the previous frame of image, that is, a first noise value, so that the current image may be further subjected to noise reduction processing according to the first noise-reduced image and the first noise value of the previous frame of image. That is, before the current image is registered by using the first noise-reduced image corresponding to the previous frame of image and the registered image is obtained, that is, before step 101, the terminal may read the first noise-reduced image and the first noise value of the previous frame of image.
It should be noted that, in the embodiment of the present application, when the terminal performs the registration processing on the current image by using the first noise-reduced image of the previous frame of image, the current image and the first noise-reduced image of the previous frame of image may be registered by using a current image registration technology. The terminal can perform registration processing according to a plurality of image registration methods, such as an optical flow method, a block search method, a feature matching method, and the like, and an image registration scheme with a good effect is adopted here, and a specific registration method is not limited.
It is to be understood that in the embodiments of the present application, the purpose of the registration process performed by the terminal on the current image and the first noise-reduced image is to align the image contents of the current image and the first noise-reduced image. That is, in the present application, the image content of the registered image obtained by the terminal registration corresponds to the image content of the first noise-reduced image.
And 102, calculating a noise estimation value of the current image, and determining a fusion weight according to the first noise value and the noise estimation value of the previous frame of image.
In the embodiment of the application, the terminal performs registration processing on the current image by using the first noise reduction image of the previous frame of image, after the registered image is obtained, the noise estimation value of the current image may be calculated first, and then the fusion weight is further determined according to the first noise value and the noise estimation value of the previous frame of image.
It should be noted that, in the embodiment of the present application, the first noise value of the previous frame image may be used to estimate the noise of the first noise-reduced image. That is, the first noise value represents the noise intensity of the first noise-reduced image, that is, represents the noise intensity after the noise reduction processing is performed on the previous frame image.
Further, in the embodiment of the present application, when the terminal calculates the noise estimation value of the current image, the noise estimation may be performed on the current image by using a preset noise model. The preset noise model is corresponding to a shooting device for shooting a video to be denoised, and therefore, the preset noise model can be used for determining the relation between a noise value and a signal of the shooting device. That is, in the present application, for a fixed photographing apparatus, a preset noise model may be correspondingly obtained or calculated.
It should be noted that, in the embodiment of the present application, when calculating the noise estimation value of the current image, the terminal may determine the signal strength corresponding to the current image, and then input the signal strength into the preset noise model, so that the noise estimation value may be output.
Further, in the embodiment of the present application, after calculating the noise estimation value of the current image, the terminal may continue to calculate the fusion weight according to the first noise value and the noise estimation value of the previous frame image, where the fusion weight may be used to perform fusion noise reduction processing on the current image.
Specifically, in the embodiment of the application, when determining the fusion weight according to the first noise value and the noise estimation value of the previous frame image, the terminal may first calculate to obtain the fusion parameter by using the first noise value and the noise estimation value, and then further determine the fusion weight according to the fusion parameter.
Further, in the embodiment of the present application, when the terminal calculates the fusion weight, the terminal further needs to compare pixel values of the first noise-reduced image and the registered image, and then calculate the fusion weight according to a comparison result obtained after the comparison.
Specifically, in the present application, when the terminal compares the pixel values of the first noise-reduced image and the registered image, the same pixel coordinate, that is, the first pixel coordinate, may be selected first, and the first pixel and the second pixel on the first pixel coordinate are determined in the first noise-reduced image and the registered image, respectively. Fig. 3 is a schematic diagram of a pixel point, and as shown in fig. 3, the terminal may determine a pixel point a corresponding to a first pixel coordinate in the first noise-reduced image as a first pixel, and at the same time, the terminal may determine a pixel point B corresponding to the first pixel coordinate in the registered image as a second pixel.
It can be understood that, in the embodiment of the present application, since the terminal performs the image registration processing on the current image by using the first noise-reduced image, so that the registered image and the first noise-reduced image are aligned with each other, for the same pixel coordinate, the terminal may determine two different pixels corresponding to each other in the first noise-reduced image and the registered image, respectively.
Further, in the embodiment of the present application, after the terminal determines the first pixel and the second pixel from the first noise-reduced image and the registered image, respectively, the terminal may continue to acquire the first pixel value of the first pixel and the second pixel value of the second pixel, so that the comparison of the pixel values of the first noise-reduced image and the registered image may be further implemented according to the first pixel value and the second pixel value.
It should be noted that, in the embodiment of the present application, when the terminal compares the pixel values of the first noise-reduced image and the registered image based on the first pixel value and the second pixel value, the terminal may calculate a difference parameter between the two pixel values, where the difference parameter may be an absolute value of a difference between the first pixel value and the second pixel value.
That is to say, in the embodiment of the present application, before determining the fusion weight according to the fusion parameter, the terminal needs to obtain a first pixel value of the first pixel and a second pixel value of the second pixel from the first noise-reduced image and the registered image based on the first pixel coordinate, then when determining the fusion weight, a difference parameter between the first pixel value and the second pixel value may be calculated, and then the difference parameter and the fusion parameter are input to the preset weight model, so that the fusion weight may be obtained.
It is understood that, in the embodiment of the present application, the preset weight model may be a specific function stored in the terminal and characterizing the fusion weight and the pixel difference value. For the preset weight model, a principle that the smaller the pixel difference is, the larger the fusion weight is needs to be followed, that is, the difference parameter and the fusion weight are in inverse proportion.
In the embodiment of the present application, the fusion weight is a natural number greater than or equal to 0 and less than or equal to 1.
And 103, acquiring a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight.
In the embodiment of the application, after determining the fusion weight according to the first noise value and the noise estimation value of the previous frame of image, the terminal may obtain the second noise-reduced image and the second noise value corresponding to the current image respectively by using the fusion weight.
It is to be understood that, in the embodiment of the present application, the second noise-reduced image may be an image obtained after performing the fusion noise reduction processing on the current image, and the second noise value may be used to estimate the noise of the second noise-reduced image. That is, the second noise value represents the noise intensity of the second noise-reduced image, that is, represents the noise intensity after the noise reduction processing is performed on the current image.
Further, in the embodiment of the application, when the terminal obtains the second noise-reduced image corresponding to the current image by using the fusion weight, the registered image may be specifically subjected to fusion noise-reduction processing according to the fusion weight, so that the second noise-reduced image may be obtained.
Specifically, in the embodiment of the application, when the terminal performs the fusion denoising processing on the registered image according to the fusion weight to obtain the second denoised image, the fusion pixel value corresponding to the first pixel coordinate may be determined according to the fusion weight, the first pixel value and the second pixel value, and then other pixel coordinates except the first coordinate in the registered image and the first denoised image may be traversed to obtain other pixel values corresponding to the other pixel coordinates, so that the second denoised image may be generated based on the fusion pixel value and the other pixel values.
That is to say, in the embodiment of the present application, the terminal may first obtain a fused pixel value corresponding to a pixel coordinate by fusing, based on one pixel coordinate, pixel value difference parameters of two corresponding pixels on the first noise-reduced image and the registered image corresponding to the pixel coordinate, and then determine the pixel value difference parameters of corresponding pixels on other pixel coordinates on the first noise-reduced image and the registered image by using the same fusion method, so as to obtain other fused pixel values corresponding to other pixel coordinates, and finally generate a fused noise-reduced image corresponding to the current image, that is, a second noise-reduced image, by using all the fused pixel values corresponding to all the pixel coordinates, so as to complete noise reduction processing of the current image.
As can be appreciated. In the embodiment of the application, the terminal performs registration processing on the current image by using the first noise-reduced image, so that the first noise-reduced image and the registered image have mutually aligned image contents, and therefore, the second noise-reduced image obtained based on the first noise-reduced image and the registered image also has mutually aligned image contents with the two images.
Further, in the embodiment of the application, the terminal obtains the second noise value corresponding to the current image by using the fusion weight, and specifically, the second noise value may be generated according to the fusion weight, the first noise value, and the noise estimation value. That is, in the present application, the noise estimation value is used to determine the noise intensity of the current image, and the second noise value is used to determine the noise intensity of the second noise-reduced image obtained by reducing the noise of the current image. Therefore, after the fusion noise reduction of the current image is completed, the second noise value can be selected as the noise reduction result and input into the noise reduction processing flow of the next frame image. Meanwhile, the second noise reduction image is also required to be used as a noise reduction result, and the noise reduction processing of the next frame image is performed.
Step 104, continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is the next image which is continuous with the current image in the video to be denoised.
In the embodiment of the application, after the terminal obtains the second noise-reduced image and the second noise value corresponding to the current image by using the fusion weight, that is, after the fusion noise reduction of the current image is completed according to the above steps 101 to 103, the terminal may further perform noise reduction processing on the next frame image according to the second noise-reduced image and the second noise value until each frame image of the video to be subjected to noise reduction is traversed, so as to complete the noise reduction processing in the video to be subjected to noise reduction.
It can be understood that, in the embodiment of the present application, after the terminal completes the fusion denoising processing on the current image and obtains the second denoised image and the second noise value, the terminal may continue to perform the registration processing on the next frame image by using the second denoised image to obtain the registered image; then calculating a noise estimation value of the next frame of image, and determining a fusion weight according to the second noise value and the noise estimation value; and finally, obtaining a third noise reduction image and a third noise value corresponding to the next frame image by using the fusion weight, completing the fusion noise reduction processing of the next frame image, taking the third noise reduction image and the third noise value as the noise reduction result of the next frame image, and continuing inputting the value in the next fusion noise reduction processing flow until the noise reduction processing of the video to be subjected to noise reduction is completed.
That is to say, in the embodiment of the present application, after traversing each frame image of the video to be denoised by the terminal according to the method in the above step 101 to step 104, all the denoised images corresponding to all the frame images may be obtained, and then the terminal may continue to generate the denoised video based on all the denoised images along the time axis.
Fig. 4 is a schematic diagram of an implementation flow of the noise reduction method, as shown in fig. 4, after the terminal obtains the second noise reduction image and the second noise value corresponding to the current image by using the fusion weight, that is, after step 103, the method for performing noise reduction processing by the terminal may further include the following steps:
and 105, storing the second noise reduction image, and updating the first noise value by using the second noise value.
That is to say, in the present application, after the terminal completes the fusion denoising processing on each frame of image and obtains the denoised image and the noise value, the terminal may store the denoised image for the subsequent generation of the denoised video, and at the same time, the terminal may directly update the noise value corresponding to the previous frame by using the new noise value. For example, after the terminal finishes the noise reduction processing on the current image and obtains the second noise reduction image and the second noise value, the second noise reduction image may be stored, and at the same time, the terminal may update the first noise value with the second noise value.
In an embodiment of the present application, further, fig. 5 is a schematic view of an implementation flow of a noise reduction method, as shown in fig. 5, the method for the terminal to perform noise reduction processing on the next frame image according to the second noise reduction image and the second noise value until each frame image in the video to be noise reduced is traversed, that is, before step 104, may further include the following steps:
and step 106, if the current image is the first frame image of the video to be denoised, determining a second noise value according to a preset noise model, and determining the current image as a second denoised image.
That is to say, for a first frame video in the video to be denoised, the terminal may directly use the first frame video as a second denoised video after denoising, and at the same time, determine a second noise value according to a preset noise model, and then input the second denoised image and the second noise value into a denoising process of a next frame image, so as to complete denoising processing of the second frame image in the video to be denoised.
In an embodiment of the application, based on the denoising method proposed in the above steps 101 to 106, when the terminal performs denoising processing on the video to be denoised, the terminal may estimate noise of the current image by using a preset noise model, then further determine a fusion weight for fusion denoising according to the noise estimation value and the first noise value degree of the previous frame image, and perform fusion denoising on the registered image subjected to registration processing on the first denoised image based on the previous frame image by using the fusion weight, so as to obtain a second denoised image and a second noise value of the current image, and further, the terminal may further continue to perform fusion denoising on the next frame image by using the second denoised image and the second noise value until the denoising processing on the video to be denoised is completed. The fusion weight can be more accurately determined by utilizing the noise estimation value estimated by the preset noise model, so that fusion noise reduction can be performed by adopting a higher weight principle based on pixels with lower noise, and the video noise reduction effect and the picture purity are improved.
According to the noise reduction method provided by the embodiment of the application, a terminal carries out registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; the previous frame of image is a previous image which is continuous with the current image in the video to be denoised; calculating a noise estimation value of a current image, and determining a fusion weight according to a first noise value and the noise estimation value of a previous frame of image; obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight; continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is the next image which is continuous with the current image in the video to be denoised. That is to say, when the terminal performs noise reduction processing on the video to be noise reduced, the fusion weight can be determined by using the noise reduction image and the noise value after the noise reduction of the previous frame of image and combining with the noise estimation value of the current image, so that the fusion weight can be used to complete the fusion noise reduction processing on the current image, and the noise reduction image and the noise value after the noise reduction of the current image can be continuously used to continue the noise reduction on the next frame of image until each frame of image in the video to be noise reduced is traversed. That is to say, in the denoising method provided by the present application, the fusion weight is determined based on the noise value corresponding to the previous frame image and the noise value corresponding to the current image, so that when denoising is performed, the fusion denoising effect of the multi-frame video image can be effectively improved, and the video quality is improved.
Based on the above embodiment, in another embodiment of the present application, the terminal may perform noise estimation on the current image by using a preset noise model, so as to calculate and obtain a noise estimation value of the current image. The preset noise model is corresponding to a shooting device for shooting a video to be denoised, specifically, for a fixed shooting device, a corresponding preset noise model may be obtained, and for example, in the present application, the preset noise model conforms to a normal distribution and may be represented as follows:
y～N(μ＝x，σ2＝λread+λshotX) (1)
wherein X is the intensity of the real signal, and for a specific shooting device, the noise can be mainly divided into a read noise and a shot noise. With reference to the following formula, the model parameter λ corresponding to the noise is readreadModel parameter lambda corresponding to shot noiseshotCan be measured in practice to yield:
Figure BDA0002309717420000091
λshot＝gaσa(3)
wherein gd is the digital gain of the camera signal, and ga is the analog gain of the camera signal.
Further, in the present application, the preset noise model may estimate the noise of the current image, and the preset noise model may be calculated by photographing a chart in a dark room, or directly obtained from a manufacturer of the photographing apparatus.
It is understood that, in the embodiment of the present application, the terminal may determine the noise intensity by using the standard deviation (standard deviation) of the signal of the current image, and therefore, for a pixel of the current image with the signal intensity X, the signal intensity X may be input into the preset noise model of the above formula (1), and the standard deviation corresponding to the pixel may be obtained through calculation, so that the noise estimation value of the current image may be further obtained. That is, since the preset noise model may be used to determine a relationship between a noise value and a signal of a shooting device that shoots a video to be denoised, when calculating the noise estimation value, the terminal may first determine a signal strength corresponding to the current image, so as to obtain the noise estimation value based on the preset noise model calculation.
In the embodiment of the present application, further, when the terminal fuses the first noise-reduced image and the registered image, the terminal may determine a fused pixel value corresponding to the first pixel coordinate according to the fused weight, the first pixel value, and the second pixel value, for example, as a pixel point a and a pixel point B in fig. 3, the terminal may determine the fused pixel value corresponding to the first pixel coordinate based on the pixel value PAAnd a pixel value PBFusing the pixel point A and the pixel point B, so as to obtain a pixel value P of a fused pixel value, which is specifically as follows:
P＝w×PA+(1-w)×PB(4)
wherein, PAAnd PBThe pixel values of the pixel point A and the pixel point B are respectively, and w is the fusion weight.
It should be noted that, in the embodiment of the present application, the fusion weight may be determined by a difference parameter of pixel values of two fused pixels, specifically, in the present application, when the terminal calculates the fusion weight, the terminal needs to compare the pixel values of the first noise-reduced image and the registered image. Specifically, when comparing the pixel values of the first noise-reduced image and the registered image, for the first noise-reduced image and the registered image, the absolute value of the difference between the pixel point a and the pixel point B of the first pixel coordinate and the corresponding pixel value may be used as a difference parameter, that is, the absolute value of | P isA-PBL as PAAnd PBThe difference parameter between them.
For example, in the present application, the terminal may input a difference parameter between the first pixel value and the second pixel value and the fusion parameter into a preset weight model to calculate the fusion weight. The preset weight model may be a specific function stored in the terminal and representing the fusion weight and the pixel difference value. For the preset weight model, a principle that the smaller the pixel difference value is, the larger the fusion weight is needs to be followed, for example, the terminal may calculate the fusion weight w by the following formula:
Figure BDA0002309717420000092
where λ is the fusion parameter and d is the difference parameter between the first pixel value and the second pixel value, illustratively, d ═ PA-PBL. It can be seen that the difference parameter and the fusion weight are inversely proportional. The terminal can adjust the changing relationship between d and w through sigma.
It should be noted that in the examples of the present application, w must be in the range of [0, 1 ].
For example, in the embodiment of the present application, the fusion parameter may be obtained by calculating the first noise value and the noise estimation value, and for example, in the present application, the terminal may calculate the fusion parameter λ by using the following formula:
Figure BDA0002309717420000101
wherein S isAIs the standard deviation of the signal intensity of the pixel A, that is, SAIs the noise value, S, corresponding to the first pixel in the first noise-reduced imageBIs the standard deviation of the signal intensity of the pixel B, that is, SBA noise value corresponding to a second pixel in the registered image.
That is, in the present application, the fusion parameter λ may be obtained by calculating a first noise value of the first noise-reduced image and a noise estimation value of the registered image.
In the embodiment of the application, further, when the terminal obtains the second noise value corresponding to the current image by using the fusion weight, the terminal may calculate to obtain the second noise value according to the fusion weight, the first noise value and the noise estimation value.
For example, in the present application, the standard deviation of the pixel may be used to characterize the noise value S of the pixel in the second noise-reduced image after the fusion noise reduction of the current image, which is calculated as follows:
Figure BDA0002309717420000102
therefore, in the application, the terminal may first obtain a fused pixel value corresponding to the pixel coordinate by fusing pixel value difference parameters of two corresponding pixels on the first noise-reduced image and the registered image corresponding to the pixel coordinate based on one pixel coordinate, and then determine the pixel value difference parameters of corresponding pixels on other pixel coordinates on the first noise-reduced image and the registered image by using the same fusion method to obtain other fused pixel values corresponding to other pixel coordinates, and finally generate a fused noise-reduced image corresponding to the current image, that is, a second noise-reduced image, by using all the fused pixel values corresponding to all the pixel coordinates, so as to complete the noise reduction processing of the current image.
According to the noise reduction method provided by the embodiment of the application, a terminal carries out registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; the previous frame of image is a previous image which is continuous with the current image in the video to be denoised; calculating a noise estimation value of a current image, and determining a fusion weight according to a first noise value and the noise estimation value of a previous frame of image; obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight; continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is the next image which is continuous with the current image in the video to be denoised. That is to say, when the terminal performs noise reduction processing on the video to be noise reduced, the fusion weight can be determined by using the noise reduction image and the noise value after the noise reduction of the previous frame of image and combining with the noise estimation value of the current image, so that the fusion weight can be used to complete the fusion noise reduction processing on the current image, and the noise reduction image and the noise value after the noise reduction of the current image can be continuously used to continue the noise reduction on the next frame of image until each frame of image in the video to be noise reduced is traversed. That is to say, in the denoising method provided by the present application, the fusion weight is determined based on the noise value corresponding to the previous frame image and the noise value corresponding to the current image, so that when denoising is performed, the fusion denoising effect of the multi-frame video image can be effectively improved, and the video quality is improved.
Based on the foregoing embodiment, in another embodiment of the present application, fig. 6 is a structural block diagram of fusion noise reduction processing, and as shown in fig. 6, when a terminal performs noise reduction on a current image in a video to be noise reduced, a noise reduction result of a previous frame of image, including a first noise reduction image and a first noise value corresponding to the previous frame of image, may be obtained first, and then image registration processing is performed on the current image by using the first noise reduction image (step 201), so that a registered image corresponding to the current image may be obtained, meanwhile, the terminal may perform noise estimation on the current image based on a preset noise model (step 202), obtain a noise estimation value, and further calculate and obtain a fusion weight based on the noise estimation value in combination with the first noise value of the first noise reduction image.
It should be noted that, when performing the calculation of the fusion weight, the noise estimation value and the first noise value may be used to calculate and obtain the fusion parameter (step 203), and meanwhile, the first pixel and the second pixel on the same pixel coordinate may be determined in the first noise-reduced image and the registered image respectively, and the pixel value of the first pixel and the second pixel may be compared (step 204) to obtain the difference parameter of the pixel value, so that the difference parameter and the fusion parameter may be input to the preset weight model to perform the weight calculation (step 205), and the fusion weight may be obtained.
Further, in the embodiment of the present application, after the terminal calculates and obtains the fusion weight, on one hand, the registered image may be subjected to fusion denoising processing according to the fusion weight (step 206), so as to obtain a second denoised image; on the other hand, a noise calculation may be performed based on the fusion weight, the first noise value, and the noise estimation value (step 207), and a second noise value may be generated. So that a noise reduction result of the current image, i.e., the second noise-reduced image and the second noise value, can be obtained.
It can be understood that, in the present application, after the terminal completes the fusion denoising processing on the current image and obtains the second denoised image and the second noise value, the terminal may continue to complete the fusion denoising processing on the next frame image by using the second denoised and the second noise value until each frame image of the video to be denoised is traversed, may obtain all denoised images corresponding to all frame images, and then may continue to generate the video after denoising based on all the denoised images along the sequence of the time axis.
In an embodiment of the present application, further, fig. 7 is a schematic diagram of merging denoising processing, and as shown in fig. 7, the denoising method provided in the present application may arrange multiple frames of images acquired by a shooting device along a time axis to obtain an image sequence, and select a denoising result of a previous frame of image, that is, a first denoising image and a first noise value, when denoising processing is performed. After the first noise-reduced image and the current image are subjected to image registration (step 301), a registered image is obtained, then fusion noise reduction is carried out on the registered image by using the first noise value and the first noise-reduced image (step 302), and a second noise-reduced image and a second noise value of the current image are obtained.
It is understood that, in the present application, after obtaining the second noise reduction image and the second noise value of the current image, the terminal may store the second noise reduction map while updating the first noise value with the second noise value.
Further, in the embodiment of the present application, the second noise reduction image and the second noise value will participate in the noise reduction processing as an input of the noise reduction processing of the next frame image. Iteration is carried out in such a mode until the whole video to be denoised is processed completely, and the corresponding denoised video is obtained.
Therefore, in the application, when the terminal performs noise reduction processing on the video to be subjected to noise reduction, the noise of the current image can be estimated by using a preset noise model, then the determination is further performed according to the noise estimation value and the fusion weight of the first noise value fusion noise reduction of the previous frame of image, and the fusion noise reduction is performed on the registered image subjected to registration processing on the basis of the first noise reduction image of the previous frame of image by using the fusion weight, so as to obtain the second noise reduction image and the second noise value of the current image. The fusion weight can be more accurately determined by utilizing the noise estimation value estimated by the preset noise model, so that fusion noise reduction can be performed by adopting a higher weight principle based on pixels with lower noise, and the video noise reduction effect and the picture purity are improved.
According to the noise reduction method provided by the embodiment of the application, a terminal carries out registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; the previous frame of image is a previous image which is continuous with the current image in the video to be denoised; calculating a noise estimation value of a current image, and determining a fusion weight according to a first noise value and the noise estimation value of a previous frame of image; obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight; continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is the next image which is continuous with the current image in the video to be denoised. That is to say, when the terminal performs noise reduction processing on the video to be noise reduced, the fusion weight can be determined by using the noise reduction image and the noise value after the noise reduction of the previous frame of image and combining with the noise estimation value of the current image, so that the fusion weight can be used to complete the fusion noise reduction processing on the current image, and the noise reduction image and the noise value after the noise reduction of the current image can be continuously used to continue the noise reduction on the next frame of image until each frame of image in the video to be noise reduced is traversed. That is to say, in the denoising method provided by the present application, the fusion weight is determined based on the noise value corresponding to the previous frame image and the noise value corresponding to the current image, so that when denoising is performed, the fusion denoising effect of the multi-frame video image can be effectively improved, and the video quality is improved.
Based on the foregoing embodiments, in yet another embodiment of the present application, fig. 8 is a schematic diagram of a composition structure of a terminal, and as shown in fig. 8, a terminal 10 according to an embodiment of the present application may include an obtaining unit 11, a calculating unit 12, a determining unit 13, a denoising unit 14, a storage unit 15, and an updating unit 16.
The acquiring unit 11 is configured to perform registration processing on a current image by using a first noise reduction image of a previous frame of image, and obtain a registered image; the previous frame image is a previous image which is continuous with the current image in the video to be denoised;
the calculating unit 12 is configured to calculate a noise estimation value of the current image;
the determining unit 13 is configured to determine a fusion weight according to a first noise value of the previous frame of image and the noise estimation value;
the obtaining unit 11 is further configured to obtain a second noise-reduced image and a second noise value corresponding to the current image by using the fusion weight;
the denoising unit 14 is configured to continue denoising the next frame of image according to the second denoising image and the second noise value until each frame of image in the video to be denoised is traversed; and the next frame image is a subsequent image which is continuous with the current image in the video to be denoised.
Further, in the embodiment of the present application, the calculating unit 12 is specifically configured to determine a signal intensity corresponding to the current image; inputting the signal intensity into a preset noise model, and outputting the noise estimation value; the preset noise model is used for determining the relation between a noise value and a signal of a shooting device for shooting the video to be denoised.
Further, in an embodiment of the present application, the determining unit 13 is specifically configured to calculate and obtain a fusion parameter based on the first noise value and the noise estimation value; and determining the fusion weight according to the fusion parameter.
Further, in an embodiment of the present application, the determining unit 13 is further configured to determine a first pixel in the first noise-reduced image and a second pixel in the registered image before determining the fusion weight according to the fusion parameter; the first pixel and the second pixel are two pixels corresponding to a first pixel coordinate;
the obtaining unit 11 is further configured to obtain a first pixel value of the first pixel, and obtain a second pixel value of the second pixel.
Further, in an embodiment of the present application, the determining unit 13 is further specifically configured to calculate a difference parameter between the first pixel value and the second pixel value; and inputting the difference parameter and the fusion parameter into a preset weight model, and outputting the fusion weight.
Further, in an embodiment of the present application, the obtaining unit 11 is specifically configured to perform fusion denoising processing on the registered image according to the fusion weight, so as to obtain the second denoised image.
Further, in an embodiment of the present application, the obtaining unit 11 is further specifically configured to determine a fusion pixel value corresponding to the first pixel coordinate according to the fusion weight, the first pixel value, and the second pixel value; traversing other pixel coordinates in the registered image and the first noise reduction image to obtain other pixel values corresponding to the other pixel coordinates; generating the second noise-reduced image based on the fused pixel value and the other pixel values.
Further, in an embodiment of the present application, the obtaining unit 11 is further specifically configured to generate the second noise value according to the fusion weight, the first noise value, and the noise estimation value.
Further, in an embodiment of the present application, the storage unit 15 is configured to store a second noise-reduced image after obtaining the second noise-reduced image and the second noise value corresponding to the current image by using the fusion weight.
Further, in an embodiment of the present application, the updating unit 16 is further configured to update the first noise value with a second noise value after obtaining the second noise-reduced image and the second noise value corresponding to the current image by using the fusion weight.
Further, in an embodiment of the present application, the obtaining unit 11 is further configured to perform registration processing on a current image by using a first noise-reduced image corresponding to an image of a previous frame, and read the first noise-reduced image and the first noise value before obtaining a registered image.
Further, in an embodiment of the present application, the determining unit 13 is further configured to continue to perform noise reduction processing on a next frame of image according to the second noise-reduced image and the second noise value until each frame of image in the video to be noise-reduced is traversed, and if the current image is the first frame of image of the video to be noise-reduced, determine the second noise value according to the preset noise model, and determine the current image as the second noise-reduced image.
Fig. 9 is a schematic diagram of a composition structure of a terminal, and as shown in fig. 9, the terminal 10 according to the embodiment of the present application may further include a processor 17, a memory 18 storing executable instructions of the processor 17, and further, the terminal 10 may further include a communication interface 19, and a bus 110 for connecting the processor 17, the memory 18, and the communication interface 19.
In an embodiment of the present application, the Processor 17 may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a ProgRAMmable Logic Device (PLD), a Field ProgRAMmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understood that the electronic devices for implementing the above processor functions may be other devices, and the embodiments of the present application are not limited in particular. The terminal 10 may further comprise a memory 18, which memory 18 may be connected to the processor 17, wherein the memory 18 is adapted to store executable program code comprising computer operating instructions, and wherein the memory 18 may comprise a high speed RAM memory and may further comprise a non-volatile memory, such as at least two disk memories.
In the embodiment of the present application, the bus 110 is used to connect the communication interface 19, the processor 17, and the memory 18 and the intercommunication among these devices.
In an embodiment of the present application, the memory 18 is used for storing instructions and data.
Further, in an embodiment of the present application, the processor 17 is configured to perform registration processing on the current image by using the first noise-reduced image of the previous frame of image, so as to obtain a registered image; the previous frame image is a previous image which is continuous with the current image in the video to be denoised; calculating a noise estimation value of the current image, and determining a fusion weight according to a first noise value of the previous frame of image and the noise estimation value; obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight; continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is a subsequent image which is continuous with the current image in the video to be denoised.
In practical applications, the Memory 18 may be a volatile Memory (volatile Memory), such as a Random-Access Memory (RAM); or a non-volatile Memory (non-volatile Memory), such as a Read-Only Memory (ROM), a flash Memory (flash Memory), a Hard disk (Hard disk Drive, HDD) or a Solid-State Drive (SSD); or a combination of the above types of memories and provides instructions and data to the processor 17.
In addition, each functional module in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware or a form of a software functional module.
Based on the understanding that the technical solution of the present embodiment essentially or a part contributing to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium, and include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the method of the present embodiment. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
According to the terminal provided by the embodiment of the application, the terminal carries out registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; the previous frame of image is a previous image which is continuous with the current image in the video to be denoised; calculating a noise estimation value of a current image, and determining a fusion weight according to a first noise value and the noise estimation value of a previous frame of image; obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight; continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is the next image which is continuous with the current image in the video to be denoised. That is to say, when the terminal performs noise reduction processing on the video to be noise reduced, the fusion weight can be determined by using the noise reduction image and the noise value after the noise reduction of the previous frame of image and combining with the noise estimation value of the current image, so that the fusion weight can be used to complete the fusion noise reduction processing on the current image, and the noise reduction image and the noise value after the noise reduction of the current image can be continuously used to continue the noise reduction on the next frame of image until each frame of image in the video to be noise reduced is traversed. That is to say, in the denoising method provided by the present application, the fusion weight is determined based on the noise value corresponding to the previous frame image and the noise value corresponding to the current image, so that when denoising is performed, the fusion denoising effect of the multi-frame video image can be effectively improved, and the video quality is improved.
An embodiment of the present application provides a computer-readable storage medium, on which a program is stored, which when executed by a processor implements the noise reduction method as described above.
Specifically, the program instructions corresponding to a noise reduction method in the present embodiment may be stored on a storage medium such as an optical disc, a hard disc, a usb disk, etc., and when the program instructions corresponding to a noise reduction method in the storage medium are read or executed by an electronic device, the method includes the following steps:
performing registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; the previous frame image is a previous image which is continuous with the current image in the video to be denoised;
calculating a noise estimation value of the current image, and determining a fusion weight according to a first noise value of the previous frame of image and the noise estimation value;
obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight;
continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is a subsequent image which is continuous with the current image in the video to be denoised.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of implementations of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart block or blocks and/or flowchart block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks in the flowchart and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application.

Claims (14)

1. A method of noise reduction, the method comprising:
performing registration processing on a current image by using a first noise reduction image of a previous frame of image to obtain a registered image; the previous frame image is a previous image which is continuous with the current image in the video to be denoised;
calculating a noise estimation value of the current image, and determining a fusion weight according to a first noise value of the previous frame of image and the noise estimation value;
obtaining a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight;
continuing to perform noise reduction processing on the next frame of image according to the second noise reduction image and the second noise value until each frame of image in the video to be subjected to noise reduction is traversed; and the next frame image is a subsequent image which is continuous with the current image in the video to be denoised.
2. The method of claim 1, wherein the calculating the noise estimate for the current image comprises:
determining the signal intensity corresponding to the current image;
inputting the signal intensity into a preset noise model, and outputting the noise estimation value; the preset noise model is used for determining the relation between a noise value and a signal of a shooting device for shooting the video to be denoised.
3. The method of claim 2, wherein determining the fusion weight according to the first noise value of the previous frame of image and the noise estimation value comprises:
calculating to obtain a fusion parameter based on the first noise value and the noise estimation value;
and determining the fusion weight according to the fusion parameter.
4. The method of claim 3, wherein prior to determining the fusion weights based on the fusion parameters, the method further comprises:
determining a first pixel in the first noise-reduced image and a second pixel in the registered image; the first pixel and the second pixel are two pixels corresponding to a first pixel coordinate;
a first pixel value of the first pixel is obtained, and a second pixel value of the second pixel is obtained.
5. The method of claim 4, wherein determining the fusion weight according to the fusion parameter comprises:
calculating a difference parameter between the first pixel value and the second pixel value;
and inputting the difference parameter and the fusion parameter into a preset weight model, and outputting the fusion weight.
6. The method according to claim 4 or 5, wherein the obtaining of the second noise-reduced image corresponding to the current image by using the fusion weight comprises:
and performing fusion noise reduction processing on the registered image according to the fusion weight to obtain the second noise reduction image.
7. The method according to claim 6, wherein the performing the fusion denoising processing on the registered image according to the fusion weight to obtain the second denoised image comprises:
determining a fusion pixel value corresponding to the first pixel coordinate according to the fusion weight, the first pixel value and the second pixel value;
traversing other pixel coordinates in the registered image and the first noise reduction image to obtain other pixel values corresponding to the other pixel coordinates;
generating the second noise-reduced image based on the fused pixel value and the other pixel values.
8. The method according to claim 3, wherein the obtaining a second noise value corresponding to the current image by using the fusion weight comprises:
and generating the second noise value according to the fusion weight, the first noise value and the noise estimation value.
9. The method according to claim 1, wherein after obtaining the second noise-reduced image and the second noise value corresponding to the current image by using the fusion weight, the method further comprises:
and storing the second noise reduction image, and updating the first noise value by using the second noise value.
10. The method of claim 1, wherein before the current image is registered by using the first noise-reduced image corresponding to the previous frame of image and the registered image is obtained, the method further comprises:
reading the first noise reduced image and the first noise value.
11. The method according to claim 1, wherein the denoising processing of the next frame of image according to the second denoised image and the second noise value is continued until each frame of image in the video to be denoised is traversed, and the method further comprises:
and if the current image is the first frame image of the video to be denoised, determining the second noise value according to the preset noise model, and determining the current image as the second denoised image.
12. A terminal, characterized in that the terminal comprises: an acquisition unit, a calculation unit, a determination unit, a noise reduction unit,
the acquiring unit is used for carrying out registration processing on the current image by utilizing the first noise reduction image of the previous frame of image to obtain a registered image; the previous frame image is a previous image which is continuous with the current image in the video to be denoised;
the computing unit is used for computing a noise estimation value of the current image;
the determining unit is used for determining a fusion weight according to a first noise value of the previous frame image and the noise estimation value;
the obtaining unit is further configured to obtain a second noise reduction image and a second noise value corresponding to the current image by using the fusion weight;
the denoising unit is used for continuing to perform denoising processing on the next frame of image according to the second denoising image and the second noise value until each frame of image in the video to be denoised is traversed; and the next frame image is a subsequent image which is continuous with the current image in the video to be denoised.
13. A terminal, characterized in that the terminal comprises a processor, a memory storing instructions executable by the processor, which instructions, when executed by the processor, implement the method according to any of claims 1-11.
14. A computer-readable storage medium, on which a program is stored, for use in a terminal, characterized in that the program, when executed by a processor, implements the method according to any one of claims 1-11.
CN201911253681.1A 2019-12-09 2019-12-09 Noise reduction method, terminal and storage medium Pending CN111127347A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201911253681.1A CN111127347A (en) 2019-12-09 2019-12-09 Noise reduction method, terminal and storage medium
PCT/CN2020/121645 WO2021114868A1 (en) 2019-12-09 2020-10-16 Denoising method, terminal, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911253681.1A CN111127347A (en) 2019-12-09 2019-12-09 Noise reduction method, terminal and storage medium

Publications (1)

Publication Number Publication Date
CN111127347A true CN111127347A (en) 2020-05-08

Family

ID=70498037

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911253681.1A Pending CN111127347A (en) 2019-12-09 2019-12-09 Noise reduction method, terminal and storage medium

Country Status (2)

Country Link
CN (1) CN111127347A (en)
WO (1) WO2021114868A1 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111583151A (en) * 2020-05-09 2020-08-25 浙江大华技术股份有限公司 Video denoising method and device, and computer readable storage medium
WO2021114868A1 (en) * 2019-12-09 2021-06-17 Oppo广东移动通信有限公司 Denoising method, terminal, and storage medium
CN113344820A (en) * 2021-06-28 2021-09-03 Oppo广东移动通信有限公司 Image processing method and device, computer readable medium and electronic equipment
CN113375808A (en) * 2021-05-21 2021-09-10 武汉博宇光电系统有限责任公司 Infrared image non-uniformity correction method based on scene
CN113674209A (en) * 2021-07-20 2021-11-19 浙江大华技术股份有限公司 Video noise detection method, terminal device and computer storage medium
CN113689373A (en) * 2021-10-21 2021-11-23 深圳市慧鲤科技有限公司 Image processing method, device, equipment and computer readable storage medium
CN114363693A (en) * 2020-10-13 2022-04-15 华为技术有限公司 Image quality adjusting method and device
CN114679553A (en) * 2020-12-24 2022-06-28 华为技术有限公司 Video noise reduction method and device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014078985A1 (en) * 2012-11-20 2014-05-30 Thomson Licensing Method and apparatus for image regularization
CN104065854A (en) * 2014-06-18 2014-09-24 联想(北京)有限公司 Image processing method and electronic device
CN109743473A (en) * 2019-01-11 2019-05-10 珠海全志科技股份有限公司 Video image 3 D noise-reduction method, computer installation and computer readable storage medium
US20190279401A1 (en) * 2018-03-07 2019-09-12 Novaquark Non-local computation of gradient noise for computer-generated digital images

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2006025396A1 (en) * 2004-09-03 2008-05-08 松下電器産業株式会社 Image processing apparatus and image processing program
CN103067661B (en) * 2013-01-07 2017-12-05 华为终端有限公司 Image processing method, device and camera terminal
CN104700405B (en) * 2015-03-05 2017-11-28 苏州科达科技股份有限公司 A kind of foreground detection method and system
CN107046648B (en) * 2016-02-05 2019-12-10 芯原微电子(上海)股份有限公司 Device and method for rapidly realizing video noise reduction of embedded HEVC (high efficiency video coding) coding unit
CN109639929B (en) * 2019-01-11 2020-10-30 珠海全志科技股份有限公司 Image noise reduction method, computer device and computer readable storage medium
CN111127347A (en) * 2019-12-09 2020-05-08 Oppo广东移动通信有限公司 Noise reduction method, terminal and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014078985A1 (en) * 2012-11-20 2014-05-30 Thomson Licensing Method and apparatus for image regularization
CN104065854A (en) * 2014-06-18 2014-09-24 联想(北京)有限公司 Image processing method and electronic device
US20190279401A1 (en) * 2018-03-07 2019-09-12 Novaquark Non-local computation of gradient noise for computer-generated digital images
CN109743473A (en) * 2019-01-11 2019-05-10 珠海全志科技股份有限公司 Video image 3 D noise-reduction method, computer installation and computer readable storage medium

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021114868A1 (en) * 2019-12-09 2021-06-17 Oppo广东移动通信有限公司 Denoising method, terminal, and storage medium
CN111583151A (en) * 2020-05-09 2020-08-25 浙江大华技术股份有限公司 Video denoising method and device, and computer readable storage medium
CN111583151B (en) * 2020-05-09 2023-05-12 浙江大华技术股份有限公司 Video noise reduction method and device, and computer readable storage medium
CN114363693A (en) * 2020-10-13 2022-04-15 华为技术有限公司 Image quality adjusting method and device
CN114679553A (en) * 2020-12-24 2022-06-28 华为技术有限公司 Video noise reduction method and device
CN113375808A (en) * 2021-05-21 2021-09-10 武汉博宇光电系统有限责任公司 Infrared image non-uniformity correction method based on scene
CN113375808B (en) * 2021-05-21 2023-06-02 武汉博宇光电系统有限责任公司 Infrared image non-uniformity correction method based on scene
CN113344820A (en) * 2021-06-28 2021-09-03 Oppo广东移动通信有限公司 Image processing method and device, computer readable medium and electronic equipment
CN113344820B (en) * 2021-06-28 2024-05-10 Oppo广东移动通信有限公司 Image processing method and device, computer readable medium and electronic equipment
CN113674209A (en) * 2021-07-20 2021-11-19 浙江大华技术股份有限公司 Video noise detection method, terminal device and computer storage medium
CN113689373A (en) * 2021-10-21 2021-11-23 深圳市慧鲤科技有限公司 Image processing method, device, equipment and computer readable storage medium

Also Published As

Publication number Publication date
WO2021114868A1 (en) 2021-06-17

Similar Documents

Publication Publication Date Title
CN111127347A (en) Noise reduction method, terminal and storage medium
JP7003238B2 (en) Image processing methods, devices, and devices
CN109040575B (en) Panoramic video processing method, device, equipment and computer readable storage medium
EP3099044B1 (en) Multi-frame noise reduction method and terminal
US9615039B2 (en) Systems and methods for reducing noise in video streams
AU2011216119B2 (en) Generic platform video image stabilization
US8508605B2 (en) Method and apparatus for image stabilization
CN107749987B (en) Digital video image stabilization method based on block motion estimation
CN109963048B (en) Noise reduction method, noise reduction device and noise reduction circuit system
CN109978774B (en) Denoising fusion method and device for multi-frame continuous equal exposure images
JP6308748B2 (en) Image processing apparatus, imaging apparatus, and image processing method
Wang et al. An asynchronous kalman filter for hybrid event cameras
CN114586337B (en) Video anti-shake optimization processing method and device and electronic equipment
CN111641835A (en) Video processing method, video processing device and electronic equipment
CN112215875A (en) Image processing method, device and electronic system
JP2003134385A (en) Image synthesizing device
CN110889809A (en) Image processing method and device, electronic device and storage medium
KR102003460B1 (en) Device and Method for dewobbling
CN115004227A (en) Image processing method, device and equipment
Liang et al. Coherent Event Guided Low-Light Video Enhancement
JP6221333B2 (en) Image processing apparatus, image processing circuit, and image processing method
CN115546043B (en) Video processing method and related equipment thereof
JP2015177528A (en) image processing apparatus, image processing method, and program
JP6120632B2 (en) Image processing apparatus, image processing method, and image processing program
CN112132879B (en) Image processing method, device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination