CN111353948A

CN111353948A - Image noise reduction method, device and equipment

Info

Publication number: CN111353948A
Application number: CN201811583920.5A
Authority: CN
Inventors: 李松南; 马岚; 俞大海
Original assignee: TCL Research America Inc
Current assignee: TCL Research America Inc
Priority date: 2018-12-24
Filing date: 2018-12-24
Publication date: 2020-06-30
Anticipated expiration: 2038-12-24
Also published as: CN111353948B

Abstract

An image noise reduction method includes: acquiring a multi-frame image, and determining a basic frame and an adjacent frame included in the multi-frame image; calculating a motion vector of the adjacent frame according to a base frame, and transforming the adjacent frame into an alignment image for aligning the base frame according to the motion vector; and fusing the aligned image and the basic frame through a convolutional neural network to obtain a noise-reduced image. Because the noise-reduced image is formed by automatically fusing the basic frame and the aligned image through the convolutional neural network, the noise can be more accurately reduced, the real scene content is effectively reserved, and the quality of the noise-reduced image is greatly improved.

Description

Image noise reduction method, device and equipment

Technical Field

The present application relates to the field of image processing, and in particular, to an image denoising method, apparatus and device.

Background

Due to popularization of smart phones, continuous improvement of quality of mobile phone camera hardware and portability of mobile phone photographing, more and more people use mobile phones to photograph, edit and share pictures and video contents of the people. Therefore, it is becoming more and more important to improve the quality of pictures taken by mobile phones.

The picture quality of the pictures taken by the mobile phone is influenced by various phonemes, such as noise, resolution, definition, color fidelity and the like, wherein the noise is a very key influencing phoneme. The noise sources in the mobile phone pictures are various, such as photon shot noise, dark current noise, dead pixel, fixed pattern noise, readout noise and the like. Photon shot noise in noise is a main source of noise and is restricted by physical laws, and always exists no matter how far the hardware technology develops. Therefore, at present, an algorithm is generally designed to distinguish shot contents from noise, so as to reduce the noise intensity in a mobile phone image.

In the currently used single-frame image noise reduction algorithm, theoretical derivation proves that the image signal-to-noise ratio (SNR) can be effectively improved by increasing the luminous flux. There are various ways to increase the light flux, one of which is to increase the exposure time. However, in the case of a hand-held camera, increasing the exposure time causes a shake blur in the picture. Therefore, the mainstream method in the industry at present increases the exposure time by means of multi-frame fusion in a phase-changing manner, so as to achieve the purpose of improving the signal-to-noise ratio of the image. However, when the multi-frame fusion method is used for denoising, noise cannot be accurately reduced, and real scene content cannot be effectively reserved.

Disclosure of Invention

In view of this, embodiments of the present application provide an image denoising method, apparatus and device, so as to solve the problems in the prior art that when denoising is performed in a multi-frame fusion manner, noise cannot be accurately reduced, and real scene content cannot be effectively retained.

A first aspect of an embodiment of the present application provides an image denoising method, including:

acquiring a multi-frame image, and determining a basic frame and an adjacent frame included in the multi-frame image;

calculating a motion vector of the adjacent frame according to a base frame, and transforming the adjacent frame into an alignment image for aligning the base frame according to the motion vector;

and fusing the aligned image and the basic frame through a convolutional neural network to obtain a noise-reduced image.

With reference to the first aspect, in a first possible implementation manner of the first aspect, the step of determining a base frame included in the multi-frame image includes:

acquiring a main body in the multi-frame image;

and calculating the definition of the main body of the multi-frame image, and selecting the image frame with the highest definition as a basic frame.

With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the calculating a sharpness of a subject of the multi-frame image, and the selecting an image frame with a highest sharpness as a base frame includes:

converting a plurality of frame images into a brightness map;

and performing edge filtering on a preset area around the center of the main body of the multi-frame image, acquiring a response average value of the edge filtering, and selecting the image with the highest response average value as a basic frame.

With reference to the first aspect, in a third possible implementation manner of the first aspect, the calculating a motion vector of the neighboring frame according to the base frame, and transforming the neighboring frame into an alignment image for aligning the base frame according to the motion vector includes:

dividing the basic frame and the adjacent frame into a plurality of image blocks respectively;

determining a motion vector for each image block of the neighboring frame based on the motion estimation of the block;

and rearranging the image blocks in the adjacent frames according to the motion vector to obtain an aligned image.

With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the dividing the base frame and the neighboring frame into a plurality of image blocks includes:

performing multiple Gaussian downsampling on the basic frame and the adjacent frame to obtain images of each frame at different resolutions;

the block-based motion estimation, determining a motion vector for each image block of a neighboring frame, comprises:

performing motion estimation on image blocks of a basic frame and an adjacent frame of a first resolution ratio, and determining a first motion vector corresponding to the image block of the first resolution ratio;

and transmitting the first motion vector to an image block with a second resolution for motion estimation, and correcting the first motion vector to obtain a second motion vector, wherein the first resolution is lower than the second resolution.

With reference to the first aspect, in a fifth possible implementation manner of the first aspect, the fusing the aligned image and the base frame by using a convolutional neural network to obtain a noise-reduced image includes:

converting the alignment image and the basic frame into a single-color multi-channel image, and splicing the same-color channel images of the alignment image and the basic frame;

and fusing the spliced images through a convolutional neural network to obtain a noise reduction image.

With reference to the first aspect, in a sixth possible implementation manner of the first aspect, before the step of fusing the aligned image and the base frame by using a convolutional neural network to obtain a noise-reduced image, the method further includes:

obtaining M sample pictures by using the same exposure parameters;

determining a basic frame and a neighboring frame in the M sample pictures, and aligning the determined neighboring frame with the basic frame;

and selecting one or more of white balance, black level removal, lens correction, demosaicing, color space conversion, sharpening and enhancement, and carrying out image processing on the basic frame and the aligned image to obtain a noise reduction image corresponding to the sample image.

A second aspect of an embodiment of the present application provides an image noise reduction apparatus, including:

the device comprises a frame image acquisition unit, a frame image acquisition unit and a frame image processing unit, wherein the frame image acquisition unit is used for acquiring a plurality of frames of images and determining a basic frame and a nearby frame which are included in the plurality of frames of images;

the alignment unit is used for calculating a motion vector of the adjacent frame according to a basic frame and converting the adjacent frame into an alignment image aligned with the basic frame according to the motion vector;

and the fusion unit is used for fusing the aligned image and the basic frame through a convolutional neural network to obtain a noise-reduced image.

A third aspect of embodiments of the present application provides an image noise reduction apparatus, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the method according to any one of the first aspect when executing the computer program.

A fourth aspect of embodiments of the present application provides a computer-readable storage medium, in which a computer program is stored, which, when executed by a processor, performs the steps of the method according to any one of the first aspect.

Compared with the prior art, the embodiment of the application has the advantages that: determining a basic frame and an adjacent frame according to a multi-frame image, calculating a motion vector of the adjacent frame according to the basic frame, transforming the adjacent frame by the calculated motion vector to obtain an aligned image, and fusing the basic frame and the aligned image through a convolutional neural network to obtain a noise-reduced image. Because the noise-reduced image is formed by automatically fusing the basic frame and the aligned image through the convolutional neural network, the noise can be more accurately reduced, the real scene content is effectively reserved, and the quality of the noise-reduced image is greatly improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic flow chart of an implementation of an image denoising method provided in an embodiment of the present application;

FIG. 1a is a schematic diagram of an image denoising frame according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart illustrating an implementation flow of a method for determining a base frame according to an embodiment of the present application;

fig. 3 is a schematic flow chart illustrating an implementation of a method for acquiring an alignment image according to an embodiment of the present application;

fig. 4 is a schematic flow chart of an implementation of a method for obtaining training samples according to an embodiment of the present application;

fig. 5 is a schematic diagram of an image noise reduction apparatus provided in an embodiment of the present application;

fig. 6 is a schematic diagram of an image noise reduction device provided in an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

In order to explain the technical solution described in the present application, the following description will be given by way of specific examples.

Fig. 1 is a schematic flow chart of an implementation of an image denoising method according to an embodiment of the present application, which is detailed as follows:

in step S101, a plurality of frame images are acquired, and a base frame and a neighboring frame included in the plurality of frame images are determined;

specifically, the multi-frame image may be a RAW (unprocessed) domain picture, or an RGB (red, green, and blue) three-channel picture, or may also be a Y-channel picture.

The acquired multi-frame images may be acquired in a very short time by using a Burst mode of the camera, and the multi-frame images may have the same ISP (image signal processing) processing parameters, such as the same exposure, white balance parameters, noise reduction or sharpening strength, and the like. A typical number N of input frames may vary from 3 to 8 depending on the brightness of the scene. The multi-frame image noise reduction frame diagram shown in fig. 1a shows a case where the number of input frames is 3, and the other frames are similar to this.

After acquiring a plurality of input images, we need to select a basic frame and then align all other frames, i.e. neighboring frames, with the basic frame. The selected base frame may be the sharpest of all input frames, which ensures that the final output result is sharp. In addition, in order to ensure the definition of the shot subject, the image with the clearest part of the shot subject can be selected as the basic image.

The position of the subject can be determined by determining a portrait area where a face is located, such as by face position detection, or the position of the subject can be determined from a touch point of the user on the screen. For a picture that cannot be judged to be a subject, let us assume that the center of the picture is the subject to be photographed. Assuming that we obtain the center position of the subject in the above manner, the basic frame determination flow is shown in fig. 2 and includes:

in step S201, converting a plurality of frame images into a luminance map;

the luminance map can be fused by using weighted average of a plurality of frame images comprising a plurality of colors, such as a bell image comprising four color values of GBRG.

In step S202, performing edge filtering in a predetermined area around the center of the main body of the multi-frame image, and acquiring a response average value of the edge filtering;

in the luminance map, a predetermined region around the center of the subject may be selected for edge filtering, for example, edge filtering by a sobel operator, to obtain a response average. Of course, if the subject is determined, the subject may be edge filtered directly. If the subject is uncertain, the subject may be assumed to be the center of the image, a predetermined area around the center position may be edge-filtered to obtain a response average value, and the size of the predetermined area may be set according to the size of the image, or may also be set according to the size of the subject being photographed.

In step S203, an image with the highest response average value is selected as a base frame.

Because only one basic frame is needed to be determined for the multi-frame images, the image with the highest response average value, namely the image with the clearest main body, can be selected as the basic frame, and therefore the definition of the fused noise-reduced image is guaranteed.

In step S102, calculating a motion vector of the neighboring frame according to a base frame, and transforming the neighboring frame into an alignment image for aligning the base frame according to the motion vector;

and dividing the image blocks of the basic frame and the adjacent frame, and performing motion estimation on the image blocks of the adjacent frame according to the similarity between the adjacent frame and the image blocks in the basic frame to determine motion vectors of the image blocks in the adjacent frame. After the motion vector of the image block of the adjacent frame is determined, the image block of the adjacent frame can be rearranged according to the motion vector, so that the adjacent frame is aligned with the basic frame, and an aligned image is obtained.

Specifically, the generation flow of the alignment image may specifically include, as shown in fig. 3, the following steps:

in step S301, a base frame and an adjacent frame are divided into a plurality of image blocks, respectively;

in a preferred embodiment, the base frame and the neighboring frame may be subjected to multiple gaussian down sampling to obtain images of each frame at different resolutions. Namely, the basic frame is subjected to multiple times of Gaussian down-sampling to obtain a plurality of basic frame images with different resolutions, and the adjacent frame is subjected to multiple times of Gaussian down-sampling to obtain a plurality of adjacent frame images with different resolutions. The downsampled images of different resolutions are then respectively divided into image blocks so that the block-based motion estimation is completed in step S302.

In step S302, a motion vector of each image block of the neighboring frame is determined based on motion estimation of the block;

after obtaining a plurality of blurred images with different resolutions in step S301, motion estimation may be performed on image blocks of a base frame and an adjacent frame of a first resolution, and a first motion vector corresponding to the image block at the first resolution is determined; and transmitting the first motion vector to an image block with a second resolution for motion estimation, and correcting the first motion vector to obtain a second motion vector, wherein the first resolution is lower than the second resolution. And repeating the transmission until the original brightness image is transmitted, and correcting the original brightness image.

The method comprises the steps of using a pyramid-based block alignment mode to make a Gaussian pyramid on each frame of image to obtain a series of pictures with different resolutions, then performing block-based motion estimation on the picture with the lowest resolution to obtain a motion vector of each block, enabling the motion vector to point to a block which is most similar to the block in an adjacent frame, then transmitting the motion vector to the next adjacent picture with higher resolution, continuing motion estimation on the picture with higher resolution by taking the motion vector as a center, performing more refined correction on the motion vector, and then repeating the following process until the motion vector is transmitted to the original luminance picture of the lowest layer and is corrected.

In step S303, rearranging the image blocks in the adjacent frame according to the motion vector to obtain an aligned image.

After the motion vector is determined, rearranging the image blocks in the adjacent frame to obtain the aligned image corresponding to the adjacent frame.

The pyramid-based motion estimation method has two advantages: (1) in a low resolution picture, the signal-to-noise ratio of the image increases, and thus the motion estimation process is less affected by noise, (2) the complexity of the motion estimation process is reduced, and the search range of motion estimation is increased. It should be noted that, in addition to the pyramid block alignment method, other conventional image alignment methods are also applicable, for example, an image alignment method based on feature points, an image alignment method in a frequency domain, and the like, can be used instead of the method currently used by us. In addition, the image alignment using the deep learning method is also very good, but the complexity is also increased significantly.

After the block alignment process, a set of motion vectors can be obtained for each adjacent frame, and by using the set of motion vectors, the pixel positions of the adjacent frames can be rearranged, so that the aligned image obtained after rearrangement is similar to the basic frame. It should be noted that the similarity of the aligned image and the base frame at some positions may be very low, because the estimated motion vectors may not be accurate or the motion pattern between frames cannot be described with a simple translation. In the next step, we fuse the base frame with the aligned multi-frame image, and the method of deep learning is used to help us get good fusion effect at those positions where the alignment is inaccurate.

In step S103, the aligned image and the base frame are fused by a convolutional neural network, so as to obtain a noise-reduced image.

The convolutional neural network may be a convolutional neural network trained by sample data in advance, or may be a convolutional neural network in the training process.

When image fusion is performed, the mode of an image needs to be converted first, a base frame and an aligned image are converted from a bayer mode including a multi-color image into a single-color multi-channel mode, the color channels are spliced (coordinated), and a Convolutional Neural Network (CNN) is used for fusion processing to generate an RGB three-channel image. The output RGB image has low noise and very good image quality.

It should be noted that we can use the same image alignment method for different input frame numbers, but need to train different CNN models.

In addition, the convolutional neural network CNN process may be implemented by using a classical convolutional neural network with a pooling layer and a full Connection layer removed, such as AlexNet (alice convolutional neural network), VGG (Visual Geometry Group computer vision Group neural network), and the like, and may accelerate a training process of the network by adding a Skip Connection (Skip Connection) to learn a residual error. An existing CNN model for noise reduction (e.g., DnCNN (feed forward noise reduction convolutional neural network), etc.) may also be used to implement this processing step. It should be noted that, the present application can input multi-frame information, so that the difficulty of image denoising can be reduced, the purpose of denoising can be achieved by using relatively fewer convolution layers, and the calculation amount of the CNN process is simplified.

In the noise reduction frame diagram shown in fig. 1a, the number of acquired multi-frame images is N-3. Of course, the N may be other natural numbers greater than 3. As shown in fig. 1a, RAW domain bell images of the same exposure are acquired, and the resolution may be H × W. One of the frames is set as a base frame and the other frames are set as neighbor frames. And (4) subjecting each adjacent frame to RAW domain multi-frame alignment operation to align the adjacent frame with the base frame. After alignment, the base frame and the aligned adjacent frames are converted into a single-color multi-channel mode, each channel having a resolution of

And then splicing (Concatenate) the data together, and processing the data by using a convolutional neural network to generate a de-noised RGB three-channel image, wherein the resolution of each color channel is the same as that of the input frame, namely the resolution of the three-channel image is H x W.

Of course, before the convolutional neural network is used for fusion, the method further includes a step of acquiring a sample image to train the convolutional neural network, as shown in fig. 4, the image sample acquiring process includes:

in step S401, M sample pictures are acquired using the same exposure parameters;

m pictures can be shot by using the same exposure parameters in a mode of a handheld device, such as a mobile phone, and the larger M is, the better M is, the more the precision of the convolutional neural network training is favorably improved.

In step S402, determining a base frame and a neighboring frame in the M sample pictures, and aligning the determined neighboring frame with the base frame;

the selection of the base frame and the alignment of the adjacent frames of the sample picture can be performed according to the determination manner of the base frame and the adjacent frames in the multi-frame image and the alignment manner of the adjacent frames and the base frame in fig. 1.

In step S403, one or more of white balance, black level removal, lens correction, demosaicing, color space conversion, sharpening, and enhancement are selected, and image processing is performed on the base frame and the aligned image to obtain a noise reduction picture corresponding to the sample picture.

The traditional Image Signal Processing (ISP) process can be used for carrying out white balance, black level removal, lens correction, demosaicing, color space conversion, sharpening, enhancement and other processing on the RAW domain picture subjected to noise reduction, and finally a high-quality picture containing RGB three channels is generated and used as an output result of the convolutional neural network to train the convolutional neural network.

Since the process of preparing the training data does not need to take into account the processing time, we can use a very complex multi-frame fusion algorithm, white balance algorithm, anti-mosaic algorithm, image enhancement algorithm, etc., and use a very large number of input frames (such as setting M equal to 30) to get the ideal output image. The multi-frame deep learning noise reduction algorithm generally can only use a relatively small number of frames as input due to the need to consider the execution complexity. Therefore, our training process, in effect, trains the deep learning algorithm (M > N) using N frame images as input by using the conventional image processing algorithm with M frame images as input. Compared with the current construction mode of image data containing real noise, the method is faster and more convenient because a tripod is not needed and a plurality of frames of images are not needed to be screened.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Fig. 5 is a schematic structural diagram of an image noise reduction apparatus according to an embodiment of the present application, which is detailed as follows:

the image noise reduction device includes:

a frame image obtaining unit 501, configured to obtain a multi-frame image, and determine a base frame and an adjacent frame included in the multi-frame image;

an alignment unit 502, configured to calculate a motion vector of the neighboring frame according to a base frame, and transform the neighboring frame into an alignment image aligned with the base frame according to the motion vector;

and a fusion unit 503, configured to fuse the aligned image and the basic frame through a convolutional neural network, so as to obtain a noise-reduced image.

The image noise reduction apparatus shown in fig. 5 corresponds to the image noise reduction method shown in fig. 1.

Fig. 6 is a schematic diagram of an image noise reduction apparatus according to an embodiment of the present application. As shown in fig. 6, the image noise reduction device 6 of this embodiment includes: a processor 60, a memory 61 and a computer program 62, such as an image noise reduction program, stored in said memory 61 and executable on said processor 60. The processor 60, when executing the computer program 62, implements the steps in the various image noise reduction method embodiments described above, such as the steps 101 to 103 shown in fig. 1. Alternatively, the processor 60, when executing the computer program 62, implements the functions of each module/unit in the above-mentioned device embodiments, for example, the functions of the modules 501 to 503 shown in fig. 5.

Illustratively, the computer program 62 may be partitioned into one or more modules/units that are stored in the memory 61 and executed by the processor 60 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 62 in the image noise reduction device 6. For example, the computer program 62 may be divided into a frame image acquisition unit, an alignment unit, and a fusion unit, each unit functioning specifically as follows:

The image noise reduction device 6 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The image noise reduction device may include, but is not limited to, a processor 60, a memory 61. Those skilled in the art will appreciate that fig. 6 is merely an example of the image noise reduction device 6, and does not constitute a limitation on the image noise reduction device 6, and may include more or less components than those shown, or combine some components, or different components, for example, the image noise reduction device may further include an input-output device, a network access device, a bus, etc.

The Processor 60 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 61 may be an internal storage unit of the image noise reduction device 6, such as a hard disk or a memory of the image noise reduction device 6. The memory 61 may also be an external storage device of the image noise reduction device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the image noise reduction device 6. Further, the memory 61 may also include both an internal storage unit of the image noise reduction device 6 and an external storage device. The memory 61 is used to store the computer program and other programs and data required by the image noise reduction apparatus. The memory 61 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer-readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. . Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media which may not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims

1. An image noise reduction method, characterized by comprising:

2. The image noise reduction method according to claim 1, wherein the step of determining the base frame included in the multi-frame image includes:

acquiring a main body in the multi-frame image;

3. The image noise reduction method according to claim 2, wherein the step of calculating the sharpness of the subject of the multi-frame image, and selecting the image frame with the highest sharpness as the base frame comprises:

converting a plurality of frame images into a brightness map;

performing edge filtering in a preset area around the center of a main body of a multi-frame image to obtain a response average value of the edge filtering;

the image with the highest response average is selected as the base frame.

4. The method of claim 1, wherein the step of calculating a motion vector of the neighboring frame according to the base frame, and transforming the neighboring frame into an alignment image for aligning the base frame according to the motion vector comprises:

5. The image noise reduction method according to claim 4, wherein the step of dividing the base frame and the neighboring frame into a plurality of image blocks respectively comprises:

6. The method of claim 1, wherein the step of fusing the aligned image and the base frame by a convolutional neural network to obtain a noise-reduced image comprises:

7. The method of image noise reduction according to claim 1, wherein before the step of fusing the aligned image and the base frame by a convolutional neural network to obtain a noise-reduced image, the method further comprises:

obtaining M sample pictures by using the same exposure parameters;

determining a basic frame and adjacent frames in M sample pictures, aligning the determined adjacent frames with the basic frame, wherein M is greater than the number of frames of the multi-frame image;

8. An image noise reduction apparatus, characterized by comprising:

9. An image noise reduction apparatus comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.