WO2023125750A1 - 一种图像去噪方法、装置和存储介质 - Google Patents

一种图像去噪方法、装置和存储介质 Download PDF

Info

Publication number
WO2023125750A1
WO2023125750A1 PCT/CN2022/143154 CN2022143154W WO2023125750A1 WO 2023125750 A1 WO2023125750 A1 WO 2023125750A1 CN 2022143154 W CN2022143154 W CN 2022143154W WO 2023125750 A1 WO2023125750 A1 WO 2023125750A1
Authority
WO
WIPO (PCT)
Prior art keywords
image sequence
image
neural network
denoised
trained
Prior art date
Application number
PCT/CN2022/143154
Other languages
English (en)
French (fr)
Inventor
边巧玲
陈彬彬
黄露
陆艳青
王进
Original Assignee
虹软科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 虹软科技股份有限公司 filed Critical 虹软科技股份有限公司
Publication of WO2023125750A1 publication Critical patent/WO2023125750A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/147—Details of sensors, e.g. sensor lenses
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • This article relates to image processing technology, especially an image denoising method, device and storage medium.
  • Images will be mixed with noise during the generation process, and images captured in low light conditions will have more noise than images captured in bright light conditions. Noise brings loss to image quality, such as unclear image, image distortion and so on.
  • the present application provides an image denoising method, device and storage medium, which can remove noise contained in an image and improve image quality.
  • the image denoising method provided by this application includes:
  • a denoised image sequence is obtained according to the image sequence output by the trained neural network.
  • the method also includes:
  • the neural network Before the image sequence to be denoised is input into the trained neural network, the neural network is trained;
  • the neural network is trained, including:
  • Adjusting parameters of the neural network to be trained by calculating a preset loss target until the neural network to be trained meets a preset convergence condition.
  • constructing an adaptive training data set through acquired sample data includes:
  • each sample data includes a noisy image sequence sample, and a reference image whose noise matching the noisy image sequence sample is smaller than a preset threshold
  • the loss function used when calculating the preset loss target includes any one or more of the following:
  • L1 loss function L2 loss function
  • structural similarity loss function L1 loss function
  • perceptual loss function L2 loss function
  • frequency loss function L1 loss function
  • the manner of obtaining each sample data includes:
  • the image obtained by shooting the target object through the external device is used as a sample of the noisy image sequence
  • the way of superimposing a preset number of noisy image sequence samples includes:
  • a preset number of noisy image sequence samples are superimposed in a pixel-weighted manner, wherein the weight corresponding to the pixels forming the moving objects in the noisy image sequence samples is smaller than the weight corresponding to the pixels forming the non-moving objects.
  • the manner of obtaining each sample data includes:
  • An image obtained by shooting the same target object with an external device having an exposure time greater than a preset time threshold is used as a reference image whose noise matched with the noisy image sequence sample is less than a preset threshold.
  • the manner of obtaining each sample data includes:
  • the image obtained by shooting the target object with an external device with a lens filter is used as a sample of a noisy image sequence
  • An image obtained by shooting the same target object with an external device without a lens filter is used as a reference image whose noise matched with the noisy image sequence sample is smaller than a preset threshold.
  • the manner of obtaining each sample data includes:
  • the acquisition method of the preset noise includes:
  • the undetermined parameters of the noise distribution model are based on different photosensitive values of the reference image, pixel variance values corresponding to different photosensitive values of the reference image, and photosensitive values and pixel variances calibrated by the noise distribution model Correspondence between values is obtained.
  • inputting the image sequence to be denoised into a trained neural network includes:
  • the image sequence obtained by fusing the image blocks output from the trained neural network is used as the image sequence after denoising.
  • performing block processing on the image sequence to be denoised includes:
  • the image sequence to be denoised is sequentially divided into blocks according to position coordinates, and adjacent image blocks have overlapping regions;
  • the image blocks output from the trained neural network are fused, including:
  • Fusion processing is performed on the image block and its adjacent image blocks having an overlapping area in sequence according to the position coordinates.
  • the method also includes:
  • the trained neural network allocates image blocks to the different hardware resources according to the performance of different hardware resources, and the obtained The above-mentioned different hardware resources process the respective allocated image blocks in parallel.
  • inputting the adaptive training data set into the neural network to be trained includes:
  • the adaptive training data sets under different classes are input into the neural network to be trained.
  • the method also includes:
  • Inverse normalization processing is performed on the image sequence output from the trained neural network to obtain the denoised image sequence.
  • the embodiment of the present application also provides an image denoising device, the device comprising:
  • an acquisition unit configured to acquire an image sequence to be denoised
  • the denoising unit is configured to input the image sequence to be denoised into a trained neural network, and complete the denoising operation through the trained neural network;
  • the output unit is configured to obtain a denoised image sequence according to the image sequence output by the trained neural network.
  • the embodiment of the present application also provides a computer-readable storage medium, where one or more programs are stored in the computer-readable storage medium, and the one or more programs can be executed by one or more processors to implement the following: method as described above.
  • the embodiment of the present application also provides an image denoising device, including: a memory and a processor, the memory stores a program, and when the program is read and executed by the processor, the method as described in any one of the foregoing is implemented .
  • the embodiment of the present application implements the operation of removing image noise by using a neural network, thereby improving image quality.
  • Fig. 1 is the flowchart of the image denoising method provided by the embodiment of the present application.
  • Fig. 2 is the flow chart of the method for training the neural network provided by the embodiment of the present application.
  • Fig. 3 is a kind of noise model graph that the embodiment of the present application provides
  • FIG. 4A is a schematic diagram of block segmentation and fusion of an image sequence to be denoised according to an embodiment of the present application
  • FIG. 4B is a schematic diagram of image blocks obtained by dividing the image sequence to be denoised into blocks according to the embodiment of the present application;
  • FIG. 5 is a block diagram of an image denoising device provided in an embodiment of the present application.
  • FIG. 6 is a block diagram of another image denoising device provided in an embodiment of the present application.
  • the embodiment of the present application provides an image denoising method, as shown in FIG. 1, the method includes:
  • Step S101 acquires an image sequence to be denoised
  • Step S102 input the image sequence to be denoised into a trained neural network, and complete the denoising operation through the trained neural network;
  • the image sequence to be denoised that is input into the trained neural network can be a single frame, a single frame after multi-frame superposition, or continuous multiple frames; its data format can be various, such as RGB, YUV, or RawRGB; in addition, For the continuous multi-frame input neural network, it is necessary to maintain the alignment of adjacent frames; the embodiment of the present application supports image denoising in multiple formats, which can meet the needs of different end users;
  • Step S103 obtains a denoised image sequence according to the image sequence output by the trained neural network
  • the image sequence output from the trained neural network is a single frame; its data format can also have multiple, such as RGB, YUV, or RawRGB; if the image sequence to be denoised and the required
  • the format of the image sequence output from the trained neural network is different, and format conversion is required;
  • the embodiment of the present application implements the operation of removing image noise by using a neural network, thereby improving image quality.
  • the method further includes: before inputting the image sequence to be denoised into the trained neural network, training the neural network.
  • the steps of training the neural network include:
  • Step S201 constructs an adaptive training data set through the acquired sample data
  • the adaptive training data set is constructed through the obtained sample data, including:
  • each sample data includes a noisy image sequence sample, and a reference image whose noise matched with the noisy image sequence sample is less than a preset threshold; combine the multiple sample data in proportion Constructing the adaptive training data set, wherein the ratio of the plurality of sample data can be set, and can also be adjusted according to the type of image sequence to be denoised and the difficulty of sample data collection;
  • the reference image can be used to calculate a loss function value to determine whether the neural network to be trained meets a preset convergence condition; the matching index in the reference image whose noise matched with the noisy image sequence sample is less than a preset threshold What is important is that the noisy image sequence sample matches the reference image in terms of pixels, which means switching any frame of the noisy image sequence sample and the reference image, the two satisfy the alignment at the pixel level, and there is only a difference in image quality;
  • Step S202 inputs the adaptive training data set into the neural network to be trained
  • neural networks include but are not limited to: RNN network, fully convolutional network and Unet network;
  • Step S203 adjusts the parameters of the neural network to be trained by calculating a preset loss target until the neural network to be trained meets a preset convergence condition; for example, by adjusting the parameters of the neural network to be trained so that the calculated The loss function value becomes smaller and smaller until the preset loss function value target is reached, and the convergence of the neural network is completed.
  • L1 loss function L2 loss function
  • structural similarity loss function L1 loss function
  • perceptual loss function L2 loss function
  • frequency loss function L1 loss function
  • the network parameters can be frozen so that the network parameters are fixed, and the network parameters can run on floating-point hardware resources, including but not limited to CPU, image processing unit (Graphic Processing Unit, GPU), DSP; on hardware acceleration platforms (including but not limited to quantization platforms), quantization training and post-quantization can be performed so that network parameters can be integerized. According to different input data requirements, 8-bit quantization or 16-bit quantization of parameters and weights can be done. Quantify.
  • the post-quantization strategy is adopted, then the accuracy of the post-quantization is used to ensure the consistency of the results of the GPU and the NPU. If it is training quantization, you can add another item to the loss function: the supervised loss function of the results of the GPU model and the NPU model.
  • the supervised loss function can be L1, L2, etc.
  • the manner of obtaining each sample data includes:
  • the first way is to use the image obtained by shooting the target object through an external device as a noisy image sequence sample; by obtaining a preset number of noisy image sequence samples and superimposing the preset number of noisy image sequence samples The image is used as a reference image whose noise matched with the noisy image sequence sample is less than a preset threshold;
  • the noise can be significantly reduced, the signal-to-noise ratio can be improved, and the details can be improved. Therefore, the superimposed image can be used as a reference image.
  • the preset number can be determined according to the noise of each noisy image sequence sample, if the noise is small (less than the preset noise threshold), the preset number can be 6-10; if the noise is larger (greater than the preset noise threshold), the preset number can be 60-100.
  • the way of superimposing the preset number of noisy image sequence samples includes:
  • a preset number of noisy image sequence samples are superimposed in a pixel-weighted manner, wherein the weight corresponding to the pixels forming the moving objects in the noisy image sequence samples is smaller than the weight corresponding to the pixels forming the non-moving objects.
  • the advantage of method 1 it is suitable for many scenarios, and there is no special requirement for the acquisition tool; the disadvantage: the acquisition cost is high, the acquisition environment requirements are high, the brightness needs to be kept constant, and there is no jitter in the acquisition process.
  • Method 2 The image obtained by shooting the target object with an external device with an exposure time shorter than the preset time threshold is used as a sample of the noisy image sequence; the image obtained by shooting the same target object with an external device with an exposure time greater than the preset time threshold The image is used as a reference image whose noise matched with the noisy image sequence sample is smaller than a preset threshold.
  • EV0 means that the exposure amount is a combination corresponding to an exposure time of 1 second and an aperture of f/1.0 or an equivalent combination thereof
  • the gain value is set The smaller the gain value is, the longer the exposure time will be and the smaller the noise will be because EV0 is fixed; on the contrary, the larger the gain value will be, the shorter the exposure time will be and the greater the noise will be because EV0 is fixed.
  • GT quality is better than Method 1;
  • Disadvantages The requirements for the acquisition environment are high, the brightness needs to be kept constant, and there is no jitter in the acquisition process.
  • Method 3 The image obtained by shooting the target object with an external device with a filter lens is used as a sample of a noisy image sequence; the image obtained by shooting the same target object with an external device without a filter lens is used as the noisy image sequence samples match reference images whose noise is less than a preset threshold.
  • the lens with a filter can simulate a dark environment
  • the image obtained by shooting the target with an external device with a filter can be used as a sample of a noisy image sequence; correspondingly, the external device without a filter
  • the image captured by the device on the same target is used as a reference image.
  • the advantage of the third method it can capture extremely dark scenes; the disadvantage: there is a glare effect.
  • Method 4 Use an image with noise less than a preset threshold obtained by shooting the target with an external device as a reference image; add the acquired preset noise to the reference image as the noisy image sequence sample.
  • the acquisition method of the preset noise may include:
  • the undetermined parameters of the noise distribution model are based on different photosensitive values of the reference image, pixel variance values corresponding to different photosensitive values of the reference image, and photosensitive values and pixel variances calibrated by the noise distribution model Correspondence between values is obtained.
  • the preset noise distribution model satisfies the Poisson Gaussian distribution model:
  • Poison () represents a Poisson function
  • Normal () represents a normal distribution function
  • I represents the photosensitive value of the reference map
  • ⁇ (I) 2 represents the pixel variance value corresponding to the reference image under the sensitivity value I.
  • Figure 3 shows a graph corresponding to I ⁇ (I) 2 , in which the abscissa represents the photosensitive value I, and the ordinate represents ⁇ (I) 2 , the slope of the curve is g, and the intercept is
  • the fourth advantage of this method fast data generation, low cost, and better GT; disadvantage: only suitable for data directly output by a specific sensor.
  • the above four different sample data acquisition methods have their own advantages and disadvantages, and the process of collecting data can be combined according to requirements to obtain an adaptive training data set.
  • the highest photosensitive value can be obtained by the third method
  • the lowest photosensitive value can be obtained by the fourth method
  • the intermediate photosensitive value can be obtained by the second method
  • the first method can be used to supplement part of the data.
  • the data set obtained in this way constitutes a complete adaptive training data set.
  • inputting the image sequence to be denoised into a trained neural network includes:
  • the image sequence obtained by fusing the image blocks output from the trained neural network is used as the image sequence after denoising.
  • the image sequence to be denoised When the image sequence to be denoised is large and the computing power of the neural network is limited, the image sequence can be segmented and then input to the neural network. Similarly, when performing neural network training, the image sequence samples to be denoised can also be divided into blocks and then input to the neural network to be trained.
  • performing block processing on the image sequence to be denoised includes:
  • the image sequence to be denoised is sequentially divided into blocks according to position coordinates, and adjacent image blocks have overlapping regions;
  • the image blocks output from the trained neural network are fused, including:
  • Fusion processing is performed on the image block and its adjacent image blocks having an overlapping area in sequence according to the position coordinates.
  • Figure 4A shows a schematic diagram of block and fusion of the image sequence to be denoised; the image block obtained by dividing the image sequence to be denoised is shown in Figure 4B, and the frame marked in Figure 4B
  • the image block is the image block input to the neural network, and the part between the inner frame and the outer frame is the peripheral part; when performing image block fusion, the peripheral parts of adjacent image blocks overlap.
  • each obtained image block can be identified, such as using the center position coordinates of each image block as the identification; when image blocks are fused, the identification is performed according to the identification fusion operation.
  • the method also includes:
  • the trained neural network allocates image blocks to the different hardware resources according to the performance of different hardware resources, and the obtained The above-mentioned different hardware resources process the respective allocated image blocks in parallel.
  • the types of hardware resources may include GPU, NPU and CPU.
  • GPU and NPU can be used to run in parallel on the hardware, and the number of image blocks is allocated to GPU and NPU according to the performance ratio of GPU and NPU; for example, the total number of image blocks is 200, and the number of GPU and NPU The performance ratio is 1:2, so 66 of the image blocks can be assigned to the GPU, and the remaining image blocks can be assigned to the NPU.
  • the method also includes:
  • the normalization operation is to standardize the data, eliminate the influence of different dimensions, make the data in the same order of magnitude, and ensure the accuracy of the output results of the neural network.
  • the image sequence samples to be denoised can also be normalized and input to the neural network to be trained.
  • inputting the adaptive training data set into the neural network to be trained includes:
  • the adaptive training data sets under different classes are input into the neural network to be trained.
  • a trained neural network can be obtained for each sensitivity value, and multiple sensitivity values correspond to multiple trained neural networks; or, multiple sensitivity values correspond to a trained neural network , each sensitivity value corresponds to a channel of the trained neural network.
  • the smaller the photosensitive value, the smaller the noise, and the corresponding trained neural network or neural network The stronger the denoising ability of the channel.
  • classifying the adaptive training data set according to the photosensitive value including:
  • the noisy image sequence samples and the reference image are sequentially classified according to the photosensitive value of 100-1600, the photosensitive value of 1600-3200, the photosensitive value of 3200-6400, and the photosensitive value of 6400-12800.
  • the finer the classification the higher the accuracy, but the higher the maintenance cost of denoising; the embodiment of this application divides the photosensitivity value into the above four categories, which can realize the control of maintenance cost under the condition of taking into account the accuracy.
  • the embodiment of the present application also provides an image denoising device, as shown in FIG. 5, the device includes:
  • the acquisition unit 501 is configured to acquire an image sequence to be denoised
  • the denoising unit 502 is configured to input the image sequence to be denoised into a trained neural network, and complete the denoising operation through the trained neural network;
  • the image sequence to be denoised input into the trained neural network can be a single frame, a single frame after multiple frames are superimposed, or continuous multiple frames; its data format can be various, such as RGB, YUV, or RawRGB; this application
  • the embodiment supports image denoising in multiple formats, which can meet the needs of different end users;
  • the output unit 503 is configured to obtain a denoised image sequence according to the image sequence output by the trained neural network
  • the image sequence output from the trained neural network is a single frame; its data format can also have multiple, such as RGB, YUN, or RawRGB; if the image sequence to be denoised and the required
  • the format of the image sequence output from the trained neural network is different, and format conversion is required.
  • the embodiment of the present application implements the operation of removing image noise by using a neural network, thereby improving image quality.
  • the embodiment of the present application also provides a computer-readable storage medium, where one or more programs are stored in the computer-readable storage medium, and the one or more programs can be executed by one or more processors to implement the following: The method described in any of the previous examples.
  • the embodiment of the present application also provides an image denoising device, as shown in FIG. 6 , including: a memory 601 and a processor 602, the memory 601 stores a program, and the program is read and executed by the processor 602 , implement the image denoising method as described in any of the previous embodiments.
  • the functional modules/units in the system, and the device can be implemented as software, firmware, hardware, and an appropriate combination thereof.
  • the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components. Components cooperate to execute.
  • Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit.
  • Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).
  • computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer.
  • communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Vascular Medicine (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

一种图像去噪方法、装置和存储介质，其中，所述方法包括：获取待去噪的图像序列；将所述待去噪的图像序列输入已训练好的神经网络，通过所述已训练好的神经网络完成去噪操作；根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列。

Description

一种图像去噪方法、装置和存储介质
本申请要求于2021年12月31日递交的中国专利申请第202111672183.8号的优先权，在此全文援引该中国专利申请的内容作为本申请的一部分。
技术领域
本文涉及图像处理技术，尤指一种图像去噪方法、装置和存储介质。
背景技术
图像在生成过程中会混杂噪音，而暗光情况下拍摄的图像相对亮光情况下拍摄的图像存在的噪音会更大。噪音给图像质量带来了损失，如图像不清晰，图像失真等。
发明内容
本申请提供了一种图像去噪方法、装置和存储介质，能够去除图像中包含的噪音，提升图像质量。
本申请提供的图像去噪方法，包括：
获取待去噪的图像序列；
将所述待去噪的图像序列输入已训练好的神经网络，通过所述已训练好的神经网络完成去噪操作；
根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列。
作为一示例性实施例，所述方法还包括：
将所述待去噪的图像序列输入已训练好的神经网络之前，对神经网络进行训练；
其中，对神经网络进行训练，包括：
通过获取的样本数据构建自适应训练数据集；
将所述自适应训练数据集输入待训练神经网络；
通过计算预设的损失目标对所述待训练神经网络进行参数调整直至所述待训练神经网络满足预设的收敛条件。
作为一示例性实施例，通过获取的样本数据构建自适应训练数据集，包括：
分别获取多个样本数据，其中，每个样本数据包括带噪图像序列样本，以及与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图；
将所述多个样本数据按比例组合构建所述自适应训练数据集。
作为一示例性实施例，计算所述预设的损失目标时使用的损失函数包含以下任一种或多种：
L1损失函数，L2损失函数，结构相似损失函数，感知损失函数，以及频率损失函数。
作为一示例性实施例，获取每个样本数据的方式，包括：
将通过外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；
通过获得预设数量的带噪图像序列样本，并将所述预设数量的带噪图像序列样本进行叠加后的图像作为与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图。
作为一示例性实施例，将预设数量的带噪图像序列样本进行叠加的方式，包括：
将预设数量的带噪图像序列样本按照像素加权的方式进行叠加，其中，带噪图像序列样本中组成运动物的像素对应的权值小于组成非运动物的像素对应的权值。
作为一示例性实施例，获取每个样本数据的方式，包括：
将具有曝光时间小于预设时间阈值的外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；
将具有曝光时间大于预设时间阈值的外部设备对同一目标物进行拍摄获得的图像作为与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图。
作为一示例性实施例，获取每个样本数据的方式，包括：
将镜头带滤光镜的外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；
将镜头不带滤光镜的外部设备对同一目标物进行拍摄获得的图像作为与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图。
作为一示例性实施例，获取每个样本数据的方式，包括：
将通过外部设备对目标物进行拍摄获得的噪声小于预设阈值的图像作为与带噪图像序列样本匹配的噪声小于预设阈值的参考图；
在所述参考图上增加获取的预设噪声作为所述带噪图像序列样本。
作为一示例性实施例，所述预设噪声的获取方式包括：
根据噪音分布模型模拟获取所述预设噪声；
其中，所述噪音分布模型的待定参数根据所述参考图的不同感光值、所述参考图在不同感光值下对应的像素方差值、以及所述噪声分布模型标定的感光值与像素方差值之间的对应关系获取。
作为一示例性实施例，将所述待去噪的图像序列输入已训练好的神经网络，包括：
将所述待去噪的图像序列进行分块处理后输入已训练好神经网络；
根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列，包括：
将从所述已训练好的神经网络输出的图像块进行融合处理获得的图像序列作为去噪后的图像序列。
作为一示例性实施例，将所述待去噪的图像序列进行分块处理，包括：
将所述待去噪的图像序列按照位置坐标依次分块，且相邻的图像块存在重叠区域；
将从所述已训练好的神经网络输出的图像块进行融合处理，包括：
按照位置坐标依次将图像块和与其具有重叠区域的相邻图像块进行融合处理。
作为一示例性实施例，所述方法还包括：
将所述待去噪的图像序列进行分块处理后输入已训练好神经网络后，通过所述已训练好的神经网络根据不同硬件资源的性能将图像块分配给所述不同硬件资源，由所述不同硬件资源并行处理各自被分配的图像块。
作为一示例性实施例，将所述自适应训练数据集输入待训练神经网络，包括：
将所述自适应训练数据集按照感光值进行分类；
将处于不同类下的自适应训练数据集输入待训练神经网络。
作为一示例性实施例，所述方法还包括：
将所述待去噪的图像序列输入已训练好的神经网络之前，对所述带去噪的图像序列进行归一化处理；
根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列，包括：
将从所述已训练好的神经网络输出的图像序列逆归一化处理得到所述去噪后的图像序列。
本申请实施例还提供了一种图像去噪装置，所述装置包括：
采集单元，设置为获取待去噪的图像序列；
去噪单元，设置为将所述待去噪的图像序列输入已训练好的神经网络，通过所述已训练好的神经网络完成去噪操作；
输出单元，设置为根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列。
本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如前任一所述的方法。
本申请实施例还提供了一种图像去噪装置，包括：存储器和处理器，所述存储器存储有程序，所述程序在被所述处理器读取执行时，实现如前任一所述的方法。
本申请实施例实现了利用神经网络去除图像噪音的操作，提升了图像质量。
本申请的其它特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本申请而了解。本申请的其他优点可通过在说明书以及附图中所描述的方案来实现和获得。
附图说明
附图用来提供对本申请技术方案的理解，并且构成说明书的一部分，与本申请的实施例一起用于解释本申请的技术方案，并不构成对本申请技术方案的限制。
图1为本申请实施例提供的图像去噪方法流程图；
图2为本申请实施例提供的对神经网络进行训练的方法流程图；
图3为本申请实施例提供的一种噪声模型曲线图；
图4A为本申请实施例提供的一种对待去噪的图像序列进行分块和融合的示意图；
图4B为本申请实施例提供的对待去噪图像序列进行分块得到的图像块示意图；
图5为本申请实施例提供的一种图像去噪装置组成模块图；
图6为本申请实施例提供的另一种图像去噪装置组成模块图。
具体实施方式
本申请描述了多个实施例，但是该描述是示例性的，而不是限制性的，并且对于本领域的普通技术人员来说显而易见的是，在本申请所描述的实施例包含的范围内可以有更多的实施例和实现方案。尽管在附图中示出了许多可能的特征组合，并在具体实施方式中进行了讨论，但是所公开的特征的许多其它组合方式也是可能的。除非特意加以限制的情况以外，任何实施例的任何特征或元件可以与任何其它实施例中的任何其他特征或元件结合使用，或可以替代任何其它实施例中的任何其他特征或元件。
本申请包括并设想了与本领域普通技术人员已知的特征和元件的组合。本申请已经公开的实施例、特征和元件也可以与任何常规特征或元件组合，以形成由权利要求限定的独特的发明方案。任何实施例的任何特征或元件也可以与来自其它发明方案的特征或元件组合，以形成另一个由权利要求限定的独特的发明方案。因此，应当理解，在本申请中示出和/或讨论的任何特征可以单独地或以任何适当的组合来实现。因此，除了根据所附权利要求及其等同替换所做的限制以外，实施例不受其它限制。此外，可以在所附权利要求的保护范围内进行各种修改和改变。
此外，在描述具有代表性的实施例时，说明书可能已经将方法和/或过程呈现为特定的步骤序列。然而，在该方法或过程不依赖于本文所述步骤的特定顺序的程度上，该方法或过程不应限于所述的特定顺序的步骤。如本领域普通技术人员将理解的，其它的步骤顺序也是可能的。因此，说明书中阐述的步骤的特定顺序不应被解释为对权利要求的限制。此外，针对该方法和/或过程的权利要求不应限于按照所写顺序执行它们的步骤，本领域技术人员可以容易地理解，这些顺序可以变化，并且仍然保持在本申请实施例的精神和范围内。
本申请实施例提供了一种图像去噪方法，如图1所示，所述方法包括：
步骤S101获取待去噪的图像序列；
步骤S102将所述待去噪的图像序列输入已训练好的神经网络，通过所述已训练好的神经网络完成去噪操作；
输入已训练好的神经网络的待去噪图像序列可以为单帧，多帧叠加后的单帧，或者连续的多帧；其数据格式可以有多种，如RGB，YUV，或RawRGB；此外，针对连续的多帧输入神经网络，需要保持相邻帧对齐；本申请实施例支持对多种格式的图像去噪，能够满足不同终端用户的需求；
步骤S103根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列；
从已训练好的神经网络输出的图像序列为单帧；其数据格式也可以有多种，如RGB，YUV，或RawRGB；如果输入已训练好的神经网络的待去噪图像序列和所需的从已训练好的神经网络输出的图像序列的格式不同，需要进行格式转换；
本申请实施例实现了利用神经网络去除图像噪音的操作，提升了图像质量。
在一示例性实施例中，所述方法还包括：将所述待去噪的图像序列输入已训练好的神经网络之前，对神经网络进行训练。
在一示例性实施例中，如图2所示，对神经网络进行训练的步骤，包括：
步骤S201通过获取的样本数据构建自适应训练数据集；
其中，通过获取的样本数据构建自适应训练数据集，包括：
分别获取多个样本数据，其中，每个样本数据包括带噪图像序列样本，以及与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图；将所述多个样本数据按比例组合构建所述自适应训练数据集，其中，所述多个样本数据具有的比例可设置，并且还可以根据待去噪图像序列的类型以及样本数据采集的难易程度进行调节；
所述参考图可以用来计算损失函数值，以判断所述待训练神经网络是否满足预设的收敛条件；所述与带噪图像序列样本匹配的噪声小于预设阈值的参考图中的匹配指的是带噪图像序列样本与参考图在像素上匹配，即意味着切换带噪图像序列样本的任意一帧和参考图，两者满足像素级别上的对齐，仅存在图像质量上的区别；
步骤S202将所述自适应训练数据集输入待训练神经网络；
神经网络的类型包括但不限于：RNN网络，全卷积网络和Unet网络；
步骤S203通过计算预设的损失目标对所述待训练神经网络进行参数调整直至所述待训练神经网络满足预设的收敛条件；如，通过对所述待训练神经网络进行参数调整使得计算得到的损失函数值越来越小，直至达到预设的损失函数值目标，完成神经网络的收敛。
本申请实施例计算损失函数值时使用的损失函数可以包含以下任一种或多种：
L1损失函数，L2损失函数，结构相似损失函数，感知损失函数，以及频率损失函数。
对神经网络训练结束后，可以冻结网络参数，以使得网络参数固定下来，所述网络参数可以运行在浮点的硬件资源上，包括但不仅限于CPU，图像处理器(Graphic Processing Unit， GPU)，DSP；在硬件加速平台(包括但不仅限于量化平台)上，可以做量化训练和后量化使得网络参数可以整型化，根据不同的输入数据要求，可以做参数和权重的8位量化或者16位量化。
为保证前向性能的最优化，需要GPU和NPU同时运行网络前向的时候，需要保证GPU和NPU的结果的一致性。如果是采用后量化策略，那么通过后量化的精度来确保GPU和NPU的结果的一致性。如果是训练量化，可以在损失函数里再加上一项：GPU模型和NPU模型的结果的监督损失函数，该监督损失函数可以是L1，L2等。
在一示例性实施例中，获取每个样本数据的方式，包括：
方式一，将通过外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；通过获得预设数量的带噪图像序列样本，并将所述预设数量的带噪图像序列样本进行叠加后的图像作为与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图；
因为噪声的随机性，通过叠加预设数量的带噪图像序列样本可以明显使噪声变小，信噪比提高，细节提升，因此叠加后的图像可以用来做参考图。所述预设数量可以根据每张带噪图像序列样本的噪声确定，如噪声较小(小于预设噪声阈值)，所述预设数量可以为6-10张；如噪声较大(大于预设噪声阈值)，所述预设数量可以为60-100张。
其中，将预设数量的带噪图像序列样本进行叠加的方式，包括：
将预设数量的带噪图像序列样本按照像素加权的方式进行叠加，其中，带噪图像序列样本中组成运动物的像素对应的权值小于组成非运动物的像素对应的权值。通过降低运动物的像素对应的权值，可以实现保留图像细节的同时减小相机抖动或运动物带来的图像污染，减小在叠加后的图像中引入鬼影的可能，从而提升对神经网络的训练效果。
该方式一的优点：适用场景多，对采集工具无特别要求；缺点：采集成本高，采集环境要求高，需要保证亮度不变，采集过程没有抖动。
方式二，将具有曝光时间小于预设时间阈值的外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；将具有曝光时间大于预设时间阈值的外部设备对同一目标物进行拍摄获得的图像作为与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图。
一般来说，曝光时间越短，噪声越大；曝光时间越长，噪声越小。
可选的，可以根据增益值*曝光时间＝EV0(EV0指的是曝光量为对应于曝光时间为1秒而光圈为f/1.0的组合或其等效组合)确定曝光时间，将增益值设置得越小，由于EV0固定， 因此得到的曝光时间越长，噪声越小；反之，将增益值设置得越大，由于EV0固定，因此得到的曝光时间越短，噪声越大。
方式二的优点：GT质量相对方式一较好；缺点：采集环境要求高，需要保证亮度不变，采集过程没有抖动。
方式三，将镜头带滤光镜的外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；将镜头不带滤光镜的外部设备对同一目标物进行拍摄获得的图像作为与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图。
由于镜头带滤光镜可以模拟暗光环境，因此可以将镜头带滤光镜的外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；相应的，将镜头不带滤光镜的外部设备对同一目标物进行拍摄获得的图像作为参考图。
该方式三的优点：可采集到极暗场景；缺点：有炫光效应。
方式四，将通过外部设备对目标物进行拍摄获得的噪声小于预设阈值的图像作为参考图；在所述参考图上增加获取的预设噪声作为所述带噪图像序列样本。
其中，所述预设噪声的获取方式可以包括：
根据噪音分布模型模拟获取所述预设噪声；
其中，所述噪音分布模型的待定参数根据所述参考图的不同感光值、所述参考图在不同感光值下对应的像素方差值、以及所述噪声分布模型标定的感光值与像素方差值之间的对应关系获取。
具体的，所述预设噪声分布模型满足泊松高斯分布模型：
Figure PCTCN2022143154-appb-000001
其中，Poison()表示泊松函数，Normal()表示正态分布函数；I表示所述参考图的感光值；
根据所述参考图的不同感光值、所述参考图在不同感光值下对应的像素方差值、以及噪声分布模型
Figure PCTCN2022143154-appb-000002
获得g和
Figure PCTCN2022143154-appb-000003
其中，δ(I) 2表示所述参考图在感光值I下对应的像素方差值。图3给出了一种I～δ(I) 2对应的曲线图，图中横坐标表示感光值I，纵坐标 表示δ(I) 2，曲线的斜率是g，截距是
Figure PCTCN2022143154-appb-000004
该方式四的优点：生成数据快捷，成本低，GT较好；缺点：只适合于特定sensor直出的数据。
综上可以看出，前述四种不同的样本数据获取方式各有优缺点，采集数据的过程可以根据需求组合获取自适应训练数据集。比如raw域去噪，最高感光值可以采用第三种方式获取数据，最低感光值使用第四种方法获取数据，中间感光值可以使用第二种方法获取，以及第一种方法补充部分数据，多种方式获取到的数据集合组成了完整的自适应训练数据集。
在一示例性实施例中，将所述待去噪的图像序列输入已训练好的神经网络，包括：
将所述待去噪的图像序列进行分块处理后输入已训练好的神经网络；
根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列，包括：
将从所述已训练好的神经网络输出的图像块进行融合处理获得的图像序列作为去噪后的图像序列。
当待去噪的图像序列较大，且神经网络运算能力有限时，可以将图像序列进行分割后再输入至神经网络。同样在进行神经网络训练时，也可以将待去噪的图像序列样本进行分块处理后输入待训练神经网络。
在一示例性实施例中，将所述待去噪的图像序列进行分块处理，包括：
将所述待去噪的图像序列按照位置坐标依次分块，且相邻的图像块存在重叠区域；
将从所述已训练好的神经网络输出的图像块进行融合处理，包括：
按照位置坐标依次将图像块和与其具有重叠区域的相邻图像块进行融合处理。
图4A给出了一种对待去噪的图像序列进行分块和融合的示意图；对待去噪图像序列进行分块得到的图像块如图4B所示，图4B中标示的外框所框住的图像块为输入神经网络的图像块，内框和外框之间的部分为外围部分；在进行图像块融合时，相邻图像块的外围部分重叠。
在对待去噪的图像序列进行分块时，可以对得到的每个图像块进行标识，如将每个图像块的中心位置坐标作为所述标识；在进行图像块融合时，按照所述标识进行融合操作。
本申请通过在相邻的图像块间设置重叠区域，可以避免相邻的图像块进行融合后存在明显的分界线。
在一示例性实施例中，所述方法还包括：
将所述待去噪的图像序列进行分块处理后输入已训练好神经网络后，通过所述已训练好的神经网络根据不同硬件资源的性能将图像块分配给所述不同硬件资源，由所述不同硬件资源并行处理各自被分配的图像块。所述硬件资源的类型可以包括GPU、NPU和CPU。如，为保持前向性能的最优化，硬件上可以使用GPU和NPU并行运行，根据GPU和NPU的性能比来为GPU和NPU分配图像块数；比如图像块总数为200块，GPU和NPU的性能比是1：2，那么可以将其中的66块图像块分配给GPU，剩下的图像块分配给NPU。
在一示例性实施例中，所述方法还包括：
将所述待去噪的图像序列输入已训练好的神经网络之前，对所述带去噪的图像序列进行归一化处理；
根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列，包括：
将从所述已训练好的神经网络输出的图像序列进行逆归一化处理得到所述去噪后的图像序列。
通过归一化操作是为了对数据进行标准化处理，消除不同量纲的影响，使数据处于同一数量级，确保神经网络输出结果的准确性。同样在进行神经网络训练时，也可以将待去噪的图像序列样本进行归一化处理后输入待训练神经网络。
在一示例性实施例中，将所述自适应训练数据集输入待训练神经网络，包括：
将所述自适应训练数据集按照感光值进行分类；
将处于不同类下的自适应训练数据集输入待训练神经网络。
通过该实施例，每种感光值可以获得一种已训练好的神经网络，多种感光值对应多种已训练好的神经网络；或者，多种感光值均对应一种已训练好的神经网络，每种感光值对应该已训练好的神经网络的一个通道。感光值越大，噪声越多，对应的训练好的神经网络或神经网络的通道的去噪能力越强，同理，感光值越小，噪声越小，对应的训练好的神经网络或神经网络的通道的去噪能力越强。
可选的，将所述自适应训练数据集按照感光值进行分类，包括：
将所述带噪图像序列样本和所述参考图依次按照感光值为100-1600，感光值为1600-3200，感光值为3200-6400，感光值为6400-12800进行分类。类别划分的越细，精度越高，但去噪维护成本也越高；本申请实施例将感光值划分为以上4类，可以在兼顾精度的 条件下实现对维护成本的控制。
本申请实施例还提供了一种图像去噪装置，如图5所示，所述装置包括：
采集单元501，设置为获取待去噪的图像序列；
去噪单元502，设置为将所述待去噪的图像序列输入已训练好的神经网络，通过所述已训练好的神经网络完成去噪操作；
输入已训练好的神经网络的待去噪图像序列可以为单帧，多帧叠加后的单帧，或者连续的多帧；其数据格式可以有多种，如RGB，YUV，或RawRGB；本申请实施例支持对多种格式的图像去噪，能够满足不同终端用户的需求；
输出单元503，设置为根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列；
从已训练好的神经网络输出的图像序列为单帧；其数据格式也可以有多种，如RGB，YUN，或RawRGB；如果输入已训练好的神经网络的待去噪图像序列和所需的从已训练好的神经网络输出的图像序列的格式不同，需要进行格式转换。
本申请实施例实现了利用神经网络去除图像噪音的操作，提升了图像质量。
本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如前任一实施例所述的方法。
本申请实施例还提供了一种图像去噪装置，如图6所示，包括：存储器601和处理器602，所述存储器601存储有程序，所述程序在被所述处理器602读取执行时，实现如前任一实施例所述的图像去噪方法。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些组件或所有组件可以被实施为由处理器，如数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、 数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。

Claims (18)

  1. 一种图像去噪方法，所述方法包括：
    获取待去噪的图像序列；
    将所述待去噪的图像序列输入已训练好的神经网络，通过所述已训练好的神经网络完成去噪操作；
    根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列。
  2. 根据权利要求1所述的方法，其中，所述方法还包括：
    将所述待去噪的图像序列输入已训练好的神经网络之前，对神经网络进行训练；
    其中，对神经网络进行训练，包括：
    通过获取的样本数据构建自适应训练数据集；
    将所述自适应训练数据集输入待训练神经网络；
    通过计算预设的损失目标对所述待训练神经网络进行参数调整直至所述待训练神经网络满足预设的收敛条件。
  3. 根据权利要求3所述的方法，其中，
    通过获取的样本数据构建自适应训练数据集，包括：
    分别获取多个样本数据，其中，每个样本数据包括带噪图像序列样本，以及与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图；
    将所述多个样本数据按比例组合构建所述自适应训练数据集。
  4. 根据权利要求2所述的方法，其中，
    计算所述预设的损失目标时使用的损失函数包含以下任一种或多种：
    L1损失函数，L2损失函数，结构相似损失函数，感知损失函数，以及频率损失函数。
  5. 根据权利要求3所述的方法，其中，
    获取每个样本数据的方式，包括：
    将通过外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；
    通过获得预设数量的带噪图像序列样本，并将所述预设数量的带噪图像序列样本进行叠 加后的图像作为与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图。
  6. 根据权利要求5所述的方法，其中，
    将预设数量的带噪图像序列样本进行叠加的方式，包括：
    将预设数量的带噪图像序列样本按照像素加权的方式进行叠加，其中，带噪图像序列样本中组成运动物的像素对应的权值小于组成非运动物的像素对应的权值。
  7. 根据权利要求3所述的方法，其中，
    获取每个样本数据的方式，包括：
    将具有曝光时间小于预设时间阈值的外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；
    将具有曝光时间大于预设时间阈值的外部设备对同一目标物进行拍摄获得的图像作为与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图。
  8. 根据权利要求3所述的方法，其中，
    获取每个样本数据的方式，包括：
    将镜头带滤光镜的外部设备对目标物进行拍摄获得的图像作为带噪图像序列样本；
    将镜头不带滤光镜的外部设备对同一目标物进行拍摄获得的图像作为与所述带噪图像序列样本匹配的噪声小于预设阈值的参考图。
  9. 根据权利要求3所述的方法，其中，
    获取每个样本数据的方式，包括：
    将通过外部设备对目标物进行拍摄获得的噪声小于预设阈值的图像作为与带噪图像序列样本匹配的噪声小于预设阈值的参考图；
    在所述参考图上增加获取的预设噪声作为所述带噪图像序列样本。
  10. 根据权利要求9所述的方法，其中，
    所述预设噪声的获取方式包括：
    根据噪音分布模型模拟获取所述预设噪声；
    其中，所述噪音分布模型的待定参数根据所述参考图的不同感光值、所述参考图在不同感光值下对应的像素方差值、以及所述噪声分布模型标定的感光值与像素方差值之间的对应 关系获取。
  11. 根据权利要求1所述的方法，其中，
    将所述待去噪的图像序列输入已训练好的神经网络，包括：
    将所述待去噪的图像序列进行分块处理后输入已训练好神经网络；
    根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列，包括：
    将从所述已训练好的神经网络输出的图像块进行融合处理获得的图像序列作为去噪后的图像序列。
  12. 根据权利要求11所述的方法，其中，
    将所述待去噪的图像序列进行分块处理，包括：
    将所述待去噪的图像序列按照位置坐标依次分块，且相邻的图像块存在重叠区域；
    将从所述已训练好的神经网络输出的图像块进行融合处理，包括：
    按照位置坐标依次将图像块和与其具有重叠区域的相邻图像块进行融合处理。
  13. 根据权利要求11所述的方法，其中，所述方法还包括：
    将所述待去噪的图像序列进行分块处理后输入已训练好神经网络后，通过所述已训练好的神经网络根据不同硬件资源的性能将图像块分配给所述不同硬件资源，由所述不同硬件资源并行处理各自被分配的图像块。
  14. 根据权利要求2所述的方法，其中，
    将所述自适应训练数据集输入待训练神经网络，包括：
    将所述自适应训练数据集按照感光值进行分类；
    将处于不同类下的自适应训练数据集输入待训练神经网络。
  15. 根据权利要求1所述的方法，其中，所述方法还包括：
    将所述待去噪的图像序列输入已训练好的神经网络之前，对所述带去噪的图像序列进行归一化处理；
    根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列，包括：
    将从所述已训练好的神经网络输出的图像序列逆归一化处理得到所述去噪后的图像序列。
  16. 一种图像去噪装置，所述装置包括：
    采集单元，设置为获取待去噪的图像序列；
    去噪单元，设置为将所述待去噪的图像序列输入已训练好的神经网络，通过所述已训练好的神经网络完成去噪操作；
    输出单元，设置为根据所述已训练好的神经网络输出的图像序列得到去噪后的图像序列。
  17. 一种计算机可读存储介质，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如权利要求1至15任一所述的方法。
  18. 一种图像去噪装置，包括：存储器和处理器，所述存储器存储有程序，所述程序在被所述处理器读取执行时，实现如权利要求1至15任一所述的方法。
PCT/CN2022/143154 2021-12-31 2022-12-29 一种图像去噪方法、装置和存储介质 WO2023125750A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111672183.8 2021-12-31
CN202111672183.8A CN116433496A (zh) 2021-12-31 2021-12-31 一种图像去噪方法、装置和存储介质

Publications (1)

Publication Number Publication Date
WO2023125750A1 true WO2023125750A1 (zh) 2023-07-06

Family

ID=86998088

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/143154 WO2023125750A1 (zh) 2021-12-31 2022-12-29 一种图像去噪方法、装置和存储介质

Country Status (2)

Country Link
CN (1) CN116433496A (zh)
WO (1) WO2023125750A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117011183A (zh) * 2023-08-15 2023-11-07 上海为旌科技有限公司 一种暗场景图像降噪方法及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117372269B (zh) * 2023-12-04 2024-03-22 珠海燧景科技有限公司 一种基于学习的拜耳图像去锯齿方法、装置和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050232506A1 (en) * 2004-04-19 2005-10-20 Smith R T Enhancing images superimposed on uneven or partially obscured background
CN102693529A (zh) * 2012-05-24 2012-09-26 杭州电子科技大学 基于延迟自反馈fhn随机共振机制的灰度图像增强方法
CN108280811A (zh) * 2018-01-23 2018-07-13 哈尔滨工业大学深圳研究生院 一种基于神经网络的图像去噪方法和系统
CN110782414A (zh) * 2019-10-30 2020-02-11 福州大学 一种基于密集连接卷积的暗光图像去噪方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050232506A1 (en) * 2004-04-19 2005-10-20 Smith R T Enhancing images superimposed on uneven or partially obscured background
CN102693529A (zh) * 2012-05-24 2012-09-26 杭州电子科技大学 基于延迟自反馈fhn随机共振机制的灰度图像增强方法
CN108280811A (zh) * 2018-01-23 2018-07-13 哈尔滨工业大学深圳研究生院 一种基于神经网络的图像去噪方法和系统
CN110782414A (zh) * 2019-10-30 2020-02-11 福州大学 一种基于密集连接卷积的暗光图像去噪方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117011183A (zh) * 2023-08-15 2023-11-07 上海为旌科技有限公司 一种暗场景图像降噪方法及系统

Also Published As

Publication number Publication date
CN116433496A (zh) 2023-07-14

Similar Documents

Publication Publication Date Title
Lv et al. Attention guided low-light image enhancement with a large scale low-light simulation dataset
US11882357B2 (en) Image display method and device
US11037278B2 (en) Systems and methods for transforming raw sensor data captured in low-light conditions to well-exposed images using neural network architectures
WO2023125750A1 (zh) 一种图像去噪方法、装置和存储介质
US11457138B2 (en) Method and device for image processing, method for training object detection model
CN108229526B (zh) 网络训练、图像处理方法、装置、存储介质和电子设备
US11127117B2 (en) Information processing method, information processing apparatus, and recording medium
WO2019233297A1 (zh) 数据集的构建方法、移动终端、可读存储介质
WO2020001197A1 (zh) 图像处理方法、电子设备、计算机可读存储介质
WO2019233263A1 (zh) 视频处理方法、电子设备、计算机可读存储介质
JP5389903B2 (ja) 最適映像選択
CN112734650B (zh) 一种基于虚拟多曝光融合的不均匀光照图像增强方法
WO2020001196A1 (zh) 图像处理方法、电子设备、计算机可读存储介质
CN111064904A (zh) 一种暗光图像增强方法
US20160155241A1 (en) Target Detection Method and Apparatus Based On Online Training
WO2012170462A2 (en) Automatic exposure correction of images
JP7212554B2 (ja) 情報処理方法、情報処理装置、及びプログラム
CN111047543A (zh) 图像增强方法、装置和存储介质
CN113505702A (zh) 基于双神经网络优化的路面病害识别方法及系统
CN115984570A (zh) 一种视频去噪方法、装置、存储介质和电子装置
US11417125B2 (en) Recognition of license plate numbers from Bayer-domain image data
KR20220001417A (ko) 전자 장치 및 전자 장치의 제어 방법
CN112712004B (zh) 人脸检测系统及人脸检测方法、装置、电子设备
Li et al. Low-Light Image Enhancement Under Non-uniform Dark
Parab et al. Image Enhancement and Exposure Correction Using Convolutional Neural Network

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22915000

Country of ref document: EP

Kind code of ref document: A1