WO2021227934A1 - 信息处理设备和方法、计算机可读存储介质 - Google Patents

信息处理设备和方法、计算机可读存储介质 Download PDF

Info

Publication number
WO2021227934A1
WO2021227934A1 PCT/CN2021/092005 CN2021092005W WO2021227934A1 WO 2021227934 A1 WO2021227934 A1 WO 2021227934A1 CN 2021092005 W CN2021092005 W CN 2021092005W WO 2021227934 A1 WO2021227934 A1 WO 2021227934A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
information processing
processing device
noise
processed
Prior art date
Application number
PCT/CN2021/092005
Other languages
English (en)
French (fr)
Inventor
禹一童
王菁
许宽宏
Original Assignee
索尼集团公司
禹一童
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 索尼集团公司, 禹一童 filed Critical 索尼集团公司
Priority to JP2022569198A priority Critical patent/JP2023525846A/ja
Priority to CN202180033535.7A priority patent/CN115605908A/zh
Priority to US17/916,813 priority patent/US20230153954A1/en
Priority to EP21803938.6A priority patent/EP4152245A4/en
Publication of WO2021227934A1 publication Critical patent/WO2021227934A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10141—Special mode during image acquisition
    • G06T2207/10144—Varying exposure
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present disclosure relates to the field of information processing technology, in particular to processing the black level in an image. More specifically, it relates to an information processing device and method, and a computer-readable storage medium.
  • an information processing device including a processing circuit configured to: generate a synthetic noise image based on an input original image; and perform processing on the black level in the synthetic noise image Processing, and training a machine learning model for image denoising based on the processed synthetic noise image to obtain a trained machine learning model.
  • processing the black level in the synthesized noise image can reduce the influence of the inaccurate black level on the training machine learning model, and therefore can improve the synthesis based on the processed The denoising performance of a trained machine learning model trained on a noisy image.
  • an information processing device including a processing circuit configured to input an image to be processed into a trained machine learning model obtained by the above-mentioned information processing device, so as The image to be processed is subjected to denoising processing.
  • the information processing device can more accurately remove noise in the image to be processed.
  • an information processing method including: generating a synthetic noise image based on an input original image; and processing a black level in the synthetic noise image, and based on the processed synthetic noise Images are used to train a machine learning model for image denoising to obtain a trained machine learning model.
  • an information processing method including: inputting an image to be processed into a trained machine learning model obtained by the above-mentioned information processing device to perform denoising processing on the image to be processed.
  • FIG. 1 is a schematic diagram showing a conventional image sensor.
  • Fig. 2 is a schematic diagram showing a black reference column.
  • Fig. 3 shows a block diagram of functional modules of an information processing device according to an embodiment of the present disclosure.
  • Fig. 4 is a diagram showing an example of a color filter array of a sensor.
  • Fig. 5 is a schematic diagram showing image signal processing performed by a conventional digital camera.
  • FIG. 6 is a schematic diagram illustrating training a machine learning model based on perturbing the black level according to an embodiment of the present disclosure.
  • FIG. 7 is a flowchart showing an example of the flow of an information processing method according to an embodiment of the present disclosure.
  • Fig. 8 shows a block diagram of functional modules of an information processing device according to an embodiment of the present disclosure.
  • FIG. 9 is a flowchart showing an example of the flow of an information processing method according to an embodiment of the present disclosure.
  • FIG. 10 is a block diagram showing an example structure of a personal computer that can be adopted in an embodiment of the present disclosure.
  • the black level is first introduced in conjunction with existing image sensors and the like.
  • FIG. 1 is a schematic diagram showing a conventional image sensor.
  • photons light signals
  • the photodiode which can be equivalent to a sensor array through the lens and RGB color filter.
  • the photodiode converts the light signal into photocurrent through the photoelectric effect.
  • the generated photocurrent is amplified by an amplifier (not shown in Figure 1) and transmitted to an ADC (Analog-to-Digital Converter), where it is finally converted into a digital signal.
  • ADC Analog-to-Digital Converter
  • the photodiode Even in the absence of light, the photodiode generates a reverse direct current (ie, dark current), and therefore, the photodiode is a device that generates dark current.
  • the dark current is offset by setting the black level.
  • the black level refers to the signal level corresponding to when the image data is 0 (that is, when there is no light output) on a display device that has been calibrated.
  • the display device is, for example, a digital camera device (including a digital camera and a digital video camera), a television, a mobile phone, a display, and the like.
  • a digital camera is taken as an example to introduce the method of obtaining the black level in the prior art.
  • the black level was obtained by calibration of Black Reference Columns. Later, due to the development of electronic devices, the dark current change was not very significant.
  • the existing cameras removed the black reference column and adopted a manually set black level (the calculation method of the black level varies with different camera manufacturers. , Is usually set to a scalar, such as 512).
  • Fig. 2 is a schematic diagram showing a black reference column.
  • the black reference column is designed next to the photodiode array and cannot be exposed.
  • the black reference column is 8 rows in size, and the same photodiode as the photodiode array is also installed on the black reference column, but it is not irradiated by light.
  • the signal on the black reference column is also amplified by the amplifier, read out and converted by the ADC.
  • the camera calculates the black level based on the digital signal map obtained by the ADC through the black reference column.
  • the black level artificially set or calculated as described above does not accurately represent the dark current, but will produce a black level error ⁇ , and this error becomes more obvious under dark light.
  • FIG. 3 shows a block diagram of functional modules of an information processing device 300 according to an embodiment of the present disclosure.
  • the information processing device 300 includes: a first processing unit 302, which may be configured to generate a composite based on an input original image Noise image; and a second processing unit 304, which may be configured to process the black level in the synthesized noise image, and train a machine learning model for image denoising based on the processed synthesized noise image to obtain Trained machine learning model.
  • the original image includes RGB data in RAW format (may also be referred to as raw-RGB image or raw image).
  • the raw-RGB image can directly reflect the electrons generated by photons after passing through sensors such as CCD/CMOS (charge coupled device/complementary metal oxide semiconductor).
  • FIG. 4 is a diagram showing an example of a color filter array (CFA) of a sensor (for example, CCD/CMOS).
  • CFA color filter array
  • Sensor data in RAW format usually exists in the form of CFA.
  • CFA is a pixel array of HxW (H and W are the dimensions of the sensor, where H represents height and W represents width), where each pixel carries information about a single color channel of red, green, or blue.
  • CFA provides the following compromise: with the help of a spectrally selectable filter placed on each pixel, information about each of the three color channels is captured at a different location. In the following, let the size of the original image be H x W.
  • the machine learning model is a neural network.
  • the neural network may be a convolutional neural network, etc., for example, it may be a U-net neural network, a Senet neural network, or an SKnet neural network.
  • processing the black level in the synthesized noise image can reduce the impact on the training machine learning model due to the inaccuracy of the black level, and therefore can improve the Denoising performance of the trained machine learning model trained by synthesizing noisy images.
  • training machine learning models based on synthetic noise images saves time and effort.
  • training a machine learning model if you are collecting paired image data including the original image and the real noise image, it means that the collector needs to fix the camera for two shots and ensure that the object cannot move between the two shots.
  • the synthetic noise image it is not necessary to collect the paired data including the original image and the real noise image on the spot, but only need to synthesize the synthetic noise image based on the original image.
  • training a machine learning model based on synthetic noise images instead of training a machine learning model based on a pair of images including the original image and the real noise image, may require less manual labeling costs. The operability of synthetic noisy images makes it possible to train more robust machine learning models with less data.
  • the original image is a long exposure image taken by the image acquisition device in a long exposure state
  • the synthesized noise image is used to simulate a short exposure image taken by the image acquisition device in a dark light environment.
  • the image acquisition device may be a digital photography device (including a digital camera or a digital video camera, etc.).
  • the image acquisition device may also be a device other than a digital photography device that can obtain RGB data in RAW format.
  • the image acquisition device may be the information processing device 300, or the image acquisition device may be different from the information processing device 300 (for example, the image acquisition device may be a digital camera or a digital video camera, and the information processing device 300 may be a mobile phone or a computer, etc.).
  • the image acquisition device and the information processing device 300 are assumed to be the same digital camera.
  • long-exposure images and short-exposure images are real images taken by the image acquisition device, long-exposure images are clean and noise-free images taken under long exposure, and short-exposure images are taken under low light conditions.
  • noisy image, synthetic noise image is generated based on the original image as a long-exposure image, and is used to simulate a short-exposure image.
  • the trained machine learning model can improve the denoising performance of the short exposure image captured in the dark light environment .
  • the above-mentioned trained machine learning model can also be applied to all-weather automatic driving, night face recognition tasks, etc., for example.
  • the first processing unit 302 may be configured to perform noise processing on the original image based on the noise parameters obtained from the original image, thereby generating a synthesized noise image.
  • the noise of the original image taken by the digital camera is mainly shot noise and readout noise.
  • the aforementioned noise parameter may be a parameter related to read noise (referred to as a read noise parameter for short) ⁇ and a parameter related to shot noise (referred to as a shot noise parameter for short) ⁇ obtained from the original image.
  • a read noise parameter for short a parameter related to read noise
  • shot noise parameter for short a parameter related to shot noise
  • the first processing unit 302 may be configured to perform linearization processing on the original image based on the black level and white level obtained from the original image to obtain a linearized image of the original image; and the first processing unit 302 It can be configured to perform noise processing on the linearized image, thereby generating a synthetic noise image.
  • the white level refers to the signal level when the image data is peak white on a display device that has been calibrated.
  • the display device is, for example, a digital camera device (including a digital camera and a digital video camera), a television, a mobile phone, a display, and the like.
  • the pixel value of the pixel in the original image be RawImage
  • the white level is n
  • the black level is m
  • the pixel value y of the corresponding pixel in the linearized image of the original image (in the following, for convenience of description, sometimes The linearized image of the original image is referred to as image y) can be expressed as:
  • each pixel in the original image is limited to a value in the range of 0-1, thereby obtaining the linearized image y of the original image.
  • the gain multiple of the linearized image y can also be changed, for example, the pixels in the partial image of the linearized image y are divided by a predetermined value (for example, 100 or 300).
  • the noise n can be expressed as at least one of n ⁇ N(0,( ⁇ y+ ⁇ ))), etc., to generate a synthetic noise image with a pixel value of z (hereinafter, the synthetic noise image is simply referred to as a synthetic noise image z).
  • the second processing unit 304 may be configured to obtain predetermined image acquisition parameters from the original image, obtain a corresponding offset image based on the correspondence between the predetermined image acquisition parameters and the offset image used to process the black level, and The black level in the synthesized noise image is processed based on the obtained offset image.
  • the information processing device 300 processes the black level in the synthesized noise image based on the obtained bias image, so that the black level in the processed synthesized noise image more accurately represents the dark current, and therefore The black level error ⁇ is reduced, so that the processed synthesized noise image is closer to the real noise image.
  • the correspondence relationship is generated in the following way: in a dark light environment, based on the black image captured for each predetermined image acquisition parameter of the image acquisition device, a bias image corresponding to each predetermined image acquisition parameter is obtained .
  • the corresponding relationship generated in this way is beneficial for processing the black level in the synthesized noise image in a dark light environment.
  • the black image may be an image taken when the image acquisition component in the image acquisition device is covered by a black object.
  • a black object Those skilled in the art can also think of other ways to capture the black image, which will not be repeated here.
  • the image acquisition device is a digital photography device and the predetermined image acquisition parameters include the exposure time and sensitivity (ISO) of the digital photography device;
  • ISO exposure time and sensitivity
  • Each parameter pair of the exposure time and sensitivity of the device takes a black image, and preprocesses the black image to obtain a bias image corresponding to each parameter pair.
  • the image acquisition device may be a device that can obtain RGB data in RAW format other than a digital photography device, and correspondingly, the predetermined image acquisition parameters may be other parameters except the exposure time and sensitivity, which are not here anymore. Tired out.
  • the image acquisition device is taken as an example of a digital photography device.
  • the digital photographing device is sometimes referred to as a digital camera or a camera.
  • the reason for using the exposure time and sensitivity of the digital camera as the predetermined image acquisition parameters is that the black level error ⁇ is affected by the exposure time and sensitivity.
  • the dark current will be amplified by the amplifier, and at the same time, the black level error ⁇ will also be amplified, so the black level error ⁇ will be affected by the sensitivity; on the other hand, when the exposure time becomes longer, the photodiode is switched on and off The accumulated photocurrent and dark current before closing will increase, so the black level error ⁇ will be affected by the exposure time.
  • the camera output format For example, to obtain a digital camera, with or without lens, close the lid of the camera, set the camera output format to .RAW format file, and set the remaining camera parameters to default values; for each pair of ISO value and exposure value, shoot Get a black image. For example, assuming that the camera's exposure time is 0.1s and 1s, and the ISO value is 100 and 200, there are four sets of ISO value and exposure value pairs: (0.1s, 100), (1s, 100), (0.1s, 200), (1s, 200), therefore, a total of 4 black images can be captured.
  • preprocessing the black image includes linearizing the black image based on the black level and white level of the digital camera.
  • the black level of the digital photography device is m'
  • the white level is n'
  • the pixel in the black image is BlackFrame
  • the corresponding pixel in the bias image is BiasFrame
  • each pixel in the black image is limited to the range of 0-1, thereby obtaining a biased image.
  • the second processing unit 304 may be configured to add the obtained offset image to the synthesized noise image.
  • the second processing unit 304 may add the value of each pixel of the obtained offset image to the corresponding pixel in the synthesized noise image z to process the black level in the synthesized noise image z, thereby generating the Processed synthetic noise image.
  • the second processing unit 304 may be configured to select an image block from a predetermined position in the obtained offset image, and add the offset value calculated based on the selected image block to the synthesized noise image.
  • the height of the offset image is H and the width is W
  • the predetermined position is ps.
  • pixels whose positions are located in the range of H/2 to H/2+ps in the height direction and W/2 to W/2+ps in the width direction can be selected from the obtained offset image as the selected image block.
  • any image block can be selected from the obtained offset image as the selected image block.
  • ps can be determined according to experience or application scenarios or experiments.
  • ps can be 128.
  • ps can be determined such that the value corresponding to any image block can be selected from the obtained offset image.
  • the average value of the pixels in the selected image block can be calculated, and the average value can be added to the value of each pixel of the synthesized noise image z to process the black level in the synthesized noise image z, thereby generating processed The synthetic noise image.
  • the average value can be added to the value of each pixel of the synthesized noise image z to process the black level in the synthesized noise image z, thereby generating processed The synthetic noise image.
  • values other than the mean value can also be calculated based on the values of the pixels in the selected image block, so as to process the black level in the synthesized noise image z.
  • the second processing unit 304 may be configured to select a pixel from a predetermined position in the obtained bias image, and add the value of the selected pixel to the synthesized noise image.
  • the value of the pixel located at [H/2,W/2] can be selected from the obtained offset image, and added to the value of each pixel in the synthesized noise image z to determine the black power in the synthesized noise image z. Ping performs processing to generate a processed synthetic noise image.
  • the value of a pixel located at any position can be selected from the obtained offset image to process the black level in the synthesized noise image z.
  • the values of R, G1, B, and G2 of the processed synthetic noise image can be formed into a separate channel, and the four channels can be cascaded together.
  • the image data of each channel (the data size of each channel is H/2 ⁇ W/2) can be separately input into the machine learning model as samples.
  • the original image is linearized, and noise is added to the linearized image y to obtain the synthesized noise image z, and the synthesized noise image z is processed with black level to obtain the processed synthesized noise image .
  • the values of R, G1, B, and G2 of the linearized image y can be formed into a separate channel, and then on each channel of R, G1, B, G2 of the linearized image y Add noise separately, and then process the black level on the R, G1, B, and G2 channels respectively, and input the processed image data of each channel as samples into the machine learning model.
  • the display image corresponding to the original image and captured by the image acquisition device is used as a label image for training the machine learning model.
  • the R, G1, B, and G2 values of the label image can be formed into a separate channel to correspond to the R, G1, B, and G2 channels of the processed synthetic noise image.
  • the digital camera can be regarded as a signal conversion tool: the digital camera converts the light signal in the real scene into a digital signal (optical signal -> electrical signal -> digital signal), and the conversion process is completed by the hardware of the digital camera.
  • the digital signal obtained from the digital camera is the original image described above.
  • the digital camera performs image signal processing to convert the digital signal (original image) into an image that can be used for display (display image).
  • the original image is a RAW format image taken by the digital camera
  • the display image is The s-RGB format image displayed on the screen of the digital camera after converting the original image.
  • Fig. 5 is a schematic diagram showing image signal processing performed by a conventional digital camera.
  • the linearization process is followed by white balance, which is used to eliminate the effect of colored light sources on the color of objects. For example, photographing red apples under red light and green light respectively, the results may be completely different, but the "color" of red apples (the distribution of spectral reflections on the apple's surface) should be an objective fact. Should change with the light source. In order to obtain the original color of the object, this problem is solved by white balance. In implementation, it is done by multiplying the three multipliers [R, G, B] to the image in a dot multiplication manner.
  • demosaicing In Figure 5, white balance is followed by demosaicing.
  • the purpose of demosaicing is to turn the previous 1-channel original image into a 3-channel image that can be displayed on the display device. Its essence is a linear interpolation operation. That is, the original image of size H ⁇ W becomes three images of size H ⁇ W (images in s-RGB format).
  • demosaicing is color space conversion.
  • the three-dimensional vector (R, G, B) is used to represent the value of a certain pixel, that is, (R, G, B) is the color base used by the digital camera.
  • the color substrate used by the digital camera and the color substrate of the display device are not the same, and the purpose of the color space conversion operation is to change the color substrate of the digital camera into a substrate usable on the display device.
  • the color space conversion is realized by multiplying the demosaiced image by a 3 ⁇ 3 conversion matrix, which is a fixed value.
  • the s-RGB format image can be obtained through color space conversion. That is, the image obtained by image signal processing is an image (display image) in the s-RGB format.
  • the parameters of the machine learning model can be updated based on the display image (tag image) and the processed synthetic noise image.
  • the display image and the processed synthetic noise image can be calculated in the average distance under the second paradigm, and the parameters of the machine learning model can be updated according to back propagation, so as to obtain the trained machine learning model.
  • the second processing unit 304 may be configured to extract an offset value from a value range of offset values obeying a predetermined distribution, and perform processing on the black level of an image block in the synthesized noise image based on the extracted offset value. Process and train the machine learning model based on the processed image blocks.
  • the image block may be a 1024 ⁇ 1024 image block.
  • Processing the black level of the image block in the synthesized noise image based on the extracted offset value essentially perturbs the black level of the image block.
  • the information processing device 300 can enhance the robustness of the machine learning model to the black level by perturbing the black level.
  • the second processing unit 304 may add the extracted offset value to each pixel of the image block in the synthesized noise image z to process the black level in the image block.
  • the predetermined distribution may be a Gaussian distribution or a uniform distribution.
  • the parameters of the distribution function of the aforementioned predetermined distribution are determined based on experience or application scenarios or experiments.
  • the uniform distribution can be a uniform distribution that obeys (-10,10).
  • the second processing unit 304 may be configured to randomly extract a bias value from the above numerical range.
  • FIG. 6 is a schematic diagram illustrating training a machine learning model based on perturbing the black level according to an embodiment of the present disclosure.
  • an image block with a size of 1024 ⁇ 1024 is taken as an example for description.
  • an image block is extracted from the synthetic noise image z generated by linearizing the original image, adding noise, and so on.
  • the black level in the image block is processed by perturbation.
  • the values of R, G1, B, and G2 of the processed image block are respectively formed into a separate channel to obtain four data blocks of (1024/2) x (1024/2) size (ie, The size of each data block is 512x512) and these four channels are cascaded together.
  • the above four 512x 512 data blocks are input as samples into the machine learning model, and the original image is processed by the image signal in (5) in the display image in the s-RGB format. The corresponding image blocks are used as training labels to train the machine learning model.
  • the average distance between the image block as the sample and the image block as the label can be calculated in the second paradigm, and the parameters of the machine learning model can be updated according to back propagation, so as to obtain the trained machine learning model.
  • the image signal processing in (5) is the same as the image signal processing in FIG. 5.
  • the black level of the synthesized noise image can also be disturbed based on the extracted offset value (for example, , Add the extracted offset value to each pixel of the synthesized noise image), which will not be repeated here.
  • the extracted offset value for example, , Add the extracted offset value to each pixel of the synthesized noise image
  • the present disclosure also provides an embodiment of an information processing method.
  • FIG. 7 is a flowchart showing an example of the flow of an information processing method S700 according to an embodiment of the present disclosure.
  • the information processing method S700 starts from S702.
  • a synthesized noise image is generated based on the input original image.
  • the black level in the synthesized noise image is processed, and a machine learning model for denoising the image is trained based on the processed synthesized noise image to obtain a trained machine learning model.
  • the information processing method S700 ends at S708.
  • processing the black level in the synthesized noise image can reduce the influence of the inaccurate black level on the training machine learning model, and therefore can improve the synthesis based on the processed The denoising performance of a trained machine learning model trained on a noisy image.
  • an information processing device 800 is also provided.
  • FIG. 8 shows a functional module block diagram of an information processing device 800 according to an embodiment of the present disclosure.
  • the information processing device 800 includes: a third processing unit 802, which may be configured to input the image to be processed into The trained machine learning model obtained by the information processing device 300 or the information processing method S700 is used to perform denoising processing on the image to be processed.
  • the information processing device 800 can more accurately remove the noise in the image to be processed .
  • the image to be processed is a short-exposure image taken in a dark light environment.
  • the information processing device 800 can more accurately remove noise in a short exposure image captured in a dark light environment.
  • the present disclosure also provides an embodiment of an information processing method.
  • FIG. 9 is a flowchart showing an example of the flow of an information processing method S900 according to an embodiment of the present disclosure.
  • the information processing method S900 starts from S902.
  • the image to be processed is input to the trained machine learning model obtained by the information processing device 300 or the information processing method S700 to perform denoising processing on the image to be processed.
  • the information processing method S900 ends in S906.
  • the information processing method S900 can more accurately remove the noise in the image to be processed .
  • the present invention also proposes a program product storing machine-readable instruction codes.
  • the instruction code is read and executed by a machine, the above-mentioned method according to the embodiment of the present invention can be executed.
  • Storage media include, but are not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, and so on.
  • a computer with a dedicated hardware structure (such as the general-purpose computer 1000 shown in FIG. 10) is installed from a storage medium or a network to the program constituting the software, and various programs are installed on the computer. When, it can perform various functions and so on.
  • a central processing unit (CPU) 1001 performs various processes in accordance with a program stored in a read only memory (ROM) 1002 or a program loaded from a storage part 1008 to a random access memory (RAM) 1003.
  • ROM read only memory
  • RAM random access memory
  • data required when the CPU 1001 executes various processing and the like is also stored as necessary.
  • the CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004.
  • the input/output interface 1005 is also connected to the bus 1004.
  • the following components are connected to the input/output interface 1005: input part 1006 (including keyboard, mouse, etc.), output part 1007 (including display, such as cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.), Storage part 1008 (including hard disk, etc.), communication part 1009 (including network interface card such as LAN card, modem, etc.).
  • the communication section 1009 performs communication processing via a network such as the Internet.
  • the driver 1010 can also be connected to the input/output interface 1005 according to needs.
  • Removable media 1011 such as magnetic disks, optical disks, magneto-optical disks, semiconductor memory, etc. are mounted on the drive 1010 as needed, so that the computer program read out therefrom is installed into the storage section 1008 as needed.
  • a program constituting the software is installed from a network such as the Internet or a storage medium such as a removable medium 1011.
  • this storage medium is not limited to the removable medium 1011 shown in FIG. 10 in which the program is stored and distributed separately from the device to provide the program to the user.
  • removable media 1011 include magnetic disks (including floppy disks (registered trademarks)), optical disks (including compact disk read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including mini disks (MD) (registered Trademark)) and semiconductor memory.
  • the storage medium may be a ROM 1002, a hard disk included in the storage part 1008, etc., in which programs are stored and distributed to users along with the device containing them.
  • each component or each step can be decomposed and/or recombined.
  • decomposition and/or recombination should be regarded as equivalent solutions of the present invention.
  • the steps of performing the above-mentioned series of processing can naturally be performed in chronological order in the order of description, but do not necessarily need to be performed in chronological order. Some steps can be performed in parallel or independently of each other.
  • This technology can also be implemented as follows.
  • An information processing equipment including:
  • the processing circuit is configured as:
  • the black level in the synthetic noise image is processed, and a machine learning model for denoising the image is trained based on the processed synthetic noise image to obtain a trained machine learning model.
  • Supplement 2 The information processing device according to Supplement 1, wherein the processing circuit is configured to acquire a predetermined image acquisition parameter from the original image, based on the predetermined image acquisition parameter and the processing black level The corresponding relationship between the offset images obtains the corresponding offset image, and the black level in the synthesized noise image is processed based on the obtained offset image.
  • a bias image corresponding to each predetermined image acquisition parameter is obtained.
  • the image acquisition device is a digital photography device and the predetermined image acquisition parameters include the exposure time and sensitivity of the digital photography device;
  • Supplement 5 The information processing device according to Supplement 4, wherein the preprocessing of the black image includes: linearizing the black image based on the black level and white level of the digital photographing device .
  • Supplement 6 The information processing device according to any one of Supplements 2 to 5, wherein the processing circuit is configured to add the obtained bias image to the synthesized noise image.
  • Supplement 7 The information processing device according to any one of Supplements 2 to 5, wherein the processing circuit is configured to select an image block from a predetermined position in the obtained offset image, and based on the selected The offset value calculated for the image block is added to the synthesized noise image.
  • Supplement 8 The information processing device according to any one of Supplements 2 to 5, wherein the processing circuit is configured to select a pixel from a predetermined position in the obtained bias image, and the selected pixel Values are added to the synthesized noise image.
  • Supplement 9 The information processing device according to Supplement 1, wherein the processing circuit is configured to:
  • the black level of the image block in the synthesized noise image is processed based on the extracted offset value, and the machine learning model is trained based on the processed image block.
  • Supplement 10 The information processing device according to Supplement 9, wherein the predetermined distribution is Gaussian distribution or uniform distribution.
  • Supplement 11 The information processing device according to Supplement 9 or 10, wherein the processing circuit is configured to randomly extract the offset value from the numerical range.
  • Supplement 12 The information processing device according to any one of Supplements 1 to 11, wherein the processing circuit is configured to perform noise processing on the original image based on noise parameters obtained from the original image , Thereby generating the synthesized noise image.
  • Supplement 13 The information processing device according to Supplement 12, wherein the processing circuit is configured to:
  • the noise processing is performed on the linearized image, thereby generating the synthesized noise image.
  • Supplement 14 The information processing device according to any one of Supplements 1 to 13, wherein the original image is a long exposure image taken by an image acquisition device in a long exposure state, and the composite noise image is used To simulate the short-exposure image taken by the image acquisition device in a dark light environment.
  • the original image includes RGB data in RAW format.
  • the machine learning model is a neural network.
  • An information processing device comprising:
  • the processing circuit is configured as:
  • the image to be processed is input to a trained machine learning model obtained by the information processing device according to any one of appendix 1 to 16 to perform denoising processing on the image to be processed.
  • Supplement 18 The information processing device according to Supplement 17, wherein the image to be processed is a short exposure image taken in a dark light environment.
  • Attachment 19 An information processing method, including:
  • the black level in the synthetic noise image is processed, and a machine learning model for denoising the image is trained based on the processed synthetic noise image to obtain a trained machine learning model.
  • An information processing method including:
  • the image to be processed is input to a trained machine learning model obtained by the information processing device according to any one of appendix 1 to 16 to perform denoising processing on the image to be processed.
  • Supplement 21 A computer-readable storage medium having computer-executable instructions stored thereon, and when the computer-executable instructions are executed, the information processing method according to Supplement 19 or 20 is executed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本申请涉及一种信息处理设备和方法、计算机可读存储介质。其中，信息处理设备包括处理电路，处理电路被配置为：基于输入的原始图像生成合成噪声图像；以及对合成噪声图像中的黑电平进行处理，并基于经处理的合成噪声图像来训练用于对图像去噪的机器学习模型，以得到经训练的机器学习模型。

Description

信息处理设备和方法、计算机可读存储介质
本申请要求于2020年5月14日提交中国专利局、申请号为202010408019.5、发明名称为“信息处理设备和方法、计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及信息处理技术领域，具体地涉及对图像中的黑电平进行处理。更具体地，涉及一种信息处理设备和方法、计算机可读存储介质。
背景技术
在图像去噪领域，因为合成数据易于获取和分析，因此常使用合成数据训练基于深度学习的去噪模型。然而，研究表明，在真实场景中测试经训练后的去噪模型时，基于合成噪声图像训练得到的去噪模型不能很好地对真实场景下的图像去噪，从而影响去噪后的图像的质量。这种现象在暗光条件下会更加严重。这是由于合成噪声图像中的黑电平不准确而导致的。
发明内容
在下文中给出了关于本发明的简要概述，以便提供关于本发明的某些方面的基本理解。应当理解，这个概述并不是关于本发明的穷举性概述。它并不是意图确定本发明的关键或重要部分，也不是意图限定本发明的范围。其目的仅仅是以简化的形式给出某些概念，以此作为稍后论述的更详细描述的前序。
根据本公开的一个方面，提供了一种信息处理设备，其包括处理电路，该处理电路被配置为：基于输入的原始图像生成合成噪声图像；以及对所述合成噪声图像中的黑电平进行处理，并基于经处理的合成噪声图像来训练用于对图像去噪的机器学习模型，以得到经训练的机器学习模型。
在根据本公开的实施例的信息处理设备中，对合成噪声图像中的黑 电平进行处理能够减轻由于黑电平不准确而对训练机器学习模型造成的影响，因此能够提高基于经处理的合成噪声图像而训练得到的经训练的机器学习模型的去噪性能。
根据本公开的另一个方面，提供了一种信息处理装置，其包括处理电路，该处理电路被配置为：将待处理图像输入到通过上述信息处理设备得到的经训练的机器学习模型，以对所述待处理图像进行去噪处理。
根据本公开实施例的信息处理装置能够更准确地去除待处理图像中的噪声。
根据本公开的另一个方面，提供了一种信息处理方法，包括：基于输入的原始图像生成合成噪声图像；以及对所述合成噪声图像中的黑电平进行处理，并基于经处理的合成噪声图像来训练用于对图像去噪的机器学习模型，以得到经训练的机器学习模型。
根据本公开的另一个方面，提供了一种信息处理方法，包括：将待处理图像输入到通过上述信息处理设备得到的经训练的机器学习模型，以对所述待处理图像进行去噪处理。
依据本发明的其它方面，还提供了用于实现上述信息处理方法的计算机程序代码和计算机程序产品以及其上记录有该用于实现上述用于信息处理方法的计算机程序代码的计算机可读存储介质。
附图说明
为了进一步阐述本发明的以上和其它优点和特征，下面结合附图对本发明的具体实施方式作进一步详细的说明。附图连同下面的详细说明一起包含在本说明书中并且形成本说明书的一部分。具有相同的功能和结构的元件用相同的参考标号表示。应当理解，这些附图仅描述本发明的典型示例，而不应看作是对本发明的范围的限定。在附图中：
图1是示出现有的图像传感器的概要图。
图2是示出黑基准列的概要图。
图3示出了根据本公开实施例的信息处理设备的功能模块框图。
图4是示出传感器的滤色器阵列的示例的图。
图5是示出现有的数字相机所进行的图像信号处理的示意图。
图6是示出根据本公开实施例的基于对黑电平进行扰动来训练机器学习模型的示意图。
图7是示出根据本公开实施例的信息处理方法的流程示例的流程图。
图8示出了根据本公开实施例的信息处理装置的功能模块框图。
图9是示出根据本公开实施例的信息处理方法的流程示例的流程图。
图10是示出作为本公开实施例中可采用的个人计算机的示例结构的框图。
具体实施方式
在下文中，首先结合现有的图像传感器等介绍黑电平。
图1是示出现有的图像传感器的概要图。如图1所示，光子(光信号)经过镜头和RGB颜色滤镜照射在光电二极管(可以等效于传感器)阵列上，光电二极管通过光电效应将光信号转换为光电流，每一个光电二极管所产生的光电流会被放大器(在图1中未示出)放大并被传送至ADC(模数转换器)，在ADC中最终转换为数字信号。
即使在无光照的情况下，光电二极管也会产生反向直流电流(即，暗电流)，因此，光电二极管是产生暗电流的器件。为了解决这一问题，在现有技术中，通过设置黑电平来抵消暗电流。
黑电平是指在经过一定校准的显示装置上，图像数据为0时(即，没有光亮输出时)对应的信号电平。显示装置例如为数字摄像装置(包括数字相机和数字摄像机)、电视、手机、显示器等。下面以数字相机为例来介绍现有技术中获取黑电平的方法。起初，黑电平的获取是通过黑基准列(Black Reference Columns)标定得到的。后来，因为电子器件的发展，暗电流的变动并不是十分显著，现有相机便去除了黑基准列，并采用人工设定的黑电平(黑电平的计算方法随相机制造厂家不同而不同，通常是被设定为一个标量，例如512)。
图2是示出黑基准列的概要图。
如图2所示，黑基准列是设计在光电二极管阵列旁边，无法被曝光 的区域。一般的，黑基准列为8行大小，黑基准列上也装有和光电二极管阵列一样的光电二极管，只不过不被光照射而已。黑基准列上的信号也会被放大器放大，被读出并被进行ADC转换。最终，相机根据黑基准列经过ADC得到的数字信号图计算出黑电平。
如上所述人为设定或计算的黑电平并不能精确地代表暗电流，而是会产生黑电平误差ε，且这个误差在暗光下变得较为明显。
在下文中将结合附图对本公开的示范性实施例进行描述。为了清楚和简明起见，在说明书中并未描述实际实施方式的所有特征。然而，应该了解，在开发任何这种实际实施例的过程中必须做出很多特定于实施方式的决定，以便实现开发人员的具体目标，例如，符合与系统及业务相关的那些限制条件，并且这些限制条件可能会随着实施方式的不同而有所改变。此外，还应该了解，虽然开发工作有可能是非常复杂和费时的，但对得益于本公开内容的本领域技术人员来说，这种开发工作仅仅是例行的任务。
在此，还需要说明的一点是，为了避免因不必要的细节而模糊了本公开，在附图中仅仅示出了与根据本公开的方案密切相关的设备结构和/或处理步骤，而省略了与本公开关系不大的其它细节。
下面结合附图详细说明根据本公开的实施例。
图3示出了根据本公开实施例的信息处理设备300的功能模块框图，如图3所示，信息处理设备300包括：第一处理单元302，其可以被配置为基于输入的原始图像生成合成噪声图像；以及第二处理单元304，其可以被配置为对合成噪声图像中的黑电平进行处理，并基于经处理的合成噪声图像来训练用于对图像去噪的机器学习模型，以得到经训练的机器学习模型。
作为示例，原始图像包括RAW格式的RGB数据(也可称为raw-RGB图像或raw图像)。raw-RGB图像可以直接反映光子经过传感器例如CCD/CMOS(电荷耦合器件/互补金属氧化物半导体)后生成电子的情况。图4是示出传感器(例如，CCD/CMOS)的滤色器阵列(CFA)的示例的图。RAW格式的传感器数据通常以CFA的形式存在。CFA是HxW(H和W是传感器的维度，其中，H表示高度，W表示宽度)大小的像素阵列，其中，每个像素携带关于红、绿或蓝单个颜色通道的信息。由于落 在CMOS中的任何给定光电二极管上的光被记录为电容器中的一些电子的数量，因此，其仅可以被保存为标量值；单个像素不能保留可观测到的光的三维特性。CFA提供了以下折中：借助于置于每个像素上的谱可选择滤波器在不同的位置捕获关于三个颜色通道中的每个通道的信息。在下文中，记原始图像的大小为H x W。
作为示例，机器学习模型是神经网络。例如，神经网络可以是卷积神经网络等，例如可以是U-net神经网络、Senet神经网络或SKnet神经网络等。
在根据本公开的实施例的信息处理设备300中，对合成噪声图像中的黑电平进行处理能够减轻由于黑电平不准确而对训练机器学习模型造成的影响，因此能够提高基于经处理的合成噪声图像而训练得到的经训练的机器学习模型的去噪性能。
另外，基于合成噪声图像训练机器学习模型省时省力。在训练机器学习模型时，如果是采集包括原始图像和真实噪声图像的成对图像数据，意味着采集人员需要固定相机进行两次拍摄，并且保证两次拍摄之间物体不能移动。然而，由于有合成噪声图像，不必实地采集成包括原始图像和真实噪声图像的成对数据，而只需要基于原始图像来合成合成噪声图像即可。另外，基于合成噪声图像训练机器学习模型、而不是基于包括原始图像和真实噪声图像的成对图像来训练机器学习模型，可以需要更少的人工标注成本。合成噪声图像的可操作性使得可以使用更少的数据训练出更加稳健的机器学习模型。
作为示例，原始图像是图像获取装置在长曝光状态下拍摄得到的长曝光图像，以及合成噪声图像用于仿真图像获取装置在暗光环境下拍摄得到的短曝光图像。
例如，图像获取装置可以是数码摄影装置(包括数字相机或数字摄像机等)。本领域技术人员可以理解，图像获取装置也可以是除了数码摄影装置之外的可以获得RAW格式的RGB数据的装置。
例如，图像获取装置可以就是信息处理设备300，或者图像获取装置可以不同于信息处理设备300(例如，图像获取装置可以是数字相机或数字摄像机，信息处理设备300可以是手机或计算机等等)。在下文中，为 了方便描述，如果没有特意说明，令图像获取装置和信息处理设备300为同一数字相机。
例如，长曝光图像和短曝光图像是图像获取装置真实拍摄得到的图像，长曝光图像是在长曝光状态下拍摄得到的干净的没有噪声的图像，短曝光图像是在暗光环境下拍摄得到的有噪声的图像，合成噪声图像是基于作为长曝光图像的原始图像而生成的，用来仿真短曝光图像。
在合成噪声图像是用于仿真图像获取装置在暗光环境下拍摄得到的短曝光图像的情况下，经训练的机器学习模型对在暗光环境下拍摄得到的短曝光图像的去噪性能得以提高。上述经训练的机器学习模型例如还可应用于全天候的自动驾驶、夜间人脸识别任务等。
作为示例，第一处理单元302可以被配置为基于从原始图像获取的噪声参数，对原始图像进行噪声化处理，从而生成合成噪声图像。
例如，由数字摄像装置拍摄的原始图像的噪声主要是散粒噪声和读出噪声。上述噪声参数可以是从原始图像获取的有关读出噪声的参数(简称为读出噪声参数)α和有关散粒噪声的参数(简称为散粒噪声参数)β。本领域技术人员还可以想到噪声参数的其他示例，这里不再累述。
作为示例，第一处理单元302可以被配置为基于从原始图像获取的黑电平和白电平，对原始图像进行线性化处理，以得到原始图像的线性化后的图像；以及第一处理单元302可以被配置为对线性化后的图像进行噪声化处理，从而生成合成噪声图像。
例如，白电平是指在经过一定校准的显示装置上，图像数据为峰值白时对应的信号电平。如上所述，显示装置例如为数字摄像装置(包括数字相机和数字摄像机)、电视、手机、显示器等。
令原始图像中的像素的像素值为RawImage，白电平为n以及黑电平为m，则原始图像的线性化后的图像中的相应像素的像素值y(在下文中，为了方便描述，有时将原始图像的线性化后的图像简称为图像y)可以表示为：
Figure PCTCN2021092005-appb-000001
在表达式(1)中，max{}表示取最大值以及min{}表示取最小值。通过基于表达式(1)进行的线性化处理，将原始图像中的每个像素限制到0-1范围内的值，从而得到原始图像的线性化后的图像y。
例如，还可以改变线性化后的图像y的增益倍数，例如，将线性化后的图像y的局部图像中的像素除以预定值(例如，100或300)。
例如，可以基于从原始图像获取的读出噪声参数α和散粒噪声参数β，向图像y添加高斯噪声、泊松噪声、高斯-泊松噪声、异方差-高斯(Heteroscedastic-Gaussian)噪声(该噪声n可以表示为n～N(0,(β×y+α)))等中的至少之一，生成像素值为z的合成噪声图像(在下文中，将该合成噪声图像简称为合成噪声图像z)。
作为示例，第二处理单元304可以被配置为从原始图像获取预定图像获取参数，基于预定图像获取参数与用于处理黑电平的偏置图像之间的对应关系得到相应的偏置图像，并基于所得到的偏置图像对合成噪声图像中的黑电平进行处理。
根据本公开实施例的信息处理设备300通过基于所得到的偏置图像对合成噪声图像中的黑电平进行处理，使得经处理的合成噪声图像中的黑电平更精确地代表暗电流，因此降低了黑电平误差ε，从而使得经处理的合成噪声图像更接近真实噪声图像。
作为示例，对应关系是通过以下方式生成的：在暗光环境下，基于针对图像获取装置的每个预定图像获取参数拍摄得到的黑图像，得到与每个预定图像获取参数相对应的偏置图像。通过该方式生成的对应关系有利于在暗光环境下对合成噪声图像中的黑电平进行处理。
例如，黑图像可以是在图像获取装置中的图像获取部件被黑色物体遮盖住的情况下而拍摄的图像。本领域技术人员还可以想到拍摄得到黑图像的其他方式，这里不再累述。
作为示例，图像获取装置是数码摄影装置以及预定图像获取参数包括数码摄影装置的曝光时间和感光度(ISO)；以及对应关系是通过以下方式生成的：合上数码摄影装置的盖子，针对数码摄影装置的曝光时间和感光度的每个参数对拍摄黑图像，并且对黑图像进行预处理，以得到 与每个参数对相对应的偏置图像。
如上所述，图像获取装置可以是除了数码摄影装置之外的可以获得RAW格式的RGB数据的装置，以及相应地预定图像获取参数可以是除了曝光时间和感光度之外的其他参数，这里不再累述。在下文中以图像获取装置为数码摄影装置例来进行描述。在下文中，为了便于描述，有时将数码摄影装置称为数字相机或相机。
使用数码摄影装置的曝光时间和感光度作为预定图像获取参数的原因在于，黑电平误差ε会受到曝光时间和感光度的影响。一方面，暗电流会被放大器放大，同时，黑电平误差ε也会被放大，因此黑电平误差ε会受到感光度的影响；另一方面，当曝光时间变长时，光电二极管在开关闭合前所累积的光电流和暗电流都会变多，因此黑电平误差ε会受到曝光时间的影响。
例如，获取一数字相机，有无镜头均可，合上相机的盖子，将相机输出格式设置为.RAW格式文件，其余相机参数设置为默认值；对于ISO值和曝光值的每个对，拍摄得到一个黑图像。例如，假设相机的曝光时间有0.1s和1s可选，ISO值有100和200可选，则共存在四组ISO值和曝光值的对：(0.1s，100)、(1s，100)、(0.1s，200)、(1s，200)，因此，可共拍摄得到4个黑图像。
作为示例，对黑图像进行预处理包括：基于数码摄影装置的黑电平和白电平，对黑图像进行线性化处理。
令数码摄影装置的黑电平为m’、白电平为n’，黑图像中的像素为BlackFrame以及偏置图像中的对应像素为BiasFrame，则上述对黑图像进行的线性化处理可以表示为：
Figure PCTCN2021092005-appb-000002
在表达式(2)中，max{}表示取最大值以及min{}表示取最小值。通过基于表达式(2)进行的线性化处理，将黑图像中的每个像素限制到0-1范围内，从而得到偏置图像。
作为示例，第二处理单元304可以被配置为将所得到的偏置图像加在合成噪声图像上。
例如，第二处理单元304可以将所得到的偏置图像的每个像素的值加在合成噪声图像z中的对应像素上，来对合成噪声图像z中的黑电平进行处理，从而生成经处理的合成噪声图像。
作为示例，第二处理单元304可以被配置为从得到的偏置图像中的预定位置选取图像块，并将基于所选取的图像块而计算的偏置值加在合成噪声图像上。
例如，偏置图像的高度为H以及宽度为W，预定位置为ps。例如，可以从得到的偏置图像选取位置位于沿高度方向在H/2至H/2+ps以及沿宽度方向在W/2至W/2+ps范围的像素作为所选取的图像块。本领域技术人员可以理解，可以从得到的偏置图像中选取任意图像块来作为所选取的图像块。例如，可以根据经验或应用场景或实验等确定ps，例如ps可以为128，本领域技术人员可以理解，ps可以被确定为使得可以从得到的偏置图像中选取任意图像块所对应的值。例如，可以计算所选取的图像块中的像素的均值，将该均值加到合成噪声图像z的每个像素的值上，来对合成噪声图像z中的黑电平进行处理，从而生成经处理的合成噪声图像。本领域技术人员可以理解，还可以基于所选取的图像块中的像素的值来计算除了均值之外的其他数值，以用于对合成噪声图像z中的黑电平进行处理。
作为示例，第二处理单元304可以被配置为从得到的偏置图像中的预定位置选取像素，并将所选取的像素的值加在合成噪声图像上。
例如，可以从得到的偏置图像选取位于[H/2,W/2]处的像素的值，加在合成噪声图像z的每个像素的值上，来对合成噪声图像z中的黑电平进行处理，从而生成经处理的合成噪声图像。本领域技术人员可以理解，可以从得到的偏置图像选取位于任意位置处的像素的值，来对合成噪声图像z中的黑电平进行处理。
作为示例，可以将经处理的合成噪声图像的R、G1、B、G2的值分别组成一单独的通道，并将这四个通道级联在一起。可以将每个通道的图像数据(每个通道的数据大小为H/2×W/2)作为样本分别输入到机器学习模型。
在上文中，对原始图像进行线性化处理，在得到的线性化后的图像y上添加噪声而得到合成噪声图像z，并对合成噪声图像z进行黑电平处理从而得到经处理的合成噪声图像。作为替选，可以将线性化后的图像y的R、G1、B、G2的值分别组成一单独的通道，然后在线性化后的图像y的R、G1、B、G2的每个通道上分别添加噪声，接下来分别在R、G1、B、G2通道对黑电平进行处理，以及将经处理的每个通道的图像数据作为样本分别输入到机器学习模型。
在将上述经处理的合成噪声图像作为样本来训练机器学习模型时，将与原始图像相对应的、由图像获取装置拍摄得到的显示图像作为用于对机器学习模型进行训练的标签图像。在对机器学习模型进行训练时，可以将标签图像的R、G1、B、G2的值分别组成一单独的通道，以与经处理的合成噪声图像的R、G1、B、G2通道相对应。
下面以图像获取装置为数字相机为例来描述上述标签图像。可以将数字相机视为一种信号转换工具：数字相机将真实场景中的光信号转变为数字信号(光信号—>电信号—>数字信号)，该转换过程是由数字相机的硬件完成的。从数字相机获得的数字信号是上文所述的原始图像。然后，数字相机进行图像信号处理，以将该数字信号(原始图像)转换成可用于显示的图像(显示图像)，举例来说，原始图像是数字相机拍摄得到的RAW格式的图像，显示图像是对原始图像转换后得到的、在数字相机的屏幕上显示出来的s-RGB格式图像。
图5是示出现有的数字相机所进行的图像信号处理的示意图。
在图5中，首先对原始图像进行如表达式(1)所示的线性化处理。
在图5中，线性化处理之后是白平衡，白平衡的作用是消除有色光源对物体颜色的影响。例如，分别拍摄红光照射下和绿光照射下的红苹果，拍摄结果可能会截然不同，但是，红苹果的“颜色”(苹果表面对光谱反射的分布性质)却应该是一个客观事实，不应该随光源改变而变化。为了获得物体本来的颜色，通过白平衡来解决这一问题。在实现上，是通过将三个乘数[R,G,B]以点乘的方式乘到图像上完成的。
在图5中，白平衡之后是去马赛克，去马赛克的目的是将之前的1通道的原始图像变成可以在显示装置上显示的3通道的图像，其本质是线性插值操作。即将大小为H×W的原始图像变为3个大小为H×W的 图像(s-RGB格式的图像)。
在图5中，去马赛克之后是颜色空间转换。在数字相机中，用(R，G，B)这个三维向量表示某一个像素的值，即(R，G，B)是数字相机所用的颜色基底。然而，数字相机所用的颜色基底和显示装置的颜色基底并不相同，颜色空间转换操作的目的是将数字相机的颜色基底变化为在显示装置上可用的基底。颜色空间转换通过将去马赛克后的图像乘以一个3×3的转换矩阵实现，该矩阵是一个固定值。通过颜色空间转换可以得到s-RGB格式的图像。也就是说，通过图像信号处理得到的图像为s-RGB格式的图像(显示图像)。
例如，可以基于显示图像(标签图像)和经处理的合成噪声图像来更新机器学习模型的参数。例如，可以将显示图像和经处理的合成噪声图像在二范式下计算平均距离，根据反向传播更新机器学习模型的参数，从而得到经训练的机器学习模型。
作为示例，第二处理单元304可以被配置为从服从预定分布的偏置值的数值范围内抽取偏置值，以及基于所抽取的偏置值对合成噪声图像中的图像块的黑电平进行处理，并且基于经处理的图像块来训练机器学习模型。
例如，本领域技术人员可以根据经验或应用场景或实验预先确定图像块的大小，例如，图像块可以是1024x 1024的图像块。
基于所抽取的偏置值对合成噪声图像中的图像块的黑电平进行处理本质上是对图像块的黑电平进行扰动。
根据本公开实施例的信息处理设备300能够通过对黑电平进行的扰动增强机器学习模型对黑电平的鲁棒性。
例如，第二处理单元304可以将所抽取的偏置值加到合成噪声图像z中的图像块的每个像素上，以对该图像块中的黑电平进行处理。
作为示例，预定分布可以是高斯分布或均匀分布。例如，上述预定分布的分布函数的参数是根据经验或者应用场景或实验等而确定的。例如，均匀分布可以是服从(-10,10)的均匀分布。
作为示例，第二处理单元304可以被配置为从上述数值范围内随机抽取偏置值。
图6是示出根据本公开实施例的基于对黑电平进行扰动来训练机器学习模型的示意图。在图6中，以1024x 1024大小的图像块作为样本为例来进行描述。
如图6所示，在(1)中，从上述通过对原始图像进行线性化处理、添加噪声后等生成的合成噪声图像z中提取图像块。在(2)中，通过扰动对图像块中的黑电平进行处理。在(3)中，将经处理的图像块的R、G1、B、G2的值分别组成一单独的通道从而获得四个(1024/2)x(1024/2)大小的数据块(即，每个数据块的大小为512x 512)并将这四个通道级联在一起。在(4)中，将上述四个512x 512大小的数据块分别作为样本输入到机器学习模型，以及将原始图像经过(5)中的图像信号处理而获得的s-RGB格式的显示图像中的相应图像块作为训练的标签，来训练机器学习模型。例如，可以将作为样本的图像块和作为标签的图像块在二范式下计算平均距离，根据反向传播更新机器学习模型的参数，从而得到经训练的机器学习模型。其中，(5)中的图像信号处理与图5中的图像信号处理相同。
尽管上文中描述了对合成噪声图像中的图像块的黑电平进行扰动，但是，本领域技术人员可以理解，还可以基于所抽取的偏置值对合成噪声图像的黑电平进行扰动(例如，将所抽取的偏置值加在合成噪声图像的每个像素上)，这里不再累述。
与上述信息处理设备实施例相对应地，本公开还提供了信息处理方法的实施例。
图7是示出根据本公开实施例的信息处理方法S700的流程示例的流程图。
根据本公开实施例的信息处理方法S700从S702开始。
在S704中，基于输入的原始图像生成合成噪声图像。
有关原始图像的具体示例可参见有关信息处理设备300的相应部分(例如，第一处理单元302)的描述，这里不再累述。
在S706中，对合成噪声图像中的黑电平进行处理，并基于经处理的合成噪声图像来训练用于对图像去噪的机器学习模型，以得到经训练的机器学习模型。
有关对黑电平进行处理的具体示例可参见有关信息处理设备300的相应部分(例如，第二处理单元304)的描述，这里不再累述。
信息处理方法S700在S708结束。
在根据本公开实施例的信息处理方法S700中，对合成噪声图像中的黑电平进行处理能够减轻由于黑电平不准确而对训练机器学习模型造成的影响，因此能够提高基于经处理的合成噪声图像而训练得到的经训练的机器学习模型的去噪性能。
根据本公开的另一实施例，还提供了一种信息处理装置800。
图8示出了根据本公开实施例的信息处理装置800的功能模块框图，如图8所示，信息处理装置800包括：第三处理单元802，其可以被配置为将待处理图像输入到通过信息处理设备300或信息处理方法S700得到的经训练的机器学习模型，以对待处理图像进行去噪处理。
由于通过信息处理设备300或信息处理方法S700得到了具有提高的去噪性能的经训练的机器学习模型，因此，根据本公开实施例的信息处理装置800能够更准确地去除待处理图像中的噪声。
作为示例，待处理图像是在暗光环境下拍摄得到的短曝光图像。信息处理装置800可以更准确地去除在暗光环境下拍摄得到的短曝光图像中的噪声。
与上述信息处理装置实施例相对应地，本公开还提供了信息处理方法的实施例。
图9是示出根据本公开实施例的信息处理方法S900的流程示例的流程图。
根据本公开实施例的信息处理方法S900从S902开始。
在S904中，将待处理图像输入到通过信息处理设备300或信息处理方法S700得到的经训练的机器学习模型，以对待处理图像进行去噪处理。
信息处理方法S900在S906结束。
由于通过信息处理设备300或信息处理方法S700得到了具有提高的去噪性能的经训练的机器学习模型，因此，根据本公开实施例的信息处理方法S900能够更准确地去除待处理图像中的噪声。
以上结合具体实施例描述了本发明的基本原理，但是，需要指出的是，对本领域的技术人员而言，能够理解本发明的方法和装置的全部或者任何步骤或部件，可以在任何计算装置(包括处理器、存储介质等)或者计算装置的网络中，以硬件、固件、软件或者其组合的形式实现，这是本领域的技术人员在阅读了本发明的描述的情况下利用其基本电路设计知识或者基本编程技能就能实现的。
而且，本发明还提出了一种存储有机器可读取的指令代码的程序产品。指令代码由机器读取并执行时，可执行上述根据本发明实施例的方法。
相应地，用于承载上述存储有机器可读取的指令代码的程序产品的存储介质也包括在本发明的公开中。存储介质包括但不限于软盘、光盘、磁光盘、存储卡、存储棒等等。
在通过软件或固件实现本发明的情况下，从存储介质或网络向具有专用硬件结构的计算机(例如图10所示的通用计算机1000)安装构成该软件的程序，该计算机在安装有各种程序时，能够执行各种功能等。
在图10中，中央处理单元(CPU)1001根据只读存储器(ROM)1002中存储的程序或从存储部分1008加载到随机存取存储器(RAM)1003的程序执行各种处理。在RAM 1003中，也根据需要存储当CPU 1001执行各种处理等等时所需的数据。CPU 1001、ROM 1002和RAM 1003经由总线1004彼此连接。输入/输出接口1005也连接到总线1004。
下述部件连接到输入/输出接口1005：输入部分1006(包括键盘、鼠标等等)、输出部分1007(包括显示器，比如阴极射线管(CRT)、液晶显示器(LCD)等，和扬声器等)、存储部分1008(包括硬盘等)、通信部分1009(包括网络接口卡比如LAN卡、调制解调器等)。通信部分1009经由网络比如因特网执行通信处理。根据需要，驱动器1010也可连接到输入/输出接口1005。可移除介质1011比如磁盘、光盘、磁光盘、半导体存储器等等根据需要被安装在驱动器1010上，使得从中读出的计算机程序根据需要被安装到存储部分1008中。
在通过软件实现上述系列处理的情况下，从网络比如因特网或存储介质比如可移除介质1011安装构成软件的程序。
本领域的技术人员应当理解，这种存储介质不局限于图10所示的其 中存储有程序、与设备相分离地分发以向用户提供程序的可移除介质1011。可移除介质1011的例子包含磁盘(包含软盘(注册商标))、光盘(包含光盘只读存储器(CD-ROM)和数字通用盘(DVD))、磁光盘(包含迷你盘(MD)(注册商标))和半导体存储器。或者，存储介质可以是ROM 1002、存储部分1008中包含的硬盘等等，其中存有程序，并且与包含它们的设备一起被分发给用户。
还需要指出的是，在本发明的装置、方法和系统中，各部件或各步骤是可以分解和/或重新组合的。这些分解和/或重新组合应该视为本发明的等效方案。并且，执行上述系列处理的步骤可以自然地按照说明的顺序按时间顺序执行，但是并不需要一定按时间顺序执行。某些步骤可以并行或彼此独立地执行。
最后，还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。此外，在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上虽然结合附图详细描述了本发明的实施例，但是应当明白，上面所描述的实施方式只是用于说明本发明，而并不构成对本发明的限制。对于本领域的技术人员来说，可以对上述实施方式作出各种修改和变更而没有背离本发明的实质和范围。因此，本发明的范围仅由所附的权利要求及其等效含义来限定。
本技术还可以如下实现。
附记1.一种信息处理设备，包括：
处理电路，被配置为：
基于输入的原始图像生成合成噪声图像；以及
对所述合成噪声图像中的黑电平进行处理，并基于经处理的合成噪声图像来训练用于对图像去噪的机器学习模型，以得到经训练的机器学习模型。
附记2.根据附记1所述的信息处理设备，其中，所述处理电路被配 置为从所述原始图像获取预定图像获取参数，基于所述预定图像获取参数与用于处理黑电平的偏置图像之间的对应关系得到相应的偏置图像，并基于所得到的偏置图像对所述合成噪声图像中的黑电平进行处理。
附记3.根据附记2所述的信息处理设备，其中，
所述对应关系是通过以下方式生成的：
在暗光环境下，基于针对图像获取装置的每个预定图像获取参数拍摄得到的黑图像，得到与每个预定图像获取参数相对应的偏置图像。
附记4.根据附记3所述的信息处理设备，其中，
所述图像获取装置是数码摄影装置以及所述预定图像获取参数包括所述数码摄影装置的曝光时间和感光度；以及
所述对应关系是通过以下方式生成的：
合上所述数码摄影装置的盖子，针对所述数码摄影装置的曝光时间和感光度的每个参数对拍摄黑图像，并且对所述黑图像进行预处理，以得到与所述每个参数对相对应的偏置图像。
附记5.根据附记4所述的信息处理设备，其中，对所述黑图像进行预处理包括：基于所述数码摄影装置的黑电平和白电平，对所述黑图像进行线性化处理。
附记6.根据附记2至5中任一项所述的信息处理设备，其中，所述处理电路被配置为将所得到的偏置图像加在所述合成噪声图像上。
附记7.根据附记2至5中任一项所述的信息处理设备，其中，所述处理电路被配置为从得到的偏置图像中的预定位置选取图像块，并将基于所选取的图像块而计算的偏置值加在所述合成噪声图像上。
附记8.根据附记2至5中任一项所述的信息处理设备，其中，所述处理电路被配置为从得到的偏置图像中的预定位置选取像素，并将所选取的像素的值加在所述合成噪声图像上。
附记9.根据附记1所述的信息处理设备，其中，所述处理电路被配置为：
从服从预定分布的偏置值的数值范围内抽取偏置值，以及
基于所抽取的偏置值对所述合成噪声图像中的图像块的黑电平进行 处理，并且基于经处理的图像块来训练所述机器学习模型。
附记10.根据附记9所述的信息处理设备，其中，所述预定分布是高斯分布或均匀分布。
附记11.根据附记9或10所述的信息处理设备，其中，所述处理电路被配置为从所述数值范围内随机抽取所述偏置值。
附记12.根据附记1至11中任一项所述的信息处理设备，其中，所述处理电路被配置为基于从所述原始图像获取的噪声参数，对所述原始图像进行噪声化处理，从而生成所述合成噪声图像。
附记13.根据附记12所述的信息处理设备，其中，所述处理电路被配置为：
基于从所述原始图像获取的黑电平和白电平，对所述原始图像进行线性化处理，以得到所述原始图像的线性化后的图像；以及
对所述线性化后的图像进行所述噪声化处理，从而生成所述合成噪声图像。
附记14.根据附记1至13中任一项所述的信息处理设备，其中，所述原始图像是图像获取装置在长曝光状态下拍摄得到的长曝光图像，以及所述合成噪声图像用于仿真所述图像获取装置在暗光环境下拍摄得到的短曝光图像。
附记15.根据附记1至14中任一项所述的信息处理设备，其中，
其中，所述原始图像包括RAW格式的RGB数据。
附记16.根据附记1至15中任一项所述的信息处理设备，其中，
所述机器学习模型是神经网络。
附记17.一种信息处理装置，包括：
处理电路，被配置为：
将待处理图像输入到通过根据附记1至16中任一项所述的信息处理设备得到的经训练的机器学习模型，以对所述待处理图像进行去噪处理。
附记18.根据附记17所述的信息处理装置，其中，所述待处理图像是在暗光环境下拍摄得到的短曝光图像。
附记19.一种信息处理方法，包括：
基于输入的原始图像生成合成噪声图像；以及
对所述合成噪声图像中的黑电平进行处理，并基于经处理的合成噪声图像来训练用于对图像去噪的机器学习模型，以得到经训练的机器学习模型。
附记20.一种信息处理方法，包括：
将待处理图像输入到通过根据附记1至16中任一项所述的信息处理设备得到的经训练的机器学习模型，以对所述待处理图像进行去噪处理。
附记21.一种计算机可读存储介质，其上存储有计算机可执行指令，当所述计算机可执行指令被执行时，执行根据附记19或20所述的信息处理方法。

Claims (21)

  1. 一种信息处理设备，包括：
    处理电路，被配置为：
    基于输入的原始图像生成合成噪声图像；以及
    对所述合成噪声图像中的黑电平进行处理，并基于经处理的合成噪声图像来训练用于对图像去噪的机器学习模型，以得到经训练的机器学习模型。
  2. 根据权利要求1所述的信息处理设备，其中，所述处理电路被配置为从所述原始图像获取预定图像获取参数，基于所述预定图像获取参数与用于处理黑电平的偏置图像之间的对应关系得到相应的偏置图像，并基于所得到的偏置图像对所述合成噪声图像中的黑电平进行处理。
  3. 根据权利要求2所述的信息处理设备，其中，
    所述对应关系是通过以下方式生成的：
    在暗光环境下，基于针对图像获取装置的每个预定图像获取参数拍摄得到的黑图像，得到与每个预定图像获取参数相对应的偏置图像。
  4. 根据权利要求3所述的信息处理设备，其中，
    所述图像获取装置是数码摄影装置以及所述预定图像获取参数包括所述数码摄影装置的曝光时间和感光度；以及
    所述对应关系是通过以下方式生成的：
    合上所述数码摄影装置的盖子，针对所述数码摄影装置的曝光时间和感光度的每个参数对拍摄黑图像，并且对所述黑图像进行预处理，以得到与所述每个参数对相对应的偏置图像。
  5. 根据权利要求4所述的信息处理设备，其中，对所述黑图像进行预处理包括：基于所述数码摄影装置的黑电平和白电平，对所述黑图像进行线性化处理。
  6. 根据权利要求2至5中任一项所述的信息处理设备，其中，所述处理电路被配置为将所得到的偏置图像加在所述合成噪声图像上。
  7. 根据权利要求2至5中任一项所述的信息处理设备，其中，所述处理电路被配置为从得到的偏置图像中的预定位置选取图像块，并将基于所选取的图像块而计算的偏置值加在所述合成噪声图像上。
  8. 根据权利要求2至5中任一项所述的信息处理设备，其中，所述处理电路被配置为从得到的偏置图像中的预定位置选取像素，并将所选取的像素的值加在所述合成噪声图像上。
  9. 根据权利要求1所述的信息处理设备，其中，所述处理电路被配置为：
    从服从预定分布的偏置值的数值范围内抽取偏置值，以及
    基于所抽取的偏置值对所述合成噪声图像中的图像块的黑电平进行处理，并且基于经处理的图像块来训练所述机器学习模型。
  10. 根据权利要求9所述的信息处理设备，其中，所述预定分布是高斯分布或均匀分布。
  11. 根据权利要求9或10所述的信息处理设备，其中，所述处理电路被配置为从所述数值范围内随机抽取所述偏置值。
  12. 根据权利要求1至11中任一项所述的信息处理设备，其中，所述处理电路被配置为基于从所述原始图像获取的噪声参数，对所述原始图像进行噪声化处理，从而生成所述合成噪声图像。
  13. 根据权利要求12所述的信息处理设备，其中，所述处理电路被配置为：
    基于从所述原始图像获取的黑电平和白电平，对所述原始图像进行线性化处理，以得到所述原始图像的线性化后的图像；以及
    对所述线性化后的图像进行所述噪声化处理，从而生成所述合成噪声图像。
  14. 根据权利要求1至13中任一项所述的信息处理设备，其中，所述原始图像是图像获取装置在长曝光状态下拍摄得到的长曝光图像，以及所述合成噪声图像用于仿真所述图像获取装置在暗光环境下拍摄得到的短曝光图像。
  15. 根据权利要求1至14中任一项所述的信息处理设备，其中，
    其中，所述原始图像包括RAW格式的RGB数据。
  16. 根据权利要求1至15中任一项所述的信息处理设备，其中，
    所述机器学习模型是神经网络。
  17. 一种信息处理装置，包括：
    处理电路，被配置为：
    将待处理图像输入到通过根据权利要求1至16中任一项所述的信息处理设备得到的经训练的机器学习模型，以对所述待处理图像进行去噪处理。
  18. 根据权利要求17所述的信息处理装置，其中，所述待处理图像是在暗光环境下拍摄得到的短曝光图像。
  19. 一种信息处理方法，包括：
    基于输入的原始图像生成合成噪声图像；以及
    对所述合成噪声图像中的黑电平进行处理，并基于经处理的合成噪声图像来训练用于对图像去噪的机器学习模型，以得到经训练的机器学习模型。
  20. 一种信息处理方法，包括：
    将待处理图像输入到通过根据权利要求1至16中任一项所述的信息处理设备得到的经训练的机器学习模型，以对所述待处理图像进行去噪处理。
  21. 一种计算机可读存储介质，其上存储有计算机可执行指令，当所述计算机可执行指令被执行时，执行根据权利要求19或20所述的信息处理方法。
PCT/CN2021/092005 2020-05-14 2021-05-07 信息处理设备和方法、计算机可读存储介质 WO2021227934A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2022569198A JP2023525846A (ja) 2020-05-14 2021-05-07 情報処理装置及び方法
CN202180033535.7A CN115605908A (zh) 2020-05-14 2021-05-07 信息处理设备和方法、计算机可读存储介质
US17/916,813 US20230153954A1 (en) 2020-05-14 2021-05-07 Information processing device and method, and computer readable storage medium
EP21803938.6A EP4152245A4 (en) 2020-05-14 2021-05-07 INFORMATION PROCESSING METHOD AND DEVICE, AND COMPUTER-READABLE STORAGE MEDIUM

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010408019.5A CN113674364A (zh) 2020-05-14 2020-05-14 信息处理设备和方法、计算机可读存储介质
CN202010408019.5 2020-05-14

Publications (1)

Publication Number Publication Date
WO2021227934A1 true WO2021227934A1 (zh) 2021-11-18

Family

ID=78526429

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/092005 WO2021227934A1 (zh) 2020-05-14 2021-05-07 信息处理设备和方法、计算机可读存储介质

Country Status (5)

Country Link
US (1) US20230153954A1 (zh)
EP (1) EP4152245A4 (zh)
JP (1) JP2023525846A (zh)
CN (2) CN113674364A (zh)
WO (1) WO2021227934A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106973194A (zh) * 2015-10-14 2017-07-21 奥林巴斯株式会社 拍摄装置、图像处理装置、图像处理方法
US20170269187A1 (en) * 2016-03-15 2017-09-21 Canon Kabushiki Kaisha Image pickup apparatus having function of generating simulation image,control method therefor, and storage medium
CN110557584A (zh) * 2018-05-31 2019-12-10 杭州海康威视数字技术股份有限公司 图像处理方法及装置、计算机可读存储介质
CN110992272A (zh) * 2019-10-18 2020-04-10 深圳大学 基于深度学习的暗光图像增强方法、装置、设备及介质
CN111127336A (zh) * 2019-11-18 2020-05-08 复旦大学 一种基于自适应选择模块的图像信号处理方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106973194A (zh) * 2015-10-14 2017-07-21 奥林巴斯株式会社 拍摄装置、图像处理装置、图像处理方法
US20170269187A1 (en) * 2016-03-15 2017-09-21 Canon Kabushiki Kaisha Image pickup apparatus having function of generating simulation image,control method therefor, and storage medium
CN110557584A (zh) * 2018-05-31 2019-12-10 杭州海康威视数字技术股份有限公司 图像处理方法及装置、计算机可读存储介质
CN110992272A (zh) * 2019-10-18 2020-04-10 深圳大学 基于深度学习的暗光图像增强方法、装置、设备及介质
CN111127336A (zh) * 2019-11-18 2020-05-08 复旦大学 一种基于自适应选择模块的图像信号处理方法

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
BROOKS TIM; MILDENHALL BEN; XUE TIANFAN; CHEN JIAWEN; SHARLET DILLON; BARRON JONATHAN T.: "Unprocessing Images for Learned Raw Denoising", 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), IEEE, 15 June 2019 (2019-06-15), pages 11028 - 11037, XP033687401, DOI: 10.1109/CVPR.2019.01129 *
JIANG ZHILIN: "Research and Implementation of Dark Image Denoising and Enhancement Algorithm Based on Generative Adversarial Networks", INFORMATION & TECHNOLOGY, CHINA MASTER'S THESES FULL-TEXT DATABASE, no. 3, 15 March 2020 (2020-03-15), XP055865819 *
See also references of EP4152245A4

Also Published As

Publication number Publication date
EP4152245A1 (en) 2023-03-22
CN113674364A (zh) 2021-11-19
CN115605908A (zh) 2023-01-13
US20230153954A1 (en) 2023-05-18
JP2023525846A (ja) 2023-06-19
EP4152245A4 (en) 2023-11-01

Similar Documents

Publication Publication Date Title
Wei et al. Physics-based noise modeling for extreme low-light photography
Xu et al. Real-world noisy image denoising: A new benchmark
Liu et al. Noise estimation from a single image
JP5744437B2 (ja) 追尾装置、追尾方法及びプログラム
CN101631200B (zh) 图像处理设备和图像处理方法
EP2299721B1 (en) Image signal correcting device, imaging device, image signal correcting method, and program
CN104065854B (zh) 一种图像处理方法及一种电子设备
US20110150280A1 (en) Subject tracking apparatus, subject region extraction apparatus, and control methods therefor
JP2010045613A (ja) 画像識別方法および撮像装置
US20050169521A1 (en) Processing of mosaic digital images
WO2017101561A1 (zh) 高动态范围图像的生成方法、拍照装置和终端、成像方法
US7092579B2 (en) Calculating noise estimates of a digital image using gradient analysis
US20140240549A1 (en) Intelligent computational imaging system
CN113763271B (zh) 一种基于物理噪声模型的高质量光谱去噪方法
CN101753797A (zh) 图像信号处理设备及方法和程序
CN114979500B (zh) 图像处理方法、图像处理装置、电子设备及可读存储介质
CN111724317A (zh) Raw域视频去噪监督数据集构造方法
CN114972085B (zh) 一种基于对比学习的细粒度噪声估计方法和系统
CN113132695A (zh) 镜头阴影校正方法、装置及电子设备
Hanji et al. Noise-aware merging of high dynamic range image stacks without camera calibration
Purohit et al. Image processing framework for performance enhancement of low-light image sensors
Grundmann et al. Post-processing approach for radiometric self-calibration of video
WO2021227934A1 (zh) 信息处理设备和方法、计算机可读存储介质
US20230267708A1 (en) Method for generating a learning model, a program, and an information processing apparatus
CN113658118A (zh) 图像噪声程度估计方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21803938

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022569198

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021803938

Country of ref document: EP

Effective date: 20221214