CN113744167A - Image data conversion method and device - Google Patents
Image data conversion method and device Download PDFInfo
- Publication number
- CN113744167A CN113744167A CN202111027726.0A CN202111027726A CN113744167A CN 113744167 A CN113744167 A CN 113744167A CN 202111027726 A CN202111027726 A CN 202111027726A CN 113744167 A CN113744167 A CN 113744167A
- Authority
- CN
- China
- Prior art keywords
- image data
- image
- processed
- conversion
- data conversion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 160
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000007781 pre-processing Methods 0.000 claims abstract description 31
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000004590 computer program Methods 0.000 claims description 15
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 23
- 230000008569 process Effects 0.000 abstract description 8
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 4
- 238000013501 data transformation Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000011550 data transformation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
The invention provides an image data conversion method and device, relating to the technical field of image processing, wherein the method comprises the following steps: acquiring image data to be processed; the image data to be processed is data in a Raw format; preprocessing image data to be processed; inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the method and the device have the advantages that the image data conversion model is obtained based on sample image data training, the image data after conversion and enhancement are converted into the universal color standard image, the conversion from the Raw format image data to the sRGB image is completed, the image effect is automatically enhanced in the conversion process, the method and the device can adapt to the Raw format image data acquired by various digital sensor devices, effective image conversion and effect enhancement are completed, and the contrast, the saturation, the exposure effect and the like of the image are greatly improved compared with the traditional method.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for converting image data.
Background
Raw Image Format (Raw Image Format) images are Raw data obtained when an Image is captured by a photographing apparatus such as a single lens reflex camera, and Raw information of a digital sensor in the apparatus and metadata generated when the Image is captured are recorded. Different digital sensor devices have different schemes for converting Raw format data into universal color standard (sRGB) type data, but the conversion effect is unstable, and only Raw data can be converted, and images are not enhanced.
Various kinds of photographic application software (App) on the market currently have Internet Service Provider (ISP) processing flows capable of simulating digital sensors to convert the Raw format images, but there is no method capable of converting the Raw format images into sRGB images and enhancing the image effect at the same time on 16-bit data accuracy.
Therefore, it is an important issue to be solved in the industry at present that various Raw format data can be automatically converted and enhanced.
Disclosure of Invention
In view of the above, the present invention provides an image data conversion method and an image data conversion apparatus, so as to solve the defect in the prior art that various Raw format data are automatically converted and enhanced, achieve effective image conversion and effect enhancement, and greatly improve the contrast, saturation, exposure effect, and the like of an image.
Based on the above object, the present invention provides an image data conversion method, comprising the steps of:
acquiring image data to be processed; the image data to be processed is data in a Raw format;
preprocessing the image data to be processed;
inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a universal color standard image.
Optionally, the preprocessing the image data to be processed specifically includes the following steps:
acquiring a normalization threshold value of a pixel value of each pixel point in the image data to be processed;
obtaining a gray value corresponding to each pixel point according to the normalization threshold; the gray value of the pixel point with the normalized threshold value higher than the preset threshold value is a first gray value and is used as a first mask, and the gray value of the pixel point with the normalized threshold value not higher than the preset threshold value is a second gray value and is used as a second mask;
and reserving the first mask to obtain a single-channel mask image with image highlight region information.
Optionally, the image data conversion model is obtained by training through the following steps:
acquiring the sample image data;
preprocessing the sample image data;
performing gain processing on the preprocessed sample image data; wherein the gain processing comprises one or more of turning, rotating, translating, affine transformation, exposure, contrast adjustment, saturation adjustment and blurring;
and taking the sample image data after the gain processing as input data used for training, and training by adopting a deep learning mode to obtain the image data conversion model for generating the conversion-enhanced image data of the image data to be processed.
Optionally, an input channel of the image data conversion model is five channels, and includes original sample image data, preprocessed sample image data, and a three-channel color standard random color noise map.
Optionally, the three-channel color standard random color noise map is obtained by the following steps:
converting the sample image data to obtain an eight-bit universal color standard image;
carrying out image enhancement on the eight-bit universal color standard image, and adjusting the contrast, exposure and saturation of the image to obtain a three-channel universal color standard image;
and removing color noise from the three-channel general color standard image to obtain a three-channel color standard random color noise image.
Optionally, the image data conversion model adopts a full convolution network model.
The present invention also provides an image data conversion apparatus comprising:
the acquisition module is used for acquiring image data to be processed; the image data to be processed is data in a Raw format;
the preprocessing module is used for preprocessing the image data to be processed;
the conversion enhancement module is used for inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a universal color standard image.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the image data conversion method as described in any one of the above when executing the program.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the image data conversion method as any one of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, carries out the steps of the image data conversion method as described in any one of the above.
As can be seen from the above, the image data conversion method and apparatus provided by the present invention complete the conversion from Raw format image data to sRGB image based on Full Convolutional Network (FCN), automatically enhance the image effect during the conversion process, can adapt to Raw format image data acquired by various digital sensor devices, complete effective image conversion and effect enhancement, and greatly improve the contrast, saturation, exposure effect, etc. of the image, which are different from the conventional methods.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart illustrating an image data conversion method according to the present invention;
FIG. 2 is a flowchart illustrating a step S200 of the image data conversion method according to the present invention;
FIG. 3 is a schematic flow chart of an image data conversion model training process in the image data conversion method according to the present invention;
FIG. 4 is a flowchart illustrating a step A400 of the image data conversion method according to the present invention;
FIG. 5 is a diagram illustrating unprocessed image data to be processed according to the image data conversion method of the present invention;
FIG. 6 is a diagram of a single-channel mask image with highlight region information for an image according to the image data conversion method of the present invention;
FIG. 7 is a schematic diagram of three-channel RGB random color noise map in the image data conversion method of the present invention;
FIG. 8 is a schematic structural diagram of an image data conversion apparatus according to the present invention;
FIG. 9 is a schematic diagram of a specific structure of a pre-processing module in the image data conversion apparatus according to the present invention;
FIG. 10 is a schematic structural diagram of an image data transformation model training process in the image data transformation method according to the present invention;
FIG. 11 is a schematic diagram illustrating a specific structure of a training module in the image data conversion apparatus according to the present invention;
fig. 12 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to specific embodiments and the accompanying drawings.
It is to be noted that technical terms or scientific terms used in the embodiments of the present invention should have the ordinary meanings as understood by those having ordinary skill in the art to which the present disclosure belongs, unless otherwise defined. The use of "first," "second," and similar terms in this disclosure is not intended to indicate any order, quantity, or importance, but rather is used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
As a preferred embodiment of the present invention, there is provided an image data conversion method including the steps of:
acquiring image data to be processed; the image data to be processed is data in a Raw format;
preprocessing the image data to be processed;
inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a universal color standard image.
The present invention also provides an image data conversion apparatus comprising:
the acquisition module is used for acquiring image data to be processed; the image data to be processed is data in a Raw format;
the preprocessing module is used for preprocessing the image data to be processed;
the conversion enhancement module is used for inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a universal color standard image.
By the image data conversion method and the image data conversion device, the conversion from the Raw format image data to the sRGB image is completed based on the FCN, the image effect is automatically enhanced in the conversion process, the method and the device can adapt to the Raw format image data acquired by various digital sensor devices, the effective image conversion and the effect enhancement are completed, and the contrast, the saturation, the exposure effect and the like of the image different from the traditional method are greatly improved.
The following describes preferred embodiments of the image data conversion method and apparatus according to the present invention with reference to the accompanying drawings.
Referring to fig. 1, the present invention provides an image data conversion method, which includes the following steps:
s100, shooting images through various digital sensors, and collecting image data to be processed. The image data to be processed is data in Raw format, including but not limited to CR2, ARW, NEF data.
S200, preprocessing image data to be processed.
In step S200, the purpose of the preprocessing is to obtain a single-channel mask (mask) image highlight region with image highlight region information to protect the details of the highlight region in the original image captured by the data-transfer sensor.
S300, inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model. The image data conversion model is obtained based on sample image data training, and the image data after conversion and enhancement is an sRGB image.
Through step S300, Raw format image data collected by the digital sensor may be converted into an sRGB image, and the saturation, contrast, exposure value, and the like of the image may be enhanced. In this embodiment, the FCN model is used as the image data conversion model, and because a full convolution network model is used, no scaling is required for the input and output images.
According to the traditional method for converting the Raw format image into the eight-bit sRGB image, the finally obtained image has a large amount of color noise, the protection on the highlight area of the image is poor, and the effect of the image is not improved and enhanced. The method is based on FCN to complete the conversion from the Raw format image data to the sRGB image, automatically enhances the image effect in the conversion process, can adapt to the Raw format image data acquired by various digital sensor devices, completes effective image conversion and effect enhancement, and greatly improves the contrast, saturation, exposure effect and the like of the image different from the traditional method.
Referring to fig. 2, step S200 specifically includes the following steps:
s210, acquiring a normalized threshold value of the pixel value of each pixel point in the image data to be processed. It should be noted that the Raw format image data is single-channel grayscale data, and no color information is presented, so the unprocessed image data is a single-channel grayscale map, and the normalized threshold of the single-channel grayscale map without conversion and enhancement processing is (-1, 1).
S220, obtaining a gray value corresponding to each pixel point according to the normalized threshold; the gray value of the pixel point with the normalized threshold value higher than the preset threshold value is a first gray value and is used as a first mask, and the gray value of the pixel point with the normalized threshold value not higher than the preset threshold value is a second gray value and is used as a second mask. In this embodiment, the preset threshold is 0.96, 1 is assigned to the gray value of the pixel point whose normalized threshold is higher than 0.96, and the pixel point is used as the first mask; and assigning 0 to the gray value of the pixel point with the normalized threshold not higher than 0.96, and taking the pixel point as a second mask. In this method, a first mask is used to represent a highlight region in an image.
And S230, reserving the first mask to obtain a single-channel mask image with image highlight region information. The normalized threshold range for a single channel mask image with image highlight region information is (0, 1).
Referring to fig. 3, the image data conversion model is obtained by training the following steps:
and A100, acquiring sample image data. In the step a100, the acquisition mode is synchronized in the step S100, and the sample image data in different scenes can be acquired in consideration of the diversity requirement of the sample image data.
And A200, preprocessing sample image data. Likewise, in the present embodiment, the preprocessing manner in step a200 coincides with the preprocessing manner in step S200.
And A300, in order to obtain a network model with stronger robustness, gain processing is carried out on the preprocessed sample image data so as to add gain to the sample image data. The gain processing comprises one or more of turning, rotating, translating, affine transformation, exposure, contrast adjustment, saturation adjustment and blurring;
and A400, taking the sample image data after the gain processing as input data used for training, and training by adopting a deep learning mode to obtain an image data conversion model for generating the image data to be processed after the conversion enhancement.
The input channel of the image data conversion model is five channels, and the image data conversion model comprises original sample image data, preprocessed sample image data and a three-channel color standard random color noise map.
Referring to fig. 4 to 7, a three-channel color standard random color noise map is obtained by the following steps:
and A410, converting the sample image data to obtain an eight-bit sRGB image.
And A420, performing image enhancement on the eight-bit sRGB image through Photoshop and other software, and adjusting the contrast, exposure, saturation and the like of the image to obtain a three-channel sRGB image.
The image enhancement algorithm needs to interpolate the single-channel image into a three-channel RGB image through a demosaicing algorithm, and the sRGB image is obtained after processing in step S520.
And A430, removing color noise from the three-channel sRGB image to obtain a three-channel RGB random color noise image. The normalized threshold for the three channel RGB random color noise map is in the range (-1, 1).
The three-channel RGB random color noise map is used to improve the effect of the image data conversion model on removing color noise, and the shape of the noise map is as shown in fig. 7, because when the digital sensor captures an image, due to the photosensitive element of the digital sensor, the sensitivity of the digital sensor to a scene is different, color noise with different degrees is generated, and if the color noise is not removed through step a530, the final imaging quality of the image is affected.
Specifically, the image data conversion model is a combination of FCN-Generated Adaptive Networks (GAN) models, as shown in table 1, the image data conversion model adopts a coding and decoding structure, the upsampling of the decoding portion adopts a combination of nearest neighbor upsampling and a convolutional layer, the activation function of an output layer is Tanh, the size of an input image is 512 for example, the negetive slope of LeakyReLU is 0.2, and the discrimination network adopts the Discriminator of multi _ scale to discriminate true and false images with different resolutions respectively. In this embodiment, 3-scale discriminators are used to discriminate 512x512, 256x256, and 128x128 resolution images, and images with different resolutions are directly down-sampled by a Pooling Layer (Pooling Layer), and the optimization algorithm uses Adam to generate a learning rate of 0.0002 and discriminate a learning rate of 0.0001 for the network.
TABLE 1 detailed construction of image data conversion model
The total loss function of the image data transformation model training process can be expressed as:
where α, β, γ, μ, and σ represent weights corresponding to loss functions in the image data conversion model, where α is 1, β is 0.5, γ is 0.5, σ is 0.5, μ is 0.5, and L is LPercRepresenting the perceptual loss function.
Wherein, tanh is hyperbolic tangent processing, Output represents a network Output image, that is, an sRGB image Output after Raw format image data is enhanced, and groudtruth is a target image.
An L1_ loss function is selected to ensure the color brightness of the image, an L1_ loss function is a linear loss function, and in order to ensure that the details of the highlight area of the Raw image cannot be lost, Tanh operation is respectively carried out on the Output image Output and the GroudTruth before the L1_ loss function is calculated, so that the image can be ensured to have higher weight in the highlight area.
In order to ensure the image perception similarity, the perception loss Perceptial _ loss L based on VGG19 is introducedPerc。
To ensure image authenticity, the image data conversion model works against the Loss Gan _ Loss function to minimize the distance between the color estimate of the real image and the image color distribution that generates the network output.
Represents three different divisionsResolution is the loss function of the output of the discriminator at 512x512, 256x256, 128x128 resolution.
Where D represents the above-mentioned discriminator, x represents real data (i.e., GroudTrut), z represents network-generated data, px represents the distribution of real data samples, pz represents the distribution of network-generated data samples, E represents expectation, and L representsadvRepresenting the penalty function.
In order to ensure that the color of the highlight area is excessively harmonious with the normal image area while the highlight area is repaired and inhibited after the image is output, the image data conversion model also uses LMatConverting the color space of the input and output images into Lab space, calculating the mean and variance of the channels a and b of the input and output images, respectively calculating the L2_ loss function of the mean and L2_ loss functions of the variance of the input and output images, and summing to obtain LMat，LMatRepresenting a statistical loss function in order to ensure excessive harmony of colors in the exposed area.
The image data conversion apparatus provided by the present invention is described below, and the image data conversion apparatus described below and the image data conversion method described above may be referred to in correspondence with each other.
Referring to fig. 8, the present invention provides an image data converting apparatus, including:
the acquisition module 100 is used for shooting images through various digital sensors and acquiring image data to be processed. The image data to be processed is data in Raw format, including but not limited to CR2, ARW, NEF data.
The preprocessing module 200 is configured to preprocess the image data to be processed.
In the preprocessing module 200, the purpose of preprocessing is to obtain a single-channel mask (mask) image highlight region with image highlight region information to protect the details of the highlight region in the original data-transfer sensor acquired image.
The conversion enhancing module 300 is configured to input the preprocessed image data to be processed into the image data conversion model, so as to obtain the conversion enhanced image data output by the image data conversion model. The image data conversion model is obtained based on sample image data training, and the image data after conversion and enhancement is an sRGB image.
By the conversion enhancement module 300, Raw format image data collected by the digital sensor can be converted into an sRGB image, and the saturation, contrast, exposure value, etc. of the image are enhanced. In this embodiment, the FCN model is used as the image data conversion model, and because a full convolution network model is used, no scaling is required for the input and output images.
According to the traditional device for converting an image in a Raw format into an eight-bit sRGB image, a large amount of color noise exists in the finally obtained image, the protection on a highlight area of the image is poor, and the effect of the image is not improved and enhanced. The device completes the conversion from the Raw format image data to the sRGB image based on the FCN, automatically enhances the image effect in the conversion process, can adapt to the Raw format image data acquired by various digital sensor devices, completes effective image conversion and effect enhancement, and greatly improves the contrast, saturation, exposure effect and the like which are different from the traditional device image.
Referring to fig. 9, the preprocessing module 200 specifically includes:
the first obtaining unit 210 is configured to obtain a normalized threshold of a pixel value of each pixel in the image data to be processed. It should be noted that the Raw format image data is single-channel grayscale data, and no color information is presented, so the unprocessed image data is a single-channel grayscale map, and the normalized threshold of the single-channel grayscale map without conversion and enhancement processing is (-1, 1).
The first obtaining unit 220 is configured to obtain a gray value corresponding to each pixel point according to the normalized threshold; the gray value of the pixel point with the normalized threshold value higher than the preset threshold value is a first gray value and is used as a first mask, and the gray value of the pixel point with the normalized threshold value not higher than the preset threshold value is a second gray value and is used as a second mask. In this embodiment, the preset threshold is 0.96, 1 is assigned to the gray value of the pixel point whose normalized threshold is higher than 0.96, and the pixel point is used as the first mask; and assigning 0 to the gray value of the pixel point with the normalized threshold not higher than 0.96, and taking the pixel point as a second mask. In this device a first mask is used to represent the highlight areas in the image.
And a retaining unit 230 configured to retain the first mask to obtain a single-channel mask image with highlight region information of the image. The normalized threshold range for a single channel mask image with image highlight region information is (0, 1).
Referring to fig. 10, the image data conversion model is trained by the following modules:
a sample acquisition module 400 for acquiring sample image data. The sample acquisition module 400 is the same as the acquisition module 100 in acquisition mode, and can acquire sample image data in different scenes in consideration of the diversity requirement of the sample image data.
The sample preprocessing module 500 is configured to preprocess sample image data. Also, in the present embodiment, the preprocessing mode in the sample preprocessing module 500 is the same as that of the preprocessing module 200.
And a gain module 600, wherein in order to obtain a network model with higher robustness, the gain module 600 is configured to perform gain processing on the preprocessed sample image data to add a gain to the sample image data. The gain processing comprises one or more of turning, rotating, translating, affine transformation, exposure, contrast adjustment, saturation adjustment and blurring;
the training module 700 is configured to train the gain-processed sample image data as input data for training in a deep learning manner, so as to obtain an image data conversion model for generating conversion-enhanced image data of the image data to be processed.
The input channel of the image data conversion model is five channels, and the image data conversion model comprises original sample image data, preprocessed sample image data and a three-channel color standard random color noise map.
Referring to fig. 11, a three-channel color standard random color noise map is obtained by:
the conversion unit 710 performs conversion processing on the sample image data to obtain an eight-bit sRGB image.
The enhancement unit 720 performs image enhancement on the eight-bit sRGB image through Photoshop and other software, and adjusts the contrast, exposure, saturation and the like of the image to obtain a three-channel sRGB image.
The image enhancement algorithm needs to interpolate the single-channel image into a three-channel RGB image through a demosaicing algorithm, and the sRGB image is obtained after processing in step S520.
And the denoising unit 730 is used for removing color noise from the sRGB image of the three channels to obtain an RGB random color noise map of the three channels. The normalized threshold for the three channel RGB random color noise map is in the range (-1, 1).
The three-channel RGB random color noise image is used for improving the effect of the image data conversion model on removing color noise, because when the digital sensor shoots an image, due to the fact that the sensitivity of the digital sensor to a scene is different, color noise with different degrees can be generated, and if the color noise is not removed through the denoising unit 730, the final imaging quality of the image can be influenced.
Fig. 12 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 12: a processor (processor)810, a communication Interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication Interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call logic instructions in the memory 830 to perform an image data conversion method comprising the steps of:
s100, collecting image data to be processed; the image data to be processed is data in a Raw format;
s200, preprocessing the image data to be processed;
s300, inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion and enhancement is an sRGB image.
In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer-readable storage medium, the computer program, when executed by a processor, being capable of executing the image data conversion method provided by the above methods, the method comprising the steps of:
s100, collecting image data to be processed; the image data to be processed is data in a Raw format;
s200, preprocessing the image data to be processed;
s300, inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion and enhancement is an sRGB image.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the image data conversion method provided by the above methods, the method comprising the steps of:
s100, collecting image data to be processed; the image data to be processed is data in a Raw format;
s200, preprocessing the image data to be processed;
s300, inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion and enhancement is an sRGB image.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the idea of the invention, also features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity.
The embodiments of the invention are intended to embrace all such alternatives, modifications and variances that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, substitutions, improvements and the like that may be made without departing from the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (10)
1. An image data conversion method, characterized by comprising the steps of:
acquiring image data to be processed; the image data to be processed is data in a Raw format;
preprocessing the image data to be processed;
inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a universal color standard image.
2. The image data conversion method according to claim 1, wherein the preprocessing the image data to be processed specifically includes the following steps:
acquiring a normalization threshold value of a pixel value of each pixel point in the image data to be processed;
obtaining a gray value corresponding to each pixel point according to the normalization threshold; the gray value of the pixel point with the normalized threshold value higher than the preset threshold value is a first gray value and is used as a first mask, and the gray value of the pixel point with the normalized threshold value not higher than the preset threshold value is a second gray value and is used as a second mask;
and reserving the first mask to obtain a single-channel mask image with image highlight region information.
3. The image data conversion method according to claim 2, wherein the image data conversion model is trained by:
acquiring the sample image data;
preprocessing the sample image data;
performing gain processing on the preprocessed sample image data; wherein the gain processing comprises one or more of turning, rotating, translating, affine transformation, exposure, contrast adjustment, saturation adjustment and blurring;
and taking the sample image data after the gain processing as input data used for training, and training by adopting a deep learning mode to obtain the image data conversion model for generating the conversion-enhanced image data of the image data to be processed.
4. The image data conversion method according to claim 3, wherein the input channels of the image data conversion model are five channels including original sample image data, preprocessed sample image data, and three-channel color standard stochastic color noise maps.
5. The image data conversion method of claim 4, wherein the three-channel color standard random color noise map is obtained by:
converting the sample image data to obtain an eight-bit universal color standard image;
carrying out image enhancement on the eight-bit universal color standard image, and adjusting the contrast, exposure and saturation of the image to obtain a three-channel universal color standard image;
and removing color noise from the three-channel general color standard image to obtain a three-channel color standard random color noise image.
6. The image data conversion method according to claim 3, wherein the image data conversion model employs a full convolution network model.
7. An image data conversion apparatus characterized by comprising:
an acquisition module (100) for acquiring image data to be processed; the image data to be processed is data in a Raw format;
a preprocessing module (200) for preprocessing the image data to be processed;
the conversion enhancement module (300) is used for inputting the preprocessed image data to be processed into an image data conversion model to obtain the conversion enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a universal color standard image.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the image data conversion method according to any of claims 1 to 6 are implemented when the processor executes the program.
9. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the image data conversion method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the image data conversion method according to any one of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111027726.0A CN113744167B (en) | 2021-09-02 | 2021-09-02 | Image data conversion method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111027726.0A CN113744167B (en) | 2021-09-02 | 2021-09-02 | Image data conversion method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113744167A true CN113744167A (en) | 2021-12-03 |
| CN113744167B CN113744167B (en) | 2024-04-30 |
Family
ID=78735136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111027726.0A Active CN113744167B (en) | 2021-09-02 | 2021-09-02 | Image data conversion method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113744167B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116095256A (en) * | 2022-12-30 | 2023-05-09 | 深圳创维数字技术有限公司 | Deep learning image conversion model training method, device, equipment and medium |
| CN116095256B (en) * | 2022-12-30 | 2024-09-24 | 深圳创维数字技术有限公司 | Deep learning image conversion model training method, device, equipment and medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1110380A1 (en) * | 1998-09-01 | 2001-06-27 | Divio Inc. | Method and apparatus for edge enhancement in digital images |
| US20080089580A1 (en) * | 2006-10-13 | 2008-04-17 | Marcu Gabriel G | System and method for raw image processing using conversion matrix interpolated from predetermined camera characterization matrices |
| CN104180908A (en) * | 2014-08-07 | 2014-12-03 | 华能国际电力股份有限公司 | RAW image radiation temperature measuring device and method |
| US20180160038A1 (en) * | 2016-12-07 | 2018-06-07 | Microsoft Technology Licensing, Llc | Advanced raw conversion to produce high dynamic range, wide color gamut output |
| CN109379572A (en) * | 2018-12-04 | 2019-02-22 | 北京达佳互联信息技术有限公司 | Image conversion method, device, electronic equipment and storage medium |
| CN110084803A (en) * | 2019-04-29 | 2019-08-02 | 南京星程智能科技有限公司 | Eye fundus image method for evaluating quality based on human visual system |
| CN110211057A (en) * | 2019-05-15 | 2019-09-06 | 武汉Tcl集团工业研究院有限公司 | A kind of image processing method based on full convolutional network, device and computer equipment |
| CN110933313A (en) * | 2019-12-09 | 2020-03-27 | Oppo广东移动通信有限公司 | Dark light photographing method and related equipment |
| CN110992272A (en) * | 2019-10-18 | 2020-04-10 | 深圳大学 | Dark light image enhancement method, device, equipment and medium based on deep learning |
| CN111612722A (en) * | 2020-05-26 | 2020-09-01 | 星际（重庆）智能装备技术研究院有限公司 | Low-illumination image processing method based on simplified Unet full-convolution neural network |
| CN112258528A (en) * | 2020-11-02 | 2021-01-22 | Oppo广东移动通信有限公司 | Image processing method and device and electronic equipment |
| CN112446839A (en) * | 2020-11-30 | 2021-03-05 | 平安科技（深圳）有限公司 | Image enhancement method and device, electronic equipment and computer readable storage medium |
| CN113099121A (en) * | 2021-04-15 | 2021-07-09 | 西安交通大学 | ISP implementation method based on weak supervised learning |
-
2021
- 2021-09-02 CN CN202111027726.0A patent/CN113744167B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1110380A1 (en) * | 1998-09-01 | 2001-06-27 | Divio Inc. | Method and apparatus for edge enhancement in digital images |
| US20080089580A1 (en) * | 2006-10-13 | 2008-04-17 | Marcu Gabriel G | System and method for raw image processing using conversion matrix interpolated from predetermined camera characterization matrices |
| CN104180908A (en) * | 2014-08-07 | 2014-12-03 | 华能国际电力股份有限公司 | RAW image radiation temperature measuring device and method |
| US20180160038A1 (en) * | 2016-12-07 | 2018-06-07 | Microsoft Technology Licensing, Llc | Advanced raw conversion to produce high dynamic range, wide color gamut output |
| US20210058595A1 (en) * | 2018-12-04 | 2021-02-25 | Beijing Dajia Internet Information Technology Co., Ltd. | Method, Device, and Storage Medium for Converting Image |
| CN109379572A (en) * | 2018-12-04 | 2019-02-22 | 北京达佳互联信息技术有限公司 | Image conversion method, device, electronic equipment and storage medium |
| CN110084803A (en) * | 2019-04-29 | 2019-08-02 | 南京星程智能科技有限公司 | Eye fundus image method for evaluating quality based on human visual system |
| CN110211057A (en) * | 2019-05-15 | 2019-09-06 | 武汉Tcl集团工业研究院有限公司 | A kind of image processing method based on full convolutional network, device and computer equipment |
| CN110992272A (en) * | 2019-10-18 | 2020-04-10 | 深圳大学 | Dark light image enhancement method, device, equipment and medium based on deep learning |
| CN110933313A (en) * | 2019-12-09 | 2020-03-27 | Oppo广东移动通信有限公司 | Dark light photographing method and related equipment |
| CN111612722A (en) * | 2020-05-26 | 2020-09-01 | 星际（重庆）智能装备技术研究院有限公司 | Low-illumination image processing method based on simplified Unet full-convolution neural network |
| CN112258528A (en) * | 2020-11-02 | 2021-01-22 | Oppo广东移动通信有限公司 | Image processing method and device and electronic equipment |
| CN112446839A (en) * | 2020-11-30 | 2021-03-05 | 平安科技（深圳）有限公司 | Image enhancement method and device, electronic equipment and computer readable storage medium |
| CN113099121A (en) * | 2021-04-15 | 2021-07-09 | 西安交通大学 | ISP implementation method based on weak supervised learning |
Non-Patent Citations (3)
| Title |
|---|
| HANG ZHAO等: "Loss Functions for Image Restoration With Neural Networks", IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING, 23 December 2016 (2016-12-23) * |
| 周祥云;: "快速自适应RAW-HDR的色调映射算法的研究", 福建电脑, no. 07, 25 July 2020 (2020-07-25) * |
| 杨勇;刘惠义;: "极端低光情况下的图像增强方法", 图学学报, no. 04 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116095256A (en) * | 2022-12-30 | 2023-05-09 | 深圳创维数字技术有限公司 | Deep learning image conversion model training method, device, equipment and medium |
| CN116095256B (en) * | 2022-12-30 | 2024-09-24 | 深圳创维数字技术有限公司 | Deep learning image conversion model training method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113744167B (en) | 2024-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11457138B2 (en) | Method and device for image processing, method for training object detection model | |
| EP3477931B1 (en) | Image processing method and device, readable storage medium and electronic device | |
| WO2021022983A1 (en) | Image processing method and apparatus, electronic device and computer-readable storage medium | |
| US11526995B2 (en) | Robust use of semantic segmentation for depth and disparity estimation | |
| CN110516577B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113766125B (en) | Focusing method and device, electronic equipment and computer readable storage medium | |
| CN115442515A (en) | Image processing method and apparatus | |
| WO2021057474A1 (en) | Method and apparatus for focusing on subject, and electronic device, and storage medium | |
| CN113313661B (en) | Image fusion method, device, electronic equipment and computer readable storage medium | |
| EP3644599B1 (en) | Video processing method and apparatus, electronic device, and storage medium | |
| CN112602088B (en) | Method, system and computer readable medium for improving quality of low light images | |
| CN113888437A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| JP2002245471A (en) | Photograph finishing service for double print accompanied by second print corrected according to subject contents | |
| US20220398698A1 (en) | Image processing model generation method, processing method, storage medium, and terminal | |
| WO2022261828A1 (en) | Image processing method and apparatus, electronic device, and computer-readable storage medium | |
| CN113673474B (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| CN113438386B (en) | Dynamic and static judgment method and device applied to video processing | |
| CN117496019B (en) | Image animation processing method and system for driving static image | |
| CN111105369B (en) | Image processing method, image processing apparatus, electronic device, and readable storage medium | |
| CN112241669B (en) | Target identification method, device, system, equipment and storage medium | |
| CN112036342A (en) | Document snapshot method, device and computer storage medium | |
| CN113298829B (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| CN113744167A (en) | Image data conversion method and device | |
| KR102389284B1 (en) | Method and device for image inpainting based on artificial intelligence | |
| CN116977190A (en) | Image processing method, apparatus, device, storage medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |