CN113744167B - Image data conversion method and device - Google Patents
Image data conversion method and device Download PDFInfo
- Publication number
- CN113744167B CN113744167B CN202111027726.0A CN202111027726A CN113744167B CN 113744167 B CN113744167 B CN 113744167B CN 202111027726 A CN202111027726 A CN 202111027726A CN 113744167 B CN113744167 B CN 113744167B
- Authority
- CN
- China
- Prior art keywords
- image data
- image
- processed
- data conversion
- conversion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 146
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000007781 pre-processing Methods 0.000 claims abstract description 34
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims description 30
- 238000010586 diagram Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 13
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 5
- 238000013135 deep learning Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 5
- 238000013519 translation Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 23
- 230000008569 process Effects 0.000 abstract description 8
- 230000006870 function Effects 0.000 description 14
- 238000004891 communication Methods 0.000 description 6
- 238000013501 data transformation Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 101100409194 Rattus norvegicus Ppargc1b gene Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000008485 antagonism Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000011550 data transformation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
The invention provides a method and a device for converting image data, which relate to the technical field of image processing, and the method comprises the following steps: collecting image data to be processed; the image data to be processed are data in a Raw format; preprocessing image data to be processed; inputting the preprocessed image data to be processed into an image data conversion model to obtain converted and enhanced image data output by the image data conversion model; the invention completes the conversion from the Raw format image data to the sRGB image, automatically enhances the image effect in the conversion process, can adapt to the Raw format image data collected by various digital sensor devices, completes effective image conversion and effect enhancement, and greatly improves the contrast, saturation, exposure effect and the like of the image different from the traditional method.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for converting image data.
Background
A Raw Format (Raw Image Format) Image is Raw data obtained when a photographing apparatus such as a single-lens reflex camera acquires an Image, and Raw information of a digital sensor in the apparatus and metadata generated at the time of photographing are recorded. Different digital sensor devices have different schemes for converting Raw format data into common color standard (STANDARD RED GREEN Blue, sRGB) type data, but the conversion effect is unstable, and only the original data can be converted, and the image is not enhanced.
Various photographic application software (App) currently exists on the market, which can simulate the internet service Provider (INTERNET SERVICE Provider, ISP) processing flow of a digital sensor to convert a Raw format image, but no method for converting the Raw format image into an sRGB image on 16-bit data precision and enhancing the image effect at the same time exists.
Therefore, the automatic conversion and enhancement of various Raw format data are important issues to be solved in the industry.
Disclosure of Invention
In view of this, the present invention provides an image data conversion method and apparatus, which are used to solve the defects of automatic conversion and enhancement of various Raw format data in the prior art, and to realize effective image conversion and enhancement of the effect, and greatly improve the contrast, saturation, exposure effect, etc. of the image.
Based on the above object, the present invention provides an image data conversion method comprising the steps of:
Collecting image data to be processed; wherein, the image data to be processed is data in a Raw format;
preprocessing the image data to be processed;
inputting the preprocessed image data to be processed into an image data conversion model to obtain conversion-enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a general color standard image.
Optionally, the preprocessing the image data to be processed specifically includes the following steps:
acquiring a normalized threshold value of a pixel value of each pixel point in the image data to be processed;
obtaining a gray value corresponding to each pixel point according to the normalized threshold value; the gray value of the pixel point with the normalized threshold value higher than the preset threshold value is a first gray value and is used as a first mask, and the gray value of the pixel point with the normalized threshold value not higher than the preset threshold value is a second gray value and is used as a second mask;
And reserving the first mask to obtain a single-channel mask image with the image highlight region information.
Optionally, the image data conversion model is obtained through training of the following steps:
collecting the sample image data;
Preprocessing the sample image data;
Performing gain processing on the preprocessed sample image data; wherein the gain processing comprises one or more of flipping, rotation, translation, affine transformation, exposure, contrast adjustment, saturation adjustment, blurring;
And training the sample image data after gain processing as input data for training by adopting a deep learning mode to obtain the image data conversion model for generating the image data after conversion enhancement of the image data to be processed.
Optionally, the input channels of the image data conversion model are five channels, including original sample image data, preprocessed sample image data and a three-channel color standard random color noise map.
Optionally, the three-channel color standard random color noise diagram is obtained through the following steps:
converting the sample image data to obtain an eight-bit universal color standard image;
performing image enhancement on the eight-bit general color standard image, and adjusting the contrast, exposure and saturation of the image to obtain a three-channel general color standard image;
And removing color noise from the three-channel universal color standard image to obtain a three-channel color standard random color noise diagram.
Optionally, the image data conversion model adopts a full convolution network model.
The present invention also provides an image data conversion apparatus including:
The acquisition module is used for acquiring image data to be processed; wherein, the image data to be processed is data in a Raw format;
the preprocessing module is used for preprocessing the image data to be processed;
The conversion enhancement module is used for inputting the preprocessed image data to be processed into an image data conversion model to obtain the converted and enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a general color standard image.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the image data conversion method as described in any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the image data conversion method as described in any of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the image data conversion method as described in any one of the above.
As can be seen from the above, the image data conversion method and apparatus provided by the present invention complete the conversion from Raw format image data to sRGB image based on the full convolution network (Fully Convolutional Networks, FCN), and automatically enhance the image effect during the conversion process, so as to adapt to Raw format image data collected by various digital sensor devices, complete the effective image conversion and effect enhancement, and greatly improve the contrast, saturation, exposure effect, etc. of the image, unlike the conventional method.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an image data conversion method according to the present invention;
FIG. 2 is a flowchart illustrating a step S200 in the image data conversion method according to the present invention;
FIG. 3 is a flowchart of an image data transformation model training process in the image data transformation method of the present invention;
FIG. 4 is a flowchart illustrating a step A400 in the image data conversion method according to the present invention;
FIG. 5 is a schematic diagram of unprocessed image data to be processed in the image data conversion method of the present invention;
FIG. 6 is a schematic diagram of a single-channel mask image with image highlight region information in the image data conversion method of the present invention;
FIG. 7 is a schematic diagram of a three-channel RGB random color noise map in the image data conversion method of the present invention;
FIG. 8 is a schematic diagram of an image data conversion device according to the present invention;
FIG. 9 is a schematic diagram showing a specific configuration of a preprocessing module in the image data conversion device according to the present invention;
FIG. 10 is a schematic diagram of the training process of the image data conversion model in the image data conversion method of the present invention;
FIG. 11 is a schematic diagram showing a specific structure of a training module in the image data conversion device according to the present invention;
fig. 12 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
The present invention will be further described in detail below with reference to specific embodiments and with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent.
It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present invention should be given the ordinary meaning as understood by one of ordinary skill in the art to which the present disclosure pertains. The terms "first," "second," and the like, as used in this disclosure, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.
As a preferred embodiment of the present invention, the present invention provides an image data conversion method including the steps of:
Collecting image data to be processed; wherein, the image data to be processed is data in a Raw format;
preprocessing the image data to be processed;
inputting the preprocessed image data to be processed into an image data conversion model to obtain conversion-enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a general color standard image.
The present invention also provides an image data conversion apparatus including:
The acquisition module is used for acquiring image data to be processed; wherein, the image data to be processed is data in a Raw format;
the preprocessing module is used for preprocessing the image data to be processed;
The conversion enhancement module is used for inputting the preprocessed image data to be processed into an image data conversion model to obtain the converted and enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a general color standard image.
According to the image data conversion method and device, conversion from the Raw format image data to the sRGB image is completed based on the FCN, and the image effect is automatically enhanced in the conversion process, so that the method and device can be suitable for the Raw format image data acquired by various digital sensor devices, complete effective image conversion and effect enhancement, and greatly improve contrast, saturation, exposure effect and the like of images different from the traditional method.
The following describes a preferred embodiment of the image data conversion method and apparatus according to the present invention with reference to the accompanying drawings.
Referring to fig. 1, the present invention provides an image data conversion method, which includes the following steps:
s100, shooting images through various different digital sensors, and collecting image data to be processed. The image data to be processed is data in a Raw format, including but not limited to CR2, ARW, NEF data.
S200, preprocessing the image data to be processed.
In step S200, the purpose of the preprocessing is to obtain a single-channel mask (mask) image highlight region with image highlight region information, so as to protect details of the highlight region in the image acquired by the original data sensor.
S300, inputting the preprocessed image data to be processed into an image data conversion model to obtain the image data after conversion enhancement output by the image data conversion model. The image data conversion model is obtained based on sample image data training, and the enhanced image data is converted into an sRGB image.
Through step S300, the Raw format image data collected by the digital sensor can be converted into an sRGB image, and the saturation, contrast, exposure value, etc. of the image can be enhanced. In this embodiment, the FCN model is used as the image data conversion model, and since a full convolution network model is used, there is no need to scale the input and output images.
The traditional method for converting the Raw format image into the eight-bit sRGB image has a large amount of color noise, the highlight area of the image is not well protected, and the effect of the image is not improved and enhanced. The method is based on FCN to complete conversion from Raw format image data to sRGB image, and automatically enhances image effect in the conversion process, so that the method can be suitable for Raw format image data collected by various digital sensor devices, complete effective image conversion and effect enhancement, and greatly improve contrast, saturation, exposure effect and the like of images different from the traditional method.
Referring to fig. 2, step S200 specifically includes the following steps:
S210, acquiring a normalized threshold value of a pixel value of each pixel point in the image data to be processed. It should be noted that, the Raw format image data is one single-channel gray scale data, and no color information is presented, so that the unprocessed image data is a single-channel gray scale map, and the range of the normalized threshold of the single-channel gray scale map which is not converted and enhanced is (-1, 1).
S220, according to the normalized threshold value, a gray value corresponding to each pixel point is obtained; the gray value of the pixel point with the normalized threshold value higher than the preset threshold value is a first gray value and is used as a first mask, and the gray value of the pixel point with the normalized threshold value not higher than the preset threshold value is a second gray value and is used as a second mask. In this embodiment, the preset threshold is 0.96, a gray value of a pixel with a normalized threshold higher than 0.96 is assigned 1, and the pixel is used as a first mask; and (3) giving 0 to the gray value of the pixel point with the normalized threshold value not higher than 0.96, and taking the pixel point as a second mask. In the method, a first mask is used to represent the highlight region in the image.
And S230, reserving the first mask to obtain a single-channel mask image with the image highlight region information. The range of normalized thresholds for single channel mask images with image highlight region information is (0, 1).
Referring to fig. 3, the image data conversion model is obtained by training:
a100, collecting sample image data. The acquisition mode in step a100 is synchronous with step S100, and sample image data in different scenes can be acquired in consideration of the diversity requirement of the sample image data.
And A200, preprocessing sample image data. Similarly, in the present embodiment, the preprocessing method in step a200 is identical to the preprocessing method in step S200.
A300, in order to obtain a network model with stronger robustness, performing gain processing on the preprocessed sample image data so as to add gain to the sample image data. Wherein the gain processing comprises one or more of flipping, rotation, translation, affine transformation, exposure, contrast adjustment, saturation adjustment, and blurring;
a400, training the sample image data after gain processing as input data for training by adopting a deep learning mode to obtain an image data conversion model for generating image data after conversion enhancement of the image data to be processed.
The input channels of the image data conversion model are five channels, and the input channels comprise original sample image data, preprocessed sample image data and a three-channel color standard random color noise diagram.
Referring to fig. 4 to 7, the three-channel color standard random color noise chart is obtained by the following steps:
and A410, performing conversion processing on the sample image data to obtain an eight-bit sRGB image.
A420, performing image enhancement on the eight-bit sRGB image by using software such as Photoshop, adjusting the contrast, exposure, saturation and the like of the image, and obtaining a three-channel sRGB image.
The image enhancement algorithm needs to interpolate the single-channel image into a three-channel RGB image by a demosaicing algorithm, and the processed image is the sRGB image in step S520.
And A430, removing color noise from the three-channel sRGB image to obtain a three-channel RGB random color noise diagram. The range of normalized thresholds for the three-channel RGB random color noise map is (-1, 1).
The three-channel RGB random color noise map is used to improve the effect of the image data conversion model on removing color noise, and the shape of the noise map is shown in fig. 7, because the digital sensor generates different degrees of color noise due to different sensitivity degrees of the digital sensor to a scene when the digital sensor shoots an image, and if the color noise is not removed in step a530, the final imaging quality of the image is affected.
Specifically, the image data conversion model is a combination of FCN-Generated Antagonism Network (GAN) model, as shown in table 1, the image data conversion model adopts a coding and decoding structure, up-sampling of the decoding part adopts a combination of nearest neighbor up-sampling and convolution layer, the activation function of the output layer is Tanh, taking the size of the input image as 512 as an example, negativeSlope of LeakyReLU is 0.2, the discrimination network adopts Discriminator of multi_scale, and true and false images with different resolutions are respectively discriminated. In this embodiment, 3 scale discriminators are adopted to respectively discriminate images with 512x512, 256x256 and 128x128 resolutions, and images with different resolutions are directly sampled by a pooling layer (Pooling Layer), adam is used in an optimization algorithm, the learning rate of a generated network is 0.0002, and the learning rate of the discriminated network is 0.0001.
TABLE 1 concrete structure of image data conversion model
The total loss function of the image data transformation model training process can be expressed as:
Wherein α, β, γ, μ, σ represent weights corresponding to the loss function in the image data conversion model, respectively, where α=1, β=0.5, γ=0.5, σ=0.5, μ=0.5, and l Perc represents the perceptual loss function.
Wherein, tanh is a tanh processing, output represents a network Output image, that is, an sRGB image which is Output after the Raw format image data is enhanced, groundTruth is a target graph.
The L1_loss function is selected to ensure the color brightness of the image, the L1_loss function is a linear loss function, and before the L1_loss function is calculated, the Output images Output and GroundTruth are respectively subjected to the Tanh operation in order to ensure that the details of the high-brightness area of the Raw image are not lost, so that the image can be ensured to have higher weight in the high-brightness area.
To ensure image perceptual similarity, a perceptual loss Perceptual _loss based on VGG19, L Perc, is introduced.
To ensure image authenticity, the image data conversion model minimizes the distance between the color estimation of the real image and the image color distribution of the generated network output against the Loss gan_loss function.A penalty function representing the output of the arbiter for three different resolutions, 512x512, 256x256, 128x 128.
Where D represents the above-described arbiter, x represents the real data (i.e., groundTrut), z represents the network-generated data, px represents the distribution of real data samples, pz represents the distribution of network-generated data samples, E represents the expectation, and L adv represents the counterdamage function.
In order to ensure that the highlight region is restored and inhibited after the image is output, and simultaneously ensure that the color of the highlight region is excessively harmonious with that of the normal image region, the image data conversion model also uses L Mat to convert the color space of the input image and the output image into Lab space, calculate the mean value and variance of a channel a and a channel b of the input image and the output image, respectively calculate the L2_loss function of the mean value of the input image and the output image and the L2_loss function of the variance, and sum the L Mat,LMat to obtain the statistical loss function of the color of the exposure region.
The image data conversion apparatus provided by the present invention will be described below, and the image data conversion apparatus described below and the image data conversion method described above may be referred to correspondingly to each other.
Referring to fig. 8, the present invention provides an image data conversion apparatus, comprising:
The acquisition module 100 is used for capturing images through various different digital sensors and acquiring image data to be processed. The image data to be processed is data in a Raw format, including but not limited to CR2, ARW, NEF data.
The preprocessing module 200 is configured to preprocess image data to be processed.
In the preprocessing module 200, the purpose of preprocessing is to obtain a single-channel mask (mask) image highlight region with image highlight region information to protect details of the highlight region in the image acquired by the original data sensor.
The conversion enhancement module 300 is configured to input the preprocessed image data to be processed into the image data conversion model, and obtain the converted and enhanced image data output by the image data conversion model. The image data conversion model is obtained based on sample image data training, and the enhanced image data is converted into an sRGB image.
The conversion enhancement module 300 can convert the Raw format image data acquired by the digital sensor into an sRGB image, and enhance the saturation, contrast, exposure value, and the like of the image. In this embodiment, the FCN model is used as the image data conversion model, and since a full convolution network model is used, there is no need to scale the input and output images.
The traditional device for converting the Raw format image into the eight-bit sRGB image has a large amount of color noise in the finally obtained image, the highlight area of the image is not well protected, and the effect of the image is not improved and enhanced. The device is based on FCN to complete conversion from Raw format image data to sRGB image, and automatically enhances image effect in conversion process, and can adapt to Raw format image data collected by various digital sensor devices, complete effective image conversion and effect enhancement, and contrast, saturation, exposure effect and the like of images different from the traditional device are greatly improved.
Referring to fig. 9, the preprocessing module 200 specifically includes:
The first obtaining unit 210 is configured to obtain a normalized threshold value of a pixel value of each pixel point in the image data to be processed. It should be noted that, the Raw format image data is one single-channel gray scale data, and no color information is presented, so that the unprocessed image data is a single-channel gray scale map, and the range of the normalized threshold of the single-channel gray scale map which is not converted and enhanced is (-1, 1).
The first obtaining unit 220 is configured to obtain a gray value corresponding to each pixel according to the normalized threshold; the gray value of the pixel point with the normalized threshold value higher than the preset threshold value is a first gray value and is used as a first mask, and the gray value of the pixel point with the normalized threshold value not higher than the preset threshold value is a second gray value and is used as a second mask. In this embodiment, the preset threshold is 0.96, a gray value of a pixel with a normalized threshold higher than 0.96 is assigned 1, and the pixel is used as a first mask; and (3) giving 0 to the gray value of the pixel point with the normalized threshold value not higher than 0.96, and taking the pixel point as a second mask. In the device, a first mask is used to represent the highlight region in the image.
And a retaining unit 230, configured to retain the first mask, and obtain a single-channel mask image with image highlight region information. The range of normalized thresholds for single channel mask images with image highlight region information is (0, 1).
Referring to fig. 10, the image data conversion model is trained by the following modules:
the sample acquisition module 400 is used for acquiring sample image data. The sample acquisition module 400 has the same acquisition mode as the acquisition module 100, and can acquire sample image data under different scenes in consideration of the diversity requirement of the sample image data.
The sample preprocessing module 500 is used for preprocessing sample image data. Also, in the present embodiment, the preprocessing mode in the sample preprocessing module 500 is identical to the preprocessing mode of the preprocessing module 200.
The gain module 600, in order to obtain a network model with stronger robustness, the gain module 600 is configured to perform gain processing on the preprocessed sample image data, so as to add gain to the sample image data. Wherein the gain processing comprises one or more of flipping, rotation, translation, affine transformation, exposure, contrast adjustment, saturation adjustment, and blurring;
the training module 700 is configured to use the sample image data after the gain processing as input data for training, and perform training by adopting a deep learning manner, so as to obtain an image data conversion model for generating image data after conversion enhancement of the image data to be processed.
The input channels of the image data conversion model are five channels, and the input channels comprise original sample image data, preprocessed sample image data and a three-channel color standard random color noise diagram.
Referring to fig. 11, a three-channel color standard random color noise chart is obtained by the following modules:
the conversion unit 710 performs conversion processing on the sample image data to obtain an eight-bit sRGB image.
The enhancement unit 720 performs image enhancement on the eight-bit sRGB image by software such as Photoshop, adjusts contrast, exposure, saturation, and the like of the image, and obtains a three-channel sRGB image.
The image enhancement algorithm needs to interpolate the single-channel image into a three-channel RGB image by a demosaicing algorithm, and the processed image is the sRGB image in step S520.
And a denoising unit 730, configured to remove color noise from the three-channel sRGB image, and obtain a three-channel RGB random color noise map. The range of normalized thresholds for the three-channel RGB random color noise map is (-1, 1).
The three-channel RGB random color noise map is used to improve the effect of the image data conversion model on removing color noise, because the digital sensor generates color noise with different degrees due to different sensitivity degrees to the scene when capturing an image, and the final imaging quality of the image is affected if the color noise is not removed by the denoising unit 730.
Fig. 12 illustrates a physical structure diagram of an electronic device, as shown in fig. 12, which may include: processor 810, communication interface (Communications Interface) 820, memory 830, and communication bus 840, wherein processor 810, communication interface 820, memory 830 accomplish communication with each other through communication bus 840. Processor 810 may invoke logic instructions in memory 830 to perform an image data conversion method comprising the steps of:
s100, collecting image data to be processed; wherein, the image data to be processed is data in a Raw format;
s200, preprocessing the image data to be processed;
S300, inputting the preprocessed image data to be processed into an image data conversion model to obtain converted and enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is an sRGB image.
Further, the logic instructions in the memory 830 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the image data conversion method provided by the methods described above, the method comprising the steps of:
s100, collecting image data to be processed; wherein, the image data to be processed is data in a Raw format;
s200, preprocessing the image data to be processed;
S300, inputting the preprocessed image data to be processed into an image data conversion model to obtain converted and enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is an sRGB image.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the image data conversion method provided by the above methods, the method comprising the steps of:
s100, collecting image data to be processed; wherein, the image data to be processed is data in a Raw format;
s200, preprocessing the image data to be processed;
S300, inputting the preprocessed image data to be processed into an image data conversion model to obtain converted and enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is an sRGB image.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Those of ordinary skill in the art will appreciate that: the discussion of any of the embodiments above is merely exemplary and is not intended to suggest that the scope of the disclosure, including the claims, is limited to these examples; the technical features of the above embodiments or in the different embodiments may also be combined within the idea of the invention, the steps may be implemented in any order and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity.
The embodiments of the invention are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omission, modification, equivalent replacement, improvement, etc. of the present invention should be included in the scope of the present invention.
Claims (7)
1. An image data conversion method, characterized by comprising the steps of:
Collecting image data to be processed; wherein, the image data to be processed is data in a Raw format;
preprocessing the image data to be processed;
Inputting the preprocessed image data to be processed into an image data conversion model to obtain conversion-enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a general color standard image;
The preprocessing of the image data to be processed specifically comprises the following steps: acquiring a normalized threshold value of a pixel value of each pixel point in the image data to be processed; obtaining a gray value corresponding to each pixel point according to the normalized threshold value; the gray value of the pixel point with the normalized threshold value higher than the preset threshold value is a first gray value and is used as a first mask, and the gray value of the pixel point with the normalized threshold value not higher than the preset threshold value is a second gray value and is used as a second mask; reserving the first mask to obtain a single-channel mask image with image highlight region information;
The image data conversion model is obtained through training the following steps: collecting the sample image data; preprocessing the sample image data; performing gain processing on the preprocessed sample image data; wherein the gain processing comprises one or more of flipping, rotation, translation, affine transformation, exposure, contrast adjustment, saturation adjustment, blurring; training the sample image data after gain processing as input data for training by adopting a deep learning mode to obtain the image data conversion model for generating the image data after conversion enhancement of the image data to be processed;
the input channels of the image data conversion model are five channels, and the input channels comprise original sample image data, preprocessed sample image data and a three-channel color standard random color noise diagram.
2. The image data conversion method according to claim 1, wherein the three-channel color standard random color noise map is obtained by:
converting the sample image data to obtain an eight-bit universal color standard image;
performing image enhancement on the eight-bit general color standard image, and adjusting the contrast, exposure and saturation of the image to obtain a three-channel general color standard image;
And removing color noise from the three-channel universal color standard image to obtain a three-channel color standard random color noise diagram.
3. The image data conversion method according to claim 1, wherein the image data conversion model employs a full convolution network model.
4. An image data conversion apparatus, comprising:
An acquisition module (100) for acquiring image data to be processed; wherein, the image data to be processed is data in a Raw format;
A preprocessing module (200) for preprocessing the image data to be processed; the method specifically comprises the following steps: acquiring a normalized threshold value of a pixel value of each pixel point in the image data to be processed; obtaining a gray value corresponding to each pixel point according to the normalized threshold value; the gray value of the pixel point with the normalized threshold value higher than the preset threshold value is a first gray value and is used as a first mask, and the gray value of the pixel point with the normalized threshold value not higher than the preset threshold value is a second gray value and is used as a second mask; reserving the first mask to obtain a single-channel mask image with image highlight region information;
The conversion enhancement module (300) is used for inputting the preprocessed image data to be processed into an image data conversion model to obtain the converted and enhanced image data output by the image data conversion model; the image data conversion model is obtained based on sample image data training, and the image data after conversion enhancement is a general color standard image; the image data conversion model is obtained through training the following steps: collecting the sample image data; preprocessing the sample image data; performing gain processing on the preprocessed sample image data; wherein the gain processing comprises one or more of flipping, rotation, translation, affine transformation, exposure, contrast adjustment, saturation adjustment, blurring; training the sample image data after gain processing as input data for training by adopting a deep learning mode to obtain the image data conversion model for generating the image data after conversion enhancement of the image data to be processed; the input channels of the image data conversion model are five channels, and the input channels comprise original sample image data, preprocessed sample image data and a three-channel color standard random color noise diagram.
5. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the image data conversion method according to any one of claims 1 to 3 when the program is executed.
6. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the image data conversion method according to any one of claims 1 to 3.
7. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the image data conversion method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111027726.0A CN113744167B (en) | 2021-09-02 | 2021-09-02 | Image data conversion method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111027726.0A CN113744167B (en) | 2021-09-02 | 2021-09-02 | Image data conversion method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113744167A CN113744167A (en) | 2021-12-03 |
| CN113744167B true CN113744167B (en) | 2024-04-30 |
Family
ID=78735136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111027726.0A Active CN113744167B (en) | 2021-09-02 | 2021-09-02 | Image data conversion method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113744167B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1110380A4 (en) * | 1998-09-01 | 2002-02-20 | Divio Inc | Method and apparatus for edge enhancement in digital images |
| CN104180908A (en) * | 2014-08-07 | 2014-12-03 | 华能国际电力股份有限公司 | RAW image radiation temperature measuring apparatus and method |
| CN109379572A (en) * | 2018-12-04 | 2019-02-22 | 北京达佳互联信息技术有限公司 | Image conversion method, device, electronic equipment and storage medium |
| CN110084803A (en) * | 2019-04-29 | 2019-08-02 | 南京星程智能科技有限公司 | Eye fundus image method for evaluating quality based on human visual system |
| CN110211057A (en) * | 2019-05-15 | 2019-09-06 | 武汉Tcl集团工业研究院有限公司 | A kind of image processing method based on full convolutional network, device and computer equipment |
| CN110933313A (en) * | 2019-12-09 | 2020-03-27 | Oppo广东移动通信有限公司 | Dark light photographing method and related equipment |
| CN110992272A (en) * | 2019-10-18 | 2020-04-10 | 深圳大学 | Dark light image enhancement method, device, equipment and medium based on deep learning |
| CN111612722A (en) * | 2020-05-26 | 2020-09-01 | 星际（重庆）智能装备技术研究院有限公司 | Low-illumination image processing method based on simplified Unet full-convolution neural network |
| CN112258528A (en) * | 2020-11-02 | 2021-01-22 | Oppo广东移动通信有限公司 | Image processing method and device and electronic equipment |
| CN112446839A (en) * | 2020-11-30 | 2021-03-05 | 平安科技（深圳）有限公司 | Image enhancement method and device, electronic equipment and computer readable storage medium |
| CN113099121A (en) * | 2021-04-15 | 2021-07-09 | 西安交通大学 | ISP implementation method based on weak supervised learning |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7835569B2 (en) * | 2006-10-13 | 2010-11-16 | Apple Inc. | System and method for raw image processing using conversion matrix interpolated from predetermined camera characterization matrices |
| US10148871B2 (en) * | 2016-12-07 | 2018-12-04 | Microsoft Technology Licensing, Llc | Advanced raw conversion to produce high dynamic range, wide color gamut output |
-
2021
- 2021-09-02 CN CN202111027726.0A patent/CN113744167B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1110380A4 (en) * | 1998-09-01 | 2002-02-20 | Divio Inc | Method and apparatus for edge enhancement in digital images |
| CN104180908A (en) * | 2014-08-07 | 2014-12-03 | 华能国际电力股份有限公司 | RAW image radiation temperature measuring apparatus and method |
| CN109379572A (en) * | 2018-12-04 | 2019-02-22 | 北京达佳互联信息技术有限公司 | Image conversion method, device, electronic equipment and storage medium |
| CN110084803A (en) * | 2019-04-29 | 2019-08-02 | 南京星程智能科技有限公司 | Eye fundus image method for evaluating quality based on human visual system |
| CN110211057A (en) * | 2019-05-15 | 2019-09-06 | 武汉Tcl集团工业研究院有限公司 | A kind of image processing method based on full convolutional network, device and computer equipment |
| CN110992272A (en) * | 2019-10-18 | 2020-04-10 | 深圳大学 | Dark light image enhancement method, device, equipment and medium based on deep learning |
| CN110933313A (en) * | 2019-12-09 | 2020-03-27 | Oppo广东移动通信有限公司 | Dark light photographing method and related equipment |
| CN111612722A (en) * | 2020-05-26 | 2020-09-01 | 星际（重庆）智能装备技术研究院有限公司 | Low-illumination image processing method based on simplified Unet full-convolution neural network |
| CN112258528A (en) * | 2020-11-02 | 2021-01-22 | Oppo广东移动通信有限公司 | Image processing method and device and electronic equipment |
| CN112446839A (en) * | 2020-11-30 | 2021-03-05 | 平安科技（深圳）有限公司 | Image enhancement method and device, electronic equipment and computer readable storage medium |
| CN113099121A (en) * | 2021-04-15 | 2021-07-09 | 西安交通大学 | ISP implementation method based on weak supervised learning |
Non-Patent Citations (3)
| Title |
|---|
| Loss Functions for Image Restoration With Neural Networks;Hang Zhao等;IEEE Transactions on Computational Imaging;20161223;全文 * |
| 快速自适应RAW-HDR的色调映射算法的研究;周祥云;;福建电脑;20200725(07);全文 * |
| 极端低光情况下的图像增强方法;杨勇;刘惠义;;图学学报(04);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113744167A (en) | 2021-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109636754B (en) | Extremely-low-illumination image enhancement method based on generation countermeasure network | |
| EP3757890A1 (en) | Method and device for image processing, method and device for training object detection model | |
| CN108694705B (en) | Multi-frame image registration and fusion denoising method | |
| CN113766125B (en) | Focusing method and device, electronic equipment and computer readable storage medium | |
| CN110992272B (en) | Dark light image enhancement method, device, equipment and medium based on deep learning | |
| US9152878B2 (en) | Image processing apparatus, image processing method, and storage medium | |
| CN110493527B (en) | Body focusing method and device, electronic equipment and storage medium | |
| EP3644599B1 (en) | Video processing method and apparatus, electronic device, and storage medium | |
| WO2015184208A1 (en) | Constant bracketing for high dynamic range operations (chdr) | |
| JP7297470B2 (en) | Image processing method, image processing apparatus, program, image processing system, and method for manufacturing trained model | |
| CN112602088B (en) | Method, system and computer readable medium for improving quality of low light images | |
| CN110191287B (en) | Focusing method and device, electronic equipment and computer readable storage medium | |
| CN112150400B (en) | Image enhancement method and device and electronic equipment | |
| CN110365897B (en) | Image correction method and device, electronic equipment and computer readable storage medium | |
| CN108737797B (en) | White balance processing method and device and electronic equipment | |
| JP2006295582A (en) | Image processor, imaging apparatus, and image processing program | |
| CN113744167B (en) | Image data conversion method and device | |
| CN113298829B (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| CN113438386B (en) | Dynamic and static judgment method and device applied to video processing | |
| KR102389284B1 (en) | Method and device for image inpainting based on artificial intelligence | |
| Vazquez-Corral et al. | Angular-based preprocessing for image denoising | |
| CN113962844A (en) | Image fusion method, storage medium and terminal device | |
| CN117496019B (en) | Image animation processing method and system for driving static image | |
| JP7485356B2 (en) | Crack detection method, crack detection device and program | |
| CN112804467B (en) | Image coding method and device based on multiple CMOS sensors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |