CN114612343A - Model training and image processing method, device, equipment and medium - Google Patents

Model training and image processing method, device, equipment and medium Download PDF

Info

Publication number
CN114612343A
CN114612343A CN202210316837.1A CN202210316837A CN114612343A CN 114612343 A CN114612343 A CN 114612343A CN 202210316837 A CN202210316837 A CN 202210316837A CN 114612343 A CN114612343 A CN 114612343A
Authority
CN
China
Prior art keywords
image
defogging
sample image
model
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210316837.1A
Other languages
Chinese (zh)
Inventor
不公告发明人
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhendi Technology Co ltd
Original Assignee
Zhendi Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhendi Technology Co ltd filed Critical Zhendi Technology Co ltd
Priority to CN202210316837.1A priority Critical patent/CN114612343A/en
Publication of CN114612343A publication Critical patent/CN114612343A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06T5/73—

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)

Abstract

The application discloses a model training and image processing method, device, equipment and medium. Because the sample image pairs are collected in advance, the first defogged image corresponding to the synthesized fogging image included in any sample image pair can be acquired through the original defogging model, and a second defogged image of the sample image included in the sample image pair, and then based on the acquired first defogged image and the second defogged image, the original defogging model is trained to obtain the trained defogging model, so that the original defogging model can be trained directly according to the acquired first defogging image and the acquired second defogging image without labeling the sample image and the synthesized fogging image contained in the sample image pair, the workload required for labeling the sample image and the synthesized fogging image contained in the sample image pair is reduced, and the acquired model can be accurately defogged through the trained defogging model.

Description

Model training and image processing method, device, equipment and medium
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a medium for model training and image processing.
Background
Because a large amount of moisture and dust exist in the working environment of the intelligent device, images acquired by the intelligent device are easily blurred, and the subsequent processing of the acquired images is influenced. Therefore, how to remove the water mist in the image is a problem which is concerned more and more in recent years.
At present, a deep learning method can be adopted to remove water mist in an image, namely, an input foggy image is processed through a pre-trained defogging model, and a clear image with water mist removed is obtained. In order to obtain the defogging model, sample fogging images for training the defogging model need to be collected in advance, and the collected sample fogging images need to be labeled. Since the process of mapping the foggy image to the clear image is a pixel-level mapping process, each pixel point in the sample foggy image needs to be labeled, so that the labeling workload is increased, the pixel value of each pixel point in the foggy image in the clear image respectively needs to be obtained, the labeling difficulty is also increased, and the difficulty in obtaining the trained defogging model is high.
Disclosure of Invention
The application provides a model training and image processing method, device, equipment and medium, which are used for solving the problem that the difficulty in obtaining a trained defogged model is high due to the difficulty in labeling a sample defogged image of the trained defogged model and the large workload required in the prior art.
The application provides a model training method, which comprises the following steps:
acquiring a sample image pair; wherein the sample image pair comprises a sample image and a synthetic hazy image corresponding to the sample image;
for any sample image pair, acquiring a first defogged image corresponding to the synthesized fogging image included in the sample image pair and a second defogged image of the sample image included in the sample image pair through an original defogging model;
and training the original defogging model based on the first defogging image and the second defogging image to obtain a trained defogging model.
The application provides an image processing method based on a defogging model obtained by training with the model training method, which comprises the following steps:
acquiring an image to be processed;
and acquiring a defogged image corresponding to the image to be processed through the defogging model.
The application provides a model training device, the device includes:
an acquisition unit configured to acquire a sample image pair; wherein the sample image pair comprises a sample image and a synthetic hazy image corresponding to the sample image;
the processing unit is used for acquiring a first defogged image corresponding to the synthesized fogging image included in any sample image pair and a second defogged image of the sample image included in the sample image pair through an original defogging model;
and the training unit is used for training the original defogging model based on the first defogging image and the second defogging image so as to obtain the trained defogging model.
The application provides an image processing device based on a defogging model obtained by training according to the model training method, which comprises:
the acquisition module is used for acquiring an image to be processed;
and the processing module is used for acquiring the defogged image corresponding to the image to be processed through the defogging model.
The present application provides an electronic device comprising at least a processor and a memory, the processor being adapted to carry out the steps of the model training method as described above, or the steps of the image processing method as described above, when executing a computer program stored in the memory.
The present application provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the model training method as described above, or carries out the steps of the image processing method as described above.
Because the sample image pairs are collected in advance, a first defogged image corresponding to a synthesized fogging image included in any sample image pair and a second defogged image of the sample image included in the sample image pair can be obtained through the original defogging model subsequently, then the original defogging model is trained based on the obtained first defogged image and the obtained second defogged image to obtain the trained defogging model, so that the sample image and the synthesized fogging image included in the sample image pair are not required to be labeled, the original defogging model can be trained directly according to the obtained first defogged image and the obtained second defogged image, the workload required for labeling the sample image and the synthesized fogging image included in the sample image pair is reduced, and the difficulty in obtaining the trained defogging model is reduced, and the acquired model can be accurately defogged through the defogging model finished by training.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic diagram of a model training process provided in an embodiment of the present application;
FIG. 2 is a schematic diagram of a particular model training process provided in some embodiments of the present application;
fig. 3 is a schematic diagram of an image processing process provided in an embodiment of the present application;
fig. 4 is a schematic diagram of a specific image processing flow provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of a model training apparatus according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of another electronic device according to an embodiment of the present application.
Detailed Description
The present application will now be described in further detail with reference to the accompanying drawings, wherein like reference numerals refer to like elements throughout. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As will be appreciated by one skilled in the art, embodiments of the present application may be embodied as a system, apparatus, device, method, or computer program product. Thus, the present application may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
In this document, it is to be understood that any number of elements in the figures are provided by way of illustration and not limitation, and any nomenclature is used for differentiation only and not in any limiting sense.
In order to reduce the difficulty of obtaining a defogging model, the application provides a model training and image processing method, device, equipment and medium.
Example 1:
fig. 1 is a schematic diagram of a model training process provided in an embodiment of the present application, where the process includes:
s101: acquiring a sample image pair; wherein the sample image pair comprises a sample image and a synthetic hazy image corresponding to the sample image.
The model training method provided by the application is applied to electronic equipment, and the electronic equipment can be intelligent equipment or a server. The smart device may be, for example, a smart robot, a smart phone, a tablet computer, or the like.
In the application, a sample image pair for training the defogging model can be collected in advance, and the defogging model is trained. The sample image pair comprises a sample image and a synthesized foggy image corresponding to the sample image.
The sample image may be a foggy image or a fogless image. In the specific implementation process, the flexible setting can be performed according to the actual requirement, and is not specifically limited herein.
In one example, the sample image may be acquired by: determining the acquired original image as a sample image; and/or after adjusting the pixel values of the pixel points in the acquired original image, determining the adjusted image as a sample image.
As a possible implementation manner, if a sufficient number of original images are acquired, that is, a large number of original images acquired in different environments are included, or a large number of original images acquired in the same environment are included, the acquired original images may be determined as sample images according to the method.
As another possible implementation, if the accuracy of the defogging model is improved in order to ensure the diversity of the sample image, a large number of adjusted images can be obtained by adjusting the pixel values of the pixels in the original image, and the adjusted images are determined as the sample images.
As another possible implementation manner, the acquired original image and an adjusted image obtained by adjusting pixel values of pixel points in the acquired original image may be determined as a sample image.
In one example, adjusting the pixel values of the pixel points in the acquired original image by at least one of the following methods includes:
firstly, adjusting the pixel value of a pixel point in the original image through a preset convolution kernel;
secondly, adjusting the contrast of pixel values of pixel points in the original image;
performing brightness adjustment on pixel values of pixel points in the original image;
and fourthly, carrying out noise addition processing on the pixel values of the pixel points in the original image.
According to statistics, in a working scene of the electronic device, the more common image quality problems existing in the acquired image include: blur, exposure, over-darkness, too low contrast, noise in the picture, etc. In order to ensure the diversity of the sample images and improve the accuracy of the defogging model, the quality of the acquired original image may be adjusted in advance according to the possible quality problem in the acquired image in the working scene of the electronic device.
In an example, if it is desired to adjust the sharpness of the original image so as to obtain an adjusted image with different sharpness, the convolution calculation may be performed on the pixel values of the pixels in the original image through a preset convolution kernel, so as to adjust the pixel values of the pixels in the original image. In the process of adjusting the definition of the original image, the preset convolution kernels should be different as much as possible so as to diversify the definition of the sample image and improve the accuracy and robustness of the defogging model.
The process of adjusting the pixel value of a pixel point in an original image through a preset convolution kernel belongs to the prior art, and is not described in detail herein.
In one example, if it is desired to adjust the contrast of the original image, so as to obtain an adjusted image with different contrasts, the contrast adjustment may be performed on the pixel values of the pixels in the original image. In the process of adjusting the contrast of the original image, the adjusted images with different contrasts can be obtained as much as possible, so that the contrast of the sample image is more diversified, and the accuracy and the robustness of the defogging model are improved.
The process of adjusting the contrast of the pixel values of the pixels in the original image belongs to the prior art, and is not described herein in detail.
In one example, if it is desired to adjust the brightness of the original image, so as to obtain an adjusted image with different brightness, the brightness of the pixel values of the pixels in the original image may be adjusted. In the process of adjusting the brightness of the original image, adjusted images with different brightness can be obtained as much as possible, so that the brightness of the sample image is more diversified, and the accuracy and robustness of the defogging model are improved.
The process of adjusting the brightness of the pixel values of the pixels in the original image belongs to the prior art, and is not described herein in detail.
In an example, if it is desired to perform denoising on an original image so as to obtain an adjusted image with different noises, denoising pixel values of pixel points in the original image, that is, randomly adding noise to the original image. In the process of denoising the original image, the types of noise used should be as many as possible, such as white noise, salt and pepper noise, gaussian noise, etc., so as to diversify the sample image, thereby improving the accuracy and robustness of the defogging model.
The process of denoising the pixel values of the pixels in the original image belongs to the prior art, and is not described in detail herein.
By obtaining the sample images in the manner, the number of the sample images can be multiplied, so that a large number of sample images can be obtained quickly, and the difficulty, cost and consumed resources for obtaining the sample images are reduced. Subsequently, the sample image pair can be determined according to more sample images, so that the defogging model is trained according to the obtained sample image pair, and the trained defogging model is more accurate.
After the sample image is acquired based on the above embodiment, the synthesized foggy image corresponding to the sample image may be acquired, so that the sample image pair is determined according to the acquired sample image and the synthesized foggy image corresponding to the sample image. And subsequently training a defogging model based on the acquired sample image pair.
In an example, a synthesized fogging image corresponding to the sample image may be obtained through a preset fogging algorithm, or may be obtained by artificially adjusting the atmospheric transmittance of the sample image. Of course, the sample image may be synthesized with a preset foggy template image, so as to obtain a synthesized foggy image corresponding to the sample image.
Illustratively, for each sample image, a synthesized fogging image corresponding to the sample image is obtained based on a preset fogging algorithm; and determining the sample image pair according to the sample image and the synthesized foggy image corresponding to the sample image.
The preset fogging algorithm may be a gaussian distribution algorithm, a poisson distribution algorithm, or other distribution algorithms.
S102: for any sample image pair, acquiring a first defogged image corresponding to the synthesized fogging image included in the sample image pair and a second defogged image of the sample image included in the sample image pair through the original defogging model.
When the sample image pair is acquired based on the above-described embodiment, any sample image pair may be acquired, and the sample image and the synthesized fogging image included in the sample image pair are input to the original defogging model, respectively. Through the original defogging model, input images (including a sample image and a synthesized defogged image) can be processed, and a defogged image corresponding to the image is obtained. For example, if the input image is a synthesized foggy image, the synthesized foggy image may be processed through the original foggy image to determine a foggy image (denoted as a first foggy image) corresponding to the synthesized foggy image; if the input image is a sample image, the sample image can be processed through the original defogged image, and the defogged image (marked as a second defogged image) corresponding to the sample image is determined.
In one example, considering the correspondence between a foggy image and a defogged image corresponding to the foggy pattern can be expressed by the following formula (1):
I(x)＝t(x)*J(x)+(1-t(x))*A (1)
wherein, i (x) represents the x-th foggy image, j (x) represents the defogged image, a represents the atmospheric light of the foggy image, and t (x) represents the atmospheric transmittance of the x-th foggy image. From this formula it can be seen that there are three unknown variables, namely, atmospheric transmittance, defogged image, and atmospheric light, in the process of converting a foggy image to a defogged image. If the fog image, the atmospheric transmittance and the atmospheric light are known, the defogged image can be obtained by converting the formula. Therefore, in the present application, in order to acquire a defogged image, the original defogged model includes a network layer for acquiring atmospheric light of the defogged image (referred to as an atmospheric optical network layer), a network layer for acquiring atmospheric transmittance of the defogged image (referred to as an atmospheric transmittance network layer), and a network layer for acquiring the defogged image (referred to as an image generation network layer). The image generation network layer is respectively connected with an atmosphere transmissivity network layer and an atmosphere optical network layer, the atmosphere transmissivity network layer is used for determining the atmosphere transmissivity of the image input into the model, the atmosphere optical network layer is used for determining the atmosphere light of the image input into the model, and the image generation network layer is used for determining the defogged image corresponding to the image according to the output of the atmosphere transmissivity network layer, the output of the atmosphere optical network layer and the image input into the model.
It should be noted that the atmospheric optical network layer, the atmospheric transmittance network layer, and the image generation network layer may be deep learning networks, such as a cyclic convolution neural network, a convolution neural network, and the like. In the specific implementation process, the flexible setting can be performed according to the actual requirement, and is not specifically limited herein.
Illustratively, obtaining a first defogged image corresponding to the synthesized fogging image included in the sample image pair through the original defogging model includes:
determining the atmosphere light of the synthesized fogging image through an atmosphere optical network layer in the original defogging model; and are
Determining the atmospheric transmittance of the synthesized foggy image through an atmospheric transmittance network layer in the original defogging model;
generating a network layer from the images in the original defogging model, and determining the first defogged image based on the synthesized fogging image, the atmospheric light, and the atmospheric transmittance.
In a specific implementation process, any sample image pair is obtained, and the synthesized foggy image included in the sample image pair is input into the original defogging model. Processing the input synthetic fog image through an atmospheric optical network layer in the original defogging model, and determining atmospheric light (marked as first atmospheric light) of the synthetic fog image; and processing the input synthetic fog image through the atmosphere transmissivity network layer in the original defogging model to determine the atmosphere transmissivity (recorded as first atmosphere transmissivity) of the synthetic fog image. And acquiring a synthesized foggy image, first atmospheric light output by an atmospheric optical network layer and first atmospheric transmittance output by an atmospheric transmittance network layer through an image generation network layer in the original defogging model, and determining a first defogged image corresponding to the synthesized foggy image based on the synthesized foggy image, the first atmospheric light and the first atmospheric transmittance.
Illustratively, any sample image pair is acquired, and the sample images included in the sample image pair are input into the original defogging model. Processing the input sample image through an atmospheric optical network layer in the original defogging model, and determining atmospheric light (marked as second atmospheric light) of the sample image; and processing the input sample image through the atmospheric transmittance network layer in the original defogging model to determine the atmospheric transmittance (denoted as second atmospheric transmittance) of the sample image. And generating a network layer through the image in the original defogging model, acquiring a sample image, second atmospheric light output by the atmospheric optical network layer and second atmospheric transmittance output by the atmospheric transmittance network layer, and determining a second defogged image corresponding to the sample image based on the sample image, the second atmospheric light and the second atmospheric transmittance.
S103: and training the original defogging model based on the first defogging image and the second defogging image to obtain a trained defogging model.
Since the sample image in the sample image pair can be expressed by the following formula (2):
I1(x)＝t1(x)*J1(x)+(1-t1(x))*A1 (2)
wherein, I1(x) Representing the x-th sample image, J1(x) Representing a second defogged image, A1Atmospheric light, t, representing the sample image1(x) Atmosphere representing an image of a sampleTransmittance.
By adding the fog to the sample image, the synthesized foggy image corresponding to the obtained sample image can be represented by the following formula (3):
I2(x)＝t2(x)*J2(x)+(1-t2(x))*A2 (3)
wherein, I2(x) Representing the corresponding synthetic foggy image of the x-th sample image, J2(x) Representing a sample image, A2Atmospheric light, t, representing the combined foggy image2(x) Indicating the atmospheric transmittance of the synthesized hazy image.
In one example, since the synthesized hazy image included in any sample image pair is obtained by adding haze on the basis of the sample images included in the sample image pair, the first atmospheric transmittance of the synthesized hazy image and the second atmospheric throw ratio of the sample image can be expressed by the following formula (4):
t2(x)＝k(x)*t1(x)， (4)
wherein k (x) represents a conversion rate between the second atmospheric transmittance of the x-th sample image and the first atmospheric transmittance of the synthesized fogging image corresponding to the x-th sample image, t2(x) Represents the atmospheric transmittance, t, of the synthesized hazy image1(x) Representing the atmospheric transmittance of the sample image.
In order to make the acquired defogging model more robust, the defogging result of the defogging model corresponding to the acquired fogging images containing different concentrations of fog in the same scene is the same. Therefore, it is necessary that the corresponding first defogged image and the corresponding second defogged image are the same based on any sample image pair acquired in the above embodiment, that is, J2＝I1. J in the formula (3) can be expressed2By means of I1Instead, then substituting equation (2) into equation (3) results in the following equation (5):
I2(x)＝t2(x)*(t1(x)*J1(x)+(1-t1(x))*A1)+(1-t1(x))*A2 (5)
in view ofThe atmospheric light of the images acquired in the same scene is generally the same, i.e. A1＝A2. This condition is substituted into the above formula (5), and the formula (5) can be reduced by the above formula (5), and the reduced formula (5) is expressed by the following formula (6):
I2(x)＝t2(x)*t1(x)J1(x)+(1-t1(x)t2(x))*A1 (6)
it can be seen from the formula (6) that the synthesized hazy image included in any sample image pair can be obtained by the defogged image corresponding to the sample image included in the sample image pair, that is, the defogged image corresponding to the synthesized hazy image is the same as the defogged image corresponding to the sample image. Therefore, in the present application, the first defogged image and the second defogged image corresponding to any sample image pair obtained by the original defogging model should be the same. After the first defogged image and the second defogged image corresponding to a certain sample image pair are acquired based on the above embodiment, the loss value (referred to as the first loss value) may be determined based on the first defogged image and the second defogged image. And training the original defogging model according to the first loss value so as to adjust parameter values of parameters in the original defogging model.
In one example, the method further comprises:
for any sample image pair, acquiring first atmosphere light corresponding to a synthesized fog image included in the sample image pair and second atmosphere light of the sample image included in the sample image pair through the original defogging model;
training the original defogging model based on the first defogging image and the second defogging image comprises:
and training the original defogging model based on the first defogging image, the second defogging image, the first atmosphere light and the second atmosphere light to obtain a trained defogging model.
Considering that the atmospheric light of images in the same scene is generally the same, in this application, for any sample image pair, the first atmospheric light of the synthesized fogging image included in the sample image pair and the second atmospheric light of the sample image included in the sample image pair are acquired through the atmospheric optical network layer in the original defogging model respectively, and the acquired first atmospheric light and second atmospheric light should be the same, and based on the loss value between the first atmospheric light and the second atmospheric light, the original defogging model may be trained, and the parameter value of the parameter in the original defogging model may be adjusted.
In a specific implementation process, for any sample image pair, after a sample image and a synthesized fogging image included in the sample image pair are respectively input to an original defogging model, the input synthesized fogging image is processed through the original defogging model, a first defogging image corresponding to the synthesized fogging image and first atmospheric light of the synthesized fogging image can be obtained, and the input sample image is processed through the original defogging model, and a second defogging image corresponding to the sample image and second atmospheric light of the sample image can be obtained. A first loss value is determined based on the first and second defogged images. And determining a second loss value based on the first and second atmospheric lights. And training the original defogging model according to the second loss value and the first loss value so as to adjust parameter values of parameters in the original defogging model.
In one example tracking, a weighted sum may be determined according to the second loss value and its corresponding weighted value, and the first loss value and its corresponding weighted value. And training the original defogging model according to the weighted sums.
Since a large number of sample image pairs are collected in advance, the above steps are performed for each sample image pair, and when a preset convergence condition is satisfied, the defogging model training is completed.
The condition that the preset convergence condition is met may be that the sum of the first loss values respectively corresponding to each sample image pair of the current iteration is smaller than a preset threshold, or the iteration number of training the original defogging model reaches a set maximum iteration number. The specific implementation can be flexibly set, and is not particularly limited herein.
As a possible implementation manner, when performing original defogging model training, the sample image may be divided into a training sample and a test sample, the original defogging model is trained based on the training sample, and then the reliability of the trained defogging model is verified based on the test sample.
Because the sample image pairs are collected in advance, a first defogged image corresponding to a synthesized fogging image included in any sample image pair and a second defogged image of the sample image included in the sample image pair can be obtained through the original defogging model subsequently, then the original defogging model is trained based on the obtained first defogged image and the obtained second defogged image to obtain the trained defogging model, so that the sample image and the synthesized fogging image included in the sample image pair are not required to be labeled, the original defogging model can be trained directly according to the obtained first defogged image and the obtained second defogged image, the workload required for labeling the sample image and the synthesized fogging image included in the sample image pair is reduced, and the difficulty in obtaining the trained defogging model is reduced, the semi-supervised mode is realized to obtain the trained defogging model, and the acquired model can be accurately defogged through the trained defogging model subsequently.
Example 2:
the model training method provided by the present application is explained below by specific embodiments, and fig. 2 is a schematic diagram of a specific model training process provided by some embodiments of the present application, where the process includes:
s201: a sample image is acquired.
S202: and acquiring a synthesized foggy image corresponding to the sample image based on a preset foggy algorithm.
S203: and determining a sample image pair according to the sample image and the synthesized foggy image corresponding to the sample image.
After a large number of sample image pairs are acquired based on S201 to S203, the following steps may be performed for each sample image pair:
s204: and acquiring a first defogged image corresponding to the synthesized fogging image included by the sample image pair and first atmosphere light of the synthesized fogging image through the original defogging model.
Determining first atmospheric light of the synthesized foggy image through an atmospheric optical network layer in the original defogging model; determining a first atmospheric transmittance of the synthesized foggy image through an atmospheric transmittance network layer in the original defogging model; a network layer is generated through the images in the original defogging model, and the first defogged image is determined based on the synthesized fogging image, the first atmospheric light, and the first atmospheric transmittance.
S205: and acquiring a first defogged image corresponding to the sample image included by the sample image pair and first atmosphere light of the sample image through the original defogging model.
Determining second atmospheric light of the sample image through an atmospheric optical network layer in the original defogging model; determining a second atmospheric transmittance of the sample image through an atmospheric transmittance network layer in the original defogging model; generating a network layer through the image in the original defogging model, and determining the second defogged image based on the sample image, the second atmospheric light and the second atmospheric transmittance.
It should be noted that the steps of S204 and S205 may be interchanged, that is, S205 is executed first and then S204 is executed.
S206: and training the original defogging model based on the first defogging image, the second defogging image, the first atmosphere light and the second atmosphere light to obtain a trained defogging model.
Example 3:
the present application further provides an image processing method, and fig. 3 is a schematic diagram of an image processing process provided in an embodiment of the present application, where the process includes:
s301: and acquiring an image to be processed.
The image processing method provided by the application is applied to the electronic equipment, and the electronic equipment can be intelligent equipment or a server. The smart device may be, for example, a smart robot, a smart phone, a tablet computer, or the like.
The electronic device that performs image processing may be the same as or different from the electronic device that performs model training. In the specific implementation process, the setting can be flexibly performed according to the requirement, and is not specifically limited herein.
In an example, since the defogging model training process is generally performed in an off-line manner, the electronic device (e.g., the first server) performing the model training obtains the trained defogging model in advance based on the above-mentioned embodiment. The electronic device (e.g., the second server) that subsequently performs image processing may acquire image defogging based on the defogging model trained in the above embodiment.
In the present application, the image to be processed may be acquired by the electronic device that performs image processing itself, or may be sent by other devices.
In one example, it is considered that the image to be processed acquired by the electronic device performing image processing may be a foggy image or a fogless image. The acquired fog-free image is not required to be defogged. Therefore, in the application, after the electronic device acquires the image to be processed, whether the image to be processed is a foggy image or not can be determined. If the image to be processed is determined to be a foggy image, which indicates that the image to be processed needs to be defogged, the image to be processed may be subjected to subsequent processing based on the image processing method provided by the present application. And if the image to be processed is determined not to be the foggy image, which indicates that the image to be processed does not need to be defogged, acquiring the next image to be processed. By the method, resource consumption can be avoided for defogging the acquired fog-free image, and the calculation amount of the electronic equipment for processing the image is saved.
S302: and acquiring a defogged image corresponding to the image to be processed through the defogging model.
Based on the method in the above-described embodiments 1-2, a defogging model may be trained in advance. Since the training process of the defogging model is described in detail in the above-mentioned embodiments 1-2, it is not described herein.
In a specific implementation process, after an image to be processed is acquired, the image to be processed is input into a pre-trained defogging model. And processing the input image to be processed through the defogged image to obtain the defogged image corresponding to the image to be processed.
Exemplarily, determining the atmospheric light of the image to be processed through an atmospheric optical network layer in the defogging model; determining the atmospheric transmittance of the image to be processed through an atmospheric transmittance network layer in the defogging model; and generating a network layer through the image in the defogging model, and determining the defogged image corresponding to the image to be processed based on the image to be processed, the atmospheric light and the atmospheric transmittance.
Through the defogging model obtained by the embodiment, the acquired image to be processed can be accurately defogged.
Example 4:
the image processing method provided by the present application is described below by a specific embodiment, and fig. 4 is a schematic diagram of a specific image processing flow provided by the embodiment of the present application, taking as an example that an electronic device performing model training is the same as an electronic device performing image processing, the flow includes:
s401: and acquiring the defogging model after training.
It should be noted that the process of specifically obtaining the defogging model is described in the above embodiments, and repeated portions are not described again.
S402: and acquiring an image to be processed.
S403: and judging whether the image to be processed is a foggy image, if so, executing S404, and otherwise, executing S402.
S404: and acquiring a defogged image corresponding to the image to be processed through a pre-trained defogging model.
Example 5:
the present application provides a model training device, fig. 5 is a schematic structural diagram of a model training device provided in an embodiment of the present application, and the device includes:
an acquisition unit 51 for acquiring a sample image pair; wherein the sample image pair comprises a sample image and a synthetic hazy image corresponding to the sample image;
the processing unit 52 is configured to, for any sample image pair, obtain, through the original defogging model, a first defogged image corresponding to the synthesized fogging image included in the sample image pair, and a second defogged image of the sample image included in the sample image pair;
a training unit 53, configured to train the original defogging model based on the first defogging image and the second defogging image to obtain a trained defogging model.
In some possible embodiments, the obtaining unit 51 is specifically configured to, for each sample image, obtain a synthesized fogging image corresponding to the sample image based on a preset fogging algorithm; and determining the sample image pair according to the sample image and the synthesized foggy image corresponding to the sample image.
In some possible embodiments, the processing unit 52 is specifically configured to determine, through an atmospheric optical network layer in the original defogging model, the atmospheric light of the synthesized fogging image; determining the atmospheric transmittance of the synthesized foggy image through an atmospheric transmittance network layer in the original defogging model; generating a network layer from the images in the original defogging model, and determining the first defogged image based on the synthesized fogging image, the atmospheric light, and the atmospheric transmittance.
In some possible embodiments, the processing unit 52 is further configured to, for any of the sample image pairs, obtain, through the original defogging model, a first atmosphere light corresponding to the included synthesized fogging image of the sample image pair and a second atmosphere light corresponding to the included sample image of the sample image pair;
the training unit 53 is specifically configured to train the original defogging model based on the first defogging image and the second defogging image, and the first atmospheric light and the second atmospheric light, so as to obtain a trained defogging model.
Because the sample image pairs are collected in advance, a first defogged image corresponding to a synthesized fogging image included in any sample image pair and a second defogged image of the sample image included in the sample image pair can be obtained through the original defogging model subsequently, then the original defogging model is trained based on the obtained first defogged image and the obtained second defogged image to obtain the trained defogging model, so that the sample image and the synthesized fogging image included in the sample image pair are not required to be labeled, the original defogging model can be trained directly according to the obtained first defogged image and the obtained second defogged image, the workload required for labeling the sample image and the synthesized fogging image included in the sample image pair is reduced, and the difficulty in obtaining the trained defogging model is reduced, and the acquired model can be accurately defogged through the defogging model finished by training.
Example 6:
the present application provides an image processing apparatus, and fig. 6 is a schematic structural diagram of an image processing apparatus provided in an embodiment of the present application, where the apparatus includes:
an obtaining module 61, configured to obtain an image to be processed;
and the processing module 62 is configured to obtain a defogged image corresponding to the image to be processed through the defogging model.
In some possible embodiments, the processing module 62 is further configured to determine that the image to be processed is a foggy image before the defogged image corresponding to the image to be processed is acquired through the defogging model.
Example 7:
on the basis of the foregoing embodiments, an embodiment of the present application further provides an electronic device, and fig. 7 is a schematic structural diagram of another electronic device provided in the embodiment of the present application, as shown in fig. 7, including: the system comprises a processor 71, a communication interface 72, a memory 73 and a communication bus 74, wherein the processor 71, the communication interface 72 and the memory 73 are communicated with each other through the communication bus 74;
the memory 73 has stored therein a computer program which, when executed by the processor 71, causes the processor 71 to perform the steps of:
acquiring a sample image pair; wherein the sample image pair comprises a sample image and a synthetic hazy image corresponding to the sample image;
for any sample image pair, acquiring a first defogged image corresponding to the synthesized fogging image included in the sample image pair and a second defogged image of the sample image included in the sample image pair through an original defogging model;
and training the original defogging model based on the first defogging image and the second defogging image to obtain a trained defogging model.
Because the principle of solving the problem of the electronic device is similar to that of the model training method, the implementation of the electronic device can be referred to in embodiment 1-2 of the method, and repeated details are not repeated.
Because the sample image pairs are collected in advance, a first defogged image corresponding to a synthesized fogging image included in any sample image pair and a second defogged image of the sample image included in the sample image pair can be obtained through the original defogging model subsequently, then the original defogging model is trained based on the obtained first defogged image and the obtained second defogged image to obtain the trained defogging model, so that the sample image and the synthesized fogging image included in the sample image pair are not required to be labeled, the original defogging model can be trained directly according to the obtained first defogged image and the obtained second defogged image, the workload for labeling the sample image and the synthesized fogging image included in the sample image pair is reduced, and the difficulty for obtaining the trained defogging model is reduced, and the acquired model can be accurately defogged through the defogging model finished by training.
Example 8:
on the basis of the foregoing embodiment, an embodiment of the present application further provides an electronic device, and fig. 8 is a schematic structural diagram of another electronic device provided in the embodiment of the present application, as shown in fig. 8, including: the system comprises a processor 81, a communication interface 82, a memory 83 and a communication bus 84, wherein the processor 81, the communication interface 82 and the memory 83 are communicated with each other through the communication bus 84;
the memory 83 has stored therein a computer program which, when executed by the processor 81, causes the processor 81 to perform the steps of:
acquiring an image to be processed;
and acquiring a defogged image corresponding to the image to be processed through the defogging model.
Since the principle of the electronic device for solving the problem is similar to that of the image processing method, the implementation of the electronic device can be referred to in embodiments 3-4 of the method, and repeated details are not repeated.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface 82 is used for communication between the above-described electronic apparatus and other apparatuses. The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital instruction processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.
Example 9:
on the basis of the foregoing embodiments, the present application further provides a computer-readable storage medium, in which a computer program executable by a processor is stored, and when the program is run on the processor, the processor is caused to execute the following steps:
acquiring a sample image pair; wherein the sample image pair comprises a sample image and a synthetic hazy image corresponding to the sample image;
for any sample image pair, acquiring a first defogged image corresponding to the synthesized fogging image included in the sample image pair and a second defogged image of the sample image included in the sample image pair through an original defogging model;
and training the original defogging model based on the first defogging image and the second defogging image to obtain a trained defogging model.
Because the principle of solving the problem by the computer-readable storage medium is similar to that of the model training method, the implementation of the computer-readable storage medium can be referred to as implementation 1-2 of the method, and repeated details are not repeated.
Because the sample image pairs are collected in advance, a first defogged image corresponding to a synthesized fogging image included in any sample image pair and a second defogged image of the sample image included in the sample image pair can be obtained through the original defogging model subsequently, then the original defogging model is trained based on the obtained first defogged image and the obtained second defogged image to obtain the trained defogging model, so that the sample image and the synthesized fogging image included in the sample image pair are not required to be labeled, the original defogging model can be trained directly according to the obtained first defogged image and the obtained second defogged image, the workload required for labeling the sample image and the synthesized fogging image included in the sample image pair is reduced, and the difficulty in obtaining the trained defogging model is reduced, and the acquired model can be accurately defogged through the defogging model finished by training.
Example 10:
on the basis of the foregoing embodiments, the present application further provides a computer-readable storage medium, in which a computer program executable by a processor is stored, and when the program is run on the processor, the processor is caused to execute the following steps:
acquiring an image to be processed;
and acquiring a defogged image corresponding to the image to be processed through the defogging model.
Since the principle of solving the problem of the computer-readable storage medium is similar to that of the image processing method, the implementation of the computer-readable storage medium can be referred to as implementation 3-4 of the method, and repeated details are not repeated.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (10)

1. A method of model training, the method comprising:
acquiring a sample image pair; wherein the sample image pair comprises a sample image and a synthetic hazy image corresponding to the sample image;
for any sample image pair, acquiring a first defogged image corresponding to the synthesized fogging image included in the sample image pair and a second defogged image of the sample image included in the sample image pair through an original defogging model;
and training the original defogging model based on the first defogging image and the second defogging image to obtain a trained defogging model.
2. The method of claim 1, wherein said obtaining a sample image pair comprises:
for each sample image, acquiring a synthesized foggy image corresponding to the sample image based on a preset foggy algorithm; and determining the sample image pair according to the sample image and the synthesized foggy image corresponding to the sample image.
3. The method of claim 1, wherein obtaining a first defogged image corresponding to the synthesized fogging image included in the sample image pair via the original defogging model comprises:
determining the atmosphere light of the synthesized fogging image through an atmosphere optical network layer in the original defogging model; and are
Determining the atmospheric transmittance of the synthesized foggy image through an atmospheric transmittance network layer in the original defogging model;
generating a network layer from the images in the original defogging model, and determining the first defogged image based on the synthesized fogging image, the atmospheric light, and the atmospheric transmittance.
4. The method of claim 3, further comprising:
for any sample image pair, acquiring first atmosphere light corresponding to a synthesized fog image included in the sample image pair and second atmosphere light of the sample image included in the sample image pair through the original defogging model;
training the original defogging model based on the first defogging image and the second defogging image comprises:
and training the original defogging model based on the first defogging image, the second defogging image, the first atmosphere light and the second atmosphere light to obtain a trained defogging model.
5. An image processing method of a defogging model trained based on the model training method according to any one of claims 1 to 4, wherein said method comprises:
acquiring an image to be processed;
and acquiring a defogged image corresponding to the image to be processed through the defogging model.
6. The method according to claim 5, wherein before the obtaining of the defogged image corresponding to the image to be processed by the defogging model, the method further comprises:
and determining the image to be processed as a foggy image.
7. A model training apparatus, the apparatus comprising:
an acquisition unit configured to acquire a sample image pair; wherein the sample image pair comprises a sample image and a synthetic hazy image corresponding to the sample image;
the processing unit is used for acquiring a first defogged image corresponding to the synthesized fogging image included in any sample image pair and a second defogged image of the sample image included in the sample image pair through an original defogging model;
and the training unit is used for training the original defogging model based on the first defogging image and the second defogging image so as to obtain the trained defogging model.
8. An image processing apparatus for a defogging model trained based on the model training method according to any one of claims 1 to 4, wherein said apparatus comprises:
the acquisition module is used for acquiring an image to be processed;
and the processing module is used for acquiring the defogged image corresponding to the image to be processed through the defogging model.
9. An electronic device, characterized in that the electronic device comprises at least a processor and a memory, the processor being adapted to carry out the steps of the model training method according to any one of claims 1-4, or the steps of the image processing method according to claim 5, when executing a computer program stored in the memory.
10. A computer-readable storage medium, characterized in that it stores a computer program which, when being executed by a processor, carries out the steps of the model training method as claimed in any one of claims 1 to 4, or the steps of the image processing method as claimed in claim 5.
CN202210316837.1A 2022-03-28 2022-03-28 Model training and image processing method, device, equipment and medium Pending CN114612343A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210316837.1A CN114612343A (en) 2022-03-28 2022-03-28 Model training and image processing method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210316837.1A CN114612343A (en) 2022-03-28 2022-03-28 Model training and image processing method, device, equipment and medium

Publications (1)

Publication Number Publication Date
CN114612343A true CN114612343A (en) 2022-06-10

Family

ID=81866443

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210316837.1A Pending CN114612343A (en) 2022-03-28 2022-03-28 Model training and image processing method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN114612343A (en)

Similar Documents

Publication Publication Date Title
CN108122234B (en) Convolutional neural network training and video processing method and device and electronic equipment
JP6613605B2 (en) Method and system for restoring depth value of depth image
JP5954063B2 (en) Image fog removal method and system
US9330446B2 (en) Method and apparatus for processing image
JP6866889B2 (en) Image processing equipment, image processing methods and programs
CN109558901B (en) Semantic segmentation training method and device, electronic equipment and storage medium
WO2020010638A1 (en) Method and device for detecting defective pixel in image
CN109005367B (en) High dynamic range image generation method, mobile terminal and storage medium
CN110675334A (en) Image enhancement method and device
WO2021013049A1 (en) Foreground image acquisition method, foreground image acquisition apparatus, and electronic device
CN112272832A (en) Method and system for DNN-based imaging
CN113344801A (en) Image enhancement method, system, terminal and storage medium applied to gas metering facility environment
CN110211082B (en) Image fusion method and device, electronic equipment and storage medium
CN110717864B (en) Image enhancement method, device, terminal equipment and computer readable medium
CN115578286A (en) High dynamic range hybrid exposure imaging method and apparatus
JP2018133110A (en) Image processing apparatus and image processing program
CN113516697B (en) Image registration method, device, electronic equipment and computer readable storage medium
CN112465709B (en) Image enhancement method, device, storage medium and equipment
CN115880517A (en) Model training method and device and related equipment
CN114612343A (en) Model training and image processing method, device, equipment and medium
CN115719314A (en) Smear removing method, smear removing device and electronic equipment
CN112686851B (en) Image detection method, device and storage medium
CN111932514A (en) Image noise level estimation and suppression method and device and electronic equipment
CN110517204B (en) Noise elimination method and device of X-ray detector and detector
CN117151997A (en) Video restoration method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination