CN112750093A - Video image defogging method based on time sequence label transmission - Google Patents
Video image defogging method based on time sequence label transmission Download PDFInfo
- Publication number
- CN112750093A CN112750093A CN202110095486.1A CN202110095486A CN112750093A CN 112750093 A CN112750093 A CN 112750093A CN 202110095486 A CN202110095486 A CN 202110095486A CN 112750093 A CN112750093 A CN 112750093A
- Authority
- CN
- China
- Prior art keywords
- image
- defogging
- video
- video image
- foggy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000005540 biological transmission Effects 0.000 title claims abstract description 12
- 238000012549 training Methods 0.000 claims abstract description 40
- 238000012360 testing method Methods 0.000 claims abstract description 15
- 238000000605 extraction Methods 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 12
- 238000010586 diagram Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 abstract description 7
- 230000000694 effects Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 3
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 239000000443 aerosol Substances 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Studio Circuits (AREA)
Abstract
The invention discloses a video image defogging method based on time sequence label transmission, which comprises the following steps: firstly, establishing a foggy video image training set; designing a defogging network to defogge the foggy video image in the single group of sample images; thirdly, calculating a pixel-level absolute value loss function of the defogged image; fourthly, updating the weight parameter set; fifthly, taking the next group of sample images, and circulating the steps from the second step to the fourth step until the training is stopped; sixthly, defogging the actual foggy image of the first frame of the test video; seventhly, defogging the actual foggy image of the current non-first frame of the test video; and eighthly, the step seven is circulated for multiple times, and the continuous defogging of the whole video image sequence is realized. The invention uses the defogging results adjacent to the time sequence as the prior, guides the subsequent video frames to defogg, and continuously updates the guide images, thereby realizing the continuous natural change of the defogged video in the time dimension, simultaneously keeping the images capable of accurately defogging in the space dimension, and outputting clear video sequences with consistent space and time.
Description
Technical Field
The invention belongs to the technical field of video image defogging, and particularly relates to a video image defogging method based on time sequence label transmission.
Background
The video image of gathering under the bad weather of fog, haze and so on can because the effect of atmosphere scattering appears the quality degradation phenomenon, makes image color grey white, and the contrast reduces, and the object characteristic is difficult to discern, not only makes the visual effect variation, and the image sight reduces, still can lead to the understanding of image content to appear the deviation, if can lead to unable pedestrian or the lane line that detects the distant place because the influence of haze in the automatic driving field, and then causes the traffic accident. The defogging of the video image means that the adverse effect of the aerosol in the air on the video image is reduced or even eliminated by using a specific method and means, and the image is kept clear in space and transits naturally in time. Unlike single image defogging, video image defogging emphasizes normal defogging of each image, and simultaneously requires that an image sequence cannot have obvious color jump. In addition, the haze video image contains more information than a single haze, haze indication can be provided through the change of continuous frames, and the haze video image is subjected to defogging processing to obtain a clear image.
Existing video defogging algorithms are generally classified into 3 types, the first type is that videos are regarded as a single-frame image set, and defogging processing on the whole video is achieved by independently defogging each frame of image. The defogging algorithm according to a single image is mainly divided into a method based on image enhancement, a method based on a physical model and a method based on deep learning. The video defogging method based on the single frame image only carries out independent calculation on each frame image in the processing process, ignores the interframe time sequence correlation, easily causes the generated video defogging result to be discontinuous, and causes the color and brightness of the adjacent interframe defogging images to be suddenly changed in the background area. The second method is based on the fusion technology to realize video defogging. The method firstly enhances the front background area of each frame of image in the video, and then obtains the defogging result by applying the fusion technology. However, the separation algorithm of the method has dependency on the foreground and background, and usually when haze is severe, the foreground and the background cannot be accurately separated, and in addition, the contrast between the foreground and the background is easily distorted due to the respective enhancement. The third category of methods typically first evaluates a generic defogging model, which is then applied to the defogging process of subsequent frames. The method based on the universal defogging model can improve the solving efficiency of the video defogging algorithm, however, various defogging models depend on different assumption conditions, such as unobvious change of background areas in the video or equal distribution of image fog of each frame of the video, and the assumption of the real video is not satisfied, so that the effect of the method needs to be further improved.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a video image defogging method based on time sequence label transmission, which uses the defogging results adjacent to the time sequence as a priori, guides the subsequent video frames to defogg, and continuously updates the guide images, thereby realizing continuous natural change of the defogged video in the time dimension, and simultaneously keeping the images capable of accurately defogging in the space dimension, outputting clear video sequences with consistent time and space, and facilitating popularization and use.
In order to solve the technical problems, the invention adopts the technical scheme that: a video image defogging method based on time sequence label transmission is characterized by comprising the following steps:
step one, establishing a foggy video image training set, wherein the process is as follows:
step 101, using a sharp video image data set of known depth { X }iSynthesizing a fog video image training set { Y) with different haze degrees according to an atmospheric scattering modeliWhere i is a clear video picture frame number and i is 1,2A total number of frames of the clear video image and N is not less than 1000;
step 102, mixing (X)j,Yj+t,Xj+t) Forming a group of sample images, and constructing a fog video image training set { (X)j,Yj+t,Xj+t) In which, the foggy video image training set { (X)j,Yj+t,Xj+t) The number of sample image groups in the image is not less than 1000, t is a random positive integer not greater than 5, and j is a positive integer not greater than N-t;
step two, designing a defogging network to defogg the foggy video images in the single group of sample images, wherein the process is as follows:
step 201, extracting a first group of sample images from the training set of the foggy video images in the step one, overlapping the foggy video images in the group and the clear video images with frame numbers before, and sending the overlapped video images and the clear video images to an encoder to obtain a feature map f of an overlapped image size 1/81 /8The encoder comprises a plurality of downsampled convolution blocks, the convolution blocks comprising convolution layers, a batch normalization process, and an activation function;
step 202, converting the feature map f1/8Sending the data to a feature extraction module to obtain an enhanced feature map
The feature extraction module comprises a plurality of feature extraction units, and each feature extraction unit comprises a plurality of residual error modules and a jump connection positioned behind the residual error modules;
the residual error module comprises convolution block superposition identity mapping;
step 203, the characteristic diagram f1/8Deconvolution to obtain a deconvolution feature map f of the superimposed image size 1/41/4Will strengthen the feature map
Deconvolution to obtain a deconvolution enhanced feature map of the superimposed image size 1/4
Deconvoluting the feature map f1/4And deconvolution enhanced feature maps
Deconvolution is carried out after superposition to obtain a deconvolution enhanced feature map of the superposed image size 1/2
Step 204, deconvoluting the characteristic graph f1/4Deconvolution to obtain a deconvolution feature map f of the superimposed image size 1/21/2Deconvoluting the feature map f1/2And deconvolution enhanced feature maps
Deconvoluting after superposition to obtain a defogged image I of the foggy video image in the group of sample imagesd；
Step three, according to the formula
Calculating defogged image IdWherein Q is the total number of pixels of the video image with fog, Q is the number of pixels of the video image with fog, and Q is 1, 2.
Defogged image I for fog video image in sample image groupdThe pixel value of the q-th pixel point in the sequence,
the pixel value of the q-th pixel point in the clear video image with the frame number behind in the group of sample images is shown, and z is not more than a positive integer of N-t;
step four, updating the weight parameter set: to remove the fog image IdSending the pixel-level absolute value loss function L into an Adam optimizer, training and optimizing the defogging network in the step two, and updating a weight parameter set of the defogging network;
step five, taking a next group of sample images, taking the next group of sample images as a first group of sample images, and circulating the steps from step two to step four until the training reaches a preset training step number or the loss value does not decrease or the loss value is less than 0.001, stopping the training, at the moment, obtaining a final weight parameter set of the defogging network, and determining the final defogging network;
step six, defogging of an actual foggy image of a first frame of a test video;
step seven, defogging of the actual foggy image of the current non-first frame of the test video: the defogged image of the actual fogging image of the previous frame and the actual fogging image of the current non-first frame are used as input and sent into a final defogging network for forward reasoning to obtain the defogged image of the actual fogging image of the current non-first frame of the test video;
and step eight, the step seven is repeatedly circulated until the defogging of the whole video image sequence is completed, so that the continuous defogging of the whole video image sequence is realized.
The video image defogging method based on time sequence label transmission is characterized by comprising the following steps: in the fifth step, the preset training steps are 10000-20000.
The video image defogging method based on time sequence label transmission is characterized by comprising the following steps: in step 201, the encoder comprises at least three downsampled convolutional blocks; in step 202, the feature extraction module comprises 5-10 feature extraction units.
The video image defogging method based on time sequence label transmission is characterized by comprising the following steps: the set of clear video image data of known depth comprises a NYU image data set.
The video image defogging method based on time sequence label transmission is characterized by comprising the following steps: and step six, defogging the actual foggy image of the first frame of the test video by utilizing an artificial defogging mode or a single image defogging model.
Compared with the prior art, the invention has the following advantages:
1. the invention adopts the defogging image of the previous frame of the video sequence as a guide to defogge the image of the current frame, and continuously changes the guide defogging image, thereby ensuring the continuity of the defogging image of the video in the time dimension, realizing a clear video sequence with consistent space and time, and being convenient for popularization and use.
2. According to the method, the previous clear image, the current frame foggy image and the current frame clear image are combined into a group of sample images, a defogging network formed by an encoder and a feature extraction module is constructed by utilizing a deep neural network, and weight parameters of a model are obtained through optimization training, so that defogging enhancement of the video image is realized, and the method is reliable and stable and has a good using effect.
3. The method has simple steps, the deep learning model can effectively utilize the preorder defogging result to guide the defogging of the subsequent video image by synthesizing the training video and the structure of the time sequence training sample, and the method combines the time sequence priori knowledge and has the advantages of good image defogging effect, strong scene adaptability, and good defogging result space-time continuity and real-time performance; end-to-end video image defogging learning is carried out by utilizing the constructed training sample, so that the time sequence relation among video image sequences can be effectively learned, the method is suitable for defogging of videos under dynamic change scenes, and meanwhile, the time sequence range of the label images can be expanded, and continuous video defogging is realized; in order to further enhance the feature extraction capability of the encoder, feature enhancement is carried out through the continuous residual block with jump connection, so that the features of the foggy image and the time sequence label image can be effectively extracted, the extraction capability of the model on the time sequence features is enhanced, the defogging effect of the video image is improved, and the method is convenient to popularize and use.
In summary, the invention uses the defogging results adjacent to each other in time sequence as the prior, guides the subsequent video frames to defogg, and continuously updates the guide images, thereby realizing the continuous natural change of the defogged video in the time dimension, simultaneously keeping the images capable of accurately defogging in the space dimension, outputting clear video sequences with consistent time and space, and being convenient for popularization and use.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
FIG. 1 is a block diagram of the process flow of the present invention.
Detailed Description
As shown in fig. 1, a method for defogging a video image based on time-series tag delivery according to the present invention includes the following steps:
step one, establishing a foggy video image training set, wherein the process is as follows:
step 101, using a sharp video image data set of known depth { X }iSynthesizing a fog video image training set { Y) with different haze degrees according to an atmospheric scattering modeliWherein i is a clear video image frame number and i is 1,2, N is a total frame number of the clear video image and N is not less than 1000;
in this embodiment, the known depth sharp video image dataset comprises a NYU image dataset.
Step 102, mixing (X)j,Yj+t,Xj+t) Forming a group of sample images, and constructing a fog video image training set { (X)j,Yj+t,Xj+t) In which, the foggy video image training set { (X)j,Yj+t,Xj+t) The number of sample image groups in the image is not less than 1000, t is a random positive integer not greater than 5, and j is a positive integer not greater than N-t;
it should be noted that, a clear image with a certain time interval is added as a time sequence constraint when a group of sample images are constructed, and a pre-sequence image which is defogged can be used as an additional guide during testing, so that the consistency of the video image in time sequence after defogging is maintained.
Step two, designing a defogging network to defogg the foggy video images in the single group of sample images, wherein the process is as follows:
step 201, extracting a first group of sample images from the training set of the foggy video images in the step one, overlapping the foggy video images in the group and the clear video images with frame numbers before, and sending the overlapped video images and the clear video images to an encoder to obtain a feature map f of an overlapped image size 1/81 /8The encoder comprises a plurality of downsampled convolution blocks, the convolution blocks comprising convolution layers, a batch normalization process, and an activation function;
step 202, converting the feature map f1/8Sending the data to a feature extraction module to obtain an enhanced feature map
The characteristics areThe feature extraction module comprises a plurality of feature extraction units, and each feature extraction unit comprises a plurality of residual error modules and a jump connection positioned behind the residual error modules;
the residual error module comprises convolution block superposition identity mapping;
in this embodiment, in step 201, the encoder includes at least three downsampled convolution blocks; in step 202, the feature extraction module comprises 5-10 feature extraction units.
Step 203, the characteristic diagram f1/8Deconvolution to obtain a deconvolution feature map f of the superimposed image size 1/41/4Will strengthen the feature map
Deconvolution to obtain a deconvolution enhanced feature map of the superimposed image size 1/4
Deconvoluting the feature map f1/4And deconvolution enhanced feature maps
Deconvolution is carried out after superposition to obtain a deconvolution enhanced feature map of the superposed image size 1/2
Step 204, deconvoluting the characteristic graph f1/4Deconvolution to obtain a deconvolution feature map f of the superimposed image size 1/21/2Deconvoluting the feature map f1/2And deconvolution enhanced feature maps
Deconvoluting after superposition to obtain a defogged image I of the foggy video image in the group of sample imagesd；
Step three, according to the formula
Calculating defogged image IdPixel level absolute value loss ofA loss function L, wherein Q is the total number of pixels of the foggy video image, Q is the number of pixels of the foggy video image, and Q is 1, 2., Q,
defogged image I for fog video image in sample image groupdThe pixel value of the q-th pixel point in the sequence,
the pixel value of the q-th pixel point in the clear video image with the frame number behind in the group of sample images is shown, and z is not more than a positive integer of N-t;
step four, updating the weight parameter set: to remove the fog image IdSending the pixel-level absolute value loss function L into an Adam optimizer, training and optimizing the defogging network in the step two, and updating a weight parameter set of the defogging network;
step five, taking a next group of sample images, taking the next group of sample images as a first group of sample images, and circulating the steps from step two to step four until the training reaches a preset training step number or the loss value does not decrease or the loss value is less than 0.001, stopping the training, at the moment, obtaining a final weight parameter set of the defogging network, and determining the final defogging network;
in the fifth step, the number of the preset training steps is 10000-20000.
Step six, defogging of an actual foggy image of a first frame of a test video;
in this embodiment, in step six, the actual foggy image of the first frame of the test video is defogged by using an artificial defogging mode or a single image defogging model.
Step seven, defogging of the actual foggy image of the current non-first frame of the test video: the defogged image of the actual fogging image of the previous frame and the actual fogging image of the current non-first frame are used as input and sent into a final defogging network for forward reasoning to obtain the defogged image of the actual fogging image of the current non-first frame of the test video;
and step eight, the step seven is repeatedly circulated until the defogging of the whole video image sequence is completed, so that the continuous defogging of the whole video image sequence is realized.
When the method is used, the previous frame defogged image of the video sequence is used as a guide to defogg the current frame image, and the guide defogged image is continuously replaced, so that the continuity of the video defogged image on the time dimension is ensured, and a clear video sequence with consistent space-time is realized; the method comprises the steps of forming a group of sample images by using a previous clear image, a current frame foggy image and a current frame clear image, constructing a defogging network formed by an encoder and a feature extraction module by using a deep neural network, and obtaining weight parameters of a model through optimization training to realize defogging enhancement of a video image; through the construction of a synthetic training video and a time sequence training sample, the deep learning model can effectively utilize a preorder defogging result to guide the defogging of a subsequent video image, and the method combines time sequence priori knowledge and has the advantages of good image defogging effect, strong scene adaptability, and good defogging result space-time continuity and real-time performance; end-to-end video image defogging learning is carried out by utilizing the constructed training sample, so that the time sequence relation among video image sequences can be effectively learned, the method is suitable for defogging of videos under dynamic change scenes, and meanwhile, the time sequence range of the label images can be expanded, and continuous video defogging is realized; in order to further enhance the feature extraction capability of the encoder, the features of the foggy image and the time sequence label image can be effectively extracted by performing feature enhancement through the continuous residual block with jump connection, the extraction capability of the model on the time sequence features is enhanced, the defogging effect of the video image is improved, and the continuity and naturalness of the color and the contrast of the video defogged image are realized by guiding the transmission of the defogged image on the time sequence.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and all simple modifications, changes and equivalent structural changes made to the above embodiment according to the technical spirit of the present invention still fall within the protection scope of the technical solution of the present invention.
Claims (5)
1. A video image defogging method based on time sequence label transmission is characterized by comprising the following steps:
step one, establishing a foggy video image training set, wherein the process is as follows:
step 101, using a sharp video image data set of known depth { X }iSynthesizing a fog video image training set { Y) with different haze degrees according to an atmospheric scattering modeliWherein i is a clear video image frame number and i is 1,2, N is a total frame number of the clear video image and N is not less than 1000;
step 102, mixing (X)j,Yj+t,Xj+t) Forming a group of sample images, and constructing a fog video image training set { (X)j,Yj+t,Xj+t) In which, the foggy video image training set { (X)j,Yj+t,Xj+t) The number of sample image groups in the image is not less than 1000, t is a random positive integer not greater than 5, and j is a positive integer not greater than N-t;
step two, designing a defogging network to defogg the foggy video images in the single group of sample images, wherein the process is as follows:
step 201, extracting a first group of sample images from the training set of the foggy video images in the step one, overlapping the foggy video images in the group and the clear video images with frame numbers before, and sending the overlapped video images and the clear video images to an encoder to obtain a feature map f of an overlapped image size 1/81/8The encoder comprises a plurality of downsampled convolution blocks, the convolution blocks comprising convolution layers, a batch normalization process, and an activation function;
step 202, converting the feature map f1/8Sending the data to a feature extraction module to obtain an enhanced feature map
The feature extraction module comprises a plurality of feature extraction units, and each feature extraction unit comprises a plurality of residual error modules and a jump connection positioned behind the residual error modules;
the residual error module comprises convolution block superposition identity mapping;
step 203, the characteristic diagram f1/8Deconvolution to obtain a deconvolution feature map f of the superimposed image size 1/41/4Will strengthen the feature map
Deconvolution to obtain a deconvolution enhanced feature map of the superimposed image size 1/4
Deconvoluting the feature map f1/4And deconvolution enhanced feature maps
Deconvolution is carried out after superposition to obtain a deconvolution enhanced feature map of the superposed image size 1/2
Step 204, deconvoluting the characteristic graph f1/4Deconvolution to obtain a deconvolution feature map f of the superimposed image size 1/21/2Deconvoluting the feature map f1/2And deconvolution enhanced feature maps
Deconvoluting after superposition to obtain a defogged image I of the foggy video image in the group of sample imagesd；
Step three, according to the formula
Calculating defogged image IdWherein Q is the total number of pixels of the video image with fog, Q is the number of pixels of the video image with fog, and Q is 1, 2.
Defogged image I for fog video image in sample image groupdThe pixel value of the q-th pixel point in the sequence,
the pixel value of the q-th pixel point in the clear video image with the frame number behind in the group of sample images, wherein z is not more than N-tA positive integer;
step four, updating the weight parameter set: to remove the fog image IdSending the pixel-level absolute value loss function L into an Adam optimizer, training and optimizing the defogging network in the step two, and updating a weight parameter set of the defogging network;
step five, taking a next group of sample images, taking the next group of sample images as a first group of sample images, and circulating the steps from step two to step four until the training reaches a preset training step number or the loss value does not decrease or the loss value is less than 0.001, stopping the training, at the moment, obtaining a final weight parameter set of the defogging network, and determining the final defogging network;
step six, defogging of an actual foggy image of a first frame of a test video;
step seven, defogging of the actual foggy image of the current non-first frame of the test video: the defogged image of the actual fogging image of the previous frame and the actual fogging image of the current non-first frame are used as input and sent into a final defogging network for forward reasoning to obtain the defogged image of the actual fogging image of the current non-first frame of the test video;
and step eight, the step seven is repeatedly circulated until the defogging of the whole video image sequence is completed, so that the continuous defogging of the whole video image sequence is realized.
2. The method of claim 1 for defogging a video image based on time sequence label delivery, wherein: in the fifth step, the preset training steps are 10000-20000.
3. The method of claim 1 for defogging a video image based on time sequence label delivery, wherein: in step 201, the encoder comprises at least three downsampled convolutional blocks; in step 202, the feature extraction module comprises 5-10 feature extraction units.
4. The method of claim 1 for defogging a video image based on time sequence label delivery, wherein: the set of clear video image data of known depth comprises a NYU image data set.
5. The method of claim 1 for defogging a video image based on time sequence label delivery, wherein: and step six, defogging the actual foggy image of the first frame of the test video by utilizing an artificial defogging mode or a single image defogging model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110095486.1A CN112750093B (en) | 2021-01-25 | 2021-01-25 | Video image defogging method based on time sequence label transmission |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110095486.1A CN112750093B (en) | 2021-01-25 | 2021-01-25 | Video image defogging method based on time sequence label transmission |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112750093A true CN112750093A (en) | 2021-05-04 |
| CN112750093B CN112750093B (en) | 2021-10-22 |
Family
ID=75653061
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110095486.1A Active CN112750093B (en) | 2021-01-25 | 2021-01-25 | Video image defogging method based on time sequence label transmission |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112750093B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101290680A (en) * | 2008-05-20 | 2008-10-22 | 西安理工大学 | Foggy day video frequency image clarification method based on histogram equalization overcorrection restoration |
| WO2014081199A1 (en) * | 2012-11-21 | 2014-05-30 | 중앙대학교 산학협력단 | Apparatus and method for removing haze from single image using alpha matte estimation and image synthesis |
| CN106780356A (en) * | 2016-11-15 | 2017-05-31 | 天津大学 | Image defogging method based on convolutional neural networks and prior information |
| CN109300090A (en) * | 2018-08-28 | 2019-02-01 | 哈尔滨工业大学（威海） | A kind of single image to the fog method generating network based on sub-pix and condition confrontation |
| CN109360171A (en) * | 2018-10-26 | 2019-02-19 | 北京理工大学 | A kind of real-time deblurring method of video image neural network based |
| CN109360155A (en) * | 2018-08-17 | 2019-02-19 | 上海交通大学 | Single-frame images rain removing method based on multi-scale feature fusion |
| CN110097519A (en) * | 2019-04-28 | 2019-08-06 | 暨南大学 | Double supervision image defogging methods, system, medium and equipment based on deep learning |
| CN110288550A (en) * | 2019-06-28 | 2019-09-27 | 中国人民解放军火箭军工程大学 | The single image defogging method of confrontation network is generated based on priori knowledge guiding conditions |
| CN111626944A (en) * | 2020-04-21 | 2020-09-04 | 温州大学 | Video deblurring method based on space-time pyramid network and natural prior resistance |
| CN111681180A (en) * | 2020-05-25 | 2020-09-18 | 厦门大学 | Priori-driven deep learning image defogging method |
-
2021
- 2021-01-25 CN CN202110095486.1A patent/CN112750093B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101290680A (en) * | 2008-05-20 | 2008-10-22 | 西安理工大学 | Foggy day video frequency image clarification method based on histogram equalization overcorrection restoration |
| WO2014081199A1 (en) * | 2012-11-21 | 2014-05-30 | 중앙대학교 산학협력단 | Apparatus and method for removing haze from single image using alpha matte estimation and image synthesis |
| CN106780356A (en) * | 2016-11-15 | 2017-05-31 | 天津大学 | Image defogging method based on convolutional neural networks and prior information |
| CN109360155A (en) * | 2018-08-17 | 2019-02-19 | 上海交通大学 | Single-frame images rain removing method based on multi-scale feature fusion |
| CN109300090A (en) * | 2018-08-28 | 2019-02-01 | 哈尔滨工业大学（威海） | A kind of single image to the fog method generating network based on sub-pix and condition confrontation |
| CN109360171A (en) * | 2018-10-26 | 2019-02-19 | 北京理工大学 | A kind of real-time deblurring method of video image neural network based |
| CN110097519A (en) * | 2019-04-28 | 2019-08-06 | 暨南大学 | Double supervision image defogging methods, system, medium and equipment based on deep learning |
| CN110288550A (en) * | 2019-06-28 | 2019-09-27 | 中国人民解放军火箭军工程大学 | The single image defogging method of confrontation network is generated based on priori knowledge guiding conditions |
| CN111626944A (en) * | 2020-04-21 | 2020-09-04 | 温州大学 | Video deblurring method based on space-time pyramid network and natural prior resistance |
| CN111681180A (en) * | 2020-05-25 | 2020-09-18 | 厦门大学 | Priori-driven deep learning image defogging method |
Non-Patent Citations (5)
| Title |
|---|
| HANG DONG 等: "Multi-Scale Boosted Dehazing Network with Dense Feature Fusion", 《ARXIV》 * |
| YAN ZHAO SU 等: "Prior guided conditional generative adversarial network for single image dehazing", 《NEUROCOMPUTING》 * |
| YIN GAO 等: "Single Fog Image Restoration via Multi-scale Image Fusion", 《2017 3RD IEEE INTERNATIONAL CONFERENCE ON COMPUTER AND COMMUNICATIONS (ICCC)》 * |
| 王昕 等: "图像和视频的快速去雾算法研究", 《影像科学与光化学》 * |
| 白雪纯: "降质视频图像化及其应用研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112750093B (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112884064B (en) | Target detection and identification method based on neural network | |
| CN110555465B (en) | Weather image identification method based on CNN and multi-feature fusion | |
| CN111915531A (en) | Multi-level feature fusion and attention-guided neural network image defogging method | |
| CN110288550B (en) | Single-image defogging method for generating countermeasure network based on priori knowledge guiding condition | |
| CN109509156B (en) | Image defogging processing method based on generation countermeasure model | |
| CN106295645B (en) | A kind of license plate character recognition method and device | |
| CN109829868B (en) | Lightweight deep learning model image defogging method, electronic equipment and medium | |
| CN112149535B (en) | Lane line detection method and device combining SegNet and U-Net | |
| CN105898111B (en) | A kind of video defogging method based on spectral clustering | |
| CN112508814B (en) | Image tone restoration type defogging enhancement method based on unmanned aerial vehicle at low altitude visual angle | |
| CN110223251A (en) | Suitable for manually with the convolutional neural networks underwater image restoration method of lamp | |
| CN103034983A (en) | Defogging method based on anisotropic filtering | |
| CN103260043A (en) | Binocular stereo image matching method and system based on learning | |
| CN113269685A (en) | Image defogging method integrating multi-attention machine system | |
| CN112784834A (en) | Automatic license plate identification method in natural scene | |
| CN115019340A (en) | Night pedestrian detection algorithm based on deep learning | |
| CN114596233A (en) | Attention-guiding and multi-scale feature fusion-based low-illumination image enhancement method | |
| CN112750093B (en) | Video image defogging method based on time sequence label transmission | |
| CN107301625A (en) | Image defogging algorithm based on brightness UNE | |
| CN112767275B (en) | Single image defogging method based on artificial sparse annotation information guidance | |
| CN108564535A (en) | A kind of image defogging method based on deep learning | |
| CN114862707A (en) | Multi-scale feature recovery image enhancement method and device and storage medium | |
| CN113191971A (en) | Unmanned aerial vehicle image defogging method based on YUV color space | |
| CN116311002B (en) | Unsupervised video target segmentation method based on optical flow information | |
| CN114972076B (en) | Image defogging method based on layered multi-block convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |