WO2021238420A1 - Procédé de désembuage d'image, terminal et support de stockage informatique - Google Patents

Procédé de désembuage d'image, terminal et support de stockage informatique Download PDF

Info

Publication number
WO2021238420A1
WO2021238420A1 PCT/CN2021/085694 CN2021085694W WO2021238420A1 WO 2021238420 A1 WO2021238420 A1 WO 2021238420A1 CN 2021085694 W CN2021085694 W CN 2021085694W WO 2021238420 A1 WO2021238420 A1 WO 2021238420A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
defogging
processed
model
sub
Prior art date
Application number
PCT/CN2021/085694
Other languages
English (en)
Chinese (zh)
Inventor
崔永明
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2021238420A1 publication Critical patent/WO2021238420A1/fr

Links

Images

Classifications

    • G06T5/73—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the embodiments of the present application relate to the field of terminal technology, and in particular, to an image defogging method, terminal, and computer storage medium.
  • image defogging has become an important research field of computer vision.
  • image defogging methods mainly include two categories: non-physical model defogging methods and physical model defogging methods.
  • the non-physical model defogging method is essentially to enhance the contrast and color of the image;
  • the physical model defogging method is to use the atmospheric physical scattering law to establish an image restoration model.
  • the non-physical model defogging method can only improve the visual effect, and cannot improve the quality of the fogged image, and may even lose some information of the image; the physical model defogging method has a long processing time and low processing efficiency. It can be seen that the existing image defogging methods are still not mature enough to take into account the effect and efficiency of defogging processing.
  • the embodiments of the present application provide an image defogging method, terminal, and computer storage medium, which can greatly increase the processing speed while improving processing accuracy, thereby realizing high-quality and high-efficiency image defogging processing.
  • an embodiment of the present application provides an image defogging method, and the method includes:
  • the preprocessing strategy is to divide the image to be processed, divide the image to be processed to obtain a sub-image corresponding to the image to be processed;
  • an embodiment of the present application provides a terminal.
  • the terminal includes: a determining part, a dividing part, a dehazing part, and a dividing part,
  • the determining part is configured to determine the size parameter of the image to be processed after determining that the image to be processed is a fogged image; and determine the preprocessing strategy corresponding to the image to be processed according to the size parameter; wherein the preprocessing strategy is used To limit the size of the image;
  • the dividing part is configured to divide the image to be processed if the preprocessing strategy is to divide the image to be processed to obtain a sub-image corresponding to the image to be processed;
  • the defogging part is configured to perform defogging processing on the sub-image according to the image defogging model to obtain a defogging sub-image corresponding to the sub-image;
  • the dividing part is configured to perform stitching processing on the defogging sub-images according to an image stitching model to obtain a defogging image corresponding to the image to be processed.
  • an embodiment of the present application provides a terminal.
  • the terminal includes a processor and a memory storing executable instructions of the processor. When the instructions are executed by the processor, the above-mentioned Image defogging method.
  • an embodiment of the present application provides a computer-readable storage medium with a program stored thereon and applied to a terminal.
  • the program is executed by a processor, the image defogging method as described above is implemented.
  • the embodiments of the application provide an image defogging method, terminal, and computer storage medium. After determining that the image to be processed is a fogged image, the terminal determines the size parameter of the image to be processed; and determines the preprocessing strategy corresponding to the image to be processed according to the size parameter.
  • the preprocessing strategy is used to limit the image size; if the preprocessing strategy is to divide the image to be processed, the image to be processed is divided to obtain the sub-image corresponding to the image to be processed; the sub-image is defogged according to the image defogging model Process to obtain the defogging sub-image corresponding to the sub-image; perform stitching processing on the defogging sub-image according to the image stitching model to obtain the defogging image corresponding to the image to be processed. It can be seen that, in the embodiment of the present application, the terminal can use the image defogging model obtained by deep learning to defog the image to be processed.
  • the terminal can also use the image defogging.
  • the image stitching model obtained by deep learning is used for stitching processing to increase the processing speed while ensuring the processing accuracy.
  • the image defogging model and the image stitching model are obtained by the terminal's minimal network design on the CNN, so the above-mentioned image defogging method can be run in the terminal in real time.
  • the image defogging process based on the image defogging model and the image stitching model obtained by deep learning can greatly increase the processing speed while improving the processing accuracy, thereby achieving high-quality and high-efficiency images. Defogging treatment.
  • Figure 1 is a schematic diagram of a current image defogging processing method
  • Figure 2 is the first schematic diagram of the implementation process of the image defogging method
  • Figure 3 is a second schematic diagram of the implementation process of the image defogging method
  • Figure 4 is the third schematic diagram of the implementation process of the image defogging method
  • Figure 5 is a fourth schematic diagram of the implementation process of the image defogging method
  • Fig. 6 is a schematic diagram of conventional convolution processing
  • Figure 7 is a schematic diagram of Depthwise convolution processing
  • Fig. 8 is a schematic diagram of Pointwise convolution processing
  • Figure 9 is a schematic diagram five of the implementation process of the image defogging method.
  • FIG. 10 is a sixth schematic diagram of the implementation process of the image defogging method.
  • Figure 11 is a first schematic diagram of the terminal structure
  • Figure 12 is a second schematic diagram of the terminal structure.
  • the contrast of the image captured in foggy weather is low, and color distortion is caused, and the machine vision system may even fail to work normally.
  • the collected pictures are affected by suspended particulates in the atmosphere (such as fog, haze, etc.), resulting in poor picture quality, making it difficult to distinguish the features of the objects in the picture, and even affecting, for example, The quality of pictures in outdoor surveillance, target recognition and traffic navigation. Therefore, the sharpening of foggy image features is extremely important.
  • image defogging has become an important research field of computer vision.
  • traditional image processing methods or physical model defogging methods are mainly used to input fogged images and output clear images.
  • the traditional image processing method is essentially to enhance the contrast and color of the image, which can only improve the visual effect, but cannot improve the quality of the fogged image, and may even lose some information of the image;
  • the physical model defogging method is to use atmospheric physical scattering
  • the image restoration model is established according to the law, but the processing time is long and the processing efficiency is low.
  • image enhancement algorithms directly start from the perspective of image processing, by enhancing the contrast of the foggy image, highlighting the characteristics or effective information of the image, and improving the visual effect of the image to a certain extent.
  • this type of method ignores the real cause of image degradation, so for pictures with complex scenes, the quality of the picture cannot be improved, and some information of the image may even be lost.
  • the model defogging algorithm establishes an atmospheric scattering model, studies the physical principles of image degradation, and obtains the scattering effect of suspended particles in the atmosphere on the light and the influence on the picture, and restores more realistic pictures, and in complex scenes The medium dehazing effect is better, and the image information is more complete.
  • This algorithm has a good dehazing effect for pictures in non-sky areas, but the results are not ideal for bright areas with sky, and the algorithm is too computationally expensive and low in efficiency.
  • Fig. 1 is a schematic diagram of a current image defogging processing method.
  • the existing technical solution is mainly to input a fogged image into a defogging algorithm model, and output a clear image after defogging.
  • the machine learning model is generated based on traditional algorithms. It can be seen that the prior art uses traditional algorithms to process the entire fogged image, which has the following disadvantages:
  • the image defogging method proposed in this application uses AI to build an image defogging model, and at the same time , You can also assist the defogging processing of large-size images through the constructed image stitching model, so that the effect and efficiency of the defogging processing can be improved at the same time.
  • both the image defogging model and the image stitching model are obtained by simplifying the design on the basis of the convolutional neural network (Convolutional Neural Networks, CNN) network structure. -Accumulates, MACs) run in the mobile terminal to achieve end-to-end real-time image defogging.
  • the terminal can use the image defogging model obtained by deep learning to defog the image to be processed.
  • the terminal can also After the image defogging model is used to perform defogging processing on the sub-images after the image to be processed is divided, the image stitching model obtained by deep learning is used for stitching processing, so as to increase processing speed while ensuring processing accuracy.
  • the image defogging model and the image stitching model are obtained by the terminal's minimal network design on the CNN, so the above-mentioned image defogging method can be run in the terminal in real time.
  • the image defogging process based on the image defogging model and the image stitching model obtained by deep learning can greatly increase the processing speed while improving the processing accuracy, thereby achieving high-quality and high-efficiency images. Defogging treatment.
  • FIG. 2 is a schematic diagram of the implementation process of the image defogging method. As shown in FIG. It includes the following steps:
  • Step 101 After determining that the image to be processed is a fogged image, determine the size parameter of the image to be processed.
  • the terminal may first determine the size parameter of the image to be processed.
  • the above-mentioned terminal may be any terminal with communication and storage functions, such as: tablet computer, mobile phone, e-reader, remote control, personal computer (PC), notebook Terminals such as computers, in-vehicle equipment, Internet TV, wearable devices, personal digital assistants (PDA), portable media players (PMP), navigation devices, etc.
  • PC personal computer
  • notebook Terminals such as computers, in-vehicle equipment, Internet TV, wearable devices, personal digital assistants (PDA), portable media players (PMP), navigation devices, etc.
  • PDA personal digital assistants
  • PMP portable media players
  • the image to be processed may be an image stored in advance by the terminal.
  • the image to be processed may be an image in a mobile phone album; the image to be processed may also be an image collected by the terminal in real time, for example,
  • the processed image may be a preview image captured by a mobile phone;
  • the to-be-processed image may also be a frame of an image in a video recorded in real time by the terminal.
  • the to-be-processed image may be a frame of a surveillance video recorded by a surveillance device.
  • the image to be processed may be pre-stored by the terminal, may also be collected by the terminal in real time, or may be sent by the terminal receiving other devices, which is not specifically limited in this application.
  • Step 102 Determine a preprocessing strategy corresponding to the image to be processed according to the size parameter; wherein, the preprocessing strategy is used to limit the size of the image.
  • the terminal may further determine the preprocessing strategy corresponding to the image to be processed according to the size parameter.
  • the preprocessing strategy can be used to limit the image size.
  • the preprocessing strategies determined by the terminal are different.
  • the corresponding preprocessing strategy determined by the terminal may be to perform image division preprocessing on the image first, and then perform defogging processing; For smaller images, the corresponding pre-processing strategy determined by the terminal can be directly defogging.
  • the preprocessing strategy determined by the terminal is different for images to be processed with different size parameters.
  • the preprocessing strategy may include non-division processing and division processing.
  • the size parameter of the image to be processed may include the length and width of the image, where the length and width of the image may be in units of pixels or centimeters.
  • the terminal may preset a size parameter used to determine the preprocessing strategy of the image to be processed, that is, the preset size threshold.
  • the terminal when the terminal determines the preprocessing strategy corresponding to the image to be processed according to the size parameter, it may compare the size parameter of the image to be processed with a preset size threshold, so that it can further be based on the comparison result. Determine the preprocessing strategy of the image to be processed.
  • the terminal after the terminal compares the size parameter of the image to be processed with the preset size threshold, if the comparison result is that the size parameter is greater than or equal to the preset size threshold, the terminal can determine the preprocessing The strategy is to divide the image to be processed; if the comparison result is that the size parameter is less than the preset size threshold, the terminal can determine that the preprocessing strategy is not to divide the image to be processed.
  • the terminal can limit the height of the image to be processed, or limit the width of the image to be processed, or limit the height and width of the image to be processed; accordingly, the preset The size threshold may be the upper limit of the height size, the upper limit of the width size, or the upper limit of the height and the width.
  • the terminal may determine the preset size threshold as 2048 ⁇ 1536, so that the height and width of the image to be processed may be restricted according to the size threshold of 2048 ⁇ 1536.
  • Step 103 If the preprocessing strategy is to divide the image to be processed, divide the image to be processed to obtain a sub-image corresponding to the image to be processed.
  • the terminal after the terminal determines the preprocessing strategy corresponding to the image to be processed according to the size parameter, if the preprocessing strategy is to divide the image to be processed, the terminal can divide the image to be processed to obtain the subprocess corresponding to the image to be processed. image.
  • the terminal may first divide the image to be processed into multiple sub-images of equal size, and then divide the divided image into multiple sub-images. Sub-images are processed for defogging to speed up the defogging processing.
  • the terminal when the terminal divides the image to be processed, it can determine the number of sub-images obtained by the division according to the size parameter of the image to be processed, that is, for different size parameters, divide the image to be processed
  • the number of sub-images obtained later may also be different.
  • the image to be processed with the size parameter a1 is divided into 6 sub-images
  • the image to be processed with the size parameter a2 is divided into 9 sub-images, where a1 is smaller than a2.
  • the number of divisions may be preset, that is, for different size parameters, the number of sub-images obtained after dividing the image to be processed is the same.
  • the image to be processed with the size parameter a1 is divided into 9 sub-images
  • the image to be processed with the size parameter a2 is divided into 9 sub-images, where a1 is smaller than a2.
  • the terminal may divide the image to be processed into n sub-images of the same size, where n is an integer greater than 1.
  • Step 104 Perform defogging processing on the sub-image according to the image defogging model to obtain a defogging sub-image corresponding to the sub-image.
  • the terminal can divide the image to be processed and obtain the sub-images corresponding to the image to be processed, and then pair the image according to the image defogging model.
  • the image is processed for defogging, so that the defogging sub-image corresponding to the sub-image can be obtained.
  • the terminal since the terminal divides the image to be processed into n sub-images of the same size, correspondingly, the terminal can obtain n sub-images after defogging each sub-image.
  • the n sub-images correspond to n sub-images after defogging, where one sub-image corresponds to one sub-image after defogging.
  • the terminal may use a deep learning image defogging model to perform defogging processing on multiple sub-images respectively.
  • the image defogging model can be an image defogging algorithm built by the terminal based on AI.
  • the image defogging model may be obtained by the terminal training based on the convolutional neural network CNN.
  • the terminal can use MobileNet-V2 in CNN to determine the image defogging model.
  • Deep CNN networks such as ResNet and DenseNet have greatly improved the accuracy of image classification.
  • computational complexity is also an important indicator to be considered for CNN networks. Overly complex networks may be very slow.
  • some lightweight CNN networks such as MobileNet have been proposed, which have a good balance between speed and accuracy.
  • the terminal on the basis of the first lightweight convolutional neural network, can modify part of the convolutional layer to Depthwise convolution + Pointwise convolution, so that the processing effect can be guaranteed at the same time Improve image processing efficiency.
  • the first lightweight convolutional neural network can be the MobileNet-V2 network in CNN.
  • MobileNet-V2 is an improvement of MobileNet-V1. It is also a lightweight convolutional neural network. It is a deep learning network suitable for mobile terminals. Among them, because MobileNet-V2 uses depthwise separable convolution instead of traditional Convolution method, the original convolution is divided into two parts of Depthwise convolution + Pointwise convolution, so it has the advantages of fewer parameters, smaller models, and less accuracy than some traditional convolutions.
  • Depthwise that is, different channels use different convolution kernels to convolution to extract features
  • Pointwise that is, the convolution of a certain point and a certain pixel.
  • Depthwise convolution and Pointwise convolution are collectively called Depthwise Separable Convolution.
  • This structure is similar to conventional convolution operations and can be used to extract features, but compared to conventional convolution operations, its parameter amount and computational cost are lower, so This structure is more suitable for lightweight networks, such as MobileNet.
  • the terminal when the terminal performs defogging processing on sub-images according to the image defogging model, in order to improve processing efficiency, the terminal can dehaze each sub-image based on the image defogging model. Fog treatment.
  • the terminal after the terminal performs defogging processing on the sub-images according to the image defogging model, it can obtain a defogging sub-image corresponding to each sub-image.
  • the terminal when the terminal constructs the image defogging model, it may first divide the first image sample set to obtain the first training data and the first test data; wherein, the first image sample The set includes the fogged image and the clear image corresponding to the fogged image; then the first network model is constructed based on the first lightweight convolutional neural network, and the first network model is trained according to the first training data to obtain the initial defogging model ; Finally, the initial defogging model is tested according to the first test data, and then the image defogging model can be obtained.
  • Step 105 Perform splicing processing on the defogging sub-images according to the image splicing model to obtain a defogging image corresponding to the image to be processed.
  • the terminal may stitch all the defogging sub-images according to the image stitching model Through processing, a clear image corresponding to the image to be processed can be obtained, that is, the image after defogging.
  • the terminal after the terminal performs defogging processing on multiple sub-images of the image to be processed, it can obtain multiple defogging sub-images corresponding to the multiple sub-images, and further, the terminal needs to pass The multiple defogging sub-images are stitched together, and finally a frame of defogging image corresponding to the image to be processed is obtained.
  • the terminal may use a deep learning image splicing model to splice multiple defogging sub-images into corresponding defogging images.
  • the image stitching model may be an image stitching algorithm constructed by the terminal based on artificial intelligence AI.
  • the image stitching model may be obtained by the terminal training based on CNN.
  • the terminal can use ShuffleNet-V2 in CNN to determine the image stitching model.
  • the lightweight convolutional neural network CNN also includes ShuffleNet, which can also achieve a good balance between speed and accuracy.
  • the terminal can remove part of the convolutional layer, and modify the part of the convolutional layer to Depthwise convolution + Pointwise convolution.
  • the network is accelerated, so that a very small network design for CNN can be carried out based on this, and the efficiency of image processing can be improved while ensuring the splicing effect.
  • the second lightweight convolutional neural network may be the ShuffleNet-V2 network in CNN.
  • ShuffleNet-V2 is an upgraded version of ShuffleNet-V1. Under the same complexity, ShuffleNet-V2 is more accurate than ShuffleNet-V1 and MobileNet-V2.
  • the ShuffleNet-V2 version introduces a new operation: Channel Split, which divides the input channels of the module into two parts, one part is passed down directly, and the other part is for real backward calculation. At the end of the module, the number of output channels on the two branches is directly connected, thereby avoiding the element-wise sum operation in ShuffleNet-V1. Then we perform Random Shuffle operation on the final output feature maps, so that the information between the channels can communicate with each other.
  • Channel Split which divides the input channels of the module into two parts, one part is passed down directly, and the other part is for real backward calculation.
  • the number of output channels on the two branches is directly connected, thereby avoiding the element-wise sum operation in ShuffleNet-V1.
  • Random Shuffle operation on the final output feature maps, so that the information between the channels can communicate with each other.
  • the left branch does the same mapping
  • the right branch contains 3 consecutive convolutions
  • the input and output channels are the same, which conforms to the G1 principle (the same channel width can minimize the memory access cost).
  • two 1x1 convolutions are no longer group convolutions, which conforms to the G2 principle (excessive group convolutions will increase access costs), and the other two branches are equivalent to being divided into two groups.
  • the output of the two branches is no longer an Add element, but concat together, followed by a channle shuffle of the concat results of the two branches to ensure the exchange of information between the two branches.
  • concat and channel shuffle can be combined with the channel split of the next module unit to form an element-level operation, which is in line with principle G4.
  • the terminal when the terminal constructs the image stitching model, it may first divide the second image sample set to obtain the second training data and the second test data; wherein, the second image sample set Including the original image and multiple decomposed images corresponding to the original image; then constructing a second network model based on the second lightweight convolutional neural network, and training the second network model according to the second training data to obtain the initial stitching model; and finally The initial stitching model is tested according to the second test data, and then the image stitching model can be obtained.
  • the terminal can first divide the image to be processed to obtain the sub-image corresponding to the image to be processed; then perform the defogging process on the sub-image according to the image defogging model , So as to obtain the defogging sub-images corresponding to the sub-images; finally, the defogging sub-images can be stitched based on the image stitching model to obtain the defogging images.
  • Figure 3 is a schematic diagram of the second implementation process of the image defogging method. As shown in Figure 3, in the embodiment of the present application, after the terminal determines the preprocessing strategy corresponding to the image to be processed according to the size parameter, that is, after step 102, the terminal The method for image defogging processing may further include the following steps:
  • Step 106 If the preprocessing strategy is not to divide the image to be processed, then directly perform defogging processing on the image to be processed according to the image defogging model to obtain a defogging image.
  • the terminal after the terminal determines the preprocessing strategy corresponding to the image to be processed according to the size parameter, it can perform defogging processing on the image to be processed according to the image defogging model based on the preprocessing measurement strategy, so as to obtain the image to be processed
  • the corresponding clear image is the image after defogging.
  • the terminal can use the image defogging model to perform defogging processing on the image to be processed.
  • the terminal if the preprocessing strategy is not to divide the image to be processed, the terminal does not need to divide the image to be processed. Therefore, the terminal can directly remove the image to be processed according to the image defogging model. Fog processing to obtain the defogged image corresponding to the image to be processed.
  • the terminal can select different preprocessing strategies to perform defogging processing on the image to be processed, thereby improving the processing efficiency of the defogging processing.
  • Figure 4 is a schematic diagram of the third implementation process of the image defogging method. As shown in Figure 4, in the embodiment of the present application, before the terminal determines the size parameters of the image to be processed, that is, before step 101, the terminal performs image defogging processing
  • the method can also include the following steps:
  • Step 107 Perform analysis processing on the image to be processed to obtain the analysis result.
  • the terminal may first analyze and process the image to be processed to obtain the analysis result.
  • the terminal when the terminal analyzes and processes the image to be processed, it can detect feature information in the image to be processed, and then obtain the analysis result of the image to be processed based on the feature information.
  • the analysis result can be used to characterize whether the image to be processed is foggy or the degree of fogging of the image to be processed.
  • the terminal may use a pre-learned recognition model to analyze and process the image to be processed, that is, the terminal inputs the value of the image to be processed into the recognition model, and outputs the analysis result of the image to be processed.
  • Step 108 Determine whether the image to be processed is a fogged image according to the analysis result.
  • the terminal analyzes and processes the image to be processed and obtains the analysis result, it can determine whether the image to be processed is a fogged image based on the analysis result, that is, whether it is necessary to perform defogging processing on the image to be processed.
  • the terminal judges the image to be processed based on the analysis result. Whether it is a fogged image, you can directly determine a foggy image to be processed as a fogged image, and determine a non-fogged image to be processed as a non-fogged image; you can also determine an image with a higher degree of fog as a fogged image A fogged image, and an image with a low degree of fogging is determined as a non-fogged image.
  • the terminal may first extract the image to be processed to obtain the smallest RGB component of each pixel of the image to be processed.
  • the component value is stored in a grayscale image of the same size as the image to be processed, and then the grayscale image is divided into multiple 15 ⁇ 15 windows, and each window is filtered with the minimum value. After replacing all the pixels of the window with the minimum pixel value of, the dark channel image is obtained.
  • the terminal can separately make differences between all the pixel values of the dark channel image and the image to be processed, accumulate all the differences to obtain the sum of the differences, and then compare the sum of the differences with the difference threshold, if the sum of the differences is less than the difference threshold , The terminal can arbitrarily consider that the image to be processed does not need to be defogged; if the sum of the differences is greater than or equal to the difference threshold, the terminal can arbitrarily consider that the image to be processed needs to be defogged, and then judge the image to be processed as fogging image.
  • AI is used to build an image defogging model, and at the same time, the built image stitching model can also be used to assist large-size images.
  • the image defogging process can improve the effect and efficiency of the defogging process at the same time.
  • the image defogging method proposed in the present application includes accurately defogging the image through the AI image defogging algorithm, which greatly improves the effect of the defogging processing; it also includes the image stitching processing through the AI image stitching algorithm, and Angles, images in multiple scenes have better stitching effects.
  • the image defogging method using deep learning proposed in this application is more than 2 times more accurate than the traditional algorithm.
  • the large-size image can be divided into multiple sub-images for parallel processing, and then the processing results are stitched together , So the processing speed of large-size images can be increased by more than 5 times.
  • the image defogging method proposed in this application can be widely used in video surveillance, image beautification and other fields.
  • the image defogging method proposed in this application can be widely used in video surveillance, image beautification and other fields.
  • the field of surveillance when the fog is relatively large, the camera can hardly take a clear picture of the target object. After defogging the image, the target task can be clearly seen.
  • image beautification when using a mobile phone to take a selfie, if the weather is bad or the shot is blurry, the image defogging algorithm can automatically defog the image to make the image clearer.
  • This application proposes an image defogging method.
  • the size parameter of the image to be processed is determined; the preprocessing strategy corresponding to the image to be processed is determined according to the size parameter; wherein the preprocessing strategy is used In order to limit the image size; if the preprocessing strategy is to divide the image to be processed, the image to be processed is divided to obtain the sub-image corresponding to the image to be processed; the sub-image is defogged according to the image defogging model to obtain the corresponding sub-image Sub-image after defogging; the sub-image after defogging is stitched according to the image stitching model to obtain the defogging image corresponding to the image to be processed.
  • the terminal can use the image defogging model obtained by deep learning to defog the image to be processed.
  • the terminal can also use the image defogging.
  • the image stitching model obtained by deep learning is used for stitching processing to increase the processing speed while ensuring the processing accuracy.
  • the image defogging model and the image stitching model are obtained by the terminal's minimal network design on the CNN, so the above-mentioned image defogging method can be run in the terminal in real time.
  • the image defogging process based on the image defogging model and the image stitching model obtained by deep learning can greatly increase the processing speed while improving the processing accuracy, thereby achieving high-quality and high-efficiency images. Defogging treatment.
  • FIG. 5 is a schematic diagram of the fourth implementation process of the image defogging method, as shown in FIG.
  • the sub-image is subjected to defogging processing, and before the defogging sub-image corresponding to the sub-image is obtained, that is, before step 104, the method for the terminal to perform image defogging processing may further include the following steps:
  • Step 109 Divide the first image sample set to obtain first training data and first test data; wherein, the first image sample set includes a fogged image and a clear image corresponding to the fogged image.
  • Step 1010 Construct a first network model based on the first lightweight convolutional neural network, and perform training processing on the first network model according to the first training data to obtain an initial defogging model.
  • Step 1011 Perform a test process on the initial defogging model according to the first test data to obtain an image defogging model.
  • the terminal may first construct the image defogging model before performing the defogging processing on the image to be processed using the image defogging model.
  • the terminal may construct the first network model based on the first lightweight convolutional neural network, and improve the speed by modifying the processing.
  • part of the convolutional layer can be modified to Depthwise convolution + Pointwise convolution.
  • the image defogging model designed in this way can be Increase the speed without reducing the accuracy, and can run in the terminal in real time.
  • Depthwise Separable Convolution is to decompose a complete convolution operation into two steps, namely decomposition into Depthwise convolution and Pointwise convolution.
  • Fig. 6 is a schematic diagram of conventional convolution processing, as shown in Fig. 6, for a 5 ⁇ 5 pixel, three-channel color input picture (shape is 5 ⁇ 5 ⁇ 3).
  • the convolutional layer of the 3 ⁇ 3 convolution kernel (assuming the number of output channels is 4, the shape of the convolution kernel is 3 ⁇ 3 ⁇ 3 ⁇ 4), and finally output 4 Feature Maps, if there is same padding, the size and input layer Same (5 ⁇ 5), if not, the size becomes 3 ⁇ 3.
  • FIG. 7 is a schematic diagram of Depthwise convolution processing. As shown in Figure 7, for a 5 ⁇ 5 pixel, three-channel color input picture (shape is 5 ⁇ 5 ⁇ 3), Depthwise convolution first undergoes the first convolution operation Unlike conventional convolution, Depthwise is completely performed in a two-dimensional plane. The number of convolution kernels is the same as the number of channels in the previous layer (the channel and the convolution kernel correspond one-to-one). Therefore, a three-channel image generates 3 Feature maps (if there is same padding, the size is the same as the input layer and is 5 ⁇ 5).
  • the number of feature maps is the same as the number of channels in the input layer, and the feature map cannot be expanded. Moreover, this operation independently performs convolution operations on each channel of the input layer, and does not effectively use the feature information of different channels at the same spatial position. Therefore, Pointwise convolution is needed to combine these feature maps to generate a new feature map.
  • FIG 8 is a schematic diagram of Pointwise convolution processing.
  • the operation of Pointwise convolution is very similar to conventional convolution operations.
  • the size of its convolution kernel is 1 ⁇ 1 ⁇ M, and M is the channel of the previous layer. number. Therefore, the convolution operation here will perform a weighted combination of the feature map of the previous step in the depth direction to generate a new feature map.
  • MobileNet-V1 Depthwise+Pointwise, and then the linear rectification function (Rectified Linear Unit, ReLU) is activated;
  • MobileNet-V2 1 ⁇ 1 Pointwise channel expansion, Depthwise+Pointwise channel compression, and Linear activation. Among them, Depthwise is to perform convolution on a single channel, and the increase in the number of previous channels will not have much impact on its calculation.
  • Residual block of ResNet There are two convolutions in the original ResNet connection.
  • the bottleneck structure connection of ResNet is 1 ⁇ 1 channel compression + convolution + 1 ⁇ 1 channel expansion;
  • MobileNet-V2's residual Inverted Residual block 1 ⁇ 1 channel expansion + convolution + 1 ⁇ 1 channel reduction.
  • t is the multiplication factor of the input channel, which means “expansion” multiple
  • c is the number of output channels
  • n is the number of repetitions of the module
  • s is the stride.
  • the terminal may first use the MobileNet-V2 network in the convolutional neural network CNN to construct the first network model, and then obtain the image defogging model through training and testing of the first network model.
  • the terminal may first obtain the first image sample set, where the first image sample set includes a fogged image and a clear image corresponding to the fogged image. That is to say, in the first image sample set, there is a one-to-one correspondence between the fogged image and the clear image.
  • the first image sample set includes 100 fogged images and clear images corresponding to different scenes, that is, the first image sample set stores 200 frames of images.
  • the first image sample set may be collected by the terminal, or may be sent by the terminal receiving other image collection devices, which is not specifically limited in this application.
  • the terminal may divide the first image sample set to obtain the first training data and the first test data.
  • the first training data is used to train the first network model
  • the first test data is used to test the initial defogging model.
  • the first training data and the first test data both include a one-to-one correspondence between fog images and clear images.
  • the first training data and the first test data The images are not the same, that is, any group of fogged images and clear images in the first image sample set can only be divided into the first training data or the first test data, and cannot be used as the training image and the first training data at the same time.
  • the test image in the first test data is not the same, that is, any group of fogged images and clear images in the first image sample set can only be divided into the first training data or the first test data, and cannot be used as the training image and the first training data at the same time.
  • the terminal after the terminal constructs the first network model based on the first lightweight convolutional neural network, and divides the first image sample set to obtain the first training data and the first test data, the terminal can First use the first training data to train the first network model to obtain the initial defogging model; then use the first test data to test the initial defogging model to obtain the final image defogging model.
  • the terminal when the terminal uses the first training data to train the first network model, it can label the first training data, and after repeated iterations of the first training data, the recognition accuracy is finally obtained. High initial dehazing model.
  • multiple loss functions may be used for joint processing
  • the terminal may use Softmax loss and focal loss for joint training
  • Softmax loss is one of the most commonly used loss functions and is widely used in image classification and segmentation tasks. Softmax loss is a combination of softmax and cross-entropy loss, so the full name is Softmax with cross-entropy loss. In the implementation of open source frameworks such as caffe and tensorflow, the two are directly placed in one layer. Instead of separating different layers, you can make numerical calculations more stable, because the positive exponential probability may have a very large value.
  • Focal loss is mainly to solve the problem of a serious imbalance in the ratio of positive and negative samples in one-stage target detection. This loss function reduces the weight of a large number of simple negative samples in training, and can also be understood as a kind of difficult sample mining.
  • the terminal tests the initial defogging model according to the first test data to obtain the image defogging model, and may first test the initial defogging model according to the first test data. Process to generate a first test result; then modify the initial defogging model according to the first test result to obtain an image defogging model.
  • the terminal after the terminal uses the initial defogging model trained by the first training data, it can test the defogging processing effect of the initial defogging model based on the first test data. Specifically, the terminal can input the fogged image in the first test data into the initial defogging model, output the defogging image, and then compare the defogging image with the corresponding clear image to obtain the first image information. Test Results.
  • the terminal after obtaining the first test result, can modify the initial defogging model obtained by training based on the first test result, thereby improving the effect of defogging processing and obtaining optimization
  • the latter model is the image dehazing model.
  • the terminal in the embodiment of this application, can cut the network and optimize the network to make the image defogging model run quickly in the terminal with very low MACs, which can fully satisfy Real-time detection requirements of the terminal.
  • the process of designing the algorithm it can be further optimized for a variety of image scenes to improve the effect of the defogging algorithm.
  • This application proposes an image defogging method.
  • the terminal can use the image defogging model obtained by deep learning to perform defogging processing on the image to be processed.
  • the terminal can also use the image defogging model After defogging the sub-images after dividing the image to be processed, the image stitching model obtained by deep learning is used for stitching processing, so as to improve the processing speed while ensuring the processing accuracy.
  • the image defogging model and the image stitching model are obtained by the terminal's minimal network design on the CNN, so the above-mentioned image defogging method can be run in the terminal in real time.
  • the image defogging process based on the image defogging model and the image stitching model obtained by deep learning can greatly increase the processing speed while improving the processing accuracy, thereby achieving high-quality and high-efficiency images. Defogging treatment.
  • FIG. 9 is a schematic diagram of the implementation process of the image defogging method 5.
  • the post-fogging sub-images are stitched together, and before the defogging image corresponding to the image to be processed is obtained, that is, before step 105, the method for the terminal to perform image defogging processing may further include the following steps:
  • Step 1012 Divide the second image sample set to obtain second training data and second test data; where the second image sample set includes the original image and multiple decomposed images corresponding to the original image.
  • Step 1013 Construct a second network model based on the second lightweight convolutional neural network, and perform training processing on the second network model according to the second training data to obtain an initial splicing model.
  • Step 1014 Perform test processing on the initial stitching model according to the second test data to obtain an image stitching model.
  • the terminal may first construct the image stitching model before using the image stitching model to stitch the multiple defogging sub-images corresponding to the image to be processed.
  • the terminal may construct a second network model based on the second lightweight convolutional neural network, and improve the speed by modifying the processing.
  • the second lightweight convolutional neural network ShuffleNet-V2 network part of the convolutional layer can be removed, and part of the convolutional layer can be modified to Depthwise convolution + Pointwise convolution.
  • the designed image splicing model can obtain better image splicing effects and can run in real time in the terminal.
  • ShuffleNet-V1 uses a large number of 1x1 group convolutions, it violates the G2 principle (excessive group convolution will increase access costs).
  • ShuffleNet-V1 uses a bottleneck layer similar to ResNet, input and output channels The number is different, which violates the G1 principle (the same channel width can minimize the memory access cost); using too many groups at the same time also violates the G3 principle (network fragmentation will reduce the degree of parallelism); there are a large number of element levels in short-circuit connections The Add operation violates the G4 principle (element-level operations cannot be ignored).
  • ShuffleNet-V2 improves the defects of ShuffleNet-V1 through the introduction of channel split.
  • ShuffleNet-V2 no longer has a channel split for the downsampling module. Instead, each branch directly copies a copy of the input.
  • the overall network structure of ShuffleNet-V2 is very similar to ShuffleNet-V1.
  • An additional 1x1 convolutional layer is used before the GlobalPool layer to mix channel characteristics.
  • the number of channels in each block is scaled to obtain a network with different FLOPS, that is, the number of channels for each block is set, such as 0.5x, 1x , Which can adjust the complexity of the model.
  • the terminal may first use the ShuffleNet-V2 network in the convolutional neural network CNN to construct the second network model, and then obtain the image stitching model through training and testing of the second network model.
  • the terminal may first obtain the second image sample set, where the second image sample set includes the original image before decomposition, and multiple decompositions corresponding to the original image after the original image is divided. image. That is, in the second image sample set, the corresponding large-size image before decomposition and the small-size image after decomposition are stored.
  • the second image sample set includes original images and decomposed images corresponding to 100 different scenes, where one frame of original image may correspond to m frames of decomposed images, and m is an integer greater than 1.
  • the number of decomposed images corresponding to different original images in the second image sample set may be different.
  • the original image A corresponds to 6 decomposed images a1-a6, which means that Image A is divided into decomposed images a1-a6;
  • the original image B corresponds to 9 decomposed images b1-b9, that is, the decomposed images b1-b9 obtained after the original image B is divided.
  • the second image sample set may be collected and divided by the terminal, or may be sent by the terminal receiving other image collection devices, which is not specifically limited in this application.
  • the terminal may divide the second image sample set to obtain the second training data and the second test data.
  • the second training data is used to train the second network model
  • the second test data is used to test the initial splicing model.
  • the second training data and the second test data both include corresponding original images and decomposed images.
  • the images in the second training data and the second test data are different.
  • any set of original images and corresponding decomposed images in the second image sample set can only be divided into the second training data or the second test data, and cannot be used as the training image and the second test data in the second training data at the same time.
  • the test image in the test data can only be divided into the second training data or the second test data, and cannot be used as the training image and the second test data in the second training data at the same time.
  • the terminal after the terminal constructs the second network model based on the second lightweight convolutional neural network, and divides the second image sample set to obtain the second training data and the second test data, the terminal can First use the second training data to train the second network model to obtain the initial stitching model; then use the second test data to test the initial stitching model to obtain the final image stitching model.
  • the terminal when the terminal uses the second training data to train the second network model, it can mark the second training data, and after repeated iterations of the second training data, the recognition accuracy is finally obtained. High initial stitching model.
  • multiple loss functions may be used for joint processing.
  • the terminal can use Softmax loss and focal loss for joint training,
  • the terminal when the terminal performs test processing on the initial splicing model according to the second test data, and obtains the image splicing model, it may first perform the test processing on the initial splicing model according to the second test data to generate The second test result; then the initial stitching model is corrected according to the second test result to obtain an image stitching model.
  • the terminal may test the splicing processing effect of the initial splicing model based on the second test data. Specifically, the terminal may input the decomposed image in the second test data into the initial stitching model, output the stitched image, and then compare the stitched image with the corresponding original image to obtain the second test result. .
  • the terminal after obtaining the second test result, can modify the initial stitching model obtained by training based on the second test result, thereby improving the effect of stitching processing and obtaining optimized Model, that is, image stitching model.
  • the terminal in the embodiment of this application, can cut the network and optimize the network, so that the image stitching model can run quickly in the terminal with very low MACs, which can fully satisfy the terminal Real-time detection requirements.
  • the process of designing the algorithm it can be further optimized for a variety of image scenes, so as to improve the effect of the stitching algorithm.
  • This application proposes an image defogging method.
  • the terminal can use the image defogging model obtained by deep learning to perform defogging processing on the image to be processed.
  • the terminal can also use the image defogging model After defogging the sub-images after dividing the image to be processed, the image stitching model obtained by deep learning is used for stitching processing, so as to improve the processing speed while ensuring the processing accuracy.
  • the image defogging model and the image stitching model are obtained by the terminal's minimal network design on the CNN, so the above-mentioned image defogging method can be run in the terminal in real time.
  • the image defogging process based on the image defogging model and the image stitching model obtained by deep learning can greatly increase the processing speed while improving the processing accuracy, thereby achieving high-quality and high-efficiency images. Defogging treatment.
  • FIG. 10 is a sixth flowchart of image defogging processing.
  • the method for the terminal to perform defogging processing on the image to be processed may include the following steps:
  • Step 201 Determine whether the image to be processed is a fogged image, if yes, execute step 202, otherwise, execute step 207.
  • the terminal may first determine whether the image to be processed is a fogged image. Specifically, the terminal may analyze and process the image to be processed to obtain the analysis result, and then determine whether the image to be processed is a fogged image according to the analysis result.
  • Step 202 Determine the size parameter of the image to be processed.
  • the terminal needs to perform a defogging process for the image to be processed.
  • the terminal needs to determine the image size of the image to be processed, that is, determine the size of the image to be processed. Size parameters.
  • Step 203 Whether the size parameter of the image to be processed is greater than or equal to the preset size threshold, if yes, step 204 is executed, otherwise, step 208 is executed.
  • the terminal may determine the preprocessing strategy corresponding to the image to be processed according to the size parameter. Specifically, the terminal may compare the size parameter of the image to be processed with a preset size threshold, so as to further determine the corresponding Pretreatment strategy. Among them, the preprocessing strategy can be used to limit the image size. In other words, for images with different size parameters, the preprocessing strategies determined by the terminal are different.
  • the corresponding preprocessing strategy determined by the terminal may be to perform image division preprocessing on the image first, and then perform defogging processing; For smaller images, the corresponding pre-processing strategy determined by the terminal can be directly defogging.
  • Step 204 Divide the image to be processed, and obtain sub-images corresponding to the image to be processed.
  • the terminal can divide the image to be processed to obtain the sub-image corresponding to the image to be processed .
  • the terminal can first divide the to-be-processed image into multiple sub-images of equal size, and then perform defogging processing on the divided sub-images to speed up the processing. Fog treatment.
  • Step 205 Obtain a defogging sub-image corresponding to the sub-image based on the image defogging model.
  • the terminal performs defogging processing on the sub-image according to the image defogging model, and obtains the defogging sub-image corresponding to the sub-image.
  • the terminal may use a deep learning image defogging model to perform defogging processing on multiple sub-images respectively.
  • the terminal may modify part of the convolutional layer to Depthwise convolution + Pointwise convolution, which can improve the image processing efficiency while ensuring the processing effect .
  • Step 206 Use the image stitching model to stitch the defogging sub-images to obtain the defogging image.
  • the terminal may perform splicing processing on all the sub-images after defogging according to the image splicing model, so as to obtain a clear image corresponding to the image to be processed, that is, the defogging image.
  • the terminal may use a deep learning image splicing model to splice multiple defogging sub-images into corresponding defogging images.
  • the terminal can remove part of the convolutional layer, and perform network acceleration by modifying part of the convolutional layer to Depthwise convolution + Pointwise convolution. Therefore, a minimal network design for CNN can be carried out based on this, and the efficiency of image processing can be improved while ensuring the splicing effect.
  • Step 207 Jump over the fog processing flow.
  • the terminal does not need to perform a defogging process on the image to be processed.
  • Step 208 Obtain a defogging image corresponding to the image to be processed based on the image defogging model.
  • the terminal can directly perform defogging processing on the image to be processed according to the image defogging model to obtain The dehazing image corresponding to the image to be processed.
  • the method for the terminal to defog the image to be processed may further include the following steps:
  • Step 209 Perform deep learning based on the first image sample set to obtain an image defogging model.
  • the first image sample set includes a fogged image and a clear image corresponding to the fogged image.
  • the terminal divides the first image sample set into the first training data and the first test data, uses the first training data for model training to obtain the initial defogging model, and then uses the first test data to test the initial defogging model to obtain the image Dehazing model.
  • the method for the terminal to defog the image to be processed may further include the following steps:
  • Step 2010 Perform deep learning based on the second image sample set to obtain an image mosaic model.
  • the second image sample set includes the original image and multiple decomposed images corresponding to the original image.
  • the terminal divides the second image sample set into second training data and second test data, uses the second training data for model training to obtain the initial splicing model, and then uses the second test data to test the initial splicing model to obtain the image splicing model .
  • the image defogging method proposed in this application can not only be used for image beautification in mobile phone albums, but also can be used in the field of video surveillance to improve face recognition and other scenarios.
  • the image defoggeding method proposed in this application can not only be used for image beautification in mobile phone albums, but also can be used in the field of video surveillance to improve face recognition and other scenarios.
  • the fog when the fog is relatively large, the quality of the image captured by the surveillance equipment is very poor, and it is difficult to clearly see the target person in the video. After the image is defogged, the face of the target person can be better restored.
  • the image defogging method proposed in this application can also be used as an auxiliary means in face recognition to improve the effect of face recognition.
  • the use of the defogging algorithm can be significant To improve the recognition effect.
  • This application proposes an image defogging method.
  • the terminal can use the image defogging model obtained by deep learning to perform defogging processing on the image to be processed.
  • the terminal can also use the image defogging model After defogging the sub-images after dividing the image to be processed, the image stitching model obtained by deep learning is used for stitching processing, so as to improve the processing speed while ensuring the processing accuracy.
  • the image defogging model and the image stitching model are obtained by the terminal's minimal network design on the CNN, so the above-mentioned image defogging method can be run in the terminal in real time.
  • the image defogging process based on the image defogging model and the image stitching model obtained by deep learning can greatly increase the processing speed while improving the processing accuracy, thereby achieving high-quality and high-efficiency images. Defogging treatment.
  • FIG. 11 is a schematic diagram 1 of the terminal structure.
  • the terminal 10 proposed in this embodiment of the present application may include a determining part 11, a dividing part 12, The fog part 13, the splicing part 14, the analysis part 15, the judgment part 16, and the acquisition part 17.
  • the determining part 11 is configured to determine a size parameter of the image to be processed after determining that the image to be processed is a fogged image; and determine a preprocessing strategy corresponding to the image to be processed according to the size parameter; wherein, the preprocessing strategy Used to limit the image size;
  • the dividing part 12 is configured to divide the image to be processed if the preprocessing strategy is to divide the image to be processed to obtain a sub-image corresponding to the image to be processed;
  • the defogging part 13 is configured to perform defogging processing on the sub-image according to an image defogging model to obtain a defogging sub-image corresponding to the sub-image;
  • the splicing part 14 is configured to perform splicing processing on the defogging sub-images according to an image splicing model to obtain a defogging image corresponding to the image to be processed.
  • the analysis part 15 is configured to perform analysis processing on the image to be processed before determining the size parameter of the image to be processed to obtain an analysis result
  • the judging part 16 is configured to judge whether the image to be processed is a fogged image according to the analysis result.
  • the determining part 11 is specifically configured to determine that the preprocessing strategy is to divide the image to be processed if the size parameter is greater than or equal to a preset size threshold; If the size parameter is less than the preset size threshold, it is determined that the preprocessing strategy is not to divide the image to be processed.
  • the acquiring part 17 is configured to perform defogging processing on the sub-image according to the image defogging model, and before obtaining the defogging sub-image corresponding to the sub-image,
  • the first image sample set is divided to obtain first training data and first test data; wherein, the first image sample set includes a fogged image and a clear image corresponding to the fogged image; and based on the first lightweight
  • the convolutional neural network constructs a first network model, and performs training processing on the first network model according to the first training data to obtain an initial defogging model; and performing the initial defogging model on the basis of the first test data Perform test processing to obtain the image defogging model.
  • the acquiring part 17 is specifically configured to perform a test process on the initial dehazing model according to the first test data to generate a first test result; and according to the first test data The test result corrects the image defogging model to obtain the image defogging model.
  • the acquiring part 17 is further configured to perform stitching processing on the defogging sub-images according to the image stitching model, and before obtaining the defogging image corresponding to the image to be processed, Divide the second image sample set to obtain second training data and second test data; wherein, the second image sample set includes an original image and a plurality of decomposed images corresponding to the original image; and based on the second lightweight
  • the convolutional neural network constructs a second network model, and performs training processing on the second network model according to the second training data to obtain an initial stitching model; and testing the initial stitching model according to the second test data Processing to obtain the image stitching model.
  • the acquiring part 17 is specifically configured to perform test processing on the initial splicing model according to the second test data to generate a second test result; and according to the second test As a result, the initial stitching model is corrected, and the image stitching model is obtained.
  • the dehazing part 13 is further configured to determine the preprocessing strategy corresponding to the image to be processed according to the size parameter, if the preprocessing strategy is not to divide the For the image to be processed, the image to be processed is defogged directly according to the image defogging model to obtain the defogging image.
  • Fig. 12 is a second schematic diagram of the composition structure of the terminal.
  • the terminal 10 proposed in the embodiment of the present application may further include a processor 18 and a memory 19 storing executable instructions of the processor 18. Further, the terminal 10 may also It includes a communication interface 110 and a bus 111 for connecting the processor 18, the memory 19, and the communication interface 110.
  • the aforementioned processor 18 may be an application specific integrated circuit (ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD). ), Programmable Logic Device (ProgRAMmable Logic Device, PLD), Field Programmable Gate Array (Field ProgRAMmable Gate Array, FPGA), Central Processing Unit (CPU), Controller, Microcontroller, Microprocessor At least one of. It is understandable that, for different devices, the electronic devices used to implement the above-mentioned processor functions may also be other, which is not specifically limited in the embodiment of the present application.
  • the terminal 10 may also include a memory 19, which may be connected to the processor 18.
  • the memory 19 is used to store executable program code, the program code includes computer operation instructions, the memory 19 may include a high-speed RAM memory, or may also include Non-volatile memory, for example, at least two disk memories.
  • the bus 111 is used to connect the communication interface 110, the processor 18 and the memory 19, and to communicate with each other among these devices.
  • the memory 19 is used to store instructions and data.
  • the above-mentioned processor 18 is configured to determine the size parameter of the image to be processed after determining that the image to be processed is a fogged image; determine the preprocessing corresponding to the image to be processed according to the size parameter Strategy; wherein, the preprocessing strategy is used to limit the image size; if the preprocessing strategy is to divide the image to be processed, divide the image to be processed to obtain the sub-image corresponding to the image to be processed; Perform defogging processing on the sub-images according to the image defogging model to obtain the defogging sub-images corresponding to the sub-images; performing stitching processing on the defogging sub-images according to the image stitching model to obtain the to-be-processed image The corresponding image after defogging.
  • the above-mentioned memory 19 may be a volatile memory (volatile memory), such as a random-access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory.
  • volatile memory such as a random-access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory.
  • a storage device Read-Only Memory, ROM), flash memory (flash memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); or a combination of the above types of memory, and Instructions and data are provided to the processor 18.
  • the functional modules in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be realized in the form of hardware or software function module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this embodiment is essentially or correct
  • the part that the prior art contributes or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes a number of instructions to enable a computer device (which can be a personal computer).
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.
  • An embodiment of the present application proposes a terminal that can use the image defogging model obtained by deep learning to perform defogging processing on the image to be processed.
  • the terminal can also use the image defogging model After defogging the sub-images after dividing the image to be processed, the image stitching model obtained by deep learning is used for stitching processing, so as to improve the processing speed while ensuring the processing accuracy.
  • the image defogging model and the image stitching model are obtained by the terminal's minimal network design on the CNN, so the above-mentioned image defogging method can be run in the terminal in real time.
  • the image defogging process based on the image defogging model and the image stitching model obtained by deep learning can greatly increase the processing speed while improving the processing accuracy, thereby achieving high-quality and high-efficiency images. Defogging treatment.
  • the embodiment of the present application provides a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, the image defogging method as described above is realized.
  • the program instructions corresponding to an image defogging method in this embodiment can be stored on storage media such as optical disks, hard disks, USB flash drives, etc., when the program instructions corresponding to an image defogging method in the storage medium When being read or executed by an electronic device, it includes the following steps:
  • the preprocessing strategy is to divide the image to be processed, divide the image to be processed to obtain a sub-image corresponding to the image to be processed;
  • this application can be provided as a method, a terminal, or a computer program product. Therefore, this application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program codes.
  • These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are used to generate A device for realizing the functions specified in one process or multiple processes in the schematic flow chart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device realizes the functions specified in one process or multiple processes in the realization process schematic diagram and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing functions specified in one or more processes in the schematic diagram and/or one block or more in the block diagram.
  • the embodiments of the application provide an image defogging method, terminal, and computer storage medium. After determining that the image to be processed is a fogged image, the terminal determines the size parameter of the image to be processed; and determines the preprocessing strategy corresponding to the image to be processed according to the size parameter.
  • the preprocessing strategy is used to limit the image size; if the preprocessing strategy is to divide the image to be processed, the image to be processed is divided to obtain the sub-image corresponding to the image to be processed; the sub-image is defogged according to the image defogging model Process to obtain the defogging sub-image corresponding to the sub-image; perform stitching processing on the defogging sub-image according to the image stitching model to obtain the defogging image corresponding to the image to be processed. It can be seen that, in the embodiment of the present application, the terminal can use the image defogging model obtained by deep learning to defog the image to be processed.
  • the terminal can also use the image defogging.
  • the image stitching model obtained by deep learning is used for stitching processing to increase the processing speed while ensuring the processing accuracy.
  • the image defogging model and the image stitching model are obtained by the terminal's minimal network design on the CNN, so the above-mentioned image defogging method can be run in the terminal in real time.
  • the image defogging process based on the image defogging model and the image stitching model obtained by deep learning can greatly increase the processing speed while improving the processing accuracy, thereby achieving high-quality and high-efficiency images. Defogging treatment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé de désembuage, un terminal et un support de stockage informatique. Le procédé de désembuage d'image consiste à : après avoir déterminé qu'une image à traiter est une image présentant du brouillard, déterminer un paramètre de taille de l'image à traiter (101) ; déterminer, en fonction du paramètre de taille, une politique de prétraitement correspondant à l'image à traiter, la politique de prétraitement étant utilisée pour limiter la taille d'image (102) ; si la politique de prétraitement consiste à diviser l'image à traiter, diviser l'image à traiter pour obtenir des sous-images correspondant à l'image à traiter (103) ; réaliser un traitement de désembuage sur les sous-images selon un modèle de désembuage d'image pour obtenir des sous-images désembuées correspondant aux sous-images (104) ; et réaliser un traitement d'assemblage sur les sous-images désembuées selon un modèle d'assemblage d'images pour obtenir une image désembuée correspondant à l'image à traiter (105).
PCT/CN2021/085694 2020-05-29 2021-04-06 Procédé de désembuage d'image, terminal et support de stockage informatique WO2021238420A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010472428.1 2020-05-29
CN202010472428.1A CN111626960A (zh) 2020-05-29 2020-05-29 图像去雾方法、终端及计算机存储介质

Publications (1)

Publication Number Publication Date
WO2021238420A1 true WO2021238420A1 (fr) 2021-12-02

Family

ID=72260094

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/085694 WO2021238420A1 (fr) 2020-05-29 2021-04-06 Procédé de désembuage d'image, terminal et support de stockage informatique

Country Status (2)

Country Link
CN (1) CN111626960A (fr)
WO (1) WO2021238420A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114529464A (zh) * 2022-01-14 2022-05-24 电子科技大学 一种基于深度学习的水下图像恢复方法
CN115511735A (zh) * 2022-09-20 2022-12-23 北京拙河科技有限公司 一种雪场灰度画面优化方法及装置
CN116935289A (zh) * 2023-09-13 2023-10-24 长江信达软件技术（武汉）有限责任公司 一种基于视频监控的明渠漫堤检测方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111626960A (zh) * 2020-05-29 2020-09-04 Oppo广东移动通信有限公司 图像去雾方法、终端及计算机存储介质
CN112183366A (zh) * 2020-09-29 2021-01-05 重庆大学 一种高压电力线鸟巢检测方法、系统及机器可读介质
CN115937144B (zh) * 2022-12-08 2023-08-25 郑州大学 一种胸腔镜术中图像处理方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160196637A1 (en) * 2015-01-06 2016-07-07 The Regents Of The University Of California Raw sensor image and video de-hazing and atmospheric light analysis methods and systems
CN109934781A (zh) * 2019-02-27 2019-06-25 合刃科技（深圳）有限公司 图像处理方法、装置、终端设备及计算机可读存储介质
CN110322419A (zh) * 2019-07-11 2019-10-11 广东工业大学 一种遥感图像去雾方法及系统
CN110930320A (zh) * 2019-11-06 2020-03-27 南京邮电大学 一种基于轻量化卷积神经网络的图像去雾方法
CN111626960A (zh) * 2020-05-29 2020-09-04 Oppo广东移动通信有限公司 图像去雾方法、终端及计算机存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110619618B (zh) * 2018-06-04 2023-04-07 杭州海康威视数字技术股份有限公司 一种表面缺陷检测方法、装置及电子设备
CN108898562B (zh) * 2018-06-22 2022-04-12 大连海事大学 一种基于深度学习的移动设备图像去雾方法
CN109919887B (zh) * 2019-02-25 2021-06-18 中国人民解放军陆军工程大学 一种基于深度学习的无监督图像融合方法
CN110544216A (zh) * 2019-08-29 2019-12-06 北京理工大学 基于深度学习的视频去雾系统
CN111047513B (zh) * 2019-11-27 2024-01-23 中国人民解放军国防科技大学 一种用于柱面全景拼接的鲁棒性图像对齐方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160196637A1 (en) * 2015-01-06 2016-07-07 The Regents Of The University Of California Raw sensor image and video de-hazing and atmospheric light analysis methods and systems
CN109934781A (zh) * 2019-02-27 2019-06-25 合刃科技（深圳）有限公司 图像处理方法、装置、终端设备及计算机可读存储介质
CN110322419A (zh) * 2019-07-11 2019-10-11 广东工业大学 一种遥感图像去雾方法及系统
CN110930320A (zh) * 2019-11-06 2020-03-27 南京邮电大学 一种基于轻量化卷积神经网络的图像去雾方法
CN111626960A (zh) * 2020-05-29 2020-09-04 Oppo广东移动通信有限公司 图像去雾方法、终端及计算机存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114529464A (zh) * 2022-01-14 2022-05-24 电子科技大学 一种基于深度学习的水下图像恢复方法
CN115511735A (zh) * 2022-09-20 2022-12-23 北京拙河科技有限公司 一种雪场灰度画面优化方法及装置
CN116935289A (zh) * 2023-09-13 2023-10-24 长江信达软件技术（武汉）有限责任公司 一种基于视频监控的明渠漫堤检测方法
CN116935289B (zh) * 2023-09-13 2023-12-19 长江信达软件技术（武汉）有限责任公司 一种基于视频监控的明渠漫堤检测方法

Also Published As

Publication number Publication date
CN111626960A (zh) 2020-09-04

Similar Documents

Publication Publication Date Title
WO2021238420A1 (fr) Procédé de désembuage d'image, terminal et support de stockage informatique
CN109859190B (zh) 一种基于深度学习的目标区域检测方法
CN113052210B (zh) 一种基于卷积神经网络的快速低光照目标检测方法
WO2021164234A1 (fr) Procédé de traitement d'image et dispositif de traitement d'image
CN110008806B (zh) 信息处理装置、学习处理方法、学习装置及物体识别装置
CN112132156A (zh) 多深度特征融合的图像显著性目标检测方法及系统
CN109753878B (zh) 一种恶劣天气下的成像识别方法及系统
CN111046821B (zh) 一种视频行为识别方法、系统及电子设备
CN109472193A (zh) 人脸检测方法及装置
Alajarmeh et al. Real-time framework for image dehazing based on linear transmission and constant-time airlight estimation
CN111461211B (zh) 一种用于轻量级目标检测的特征提取方法及相应检测方法
CN116757988A (zh) 基于语义丰富和分割任务的红外与可见光图像融合方法
CN114565539B (zh) 一种基于线上知识蒸馏的图像去雾方法
CN114663309A (zh) 一种基于多尺度信息选择注意机制的图像去雾方法及系统
CN116757986A (zh) 一种红外与可见光图像融合方法及装置
CN114241344B (zh) 一种基于深度学习的植物叶片病虫害严重程度评估方法
US11783454B2 (en) Saliency map generation method and image processing system using the same
CN111582057B (zh) 一种基于局部感受野的人脸验证方法
CN110738624B (zh) 一种区域自适应的图像去雾系统及方法
CN115937121A (zh) 基于多维度特征融合的无参考图像质量评价方法及系统
CN115240163A (zh) 一种基于一阶段检测网络的交通标志检测方法及系统
CN113256528B (zh) 基于多尺度级联深度残差网络的低照度视频增强方法
CN112651926A (zh) 一种基于递归注意力机制的裂纹检测的方法及装置
CN112487911A (zh) 智能监控环境下基于改进yolov3的实时行人检测方法及装置
Chen et al. GADO-Net: an improved AOD-Net single image dehazing algorithm

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21813152

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21813152

Country of ref document: EP

Kind code of ref document: A1