CN117853371B - Multi-branch frequency domain enhanced real image defogging method, system and terminal - Google Patents

Abstract

The present invention provides a real image defogging method, system and terminal with multi-branch frequency domain enhancement, the method comprising: inputting a sample enhancement set into an image defogging model to perform feature extraction of a multi-branch network to obtain sample image features, performing semantic segmentation, residual attention processing and feature fusion on the sample image features to obtain fusion features; performing mixed skip connection on the residual output image to obtain a residual image, performing image restoration and frequency domain enhancement on the residual image and the fusion features to obtain a residual enhanced image and a fusion enhanced image, fusing the residual enhanced image and the fusion enhanced image to obtain a defogging image; determining a model loss according to the defogging image, and updating parameters of the image defogging model according to the model loss; inputting the image to be defogged into the image defogging model to perform defogging processing to obtain a defogging output image. The embodiment of the present invention can effectively use frequency domain information to refine the defogging effect in the defogging image, thereby improving the accuracy of image defogging.

Description

Real image defogging method, system and terminal with multi-branch frequency domain enhancement

Technical Field

The present invention relates to the field of image processing technology, and in particular to a real image defogging method, system and terminal with multi-branch frequency domain enhancement.

Background technique

Image dehazing is a low-level vision task, usually a preprocessing step to improve the performance of high-level vision tasks, such as crowd counting, object detection, or image segmentation. Generally speaking, in high-level vision tasks, clear haze-free images are required. Therefore, the image dehazing problem has attracted more and more attention from academia and industry.

In the existing image dehazing process, the mapping relationship between haze-free images and haze-containing images is generally learned based only on the image feature information of the sample data set to generate haze-free images. However, the method based only on image feature information results in poor quality of the generated haze-free images, which reduces the accuracy of image dehazing.

Summary of the invention

The purpose of the embodiments of the present invention is to provide a real image defogging method, system and terminal with multi-branch frequency domain enhancement, aiming to solve the problem of low accuracy of existing image defogging.

The embodiment of the present invention is implemented as follows: a real image defogging method with multi-branch frequency domain enhancement, the method comprising:

Acquire a sample data set, and perform data enhancement on the sample data set to obtain a sample enhancement set;

Inputting the sample enhancement set into the image dehazing model to perform feature extraction of a multi-branch network to obtain sample image features, and performing semantic segmentation on the sample image features of each branch network to obtain semantic segmentation features;

Perform residual attention processing on the semantic segmentation features of each branch network respectively to obtain mask feature maps and residual output images, and fuse the mask feature maps of each branch network to obtain fused features;

Performing a mixed skip connection on the residual output images of each branch network to obtain a residual image, and performing image restoration on the residual image and the fusion feature respectively to obtain a residual restoration image and a fusion restoration image;

Performing frequency domain enhancement on the residual restored image and the fused restored image to obtain a residual enhanced image and a fused enhanced image, and fusing the residual enhanced image and the fused enhanced image to obtain a defogging image;

Determining a model loss according to the defogging image and a real image of the sample data set, and updating parameters of the image defogging model according to the model loss until the image defogging model converges;

The image to be defogged is input to the converged image defogging model for defogging to obtain a defogged output image.

Another object of an embodiment of the present invention is to provide a real image defogging system with multi-branch frequency domain enhancement, the system comprising:

A data enhancement module is used to obtain a sample data set and perform data enhancement on the sample data set to obtain a sample enhancement set;

A feature extraction module is used to input the sample enhancement set into the image defogging model to perform feature extraction of a multi-branch network to obtain sample image features, and to perform semantic segmentation on the sample image features of each branch network to obtain semantic segmentation features;

Perform residual attention processing on the semantic segmentation features of each branch network respectively to obtain mask feature maps and residual output images, and fuse the mask feature maps of each branch network to obtain fused features;

Performing a mixed skip connection on the residual output images of each branch network to obtain a residual image, and performing image restoration on the residual image and the fusion feature respectively to obtain a residual restoration image and a fusion restoration image;

A frequency domain enhancement module, used for performing frequency domain enhancement on the residual restored image and the fused restored image to obtain a residual enhanced image and a fused enhanced image, and fusing the residual enhanced image and the fused enhanced image to obtain a defogging image;

A parameter updating module, used to determine a model loss according to the defogging image and a real image of the sample data set, and update parameters of the image defogging model according to the model loss until the image defogging model converges;

The defogging processing module is used to perform defogging processing on the image to be defogged after inputting the converged image defogging model to obtain a defogged output image.

Another object of an embodiment of the present invention is to provide a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

Another object of an embodiment of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

The embodiments of the present invention effectively improve the diversity of the sample data set by performing data enhancement on the sample data set, extract the sample image features in the sample data set in a multi-channel manner by inputting the sample enhancement set into the image defogging model for feature extraction of the multi-branch network, and effectively utilize the frequency domain information to refine the defogging effect in the residual restored image and the fused restored image by performing frequency domain enhancement on the residual restored image and the fused restored image, thereby improving the quality of the defogging image and the accuracy of the image defogging.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a real image defogging method with multi-branch frequency domain enhancement provided by a first embodiment of the present invention;

FIG2 is a schematic structural diagram of a real image defogging system with multi-branch frequency domain enhancement provided by a second embodiment of the present invention;

FIG3 is a schematic diagram of the structure of a terminal device provided in a third embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

Embodiment 1

Please refer to FIG1, which is a flowchart of a real image defogging method with multi-branch frequency domain enhancement provided by a first embodiment of the present invention. The real image defogging method with multi-branch frequency domain enhancement can be applied to any system. The real image defogging method with multi-branch frequency domain enhancement includes the following steps:

Step S10, obtaining a sample data set, and performing data enhancement on the sample data set to obtain a sample enhanced set;

Among them, by performing data enhancement on the sample data set, the number of data sets is effectively increased, so that the trained model can flexibly adapt to real-world images and have better model performance.

In this step, the formula used to perform data enhancement on the sample data set includes:

in, is the sample enhancement set,/> For the sample data set, A is a real fog-free image, A is the global atmospheric light, is the medium transmittance diagram, /> is the scene depth, /> The global atmospheric light and haze density are numerically perturbed based on the depth estimator in this step. Preferably, the global atmospheric light can be adjusted from (0.7, 1.0) to (0.5, 1.8), and the haze density can be adjusted from (0.6, 1.8) to (0.8, 2.8). By amplifying the global atmospheric light and haze density, more challenging real-world situations can be effectively covered. For example, in severe haze conditions, the supervision signal in the model input is significantly reduced due to the severe loss of information caused by dense fog. Therefore, by amplifying the global atmospheric light and haze density, it is helpful to enhance the generalization ability of the model.

The haze effect in the frequency domain is usually considered to be a spatial static signal composed of low-frequency components. This embodiment also changes the haze pattern by exchanging the low-frequency spectra of the two images without affecting the high-level semantic perception, thereby achieving image style transfer, thereby diversifying the haze pattern, but the understanding of the scene is not changed. In addition, the unique positional dependency of the haze image on the scene depth is decomposed, thereby generating more non-uniform haze patterns, further improving the diversity of the dataset.

in, For foggy images/> and foggy images/> The generated augmented image, /> and/> For a real fog-free image/> and real fog-free image/> Synthesized foggy image, /> and/> are the ath and bth real fog-free images in the sample dataset,/> is the amplitude of the Fourier transform, /> is the phase component of Fourier transform, /> is the image mask, /> A composite operation of functions.

Where H and W are the height and width of the image, and/> is the height and width of the image, /> For the ratio , used to control the migration range of low-frequency components, /> Represents the indicator function.

The range of low frequency components (i.e. haze mode) is / > The scope is replaced by / > is disturbed to maintain/> In order to reduce the semantic information in /> To avoid the negative effects of hard thresholding, a Gaussian filter can be applied to smooth the image.

In this embodiment, by performing data enhancement on the sample data set, the distribution difference between the real data and the synthetic data can be effectively reduced, thereby improving the defogging performance of the model in real scenes.

Step S20, inputting the sample enhancement set into the image defogging model to perform feature extraction of a multi-branch network to obtain sample image features, and performing semantic segmentation on the sample image features of each branch network to obtain semantic segmentation features;

Among them, multiple branch networks are set up in the image dehazing model. Each branch network extracts sample image features (shallow features) through two 3x3 convolutions and a ReLU function layer. All branch networks have a 3-layer U-Net network architecture to perform semantic segmentation on sample image features, with 3 blocks at each scale. Different attention blocks can be used to replace the naive convolution in each branch.

Step S30, performing residual attention processing on the semantic segmentation features of each branch network respectively to obtain a mask feature map and a residual output image, and fusing the mask feature maps of each branch network to obtain a fused feature;

Among them, each branch network is equipped with a residual attention module (RAM), which generates two outputs, namely, a mask feature map and a residual output image. The mask feature map is generated by the mask obtained by passing the residual output image through the sigmoid function, and the residual output image is the output image degraded by 3×3 convolution in the branch network.

Step S40, performing a mixed skip connection on the residual output images of each branch network to obtain a residual image, and performing image restoration on the residual image and the fusion feature respectively to obtain a residual restoration image and a fusion restoration image;

Among them, the residual output images of each branch network are mixed skip connection (MSC) to obtain the residual image, and the residual image and the fusion feature are respectively convolved by 3x3 to restore the image to obtain the residual restoration image and the fusion restoration image.

Step S50, performing frequency domain enhancement on the residual restored image and the fused restored image to obtain a residual enhanced image and a fused enhanced image, and fusing the residual enhanced image and the fused enhanced image to obtain a defogging image;

Optionally, frequency domain enhancement is performed on the residual restored image and the fused restored image, including:

Acquire spatial domain features of the residual restored image and the fused restored image, and determine frequency domain features according to the spatial domain features;

Determine an amplitude spectrum and a phase spectrum according to the frequency domain characteristics, and repair the amplitude spectrum to obtain an amplitude repair spectrum;

Determine a residual amplitude according to the amplitude repair spectrum and the amplitude spectrum, and determine an attention map according to the residual amplitude;

Performing a phase change on the phase spectrum according to the attention map to obtain a phase change spectrum, and determining a frequency domain enhanced real part and a frequency domain enhanced imaginary part according to the phase change spectrum and the amplitude repair spectrum;

Determine a frequency domain enhancement feature according to the frequency domain enhancement real part and the frequency domain enhancement imaginary part, and determine a spatial domain enhancement feature according to the frequency domain enhancement feature;

Performing feature enhancement on the residual restored image and the fused restored image according to the spatial domain enhancement feature to obtain the residual enhanced image and the fused enhanced image;

The formula used to determine the frequency domain features according to the spatial domain features includes:

Wherein, x is the spatial domain feature of the residual restored image and the fused restored image, are the coordinates of the residual restored image and the fused restored image in the frequency domain, /> is the frequency component of the image in the horizontal direction, /> is the frequency component of the image in the vertical direction, /> and/> is the height and width of the image, /> and/> are the vertical and horizontal coordinates of the image, /> The frequency domain features of the residual restored image and the fused restored image;

The formulas used to determine the amplitude spectrum and phase spectrum according to the frequency domain characteristics include:

in, is the real part of the frequency domain feature, /> is the imaginary part of the frequency domain feature;

in, is the amplitude spectrum, /> is the phase spectrum;

The amplitude spectrum is repaired to obtain an amplitude repair spectrum. The formula used to determine the residual amplitude according to the amplitude repair spectrum and the amplitude spectrum includes:

Among them, due to various situations such as sampling and signal damage, the amplitude may be severely distorted, so 1x1 convolution can be used to repair the amplitude. is the amplitude repair spectrum, /> is the residual amplitude between the amplitude repair spectrum and the amplitude spectrum,/> is the convolution operator,/> is a filter with a convolution kernel size of 1x1 pixels.

An attention map is determined according to the residual amplitude, and a formula used to change the phase spectrum according to the attention map includes:

in, is the attention map, GAP is the global average pooling, /> is the phase change spectrum, /> It is an element-wise product operation;

The formula used to determine the frequency domain enhancement real part and the frequency domain enhancement imaginary part according to the phase change spectrum and the amplitude repair spectrum includes:

in, is the frequency domain enhanced real part,/> Enhance the imaginary part in the frequency domain;

The formula used to determine the frequency domain enhancement feature according to the frequency domain enhancement real part and the frequency domain enhancement imaginary part includes:

in, It is the frequency domain enhancement feature of the residual enhanced image and the fusion enhanced image. In this step, the frequency domain enhancement feature is transformed into the spatial domain enhancement feature by using inverse Fourier transform.

Step S60, determining a model loss according to the defogging image and a real image of the sample data set, and updating parameters of the image defogging model according to the model loss until the image defogging model converges;

Optionally, the formula used to determine the model loss based on the dehazed image and the real image of the sample data set includes:

Wherein, X is the defogging image, Y is the real image, is the model loss,/> is the loss composed of peak signal-to-noise ratio and structural similarity index,/> is the edge loss,/> 、/> and/> is a preset constant, /> is the Laplace operator, PSNR is the peak signal-to-noise ratio, SSIM is the structural similarity, MAX ² _Y is the maximum pixel value that can be obtained in the image, MSE is the mean square error between the dehazed image and the corresponding real image, /> 、/> 、/> Respectively represent the comparison of brightness, contrast and saturation of the real image and the dehazed image,/> 、/> and/> is a hyperparameter. Preferably, /> Can be set to 0.005,/> Can be set to 0.05,/> Can be set to 10 ^-3 ,/> It is usually set to 1. S SIM measures the similarity of images from three aspects: brightness, contrast, and structure.

Step S70, inputting the image to be defogged into the converged image defogging model to perform defogging processing to obtain a defogged output image.

In this embodiment, by performing data enhancement on the sample data set, the diversity of the data in the sample data set is effectively improved, and by inputting the sample enhancement set into the image dehazing model for feature extraction of the multi-branch network, the sample image features in the sample data set are extracted in a multi-channel manner, and by performing frequency domain enhancement on the residual restored image and the fused restored image, the frequency domain information can be effectively utilized to refine the dehazing effect in the residual restored image and the fused restored image, thereby improving the quality of the dehazed image and the accuracy of image dehazing.

Embodiment 2

Please refer to FIG. 2 , which is a schematic diagram of the structure of a real image defogging system 100 with multi-branch frequency domain enhancement provided by a second embodiment of the present invention, including:

The data enhancement module 10 is used to obtain a sample data set and perform data enhancement on the sample data set to obtain a sample enhanced set.

Optionally, the formula used to perform data enhancement on the sample data set includes:

in, is the sample enhancement set,/> For the sample data set, A is a real fog-free image, A is the global atmospheric light, is the medium transmittance diagram, /> is the scene depth, /> is the fog density;

in, For/> and/> The generated augmented image, /> and/> are the ath and bth real fog-free images in the sample dataset,/> and/> For the image /> and/> Synthesized foggy image, /> is the amplitude of the Fourier transform, /> is the phase component of Fourier transform, /> is the image mask, /> A composite operation of functions.

The feature extraction module 11 is used to input the sample enhancement set into the image defogging model to perform feature extraction of the multi-branch network to obtain sample image features, and perform semantic segmentation on the sample image features of each branch network to obtain semantic segmentation features;

Perform residual attention processing on the semantic segmentation features of each branch network respectively to obtain mask feature maps and residual output images, and fuse the mask feature maps of each branch network to obtain fused features;

The residual output images of each branch network are subjected to mixed skip connection to obtain a residual image, and the residual image and the fusion feature are respectively subjected to image restoration to obtain a residual restoration image and a fusion restoration image.

The frequency domain enhancement module 12 is used to perform frequency domain enhancement on the residual restored image and the fused restored image to obtain a residual enhanced image and a fused enhanced image, and fuse the residual enhanced image and the fused enhanced image to obtain a defogging image.

Optionally, the frequency domain enhancement module 12 is further used to: obtain spatial domain features of the residual restored image and the fused restored image, and determine frequency domain features according to the spatial domain features;

Determine an amplitude spectrum and a phase spectrum according to the frequency domain characteristics, and repair the amplitude spectrum to obtain an amplitude repair spectrum;

Determine a residual amplitude according to the amplitude repair spectrum and the amplitude spectrum, and determine an attention map according to the residual amplitude;

Performing a phase change on the phase spectrum according to the attention map to obtain a phase change spectrum, and determining a frequency domain enhanced real part and a frequency domain enhanced imaginary part according to the phase change spectrum and the amplitude repair spectrum;

Determine a frequency domain enhancement feature according to the frequency domain enhancement real part and the frequency domain enhancement imaginary part, and determine a spatial domain enhancement feature according to the frequency domain enhancement feature;

Feature enhancement is performed on the residual restored image and the fused restored image according to the spatial domain enhancement feature to obtain the residual enhanced image and the fused enhanced image.

Furthermore, the formula used to determine the frequency domain feature according to the spatial domain feature includes:

Wherein, x is the spatial domain feature of the residual restored image and the fused restored image, are the coordinates of the residual restored image and the fused restored image in the frequency domain, /> is the frequency component of the image in the horizontal direction, /> is the frequency component of the image in the vertical direction, /> and/> is the height and width of the image, /> and/> are the vertical and horizontal coordinates of the image, /> The frequency domain features of the residual restored image and the fused restored image;

The formulas used to determine the amplitude spectrum and phase spectrum according to the frequency domain characteristics include:

in, is the real part of the frequency domain feature, /> is the imaginary part of the frequency domain feature;

in, is the amplitude spectrum, /> is the phase spectrum;

The amplitude spectrum is repaired to obtain an amplitude repair spectrum. The formula used to determine the residual amplitude according to the amplitude repair spectrum and the amplitude spectrum includes:

in, is the amplitude repair spectrum, /> is the residual amplitude between the amplitude repair spectrum and the amplitude spectrum,/> is the convolution operator,/> is the filter;

An attention map is determined according to the residual amplitude, and a formula used to change the phase spectrum according to the attention map includes:

in, is the attention map, GAP is the global average pooling, /> is the phase change spectrum, /> It is an element-wise product operation;

The formula used to determine the frequency domain enhancement real part and the frequency domain enhancement imaginary part according to the phase change spectrum and the amplitude repair spectrum includes:

in, is the frequency domain enhanced real part,/> Enhance the imaginary part in the frequency domain;

The formula used to determine the frequency domain enhancement feature according to the frequency domain enhancement real part and the frequency domain enhancement imaginary part includes:

in, It is the frequency domain enhancement feature of the residual enhanced image and the fusion enhanced image.

The parameter updating module 13 is used to determine the model loss according to the defogging image and the real image of the sample data set, and update the parameters of the image defogging model according to the model loss until the image defogging model converges.

Optionally, the formula used to determine the model loss based on the dehazed image and the real image of the sample data set includes:

Wherein, X is the defogging image, Y is the real image, is the model loss,/> is the loss composed of peak signal-to-noise ratio and structural similarity index,/> is the edge loss,/> 、/> and/> is a preset constant, /> is the Laplace operator, PSNR is the peak signal-to-noise ratio, SSIM is the structural similarity, MAX ² _Y is the maximum pixel value that can be obtained in the image, MSE is the mean square error between the dehazed image and the corresponding real image, /> 、/> 、/> Respectively represent the comparison of brightness, contrast and saturation of the real image and the dehazed image,/> 、/> and/> is a hyperparameter.

The defogging processing module 14 is used to input the image to be defogged into the converged image defogging model for defogging processing to obtain a defogged output image.

In this embodiment, by performing data enhancement on the sample data set, the diversity of the data in the sample data set is effectively improved. By inputting the sample enhancement set into the image dehazing model for feature extraction of the multi-branch network, the sample image features in the sample data set are extracted in a multi-channel manner. By performing frequency domain enhancement on the residual restored image and the fused restored image, the frequency domain information can be effectively used to refine the dehazing effect in the residual restored image and the fused restored image, thereby improving the quality of the dehazed image and the accuracy of image dehazing.

Embodiment 3

FIG3 is a block diagram of a terminal device 2 provided in the third embodiment of the present application. As shown in FIG3, the terminal device 2 of this embodiment includes: a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program of a real image defogging method with multi-branch frequency domain enhancement. When the processor 20 executes the computer program 22, the steps in each embodiment of the real image defogging method with multi-branch frequency domain enhancement are implemented.

Exemplarily, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, a processor 20 and a memory 21.

The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

The memory 21 may be an internal storage unit of the terminal device 2, such as a hard disk or memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Further, the memory 21 may also include both an internal storage unit and an external storage device of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 may also be used to temporarily store data that has been output or is to be output.

In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium can be non-volatile or volatile. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in computer-readable storage media can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, based on legislation and patent practices, computer-readable storage media do not include electrical carrier signals and telecommunication signals.

The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims (6)

1. A real image defogging method with multi-branch frequency domain enhancement, characterized in that the method comprises: Acquire a sample data set, and perform data enhancement on the sample data set to obtain a sample enhancement set; Inputting the sample enhancement set into the image dehazing model to perform feature extraction of a multi-branch network to obtain sample image features, and performing semantic segmentation on the sample image features of each branch network to obtain semantic segmentation features; Perform residual attention processing on the semantic segmentation features of each branch network respectively to obtain mask feature maps and residual output images, and fuse the mask feature maps of each branch network to obtain fused features; Performing a mixed skip connection on the residual output images of each branch network to obtain a residual image, and performing image restoration on the residual image and the fusion feature respectively to obtain a residual restoration image and a fusion restoration image; Performing frequency domain enhancement on the residual restored image and the fused restored image to obtain a residual enhanced image and a fused enhanced image, and fusing the residual enhanced image and the fused enhanced image to obtain a defogging image; Determining a model loss according to the dehazed image and a real image of the sample data set, and updating parameters of the image dehazed model according to the model loss until the image dehazed model converges; The image to be defogged is inputted into the image defogging model after convergence, and defogging is performed to obtain a defogged output image; Performing frequency domain enhancement on the residual restored image and the fused restored image, comprising: Acquire spatial domain features of the residual restored image and the fused restored image, and determine frequency domain features according to the spatial domain features; Determine an amplitude spectrum and a phase spectrum according to the frequency domain characteristics, and repair the amplitude spectrum to obtain an amplitude repair spectrum; Determine a residual amplitude according to the amplitude repair spectrum and the amplitude spectrum, and determine an attention map according to the residual amplitude; Performing a phase change on the phase spectrum according to the attention map to obtain a phase change spectrum, and determining a frequency domain enhanced real part and a frequency domain enhanced imaginary part according to the phase change spectrum and the amplitude repair spectrum; Determine a frequency domain enhancement feature according to the frequency domain enhancement real part and the frequency domain enhancement imaginary part, and determine a spatial domain enhancement feature according to the frequency domain enhancement feature; Performing feature enhancement on the residual restored image and the fused restored image according to the spatial domain enhancement feature to obtain the residual enhanced image and the fused enhanced image; The formula used to determine the frequency domain features according to the spatial domain features includes: ; Wherein, x is the spatial domain feature of the residual restored image and the fused restored image, are the coordinates of the residual restored image and the fused restored image in the frequency domain, /> is the frequency component of the image in the horizontal direction, /> is the frequency component of the image in the vertical direction, /> and/> is the height and width of the image, /> and/> are the vertical and horizontal coordinates of the image, /> The frequency domain features of the residual restored image and the fused restored image; The formulas used to determine the amplitude spectrum and phase spectrum according to the frequency domain characteristics include: ; in, is the real part of the frequency domain feature, /> is the imaginary part of the frequency domain feature; ; in, is the amplitude spectrum, /> is the phase spectrum; The amplitude spectrum is repaired to obtain an amplitude repair spectrum. The formula used to determine the residual amplitude according to the amplitude repair spectrum and the amplitude spectrum includes: ; in, is the amplitude repair spectrum, /> is the residual amplitude between the amplitude repair spectrum and the amplitude spectrum,/> is the convolution operator,/> is the filter; An attention map is determined according to the residual amplitude, and a formula used to change the phase spectrum according to the attention map includes: ; in, is the attention map, GAP is the global average pooling, /> is the phase change spectrum, /> It is an element-wise product operation; The formula used to determine the frequency domain enhancement real part and the frequency domain enhancement imaginary part according to the phase change spectrum and the amplitude repair spectrum includes: ; in, is the frequency domain enhanced real part,/> Enhance the imaginary part in the frequency domain; The formula used to determine the frequency domain enhancement feature according to the frequency domain enhancement real part and the frequency domain enhancement imaginary part includes: ; in, It is the frequency domain enhancement feature of the residual enhanced image and the fusion enhanced image. 2. The real image defogging method with multi-branch frequency domain enhancement according to claim 1, characterized in that the formula used for data enhancement of the sample data set includes: ; in, is the sample enhancement set,/> For the sample data set, A is a real fog-free image, A is the global atmospheric light, /> is the medium transmittance diagram, /> is the scene depth, /> is the fog density; ; in, For foggy images/> and foggy images/> The generated augmented image, /> and/> For a real fog-free image/> and real fog-free image/> Synthesized foggy image, /> and/> are the ath and bth real fog-free images in the sample dataset, is the amplitude of the Fourier transform, /> is the phase component of Fourier transform, /> is the image mask, /> A composite operation of functions. 3. The real image defogging method with multi-branch frequency domain enhancement according to claim 1, characterized in that the formula used to determine the model loss based on the defogging image and the real image of the sample data set includes: ; Wherein, X is the defogging image, Y is the real image, is the model loss,/> is the loss composed of peak signal-to-noise ratio and structural similarity index,/> is the edge loss,/> 、/> and/> is a preset constant, /> is the Laplace operator, PSNR is the peak signal-to-noise ratio, SSIM is the structural similarity, MAX ² _Y is the maximum pixel value that can be obtained in the image, MSE is the mean square error between the dehazed image and the corresponding real image, /> 、/> 、/> Respectively represent the comparison of brightness, contrast and saturation of the real image and the dehazed image,/> 、/> and/> is a hyperparameter. 4. A multi-branch frequency domain enhanced real image defogging system, characterized in that the multi-branch frequency domain enhanced real image defogging method according to any one of claims 1 to 3 is applied, and the system comprises: A data enhancement module is used to obtain a sample data set and perform data enhancement on the sample data set to obtain a sample enhancement set; A feature extraction module is used to input the sample enhancement set into the image defogging model to perform feature extraction of a multi-branch network to obtain sample image features, and to perform semantic segmentation on the sample image features of each branch network to obtain semantic segmentation features; Perform residual attention processing on the semantic segmentation features of each branch network respectively to obtain mask feature maps and residual output images, and fuse the mask feature maps of each branch network to obtain fused features; Performing a mixed skip connection on the residual output images of each branch network to obtain a residual image, and performing image restoration on the residual image and the fusion feature respectively to obtain a residual restoration image and a fusion restoration image; A frequency domain enhancement module, used for performing frequency domain enhancement on the residual restored image and the fused restored image to obtain a residual enhanced image and a fused enhanced image, and fusing the residual enhanced image and the fused enhanced image to obtain a defogging image; A parameter updating module, used to determine a model loss according to the defogging image and a real image of the sample data set, and update parameters of the image defogging model according to the model loss until the image defogging model converges; The defogging processing module is used to perform defogging processing on the image to be defogged after inputting the converged image defogging model to obtain a defogged output image. 5. The real image defogging system with multi-branch frequency domain enhancement according to claim 4, wherein the frequency domain enhancement module is further used for: Acquire spatial domain features of the residual restored image and the fused restored image, and determine frequency domain features according to the spatial domain features; Determine an amplitude spectrum and a phase spectrum according to the frequency domain characteristics, and repair the amplitude spectrum to obtain an amplitude repair spectrum; Determine a residual amplitude according to the amplitude repair spectrum and the amplitude spectrum, and determine an attention map according to the residual amplitude; Performing a phase change on the phase spectrum according to the attention map to obtain a phase change spectrum, and determining a frequency domain enhanced real part and a frequency domain enhanced imaginary part according to the phase change spectrum and the amplitude repair spectrum; Determine a frequency domain enhancement feature according to the frequency domain enhancement real part and the frequency domain enhancement imaginary part, and determine a spatial domain enhancement feature according to the frequency domain enhancement feature; Feature enhancement is performed on the residual restored image and the fused restored image according to the spatial domain enhancement feature to obtain the residual enhanced image and the fused enhanced image. 6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 3 when executing the computer program.