CN116823914A - Unsupervised focal stack depth estimation method based on all-focusing image synthesis - Google Patents

Abstract

The invention discloses an unsupervised focal stack depth estimation method based on full-focusing image synthesis, which comprises the following steps: s1, performing all-focus image calculation by using an image pyramid-based and focus measurement operator to obtain corresponding all-focus images, and fusing the obtained all-focus images to serve as supervision information; s2, carrying out high-frequency noise filtration and preliminary feature extraction on the focal stack through a three-dimensional perception module; s3, introducing a three-dimensional polarization self-attention mechanism into a focal stack, and dividing an input feature map into a channel polarization feature map and a space polarization feature map; and S4, positioning the layer where the maximum definition of the focal stack is located by adopting a layered depth probability prediction module, outputting a corresponding probability value, determining the layer where the optimal definition is located, and obtaining the full-focusing image. The method has relatively high accuracy and good generalization performance in the aspect of depth prediction, is suitable for depth estimation tasks under different scenes, and has high practicability.

Description

Unsupervised focus stack depth estimation method based on fully focused image synthesis

Technical field

The invention relates to the technical field of monocular depth estimation, and in particular to an unsupervised focus stack depth estimation method based on full-focus image synthesis.

Background technique

Supervised methods show high accuracy in depth estimation tasks, but are limited in that they require true depth values, which may be difficult to obtain in practical application scenarios. In recent years, with the continuous development of deep learning technology and continuous exploration in the field of computer vision, the field of unsupervised monocular depth estimation has made great progress. Unsupervised monocular depth estimation refers to inferring the depth information of the scene through computer vision algorithms without depth labels. Unsupervised focus stack depth estimation can be divided into two categories, namely reconstruction supervision and auxiliary supervision.

Reconstruction supervision performs supervised learning on the network through the reconstruction loss of the network to learn depth information. Unsupervised focus stack depth estimation is regarded as a special case of multi-view monocular depth estimation, and the scene is estimated by exploiting the blur difference of the focus sequence. Depth is then refocused using the focus map and estimated intermediate depth, a focus stack is output, and a reconstruction loss is used for supervised learning. However, due to the ill-posedness of the depth estimation task, reconstructing the model easily leads to multiple depth solutions competing with each other, making it difficult to determine the optimal solution. Therefore, the network structure is very unstable. At the same time, the intermediate representation is easily interpreted as the information compression encoding of the focus stack, resulting in Models have difficulty converging, so additional losses often need to be introduced to constrain the intermediate representation.

Auxiliary supervision uses some auxiliary information to guide the learning process of the network under unsupervised conditions, using full-focus images as auxiliary supervision information. This method first inputs the focus stack into the codec structure and outputs the image at each focus distance. Depth distribution probability, and combining it with the focus stack and focus distance respectively, can output a fully focused image while also obtaining a relatively rough depth map. However, this model has certain limitations, such as a large number of parameters and the need for the data set itself to provide full-focus images as supervision information, so its application is severely limited. Therefore, how to provide an unsupervised focus stack depth estimation method based on full-focus image synthesis is an urgent problem that those skilled in the art need to solve.

Contents of the invention

One purpose of the present invention is to propose an unsupervised focus stack depth estimation method based on full-focus image synthesis. The present invention shows relatively high accuracy and good generalization performance in depth prediction, and is suitable for depth in different scenarios. Estimation tasks, with high practicality.

An unsupervised focus stack depth estimation method based on full-focus image synthesis according to an embodiment of the present invention includes:

S1. Use the full-focus image synthesis method based on the image pyramid and the full-focus image synthesis method based on the focus measurement operator to calculate the full-focus image, obtain the corresponding full-focus image, and fuse the obtained full-focus images as supervision information;

S2. Use the three-dimensional perception module to perform high-frequency noise filtering and preliminary feature extraction on the focus stack to obtain the initial extraction features. At the same time, the focus stack obtains features encoding fuzzy ambiguity through the difference value calculation module. The initial extraction features and fuzzy ambiguity features are processed. Cascade, that is, the focus body is obtained;

S3. Introduce the three-dimensional polarization self-attention mechanism into the focus stack, and divide the input feature focus volume into a channel polarization feature map and a spatial polarization feature map;

S4. The above-mentioned channel polarization feature map and spatial polarization feature map use the depth probability prediction module to locate the level where the maximum sharpness of the focus stack is located, and output the corresponding probability value to determine the level where the best clarity is located, and obtain a fully focused image. .

Optionally, the image pyramid specifically includes:

Gaussian pyramid downsampling to the original image Represents the bottom layer of the Gaussian pyramid, and its resolution is/> , by defining the i-th layer of Gaussian pyramid:

;

Among them, among them, Represents the convolution operation,/> Indicates that the size is/> The convolution kernel,/> Represents the downsampling process of removing even rows and even columns of the input image;

Downsampling will be the resolution of the input image Reduce it to a quarter, and obtain the entire Gaussian pyramid by continuously iterating the above steps;

Gaussian pyramid upsampling, the original image Expand to twice the original size in each direction, fill the new rows and columns with 0, use the same convolution kernel multiplied by four as before to convolve with the enlarged image to obtain the reconstructed image;

The Laplacian pyramid is introduced into the reconstructed image, assuming Represents the third of Laplace's Pyramid/> layer:

;

in, Represents the upsampling process, that is, the image is expanded to twice the original size in each direction, and the new rows and columns are filled with 0;

original image is decomposed into a Gaussian pyramid and a Laplacian pyramid, and for each image in the focus stack, the same decomposition operation is performed to obtain a set of image pyramids.

Optionally, the image pyramid fusion process specifically includes:

Given a focus stack sequence:

;

in, Represents the spatial coordinates of the pixel point,/> Indicates the number of focus sequences, each picture corresponds to a specific focus distance;

focus stack Perform image pyramid decomposition to obtain Gaussian pyramid/> and Laplace's Pyramid/> , where,/> Represents the number of layers of the pyramid;

Pyramid of Laplace every position/> Perform focus measurement to obtain the index image corresponding to the maximum clarity/> ,/> Generated from index graph and Laplacian pyramid:

use Full focus on Laplacian Pyramid/> Upsampling is performed from top to bottom to obtain a fully focused image corresponding to the focus stack.

Optionally, the all-focus image synthesis method based on image pyramid specifically includes the input focus stack Perform image pyramid decomposition to obtain Gaussian pyramid/> and Laplace's Pyramid/> , to Laplace's Pyramid/> The regional information entropy is calculated to obtain the focus measurement sharpness measurement value of each layer. The layer with the largest sharpness measurement value is extracted as the full-focus image of the corresponding layer, and the final full-focus image is reconstructed.

Optionally, the full-focus image synthesis method based on focus measurement operators includes applying small-area neighborhood fusion operators to each focus sequence. The focus measurement sharpness metric value of each focus image is obtained, the index is maximized to determine the index corresponding to the best sharpness, and the pixel value in the focus stack is extracted according to the index as a fully focused image.

Optionally, the full-focus image synthesis method based on the focus measurement operator specifically includes:

Comprehensive features are obtained by converting vector-valued images into scalar-valued images through vector operations:

set up Represents vector-valued pixels, /> Represents a scalar-valued pixel, selecting a patch size in a vector-valued image/> , make/> is the center vector value pixel,/> for window/> vector value pixels within;

where, the vector value pixel Corresponding scalar value pixel/> Obtained by scaling the length of the difference vector within the window;

calculation window Other vectors within/> with center vector/> The difference is the difference vector/> :

;

;

;

in, Represents a scalar value formed by the dot product of the result vectors, /> Represents a local adaptive scaling factor;

;

in, Calculate the dot product between difference vectors to measure the similarity between features, Provides differential vectors/> and center vector/> The cross product length between;

The resulting scalar value image is applied to the index maximization operation to evaluate the sharpness of the image, and the pixel value at the corresponding position is extracted from the input focus stack according to the index where the best sharpness is located, and the corresponding fully focused image is obtained.

Optionally, the three-dimensional perception module completes the high-frequency noise filtering and preliminary feature extraction of the focus stack through a four-layer network structure. The three-dimensional perception module includes multiple parallel convolutions with different convolution kernel sizes and step sizes. Accumulated layers are used to capture fuzzy features at different scales;

The S2 specifically includes:

S21. Use a 3D convolution network to filter the focus stack and extract blur features;

S22. Introduce a differential value calculation module into the network structure, input the fuzzy features into the differential value calculation module, and the differential value calculation module calculates the differential values of the three RGB channels:

;

in, Represents the RGB channel difference after fusion,/> Represents different color dimensions of input features;

S23. Obtain the RGB differential features through a down-sampling layer, and fuse the RGB differential features with the fuzzy features to construct a focus volume that combines fuzzy ambiguity.

Optionally, the channel polarization feature map is obtained by performing polarization transformation on the input feature map x:

Polarization transformation converts the input feature map x into two sets of basis vectors and/> ;

in, and/> Corresponds to channel-level queries and keys;

calculate and/> similarity score/> :

;

in, Represents the activation function,/> Represents a normalized exponential function,/> ,/> and/> Represents a 1×1 three-dimensional convolution layer respectively, /> and/> Represents two tensor reshaping operators, × represents element-level multiplication operations, /> and/> with/> The number of channels between is/> ;

Use score As a weight, the input vectors are weighted and summed to obtain the channel polarization feature map that obtains the channel correlation/> :

;

in, Represents a channel-level multiplication operator.

Optionally, the spatial polarization feature map method includes:

Convert the input channel polarization feature map to Perform polarization changes to obtain two sets of polarization vectors and/> ;

in, Global pooling is performed on three channels to obtain global spatial features,/> Rearrange the pixels in the input feature map through three-dimensional convolution to enhance features in different directions in space;

Calculate the similarity matrix from two sets of polarization vectors :

;

in, and Represents the standard 1×1 three-dimensional convolution layer respectively, Represents the intermediate parameters of channel convolution, , and , × represents the matrix dot multiplication operation, Represents global pooling;

The corresponding weights are obtained through the similarity matrix, and the weights are weighted and summed with the input channel polarization features to obtain a comprehensive self-attention feature representation that associates channel and spatial features. ;

;

in Represents the spatial multiplication operator.

Optionally, the S4 specifically includes:

S41. After passing through a codec network with the pooling layer removed, the output of the focus stack depth estimation network is divided into multiple levels, each level corresponding to a specific focus distance;

S42. Apply between levels The operation determines the level where the best clarity is, obtains the best focus position, and obtains a fully focused image;

S43. Use the weighted summation of multi-layer probability values to obtain the final depth estimation result.

The beneficial effects of the present invention are:

The present invention first synthesizes a full-focus image and uses it as supervision information, and then performs depth estimation through a feature rough extraction module, a polarization self-attention module and a hierarchical depth estimation module. Using focus stacks to synthesize fully focused images is used as supervision information and utilizes the correlation ability of the self-attention mechanism to obtain scene depth, so that the present invention shows relatively high accuracy and good generalization performance in depth prediction, and is suitable for different scenes It is highly practical for depth estimation tasks.

Description of the drawings

The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. In the attached picture:

Figure 1 is an unsupervised focus stack depth estimation model in an unsupervised focus stack depth estimation method based on full-focus image synthesis proposed by the present invention;

Figure 2 is a structural block diagram of the focus measurement sharpness measurement value in an unsupervised focus stack depth estimation method based on full-focus image synthesis proposed by the present invention;

Figure 3 is a qualitative comparison diagram of full-focus image synthesis in an unsupervised focus stack depth estimation method based on full-focus image synthesis proposed by the present invention;

Figure 4 is a structural block diagram of a three-dimensional perception module in an unsupervised focus stack depth estimation method based on full-focus image synthesis proposed by the present invention;

Figure 5 is a structural block diagram of a channel difference module in an unsupervised focus stack depth estimation method based on full-focus image synthesis proposed by the present invention;

Figure 6 is a visual comparison chart of the generalization performance on DefocusNet in an unsupervised focus stack depth estimation method based on full-focus image synthesis proposed by the present invention;

Figure 7 is a visual comparison chart of generalization performance on MobileDepth in an unsupervised focus stack depth estimation method based on full-focus image synthesis proposed by the present invention.

Detailed ways

The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams that only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the structures related to the present invention.

Referring to Figure 1, an unsupervised focus stack depth estimation method based on full-focus image synthesis includes:

S1. Use the full-focus image synthesis method based on the image pyramid and the full-focus image synthesis method based on the focus measurement operator to calculate the full-focus image, obtain the corresponding full-focus image, and fuse the obtained full-focus images as supervision information;

Referring to Figure 2, this embodiment shows the process of synthesizing a full-focus image using two methods.

in the picture Represents the focus sequence, Gaussian pyramid downsampling, using the original image/> Represents the bottom layer of the Gaussian pyramid, and its resolution is/> , by defining the i-th layer of Gaussian pyramid:

;

in, Represents the convolution operation,/> Indicates that the size is/> The convolution kernel,/> Represents the downsampling process of removing even rows and even columns of the input image;

Downsampling will be the resolution of the input image Reduce it to a quarter, and obtain the entire Gaussian pyramid by continuously iterating the above steps;

;

in, Represents the upsampling process, that is, the image is expanded to twice the original size in each direction, and the new rows and columns are filled with 0;

original image is decomposed into a Gaussian pyramid and a Laplacian pyramid, and for each image in the focus stack, the same decomposition operation is performed to obtain a set of image pyramids.

In this implementation, the image pyramid fusion process specifically includes:

Given a focus stack sequence:

;

in, Represents the spatial coordinates of the pixel point,/> Indicates the number of focus sequences, each picture corresponds to a specific focus distance;

focus stack Perform image pyramid decomposition to obtain Gaussian pyramid/> and Laplace's Pyramid/> , where,/> Represents the number of layers of the pyramid;

Pyramid of Laplace every position/> Perform focus measurement to obtain the index image corresponding to the maximum clarity/> , full focus Laplacian Pyramid/> Generated from index graph and Laplacian pyramid: /> ;

use Full focus on Laplacian Pyramid/> Upsampling is performed from top to bottom to obtain a fully focused image corresponding to the focus stack.

In this embodiment, the full-focus image synthesis method based on image pyramid specifically includes the input focus stack Decompose and obtain Gaussian pyramid/> and Laplace's Pyramid/> , since the entire decomposition process is completely reversible, there is no information loss in this image transformation method, for the Laplacian pyramid/> The regional information entropy is calculated to obtain the focus measurement sharpness measurement value of each layer. The layer with the largest sharpness measurement value is extracted as the full-focus image of the corresponding layer, and the final full-focus image is reconstructed.

In this embodiment, the full-focus image synthesis method based on the focus measurement operator includes applying the small-area neighborhood fusion operator to each focus sequence. The focus measurement sharpness metric value of each focus image is obtained, the index is maximized to determine the index corresponding to the best sharpness, and the pixel value in the focus stack is extracted according to the index as a fully focused image.

The full-focus image fusion algorithm of the present invention based on image pyramid and small window fusion operator can synthesize high-quality full-focus images. The proposed model uses the global correlation structure to effectively improve the accuracy of depth prediction, and its lightweight design enables the model to have real-time reasoning capabilities.

Referring to Figure 3, in this embodiment, the full-focus image synthesis method based on the focus measurement operator specifically includes:

Comprehensive features are obtained by converting vector-valued images into scalar-valued images through vector operations:

set up Represents vector-valued pixels, /> Represents a scalar-valued pixel, selecting a patch size in a vector-valued image/> , make/> is the center vector value pixel,/> for window/> vector value pixels within;

where, the vector value pixel Corresponding scalar value pixel/> Obtained by scaling the length of the difference vector within the window;

calculation window Other vectors within/> with center vector/> The difference is the difference vector/> :

;

;

;

in, Represents a scalar value formed by the dot product of the result vectors, /> Represents a local adaptive scaling factor, /> Plays an important role in calculating scalar feature images;

;

in, Calculate the dot product between difference vectors to measure the similarity between features, Provides differential vectors/> and center vector/> The cross product length between;

The resulting scalar value image is applied to the index maximization operation to evaluate the sharpness of the image, and the pixel value at the corresponding position is extracted from the input focus stack according to the index where the best sharpness is located, and the corresponding fully focused image is obtained according to With this method, high-quality all-focus images can be synthesized from focus sequences.

S2. Use the three-dimensional perception module to perform high-frequency noise filtering and preliminary feature extraction on the focus stack to obtain the initial extraction features. At the same time, the focus stack obtains features encoding fuzzy ambiguity through the difference value calculation module. The initial extraction features and fuzzy ambiguity features are processed. Cascade, that is, the focus body is obtained;

In this implementation, the three-dimensional perception module completes high-frequency noise filtering and preliminary feature extraction of the focus stack through a four-layer network structure. The three-dimensional perception module includes multiple parallel convolution layers with different convolution kernel sizes and step sizes. Used to capture fuzzy features at different scales;

Referring to Figure 4, S2 specifically includes:

S21. Use a 3D convolution network to filter the focus stack and extract blur features;

S22. Introduce a differential value calculation module into the network structure, input the fuzzy features into the differential value calculation module, and the differential value calculation module calculates the differential values of the three RGB channels:

;

in, Represents the fused RGB channel difference, representing the different color dimensions of the input features;

S23. Obtain the RGB differential features through a down-sampling layer, and fuse the RGB differential features with the fuzzy features to construct a focus volume that combines fuzzy ambiguity.

S3. Introduce the three-dimensional polarization self-attention mechanism into the focus stack, and divide the input feature focus volume into a channel polarization feature map and a spatial polarization feature map;

In this implementation, the channel polarization feature map is obtained by performing polarization transformation on the input feature map x:

Polarization transformation converts the input feature map x into two sets of basis vectors and/> ;

in, and/> Corresponds to channel-level queries and keys;

calculate and/> similarity score/> :

;

in, Represents the activation function,/> Represents a normalized exponential function,/> ,/> and/> Represents a 1×1 three-dimensional convolution layer respectively, /> and/> Represents two tensor reshaping operators, × represents element-level multiplication operations, /> and/> with/> The number of channels between is/> ;

Use score As a weight, the input vectors are weighted and summed to obtain the channel polarization feature map that obtains the channel correlation/> :

;

in, Represents a channel-level multiplication operator.

In this implementation, the spatial polarization feature map method includes:

Convert the input channel polarization feature map to Perform polarization changes to obtain two sets of polarization vectors and/> ;

in, Global pooling is performed on three channels to obtain global spatial features,/> Rearrange the pixels in the input feature map through three-dimensional convolution to enhance features in different directions in space;

Calculate the similarity matrix from two sets of polarization vectors :

;

in, and/> Represents the standard 1×1 three-dimensional convolution layer respectively,/> Represents the intermediate parameters of channel convolution, ,/> and/> represents three tensor reshaping operations, × represents the matrix dot multiplication operation, /> Represents global pooling;

The corresponding weights are obtained through the similarity matrix, and the weights are weighted and summed with the input channel polarization features to obtain a comprehensive self-attention feature representation that associates channel and spatial features. ; ;

in Represents the spatial multiplication operator.

It should be noted that all the above convolution operations and tensor reshaping operations are performed in three channel dimensions. Therefore, the three-dimensional polarization self-attention mechanism can consider both channel correlation and spatial fuzzy correlation.

The model proposed in this invention shows good performance on smaller focus stacks while having excellent generalization capabilities.

S4. The above-mentioned channel polarization feature map and spatial polarization feature map use the depth probability prediction module to locate the level where the maximum sharpness of the focus stack is located, and output the corresponding probability value to determine the level where the best clarity is located, and obtain a fully focused image. .

In this implementation, S4 specifically includes:

S41. After passing through a codec network with the pooling layer removed, the output of the focus stack depth estimation network is divided into multiple levels, each level corresponding to a specific focus distance;

S42. Apply Softmax operation between layers to determine the layer with the best clarity, obtain the best focus position, and obtain a fully focused image;

During testing, the blur information in the input focus sequence is used to determine the level at which the target depth is located, and the probability density function of the corresponding level is used to calculate the depth probability value.

S43. Use the weighted summation of multi-layer probability values to obtain the final depth estimation result.

In Example 1:

This invention is quantified on 4D Light Field, DefocusNet and FlyingThings3D datasets:

As can be seen from Table 1 above, the proposed full-focus image synthesis method can synthesize a more accurate full-focus image from a smaller focus stack.

The above Tables 2 to 4 are the quantitative comparison results between the present invention and the latest methods on the 4D Light Field, DefocusNet and FlyingThings3D data sets.

As can be seen from Tables 1 to 4 above, the results on the 4D Light Field data set show that the present invention improves the MSE and RMSE indicators by 42.5% and 26.3% respectively in unsupervised depth estimation compared to the AiFDepthNet method. In comparison with supervised methods, this method surpasses most supervised methods including VDFF, PSPNet, and DDFF. Even compared with the DefocusNet method, the performance difference in MSE and RMSE is only 15.0% and 4.6%. The results on the DefocusNet data set and FlyingThings3D data set show that compared with the AiFDepthNet method, this method achieves higher accuracy in MAE, MSE, and RMSE indicators. Compared with the 16M parameter size of the AiFDepthNet method, the parameter size of this method is also smaller, 3.3M, and has higher computational efficiency.

The present invention first synthesizes a full-focus image and uses it as supervision information, and then performs depth estimation through a feature rough extraction module, a polarization self-attention module and a hierarchical depth estimation module. Using focus stacks to synthesize fully focused images is used as supervision information and utilizes the correlation ability of the self-attention mechanism to obtain scene depth, so that the present invention shows relatively high accuracy and good generalization performance in depth prediction, and is suitable for different scenes It is highly practical for depth estimation tasks.

The above are only preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can, within the technical scope disclosed in the present invention, implement the technical solutions of the present invention. Equivalent substitutions or changes of the inventive concept thereof shall be included in the protection scope of the present invention.

Claims (10)

1. An unsupervised focal stack depth estimation method based on all-focus image synthesis, comprising:

s1, performing all-focus image calculation by using an all-focus image synthesis method based on an image pyramid and an all-focus image synthesis method based on a focus measurement operator to obtain corresponding all-focus images, and fusing the obtained all-focus images to serve as supervision information;

s2, performing high-frequency noise filtration and preliminary feature extraction on the focal stack through a three-dimensional perception module to obtain primary extraction features, simultaneously obtaining coded ambiguity features through a differential value calculation module, and cascading the primary extraction features and the ambiguity features to obtain a focal body;

s3, introducing a three-dimensional polarization self-attention mechanism into a focal stack, and dividing an input characteristic focal body into a channel polarization characteristic diagram and a space polarization characteristic diagram;

s4, positioning the layer where the maximum definition of the focal stack is located through the depth probability prediction module by the channel polarization feature map and the space polarization feature map, outputting a corresponding probability value, determining the layer where the optimal definition is located, and obtaining the full-focusing image.

2. The method for estimating depth of an unsupervised focal stack based on full-focus image synthesis according to claim 1, wherein the image pyramid specifically comprises:

downsampling with Gaussian pyramid to original imageRepresents the bottom layer of the Gaussian pyramid with resolution +.>By defining the gaussian pyramid of the i-th layer:

;

wherein ,representing convolution operations +.>Representing a size of +.>Is a convolution kernel of->A downsampling process that removes even rows and even columns of the input image;

downsampling the resolution of an input imageReducing the height to one fourth, and obtaining the whole Gaussian pyramid by continuously iterating the steps;

gaussian pyramid upsampling to image the originalExpanding twice in each direction, filling the newly added rows and columns with 0, and convolving the newly added rows and columns with the amplified image by multiplying the newly added rows and columns by four by the convolution kernel which is the same as the previous row to obtain a reconstructed image;

introducing Laplacian pyramid into the reconstructed image, and settingRepresents the +.o of the Laplacian pyramid>Layer (c):

;

wherein ,representing the upsampling process, i.e., expanding the image twice as much as it was in each direction, with the newly added rows and columns filled with 0's;

original imageIs decomposed into a gaussian pyramid and a laplacian pyramid, and the same decomposition operation is performed for each image in the focal stack, resulting in a set of image pyramids.

3. The method for estimating the depth of an unsupervised focal stack based on the synthesis of an all-focused image according to claim 2, wherein the process of merging the image pyramids specifically comprises:

given a focal stack sequence:

；

wherein ,representing the spatial coordinates of the pixel points, +.>Representing the number of focusing sequences, wherein each picture corresponds to a specific focusing distance;

focal stackDecomposing the image pyramid to obtain Gaussian pyramid +.>Laplacian pyramid, wherein ,/>Representing the number of layers of the pyramid;

laplacian pyramidIs +.>Performing focus measurement to obtain index map corresponding to maximum definition +.>，/>The all-focus Laplacian pyramid is generated by an index map and the Laplacian pyramid:;

by means ofFor all-focus Laplacian pyramid->And up-sampling from top to bottom to obtain an all-focusing image corresponding to the focal stack.

4. A non-focusing image synthesis based on claim 3The method for estimating the depth of the supervised focal stack is characterized by specifically comprising the steps of focusing the input focal stackDecomposing to obtain Gaussian pyramid->And Laplacian pyramid->Laplacian pyramidAnd carrying out regional information entropy calculation to obtain a focus measurement definition measurement value of each layer, extracting a layer with the maximum definition measurement value as an all-focusing image of the corresponding layer, and reconstructing to obtain a final all-focusing image.

5. An unsupervised focal stack depth estimation method based on full focus image synthesis according to claim 3, wherein the full focus image synthesis method based on focus measurement operator comprises applying a small region neighborhood fusion operator to each focus sequenceAnd obtaining focus measurement definition values of all focus images, carrying out index maximization to determine an index corresponding to the optimal definition, and extracting pixel values in a focus stack according to the index to serve as an all-focus image.

6. The method for estimating depth of an unsupervised focal stack based on the composition of an all-focused image according to claim 5, wherein the method for composing an all-focused image based on a focus measurement operator specifically comprises:

the vector value image is converted into a scalar value image through vector operation to obtain comprehensive characteristics:

is provided withRepresenting vector value pixels, ">Representing scalar value pixels, selecting a tile size +.>Make->For the center vector value pixel, ">For window->Vector value pixels within;

wherein the vector value pixelsCorresponding scalar value pixels +.>Obtaining by scaling the differential vector length in the window;

computing windowInner other vector->And center vector->The difference results in a difference vector->：

;

;

;

wherein ,scalar value representing dot product formation of result vector, < ->Representing a local adaptive scaling factor;

；

wherein ,the dot product between the differential vectors is calculated to measure the similarity between the features,providing a differential vector->And center vector->Cross-product length between;

and applying the obtained scalar value image to index maximization operation to evaluate the definition of the image, and extracting the pixel value of the corresponding position from the input focal stack according to the index of the optimal definition to obtain the corresponding all-focusing image.

7. The method for estimating depth of an unsupervised focal stack based on full-focus image synthesis according to claim 1, wherein the three-dimensional perception module is configured to complete high-frequency noise filtering and preliminary feature extraction of the focal stack through a four-layer network structure, and the three-dimensional perception module comprises a plurality of parallel convolution layers with different convolution kernel sizes and step sizes for capturing fuzzy features on different scales;

the step S2 specifically comprises the following steps:

s21, filtering the focal stack by using a 3D convolution network to extract fuzzy features;

s22, introducing a differential value calculation module into the network structure, inputting the fuzzy characteristic into the differential value calculation module, and calculating differential values of RGB three channels by the differential value calculation module:

；

wherein ,representing the fused RGB channel difference, +.>Different color dimensions representing input features;

s23, obtaining RGB differential features through a downsampling layer, fusing the RGB differential features and the fuzzy features, and constructing a focus body fused with fuzzy ambiguity.

8. The method for estimating depth of an unsupervised focal stack based on full-focus image synthesis according to claim 1, wherein the channel polarization feature map is obtained by performing polarization transformation on an input feature map x:

the polarization transformation converts the input feature map x into two sets of basis vectors and />；

wherein , and />Query and key corresponding to channel level;

calculation of and />Similarity score +.>：

;

wherein ,representing an activation function->Representing normalized exponential function, ++>、/> and />Respectively representing 1 x 1 three-dimensional convolution layers, < >> and />Representing two tensor remodelling operators, x represents multiplication at element level, +.> and />And->The number of channels between is->；

Score for useAs weights, the input vectors are weighted and summed to obtain a channel polarization characteristic map of the channel correlation>：

;

wherein ,representing a channel-level multiply operator.

9. The method for estimating depth of an unsupervised focal stack based on full focus image synthesis according to claim 8, wherein the method for spatially polarizing the feature map comprises:

to input the channel polarization characteristic diagramPerforming polarization change to obtain two sets of polarization vectors and /> wherein ,/>Global spatial feature acquisition by global pooling of three channels +.>Rearranging pixels in an input feature map through three-dimensional convolution to enhance features in different directions in space;

calculating a similarity matrix from two sets of polarization vectors：

;

wherein , and />Respectively representing standard 1 x 1 three-dimensional convolution layers,/->Intermediate parameters representing the convolution of the channel,/->、 and />Representing three tensor remodelling operations, x represents matrix dot product operation, +.>Representing global pooling;

corresponding weights are obtained through the similarity matrix, weighted summation is carried out on the weights and the input channel polarization characteristics, and the comprehensive self-attention characteristic representation associated with the channel and the spatial characteristics is obtained；

;

wherein Representing the spatial multiplication operator.

10. The method for estimating the depth of an unsupervised focal stack based on the synthesis of an all-in-focus image according to claim 1, wherein S4 specifically comprises:

s41, after passing through a codec network with a pooling layer removed, dividing the output of a focal stack depth estimation network into a plurality of layers, wherein each layer corresponds to a specific focusing distance;

s42, application between layersThe hierarchy where the best definition is located is determined through operation, the best focusing position is obtained, and an all-focusing image is obtained;

s43, obtaining a final depth estimation result by using a multi-layer probability value weighted summation mode.