CN117689760A - OCT axial super-resolution method and system based on histogram information network - Google Patents

Abstract

The invention provides an OCT axial super-resolution method and system based on a histogram information network, relates to the field of image processing, and aims at the problem that the super-resolution effect is affected by the amplified learning error when histogram information is directly used, and a selective feature kernel is designed to strengthen axial features so as to adapt to the introduction of the histogram information. The histogram information is introduced to carry out statistical distribution of pixels, so that a high quantitative evaluation index is obtained, and meanwhile, the sensory difference problem between the result and the true value is considered, so that the result is more similar to the sensory effect of high axial resolution.

Description

OCT axial super-resolution method and system based on histogram information network

Technical field

The invention relates to the field of image processing, and in particular to an OCT axial super-resolution method and system based on histogram information network.

Background technique

Optical coherence tomography technology is a new imaging method (Optical coherencetomography, OCT). It has made breakthrough progress in the industrial field with its high axial resolution in the depth direction and three-dimensional imaging advantages, such as three-dimensional tomography of memory cards. , scratch defect detection of transparent materials, etc. High axial resolution is achieved by increasing the spectral width of the light source and reducing the central wavelength. Therefore, some researchers have achieved high axial resolution through hardware improvements, such as using supercontinuum light sources. However, This method significantly increases the complexity of equipment design and product cost.

In the existing technology, deep learning is often used to directly perform axial super-resolution on OCT images, thereby obtaining image effects close to those captured by high-performance OCT equipment. At present, most methods for improving the image spatial domain or frequency domain only consider the structure of the neural network and often ignore the statistical information of the image pixel distribution. The histogram can reflect various information of the OCT grayscale image through the pixel type distribution. , therefore, introducing histogram information to provide statistical guidance on pixel distribution in different areas of the image can not only improve the quantitative evaluation index of OCT images, but also adjust the light and shadow structure such as contrast in specific areas, solving the problems of super-resolution and high-axis in existing OCT images. There is a problem of sensory differences in axial resolution, and a qualitative and intuitive effect closer to high axial resolution images is obtained. In addition to introducing the statistical information of the histogram, due to the roll-off of the OCT equipment, this will cause the light intensity in the axial direction to continuously decrease when the equipment collects images, thus causing high and low axial resolutions along the A-line direction. (Axis direction) The pixel distribution changes greatly and the similarity decreases. Directly introducing histogram information to provide pixel distribution guidance for OCT with low axial resolution will amplify learning errors, reduce learning efficiency, and affect quantitative evaluation indicators and qualitative intuitive effects.

Contents of the invention

The purpose of the present invention is to provide an OCT axial super-resolution method and system based on histogram information network in view of the shortcomings of the existing technology, and to design a selective feature kernel to enhance the axial features and thereby adapt to the histogram information. Introducing histogram information for statistical distribution of pixels to obtain high quantitative evaluation indicators, while taking into account the sensory difference between the results and the true value, making the results closer to the sensory effect of high axial resolution .

The first purpose of the present invention is to provide an OCT axial super-resolution method based on histogram information network, adopting the following scheme:

include:

Obtain the OCT image data to be processed for axial super-resolution and input the axial super-resolution processing model;

The axial super-resolution processing model outputs OCT images with improved axial resolution;

Among them, establishing an axial super-resolution processing model includes:

Obtain the original image data and perform image reconstruction and image reconstruction with spectral cropping, respectively, to obtain high axial resolution images and low axial resolution images;

The low axial resolution image input is based on the histogram information network, which performs selective feature enhancement and histogram information coupling. After multiple fusions, the final feature map is obtained, and the final feature map is reconstructed to output the final reconstructed image;

The final reconstructed image and the high axial resolution image are used for training the axial super-resolution processing model, and the trained axial super-resolution processing model is obtained.

Further, the spectral cropping is image reconstruction by adding spectral cropping to each A-line of the original image data through a Gaussian window to obtain a low axial resolution image.

Further, the selective feature enhancement includes:

Three sets of convolutions are extracted in parallel to extract the features of the input low axial resolution image, and the resulting three sets of feature maps are channel fused;

And the feature map information in different directions is aggregated, and the feature map F1 is output after processing.

Furthermore, when performing channel fusion on three groups of feature maps, parameters are set for the number of feature maps in each group to obtain feature maps with different numbers of weights and strengthen the feature maps in the axial direction.

Further, after aggregation, the vector Z is obtained through convolution and activation function; the feature map F1 is obtained by combining the vector Z and the feature map.

Further, the histogram information coupling includes:

Use the feature map F1 output by selective feature enhancement as the input image, and transform it to obtain a frequency domain spectrogram;

The spectrogram is processed in four parallel ways;

The spectrogram extracts and couples the histogram information through the first and second channels, multiplies it with the resized spectrogram in the third channel and resizes it again, and resizes it again from the fourth channel. The plasticized result is combined with the initial spectrogram and then transformed, and the feature map is output;

The output feature map is fused with the feature map F1 to obtain the feature map F2.

Further, the input spectrogram extracts histogram information in the first channel, uses linear mapping to expand its size and transposes; the input spectrogram extracts histogram information in the second channel, uses linear mapping to expand its size but does not translate it. settings, and then the two are multiplied by matrices and the information is coupled through function activation.

Further, the feature map F2 is used as the input image, the processing flow of the feature map F1 is repeated, and the feature map input after the feature map F2 is processed is fused with the feature map F2 to obtain the feature map F3.

Further, the low axial resolution image, feature map F1, feature map F2, and feature map F3 are channel-merged to obtain feature map F4 as the final feature map, which is reconstructed through convolution, and the final reconstructed image is output.

The second purpose of the present invention is to provide an OCT axial super-resolution system based on histogram information network, including:

The data acquisition module is configured to: acquire OCT image data to be subjected to axial super-resolution, and input the axial super-resolution processing model;

The axial super-resolution processing module is configured as follows: the axial super-resolution processing model outputs the OCT image with improved axial resolution;

Among them, establishing an axial super-resolution processing model includes:

Obtain the original image data and perform image reconstruction and image reconstruction with spectral cropping, respectively, to obtain high axial resolution images and low axial resolution images;

The low axial resolution image input is based on the histogram information network, which performs selective feature enhancement and histogram information coupling. After multiple fusions, the final feature map is obtained, and the final feature map is reconstructed to output the final reconstructed image;

The final reconstructed image and the high axial resolution image are used for training the axial super-resolution processing model, and the trained axial super-resolution processing model is obtained.

Compared with the existing technology, the advantages and positive effects of the present invention are:

(1) In view of the problem that directly using histogram information will amplify the learning error and affect the super-resolution effect, a selective feature kernel is designed to strengthen the axial features and adapt to the introduction of histogram information. Histogram information is introduced to carry out statistical distribution of pixels to obtain high quantitative evaluation indicators. At the same time, the sensory difference between the results and the true value is taken into account, so that the results are closer to the sensory effect of high axial resolution.

(2) The trained model can be applied to low axial resolution OCT systems to improve the axial resolution of the OCT images output by it.

Description of the drawings

The description and drawings that constitute a part of the present invention are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

Figure 1 is a schematic flow chart of the OCT axial super-resolution method based on histogram information network in Embodiments 1 and 2 of the present invention.

Figure 2 is a schematic flow chart of low axial resolution image input based on histogram information network in Embodiments 1 and 2 of the present invention.

Figure 3 is a schematic flowchart of enhancing axial features of OCT images in Embodiments 1 and 2 of the present invention.

Figure 4 is a schematic flowchart of fusing histogram information in Embodiments 1 and 2 of the present invention.

Detailed ways

Example 1

In a typical embodiment of the present invention, as shown in Figures 1 to 4, an OCT axial super-resolution method based on histogram information network is provided.

In this embodiment, in view of the problem of sensory differences existing in the super-resolution of existing OCT images, a high quantitative evaluation index is obtained by introducing histogram information and considering real physical factors, so that the super-resolved image is closer to the high-axis towards resolution sensory effects.

Next, the OCT axial super-resolution method based on histogram information network will be described in detail with reference to the accompanying drawings.

Referring to Figure 1, the OCT axial super-resolution method based on histogram information network includes:

Obtain the OCT image data to be processed for axial super-resolution and input the axial super-resolution processing model;

The axial super-resolution processing model outputs OCT images with improved axial resolution;

Among them, establishing an axial super-resolution processing model includes:

Obtain the original image data and perform image reconstruction and image reconstruction with spectral cropping, respectively, to obtain high axial resolution images and low axial resolution images;

The low axial resolution image input is based on the histogram information network, which performs selective feature enhancement and histogram information coupling. After multiple fusions, the final feature map is obtained, and the final feature map is reconstructed to output the final reconstructed image;

The final reconstructed image and the high axial resolution image are used for training the axial super-resolution processing model, and the trained axial super-resolution processing model is obtained.

In this embodiment, as shown in Figure 1, original image data is first obtained through the OCT device. Image reconstruction algorithms are performed directly on the raw data to obtain high axial resolution images. Spectral crop each A-line of the original data through a Gaussian window to obtain a low axial resolution image, and create an image data pair of high and low axial resolution images as a data set. Then the low axial resolution image is input into the histogram information network, as shown in Figure 2. The image is first input into the selective feature enhancement module, and then the feature map F1 is obtained. The feature map F1 is input into the histogram information module, and its output The result is channel-fused with F1 to obtain the feature map F2. F2 is input into the histogram information module, and its output result is channel-fused with F2. Finally, the input image, the output result of the selective feature enhancement module, and the results after the two fusions are combined. The four groups perform channel fusion, and the output results are obtained after 3×3 convolution reconstruction.

Through the above process, the axial super-resolution processing model is trained, and then the training results are applied to the low axial resolution OCT system to obtain the OCT image data to be performed for axial super-resolution and input into the loaded trained axial super-resolution In the low axial resolution OCT system of the rate processing model, the axial resolution of the OCT image output by the low axial resolution OCT system is improved.

The above content will be explained in detail with reference to the accompanying drawings.

Step 1. Data set preparation

Obtain raw image data through OCT equipment. Image reconstruction algorithms are performed directly on the raw data to obtain high axial resolution images. The original data is spectrally cropped for each A-line of the original data through a Gaussian window, and then the image is reconstructed to obtain a low axial resolution image to create a data pair consisting of a high axial resolution image and a low axial resolution image.

Step 2. Enhance the axial features of the OCT image

Step 2.1

As shown in Figure 2, the low axial resolution image is input, and the features of the input image are extracted through three sets of convolutions of 3×1, 1×3, and 3×3 in parallel. The three sets of feature maps are channel fused through the Concat operation. The fusion When setting parameters for the number of feature maps in each group to obtain feature maps with different numbers of weights to strengthen the feature maps in the axial direction, the weight settings of the three groups of convolutions are K1, K2, and K3 respectively. Corresponding to the need to strengthen the axial direction, The weights are set to K1>K2, and K1>K3, and are continuously learned and adjusted to the optimal state during network training.

Step 2.2

After the maximum global pooling size, the feature map size is reduced to 1×1 to achieve the aggregation of feature map information in different directions.

Step 2.3

Then after 1×1 convolution and softmax activation function, the vector Z is obtained.

Step 2.4

Then Z is element-wise multiplied by the feature map that has passed through the 3×3 convolution kernel, and then the output image is obtained, which is F1 in Figure 2.

Step 3. Introduce histogram information

Step 3.1

The specific process is shown in Figure 3. The output image of the previous step is used as the input image of this step. The input image undergoes fast Fourier transformation to obtain the frequency domain spectrogram. The spectrogram will be processed in four parallel ways.

Step 3.2

Extract the histogram feature information as shown in Formula 1, input the spectrogram f(x,y), calculate the number of pixel values i in f(x,y), the result is H(i), consisting of 256 H(i ) constitutes the histogram vector v.

Formula 1:

where f(x,y) is the input spectrogram, x,y is the pixel coordinates, is the Dirac function, i is the size of the pixel value, and H(i) is the number of pixels with value i.

Among them, H(i) is the number of pixels with value i, and v represents a total of 256 data forming a histogram vector.

The above process of extracting histogram information is collectively called the histogram vector extraction F _hist function.

First, from top to bottom, the first and second paths are used to extract histogram information and perform information coupling. The first way is to extract the histogram information of the input spectrogram through the F _hist function, and then use linear mapping to expand its size and transpose, and the result is (1, H W), where H is the image height and W is the image width. The same applies to the following. The second pass extracts the histogram information of the input spectrum image through the F _hist function, and then uses linear mapping to expand its size but not transpose, and the result is (H × W, 1). Then the two are multiplied by the matrix and activated by the softmax function to obtain the result (H×W, H×W).

Step 3.3

The third path reshapes the spectrogram 1 size to (1,H×W), where H is the image height and W is the image width. Perform matrix multiplication and size reshaping with the result of step 3.2 to obtain the result of image size (H, W).

Step 3.4

Then the third path result is combined with the residual of the initial spectrogram 1 to increase image information, and finally the result is obtained as output through inverse Fourier transform.

Step 4: Fusion of histogram information and feature map

F1 and the result of step 3 are fused through channels to obtain the input image F2 of the next step.

Step 5

Repeat the same operations of steps 3 and 4 as the input image F2 to obtain the output image F3.

Step 6

The input image, F1, F2 and F3 are channel-merged to obtain the feature map F4, which is then reconstructed through 3×3 convolution to finally output the image.

Step 7. Network training

The network training process of the axial super-resolution processing model is the same as the conventional deep learning neural network training process.

Example 2

In another typical implementation of the present invention, as shown in Figures 1 to 4, an OCT axial super-resolution system based on histogram information network is provided.

The OCT axial super-resolution system based on histogram information network includes:

The data acquisition module is configured to: acquire OCT image data to be subjected to axial super-resolution, and input the axial super-resolution processing model;

The axial super-resolution processing module is configured as follows: the axial super-resolution processing model outputs the OCT image with improved axial resolution;

Among them, establishing an axial super-resolution processing model includes:

Obtain the original image data and perform image reconstruction and image reconstruction with spectral cropping, respectively, to obtain high axial resolution images and low axial resolution images;

The low axial resolution image input is based on the histogram information network, which performs selective feature enhancement and histogram information coupling. After multiple fusions, the final feature map is obtained, and the final feature map is reconstructed to output the final reconstructed image;

The final reconstructed image and the high axial resolution image are used for training the axial super-resolution processing model, and the trained axial super-resolution processing model is obtained.

The working process of the OCT axial super-resolution system based on the histogram information network is as described in Embodiment 1 and will not be described again here.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims (10)

1. The OCT axial super-resolution method based on the histogram information network is characterized by comprising the following steps of:

acquiring OCT image data to be subjected to axial super-resolution, and inputting an axial super-resolution processing model;

outputting an OCT image with improved axial resolution by the axial super-resolution processing model;

wherein, establishing the axial super-resolution processing model comprises:

acquiring original image data, and respectively carrying out image reconstruction and image reconstruction with increased spectrum clipping to obtain a high axial resolution image and a low axial resolution image;

the low axial resolution image input is based on a histogram information network, selective feature strengthening and histogram information coupling are carried out, a final feature map is obtained after multiple times of fusion, the final feature map is reconstructed, and a final reconstructed image is output;

and using the final reconstructed image and the high axial resolution image for training an axial super resolution processing model to obtain the trained axial super resolution processing model.

2. The OCT axial super-resolution method based on a histogram information network of claim 1, wherein the spectral clipping is an image reconstruction by increasing spectral clipping of each a-line of the original image data through a gaussian window to obtain a low axial resolution image.

3. The histogram information network based OCT axial super-resolution method of claim 1, wherein the selective feature enhancement comprises:

extracting the characteristics of the input low axial resolution images by three groups of convolutions in parallel, and executing channel fusion on the obtained three groups of characteristic images;

and aggregating the feature map information in different directions, and outputting a feature map F1 after processing.

4. A histogram information network based OCT axial super-resolution method according to claim 3, wherein when channel fusion is performed on three sets of feature maps, parameters are set for the number of feature maps in each set to obtain feature maps with different number weights, and feature maps in the axial direction are enhanced.

5. A histogram information network based OCT axial super resolution method according to claim 3, characterized in that after aggregation, vector Z is obtained by convolution and activation functions; and combining the vector Z and the feature map to obtain a feature map F1.

6. The histogram information network based OCT axial super resolution method of claim 1, wherein the histogram information coupling comprises:

taking the characteristic diagram F1 of the selective characteristic strengthening output as an input image, and transforming to obtain a frequency spectrum diagram of a frequency domain;

the spectrogram is processed in parallel by four paths;

extracting histogram information from the spectrogram through a first path and a second path, performing information coupling, multiplying the histogram information with the spectrogram subjected to the medium-size reshaping in a third path, performing re-size reshaping, performing residual combination on the re-size reshaped result and the initial spectrogram from a fourth path, and then converting to output a feature map;

and fusing the output characteristic diagram with the characteristic diagram F1 to obtain a characteristic diagram F2.

7. The OCT axial super-resolution method based on a histogram information network of claim 6, wherein the input spectrogram extracts the histogram information in the first path, expands and transposes its size using linear mapping; the input spectrogram extracts the histogram information in the second path, expands the size of the histogram information by using linear mapping but not transposes the histogram information, and then multiplies the histogram information by a matrix and activates the histogram information by a function to realize information coupling.

8. The method for axially super-resolution OCT based on a histogram information network according to claim 6, wherein the feature map F2 is used as an input image, the processing procedure of the feature map F1 is repeated, and the feature map F3 is obtained by fusing the feature map F2 with the feature map F2 which is input after the feature map F2 is processed.

9. The method for axially super-resolution OCT based on a histogram information network according to claim 8, wherein the low-axial-resolution image, the feature map F1, the feature map F2, and the feature map F3 are channel-combined to obtain a feature map F4 as a final feature map, and the final feature map is reconstructed by convolution to output a final reconstructed image.

10. OCT axial super-resolution system based on histogram information network, characterized by comprising:

a data acquisition module configured to: acquiring OCT image data to be subjected to axial super-resolution, and inputting an axial super-resolution processing model;

an axial super-resolution processing module configured to: outputting an OCT image with improved axial resolution by the axial super-resolution processing model;

wherein, establishing the axial super-resolution processing model comprises:

acquiring original image data, and respectively carrying out image reconstruction and image reconstruction with increased spectrum clipping to obtain a high axial resolution image and a low axial resolution image;

the low axial resolution image input is based on a histogram information network, selective feature strengthening and histogram information coupling are carried out, a final feature map is obtained after multiple times of fusion, the final feature map is reconstructed, and a final reconstructed image is output;

and using the final reconstructed image and the high axial resolution image for training an axial super resolution processing model to obtain the trained axial super resolution processing model.