WO2023142781A1 - 图像三维重建方法、装置、电子设备及存储介质 - Google Patents

图像三维重建方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023142781A1
WO2023142781A1 PCT/CN2022/140102 CN2022140102W WO2023142781A1 WO 2023142781 A1 WO2023142781 A1 WO 2023142781A1 CN 2022140102 W CN2022140102 W CN 2022140102W WO 2023142781 A1 WO2023142781 A1 WO 2023142781A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional
network
image
feature information
layer
Prior art date
Application number
PCT/CN2022/140102
Other languages
English (en)
French (fr)
Inventor
胡颖
谭志强
李世博
张立海
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2023142781A1 publication Critical patent/WO2023142781A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects

Definitions

  • the present application belongs to the technical field of image processing, and in particular relates to an image three-dimensional reconstruction method, device, electronic equipment and storage medium.
  • Image three-dimensional reconstruction refers to inferring the three-dimensional shape of the object in the image from a single or multiple two-dimensional images, so as to reconstruct an image containing three-dimensional information of the object.
  • image 3D reconstruction algorithms have been widely used in many fields, such as robot navigation, 3D modeling, object recognition, scene recognition, medical diagnosis, etc.
  • image 3D reconstruction method that can guarantee efficiency and accuracy.
  • the embodiments of the present application provide a method, device, electronic device, and storage medium for image three-dimensional reconstruction, so as to solve the problem of how to efficiently and accurately implement three-dimensional image reconstruction in the prior art.
  • the first aspect of the embodiments of the present application provides a method for three-dimensional image reconstruction, including:
  • the 3D feature information is input into a trained image generation network for processing to obtain a target 3D reconstructed image; wherein, the image generation network includes a 3D point-by-point convolution layer and a 3D deconvolution layer in sequence.
  • the 3D deconvolution layer of the image generation network includes a first preset number of 3D convolution kernels of different scales.
  • the inputting the two-dimensional image into a trained feature extraction network for processing to obtain two-dimensional feature information corresponding to the two-dimensional image includes:
  • the two-dimensional image is input into a trained feature extraction network for processing to obtain two-dimensional feature information of at least two different scales; wherein, the feature extraction network includes at least two network layers connected in sequence, and the different scales of the The two-dimensional feature information corresponds to feature information output by different network layers;
  • the input of the two-dimensional feature information into the trained dimension transformation network for processing to obtain three-dimensional feature information includes:
  • Each of the two-dimensional feature information is input into the trained dimension transformation network for processing, and each three-dimensional feature information corresponding to each of the two-dimensional feature information is obtained.
  • the feature extraction network includes a first CBAM network layer, N ECA network layers, and a second CBAM network layer connected in sequence, and the ECA network layer includes a residual block;
  • the two-dimensional feature information of different scales includes N two-dimensional feature information output by the N ECA network layers and two-dimensional feature information output by the second CBAM network layer; wherein, the first CBAM network layer The input is the two-dimensional image, the input of the first ECA network layer is the output of the first CBAM network layer, and the input of the i-th ECA network layer is the i-1th ECA network layer , the input of the second CBAM network layer is the output of the last ECA network layer; N is a positive integer greater than 1, and i is any positive integer greater than 1 and less than or equal to N.
  • the dimension transformation network includes N+1 transformation sub-networks
  • the input of the j-th transformation sub-network is the output of the j-th ECA network layer, and the output of the j-th transformation sub-network
  • the input of the last said transformation sub-network is the output of the second CBAM network layer
  • the input of the last said transformation sub-network The output is the three-dimensional feature information converted from the two-dimensional feature information output by the second CBAM network layer
  • j is a positive integer less than or equal to N.
  • the image generation network includes N+1 feature generation network layers and target convolutional network layers; the input of the first feature generation network layer is the output of the first transformation sub-network, and the mth The input of the feature generation network layer includes the output of the mth transformation subnetwork and the output of the m-1th transformation subnetwork; the input of the target convolutional network layer is the last transformation subnetwork The output of the network, the output of the target convolutional network layer is the target three-dimensional reconstructed image; wherein, m is any positive integer greater than 1 and less than or equal to N+1.
  • the method for three-dimensional image reconstruction is applied to three-dimensional reconstruction of X-ray images, and before the acquisition of two-dimensional images, it also includes:
  • the feature extraction network, dimension transformation network and image generation network are trained to obtain the trained feature extraction network, the trained dimension transformation network and the trained Image generation network.
  • the second aspect of the embodiments of the present application provides a three-dimensional image reconstruction device, including:
  • an acquisition unit configured to acquire a two-dimensional image
  • a feature extraction unit configured to input the two-dimensional image into a trained feature extraction network for processing to obtain two-dimensional feature information corresponding to the two-dimensional image;
  • a dimension transformation unit configured to input the two-dimensional feature information into a trained dimension transformation network for processing to obtain three-dimensional feature information
  • a reconstruction unit configured to input the 3D feature information into a trained image generation network for processing to obtain a target 3D reconstruction image; wherein, the image generation network includes a 3D point-by-point convolution layer and a 3D deconvolution layer in sequence.
  • the third aspect of the embodiments of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor.
  • the processor executes the computer program
  • the electronic device is made to implement the steps of the method for three-dimensional image reconstruction.
  • the fourth aspect of the embodiments of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the electronic device realizes the image three-dimensional Steps of the reconstruction method.
  • a fifth aspect of the embodiments of the present application provides a computer program product, which, when the computer program product is run on an electronic device, causes the electronic device to execute the image three-dimensional reconstruction method described in any one of the above first aspects.
  • the embodiment of the present application has the following beneficial effects: in the embodiment of the present application, after the two-dimensional image is acquired, the two-dimensional image is input into the trained feature extraction network for processing, and the corresponding two-dimensional image is obtained. Two-dimensional feature information; then input the two-dimensional feature information into the trained dimension transformation network for processing to obtain three-dimensional feature information; after that, input the three-dimensional feature information into the trained image generation network for processing, and the target three-dimensional reconstruction image can be obtained . Since the image generation network includes a three-dimensional deconvolution layer, accurate upsampling can be achieved through the three-dimensional deconvolution layer, which compensates for the loss of image resolution caused by the convolution operation downsampling process in the feature extraction network, and improves the image quality.
  • the image generation network also includes a three-dimensional pointwise convolution layer before the three-dimensional deconvolution layer, by adding the three-dimensional pointwise convolution layer, more details can be obtained in the generated image , at the same time, it can effectively reduce the number of redundant features input to the subsequent 3D deconvolution layer, reduce the calculation amount of the subsequent 3D deconvolution process, improve the calculation efficiency, and then improve the image reconstruction efficiency. Therefore, the three-dimensional image reconstruction method in the embodiment of the present application can improve the image reconstruction efficiency while ensuring the image reconstruction effect, and efficiently and accurately realize the three-dimensional image reconstruction.
  • FIG. 1 is a schematic diagram of an implementation flow of a method for three-dimensional image reconstruction provided in an embodiment of the present application
  • Fig. 2 is a schematic structural diagram of a feature generation network layer provided by an embodiment of the present application.
  • Fig. 3 is a schematic structural diagram of a CBAM network layer provided by an embodiment of the present application.
  • Fig. 4 is a schematic structural diagram of an ECA network layer provided by an embodiment of the present application.
  • Fig. 5 is an example diagram of the image three-dimensional reconstruction network provided by the embodiment of the present application.
  • FIG. 6 is a schematic diagram of a three-dimensional image reconstruction device provided by an embodiment of the present application.
  • Fig. 7 is a schematic diagram of an electronic device provided by an embodiment of the present application.
  • the term “if” may be construed as “when” or “once” or “in response to determining” or “in response to detecting” depending on the context .
  • the phrase “if determined” or “if [the described condition or event] is detected” may be construed, depending on the context, to mean “once determined” or “in response to the determination” or “once detected [the described condition or event] ]” or “in response to detection of [described condition or event]”.
  • image 3D reconstruction is usually based on convolutional neural network (Convolutional Neural Network) Networks, CNN).
  • CNN convolutional Neural Network
  • the convolutional neural network will cause a certain loss of resolution during the network sampling process, and the image reconstruction task has a large amount of calculation, the accuracy and efficiency of the current 3D image reconstruction are low.
  • the embodiment of the present application provides a three-dimensional image reconstruction method, device, electronic equipment and storage medium, after acquiring the two-dimensional image, input the two-dimensional image into the trained feature extraction network for processing, Obtain the two-dimensional feature information corresponding to the two-dimensional image; then input the two-dimensional feature information into the trained dimension transformation network for processing to obtain three-dimensional feature information; after that, input the three-dimensional feature information into the image generation network to perform three-dimensional point-by-point convolution
  • the target 3D reconstructed image can be obtained through product processing and 3D deconvolution processing.
  • the image generation network includes a three-dimensional deconvolution layer, accurate upsampling can be achieved through the three-dimensional deconvolution layer, which compensates for the loss of image resolution caused by the convolution operation downsampling process in the feature extraction network, and improves the image quality.
  • Reconstruction effect since the image generation network also includes a three-dimensional pointwise convolution layer before the three-dimensional deconvolution layer, by adding the three-dimensional pointwise convolution layer, more details can be obtained in the generated image , at the same time, it can effectively reduce the number of redundant features input to the subsequent 3D deconvolution layer, reduce the calculation amount of the subsequent 3D deconvolution process, improve the calculation efficiency, and then improve the image reconstruction efficiency. Therefore, the three-dimensional image reconstruction method in the embodiment of the present application can improve the image reconstruction efficiency while ensuring the image reconstruction effect, and efficiently and accurately realize the three-dimensional image reconstruction.
  • the image three-dimensional reconstruction method of the embodiment of the present application may be applied in a medical scene.
  • the patient's X-ray image can be taken, and based on the two-dimensional X-ray image, the three-dimensional image reconstruction method of the embodiment of the present application can be used to reconstruct the target three-dimensional reconstruction image.
  • the target three-dimensional reconstruction image can be a human body CT sequence images of tissue spatial information.
  • X-ray images refer to the images that can present different tissue parts of the human body generated by using X-rays (also known as Roentgen rays) to irradiate parts of the human body and using the principle that different tissues of the human body absorb X-rays differently;
  • X-ray images Compared with the CT scanning process, the acquisition process of the method requires less radiation to the patient, and the two-dimensional X-ray image actually contains hidden spatial information. Therefore, the three-dimensional reconstruction of the X-ray image is carried out by the image three-dimensional reconstruction method of the embodiment of the present application.
  • a target three-dimensional reconstructed image containing three-dimensional spatial information of human tissue can be efficiently and accurately reconstructed with less impact on the human body.
  • This method can be applied to the task of three-dimensional reconstruction of X-ray images of human spine, teeth, lungs, chest and various human tissues, which provides great convenience for doctors' work.
  • Fig. 1 shows a schematic flowchart of a method for three-dimensional reconstruction of an image provided by an embodiment of the present application.
  • the method for three-dimensional reconstruction of an image is applied to an electronic device, and the electronic device includes but is not limited to a computer, a tablet computer, an image capturing device, and the like.
  • the three-dimensional image reconstruction method shown in Figure 1 is described in detail as follows:
  • the two-dimensional image in the embodiment of the present application can represent information such as shadow, occlusion, and relative size through two-dimensional pixels, and reflect the relative space of the object to be photographed.
  • the two-dimensional image is an image obtained by projection radiography, and pixels in the two-dimensional image imply information about transparent volume structures other than opaque surfaces.
  • the two-dimensional image may be an X-ray image.
  • the electronic device in the embodiment of the present application can communicate with the image capturing device, and receive the two-dimensional image transmitted by the image capturing device after the image capturing device captures an object to generate a two-dimensional image.
  • the electronic device in the embodiment of the present application may directly be an image capture device, and after receiving a capture instruction, it starts to capture objects in a designated capture area to generate the two-dimensional image.
  • the image capturing device may be an X-ray machine.
  • the two-dimensional image is input into a trained feature extraction network for processing to obtain two-dimensional feature information corresponding to the two-dimensional image.
  • the electronic device includes a trained image three-dimensional reconstruction network
  • the image three-dimensional reconstruction network is a neural network
  • the trained image three-dimensional reconstruction network specifically includes a trained feature extraction network, a trained dimension transformation network and the trained image generation network in three parts.
  • the feature extraction network is specifically a deep convolutional network including a convolutional layer, and the feature information of the two-dimensional image can be extracted through the convolution operation of the convolutional layer.
  • the feature extraction network usually includes convolution processing of multiple network layers, and a downsampling process is included in the feature extraction, so that the finally extracted two-dimensional feature information contains deep feature information whose scale is smaller than the input scale.
  • the two-dimensional feature information is input into a trained dimension transformation network for processing to obtain three-dimensional feature information.
  • the dimension transformation network is specifically a neural network including a dimension transformation function and an activation function.
  • the two-dimensional feature information extracted by the feature extraction network includes two-dimensional feature information of size (C, H, W) (where C represents channel information, H represents image height information, and W represents image width information)
  • the two-dimensional feature information is processed by the dimension conversion function, and the initial three-dimensional feature information with dimensions (C, 1, H, W) can be obtained.
  • the initial three-dimensional feature information (C, 1, H, W) can be further processed through a deconvolution layer with a kernel size of D ⁇ 1 ⁇ 1, an activation function, and a normalization function (such as a batch normalization function) to obtain the The three-dimensional feature information corresponding to the two-dimensional feature information.
  • D is preset depth information.
  • the size of D may be equal to the current number of channels, or may be determined according to the current number of network layers.
  • the aforementioned activation function can be a linear rectification function (Linear rectification function, ReLU).
  • the 3D feature information is input into a trained image generation network for processing to obtain a target 3D reconstructed image; wherein the image generation network includes a 3D pointwise convolution layer and a 3D deconvolution layer in sequence.
  • the image generation network is a deep convolutional network capable of reconstructing based on the 3D feature information of the image to obtain a 3D reconstructed image of the target map including the 3D information of the image.
  • the image generation network specifically includes a three-dimensional pointwise convolution layer and a three-dimensional deconvolution layer.
  • the three-dimensional feature information is input into the trained image generation network, and the single-point feature extraction is performed through the three-dimensional point-by-point convolution layer.
  • the convolution dimension is reduced; after that, the feature information output by the 3D point-by-point convolution layer is input into the 3D deconvolution layer for processing.
  • the information is converted into high-scale feature information for output. After the high-scale feature information is obtained, based on the high-scale feature information and the convolution operation, a target 3D reconstructed image containing 3D feature information is generated.
  • the target 3D reconstructed image is a 3D image
  • the target 3D reconstructed image is a sequence of multiple images containing 3D volume information of the object.
  • the target three-dimensional reconstructed image may be a CT sequence image including volume information of human tissue.
  • the image generation network includes a three-dimensional deconvolution layer, accurate upsampling can be achieved through the three-dimensional deconvolution layer, which compensates for the loss of image resolution caused by the convolution operation downsampling process in the feature extraction network, and improves the image quality.
  • the image generation network also includes a three-dimensional pointwise convolution layer before the three-dimensional deconvolution layer, by adding the three-dimensional pointwise convolution layer, more details can be obtained in the generated image , at the same time, it can effectively reduce the number of redundant features input to the subsequent 3D deconvolution layer, reduce the calculation amount of the subsequent 3D deconvolution process, improve the calculation efficiency, and then improve the image reconstruction efficiency. Therefore, the three-dimensional image reconstruction method in the embodiment of the present application can improve the image reconstruction efficiency while ensuring the image reconstruction effect, and efficiently and accurately realize the three-dimensional image reconstruction.
  • the feature extraction network includes a residual block and/or an attention module.
  • the feature extraction network described above may include a residual block.
  • the network structure of the feature extraction network may be a ResNet structure (a Residual Network structure), such as a ResNet34 structure.
  • the input of the feature extraction network at this time can be a two-dimensional image with a size of 128 ⁇ 128.
  • the first layer of the network is composed of a convolutional layer with a kernel size of 7 ⁇ 7 and a step size of 2.
  • the second to fifth layers It consists of 4 residual blocks, and it contains two convolutional layers with a convolution kernel of 3 ⁇ 3.
  • the number of channels of the convolutional layers in each residual block is kept the same to ensure that the shortcut path and the residual path can maintain the same size during the element-wise addition operation.
  • the intermediate feature information can be directly connected to the final residual block through a long connection, so that the subsequent network layer can learn the previous intermediate feature information, and reduce the information of the feature information when it passes through each layer of convolutional layers.
  • the problem of loss and loss improves the accuracy of feature extraction and also solves the problem of gradient disappearance in the training process.
  • the feature extraction network described above further includes an attention module.
  • the two-dimensional image is first processed by the attention module to obtain the attention feature information of the two-dimensional image; then the two-dimensional image is convoluted based on the attention feature information processing to obtain the two-dimensional feature information corresponding to the two-dimensional image.
  • the attention module the extraction of effective features in the two-dimensional image can be strengthened, the extraction of redundant useless information can be reduced, the efficiency of image reconstruction can be improved, and the redundant features of the finally reconstructed image can be reduced.
  • the 3D deconvolution layer of the image generation network includes a first preset number of 3D convolution kernels of different scales.
  • the image generation network specifically includes at least one feature generation network layer
  • the feature generation network sequentially includes a three-dimensional pointwise convolution layer and a three-dimensional deconvolution layer
  • the three-dimensional deconvolution layer includes the first pre- Set the number of three-dimensional convolution kernels of different scales.
  • the first preset number is greater than 1, and may be 2, 3, 4, etc., and is set according to actual needs.
  • the feature generation network layer is shown in FIG. 2 .
  • the 3D feature information generated based on the dimension transformation network is input into a 3D point-by-point convolution layer with a size of 1 ⁇ 1 ⁇ 1, after being processed by the activation layer ReLu and the normalization layer Batch Normalization, it is input into the 3D deconvolution layer.
  • the product layer contains two branches, where the upper branch contains deconvolution with a convolution kernel of 5 ⁇ 5 ⁇ 5, and the lower branch contains deconvolution with a convolution kernel of 3 ⁇ 3 ⁇ 3. These two deconvolution After the layer, the activation layer ReLu and the normalization layer Batch Normalization are also connected.
  • each branch contains convolution kernels of different scales, so that information of different scales can be generated during deconvolution, and richer and more accurate Detailed feature information improves the accuracy of 3D image reconstruction.
  • the inputting the two-dimensional image into a trained feature extraction network for processing to obtain two-dimensional feature information corresponding to the two-dimensional image includes:
  • the two-dimensional image is input into a trained feature extraction network for processing to obtain two-dimensional feature information of at least two different scales; wherein, the feature extraction network includes at least two network layers connected in sequence, and the different scales of the The two-dimensional feature information corresponds to feature information output by different network layers;
  • the input of the two-dimensional feature information into the trained dimension transformation network for processing to obtain three-dimensional feature information includes:
  • Each of the two-dimensional feature information is input into the trained dimension transformation network for processing, and each three-dimensional feature information corresponding to each of the two-dimensional feature information is obtained.
  • the feature extraction network specifically includes at least two network layers, and each network layer has a convolution operation. After the two-dimensional image is input into the trained feature extraction network, it undergoes convolution operations and down-sampling operations of different network layers in sequence to obtain two-dimensional feature information of different scales from large to small, that is, two-dimensional feature information of different scales are the feature information output by different network layers.
  • the two-dimensional feature information input by the shallow network has a large scale and strong semantic information representation ability, but lacks spatial and geometric detail information; the two-dimensional feature information output by the deep network has a small scale and has Strong geometric detail information representation ability, but weak semantic representation ability.
  • each two-dimensional feature information is input into the trained dimension transformation network to perform dimension conversion processing respectively, and each three-dimensional feature information corresponding to each of the two-dimensional feature information is obtained.
  • the feature information output by different network layers of the feature extraction network can be obtained, two-dimensional feature information of different scales can be obtained, and the three-dimensional feature information corresponding to these two-dimensional feature information can be generated through the dimension transformation network, so that Afterwards, these three-dimensional feature information is input into the image generation network for processing, which can fuse the three-dimensional features of different scales of the image, so that the semantic information and deep geometric details of the image can be accurately reconstructed, so the final reconstructed image can be made more accurate. Clear and accurate.
  • the feature extraction network includes a first CBAM network layer, N ECA network layers, and a second CBAM network layer connected in sequence, and the ECA network layer includes a residual block;
  • the two-dimensional feature information of different scales includes N two-dimensional feature information output by the N ECA network layers and two-dimensional feature information output by the second CBAM network layer; wherein, the first CBAM network layer The input is the two-dimensional image, the input of the first ECA network layer is the output of the first CBAM network layer, and the input of the i-th ECA network layer is the i-1th ECA network layer , the input of the second CBAM network layer is the output of the last ECA network layer; N is a positive integer greater than 1, and i is any positive integer greater than 1 and less than or equal to N.
  • the feature extraction network in the embodiment of the present application specifically includes a convolutional block attention module (Convolutional Block Attention module, CBAM) network layer and efficient channel attention module (Efficient channel Attention module) network layer, these two attention modules are lightweight attention modules, which enable efficient and accurate feature extraction of two-dimensional images without increasing network complexity.
  • CBAM Convolutional Block Attention module
  • Efficient channel Attention module efficient channel Attention module
  • Figure 3 the structure of the CBAM network layer of the embodiment of the present application is shown in Figure 3, which includes a channel attention module and a spatial attention module, so that the two-dimensional image can be extracted more accurately and effectively based on the channel attention and spatial attention. characteristic information.
  • the structure of the ECA network layer in the embodiment of the present application is shown in Figure 4, with two residual blocks at the beginning and the end, and two convolution layers with convolution kernels of 1 ⁇ 1 ⁇ C in the middle (where C is the channel number).
  • the ECA is a local cross-channel interaction module that does not reduce the feature dimension, and obtains local cross-channel interaction information by combining each channel with its K adjacent channels. Where K is a positive integer, specifically through the preset function of the channel Adaptive OK.
  • the feature extraction network in the embodiment of the present application is specifically improved on the basis of the ResNet34 network structure.
  • the first layer and the last layer of the feature extraction network are both CBAM network layers (in order to show the difference, they are respectively referred to as the first The CBAM network layer and the second CBAM network layer) can improve the feature adaptive extraction ability under the premise of ensuring that the overall network structure is not affected, and the scale of input features and output features is consistent with the scale of the original ResNet34 network.
  • the middle network layer may include a second preset number of ECA network layers to efficiently obtain local cross-channel interaction information without reducing feature dimension and network complexity.
  • the feature extraction network in the embodiment of the present application specifically includes a first CBAM network layer, N ECA network layers, and a second CBAM network layer in sequence.
  • N is a positive integer greater than 1, such as 2, 3, 4, etc.
  • the two-dimensional image input to the feature extraction network is used as the input of the first CBAM network layer, and the output of the first CBAM network layer is used as the initial feature information X0. Since the first CBAM network layer is the first network layer, the feature information output by the first CBAM network layer is relatively shallow and not enough to effectively represent the two-dimensional image features. At this time, it is not used as two-dimensional feature information. Instead, it is passed to the next network layer for processing, that is, it is input to the first ECA network layer (such as ECA network layer 1 shown in Figure 5) for processing.
  • ECA network layer such as ECA network layer 1 shown in Figure 5
  • the input of the first ECA network layer is the output of the first CBAM network layer connected before the first ECA network layer.
  • the ECA network layer after the first ECA network layer that is, the i-th ECA network layer
  • its input is the output of the i-1th ECA network layer.
  • one branch of its output is input to the next network layer (that is, the next ECA network or the second CBAM network layer), and the information of the other branch of its output is one Two-dimensional feature information. That is, N ECA network layers respectively output N two-dimensional feature information correspondingly.
  • the last network layer of the feature extraction network that is, the second CBAM network layer
  • its input is the feature information output by the last ECA network layer among the N ECA network layers
  • its output is the last two-dimensional feature information with the smallest scale.
  • the two-dimensional feature information of different scales finally output by the feature extraction network includes the two-dimensional feature information output by each ECA network layer and the two-dimensional feature information output by the second CBAM network, a total of N+1 two-dimensional features information.
  • the feature extraction network is composed of the CBAM network layer and the ECA network layer in an orderly manner.
  • the processing of the CBAM network layer and the ECA network layer it is possible to reduce the weight of the network and reduce the amount of calculation. Effectively realize the feature extraction of two-dimensional images, thereby improving the efficiency and accuracy of image three-dimensional reconstruction.
  • the dimension transformation network includes N+1 transformation sub-networks
  • the input of the j-th transformation sub-network is the output of the j-th ECA network layer, and the output of the j-th transformation sub-network
  • the input of the last said transformation sub-network is the output of the second CBAM network layer
  • the input of the last said transformation sub-network The output is the three-dimensional feature information converted from the two-dimensional feature information output by the second CBAM network layer
  • j is a positive integer less than or equal to N.
  • the dimension transformation network includes N+1 transformation sub-networks, and each transformation sub-network corresponds to dimensioning a two-dimensional feature information. Conversion processing.
  • each transformation sub-network includes the dimension transformation function described in step S103 above, a deconvolution layer with a kernel size of D ⁇ 1 ⁇ 1, an activation function and a normalization function.
  • any one of the first N transformation sub-networks that is, the j-th transformation sub-network, its input is the output of the j-th ECA network layer, and its output is the binary output of its corresponding j-th ECA network layer.
  • the three-dimensional feature information obtained by dimensionally transforming the three-dimensional feature information.
  • the last transformation subnetwork that is, the N+1th transformation subnetwork
  • its input is the output of the second CBAM network layer
  • its output is the dimension conversion of the two-dimensional feature information output by its corresponding second CBAM network layer
  • the obtained three-dimensional feature information is obtained three-dimensional feature information.
  • each transformation sub-network in the dimension transformation network processes each two-dimensional characteristic information respectively, and can accurately generate each three-dimensional characteristic information corresponding to each two-dimensional characteristic information one-to-one, thereby improving the image three-dimensional reconstruction. accuracy.
  • the image generation network includes N+1 feature generation network layers and target convolutional network layers; the input of the first feature generation network layer is the output of the last transformation sub-network, and the mth The input of the feature generation network layer includes the output of the penultimate mth transformation subnetwork and the output of the m-1th transformation subnetwork; the input of the target convolutional network layer is the last transformation subnetwork The output of the network, the output of the target convolutional network layer is the target three-dimensional reconstructed image; wherein, m is any positive integer greater than 1 and less than or equal to N+1.
  • the image generation network in the embodiment of the present application specifically includes a feature generation network layer for upsampling and fusing 3D feature information of images, and a target convolutional network layer for finally reconstructing target 3D feature information to obtain a target 3D reconstructed image.
  • the number of feature generation network layers is consistent with the number of three-dimensional feature information, which is N+1.
  • the scales of the N+1 two-dimensional feature information output by the feature extraction network gradually become smaller from to , correspondingly, transformation subnetwork 1 ⁇ transformation subnetwork
  • the scales are arranged in order from large to small.
  • the input of the first feature generation network layer is directly the smallest-scale three-dimensional feature information Y N+1 , which corresponds to the last transformation subnetwork (that is, transformation subnetwork N+1) Output.
  • each feature generation network layer after the first feature generation network layer, that is, the mth feature generation network layer its input contains two branches, and one input is the penultimate mth transformation subnetwork (that is, the positive number The output of N+1-m transformation subnetworks) and the other input is the output of the m-1th feature generation network layer.
  • the 3D feature information obtained by upsampling the 3D feature information of the previous scale and the 3D feature information of the current scale are input into the feature generation network layer for 3D point-by-point convolution and 3D deconvolution processing to achieve two
  • the fusion of three-dimensional feature information of three scales finally obtains the target three-dimensional feature information.
  • the three-dimensional feature information of the target is a fusion of the three-dimensional feature information of different scales.
  • the target 3D feature information can fuse the shallow semantic information and deep geometric detail information of the image.
  • the 3D feature information of the target is generated, the 3D feature information of the target is input to the target convolutional network layer for processing, and a 3D reconstructed image of the target with clear details and clear boundaries can be obtained in the same scale as the original 2D image.
  • the target 3D feature can be accurately generated information, so that a clear three-dimensional reconstructed image of the target can be accurately obtained based on the three-dimensional feature information of the target.
  • the method for three-dimensional image reconstruction is applied to three-dimensional reconstruction of X-ray images, and before the acquisition of two-dimensional images, it also includes:
  • the feature extraction network, dimension transformation network and image generation network are trained to obtain the trained feature extraction network, the trained dimension transformation network and the trained Image generation network.
  • the neural network composed of the feature extraction network, the dimension transformation network and the image generation network may be called an image three-dimensional reconstruction network.
  • the image three-dimensional reconstruction method in the embodiment of the present application is specifically applied to the three-dimensional reconstruction of X-ray images, that is, when the two-dimensional image in the embodiment of the present application is an X-ray image, then correspondingly, the image three-dimensional reconstruction network needs to be based on the X-ray sample image Get trained.
  • the X-ray sample image can be constructed based on the CT image data set and the style transfer algorithm, and then the training of the image three-dimensional reconstruction network can be accurately realized based on the constructed X-ray sample image.
  • a preset number of CT sample images can be acquired from a preset CT image data set, and the preset number can be determined according to actual training requirements, for example, 1000.
  • the preset style transfer algorithm (such as the style transfer neural network obtained through pre-training) can be used to perform style transfer processing on each two-dimensional projection image , so that each two-dimensional projection image can imitate the characteristics of the original X-ray image, and generate a preset number of X-ray sample images.
  • the preset number of X-ray sample images can be input into the image three-dimensional reconstruction network, and the feature extraction network, dimension transformation network and image generation network are trained end-to-end. Finally, a trained image three-dimensional reconstruction network is obtained, that is, a trained feature extraction network, a trained dimension transformation network and a trained image generation network are obtained.
  • the feature extraction network and dimension transformation can be accurately performed based on these X-ray sample images.
  • Network and image generation network are trained, and finally the trained network is accurately generated, so that the subsequent three-dimensional image reconstruction can be realized efficiently and accurately based on these networks.
  • the feature extraction network includes the first CBAM network layer, the 3-layer ECA network layer and the second CBAM network layer connected in sequence.
  • the initial feature information X0 is obtained.
  • the initial characteristic information X0 is used as the input of the first ECA network layer, namely ECA network layer 1.
  • the two-dimensional feature information X1 obtained after the ECA network layer 1 processes the initial feature information X0 is divided into two branches, one branch is input to the corresponding transformation sub-network 1 in the dimension transformation network for dimension transformation processing, and the other branch is input To the next ECA network layer, that is, ECA network layer 2 for processing.
  • the obtained two-dimensional feature information X2 is divided into two branches, and one branch is input to the corresponding transformation sub-network 2 in the dimension transformation network for dimension transformation processing, the other branch is input to the next ECA network layer, that is, ECA network layer 3 for processing;
  • ECA network layer 3 processes the two-dimensional feature information input by ECA network layer 2
  • the obtained two-dimensional feature information X3 is divided into two branches , one branch is input to the corresponding transformation sub-network 3 in the dimension transformation network for dimension transformation processing, and the other branch is input to the next network layer, that is, the second CBAM network layer for processing;
  • the second CBAM network layer inputs to the ECA network layer 3
  • the two-dimensional feature information is processed to obtain the two-dimensional feature information X4, which is directly input to the transformation sub-network 4 in the dimension transformation network for processing.
  • the scales are sequentially changed from large to small; correspondingly, the scales of the four three-dimensional feature information output by the dimension transformation network are also sequentially changed from large to small.
  • the last transformation sub-network that is, the smallest-scale three-dimensional feature information Y4 output by the transformation sub-network 4 is first input into the first feature generation network layer, that is, the feature generation network layer 1 performs three-dimensional Point-by-point convolution processing and deconvolution processing to obtain scale-amplified three-dimensional feature information Y4'; input the Y4' and three-dimensional feature information Y3 into the feature generation network layer 2 for processing, and obtain scale-amplified three-dimensional feature information Y3'; Input the Y3' and the three-dimensional feature information Y2 into the feature generation network layer 3 for processing, and obtain the scale-amplified three-dimensional feature information Y2'; input the Y2' and the three-dimensional feature information Y1 into the feature generation network layer 4 for processing, then
  • the target three-dimensional feature information whose scale size matches the scale size of the two-dimensional feature information X0 output by the first CBAM
  • the image 3D reconstruction network based on the lightweight attention module, residual block and multi-scale feature fusion, while greatly reducing the required computing resources, the accuracy of the reconstructed image is guaranteed, and at the same time, the 3D step-by-step reconstruction of the feature-based network can be achieved.
  • the point convolution layer and the 3D deconvolution layer can accurately and efficiently improve the pixel loss in the reconstruction process, so that the 3D reconstruction of the image can be realized efficiently and accurately, and a clear and accurate 3D reconstruction image of the target can be obtained.
  • Figure 3 shows a schematic structural diagram of a three-dimensional image reconstruction device provided by the embodiment of the present application. For the convenience of description, only the parts related to the embodiment of the present application are shown:
  • the image three-dimensional reconstruction device includes: an acquisition unit 61 , a feature extraction unit 62 , a dimension transformation unit 63 , and a reconstruction unit 64 . in:
  • An acquiring unit 61 configured to acquire a two-dimensional image.
  • the feature extraction unit 62 is configured to input the two-dimensional image into a trained feature extraction network for processing to obtain two-dimensional feature information corresponding to the two-dimensional image; the feature extraction network includes a convolutional layer.
  • the dimension transformation unit 63 is configured to input the two-dimensional feature information into the trained dimension transformation network for processing to obtain three-dimensional feature information.
  • the reconstruction unit 64 is configured to input the 3D feature information into a trained image generation network for processing to obtain a target 3D reconstructed image; wherein the image generation network includes a 3D pointwise convolution layer and a 3D deconvolution layer in sequence.
  • the 3D deconvolution layer of the image generation network includes a first preset number of 3D convolution kernels of different scales.
  • the feature extraction unit 62 is specifically configured to input the two-dimensional image into a trained feature extraction network for processing to obtain at least two two-dimensional feature information of different scales; wherein the feature extraction network includes At least two network layers connected in sequence, the two-dimensional feature information of different scales correspond to feature information output by different network layers;
  • the dimension transformation unit 63 is specifically configured to input each of the two-dimensional feature information into a trained dimension transformation network for processing, and obtain each of the three-dimensional feature information corresponding to each of the two-dimensional feature information.
  • the feature extraction network includes a first CBAM network layer, N ECA network layers, and a second CBAM network layer connected in sequence, and the ECA network layer includes a residual block;
  • the two-dimensional feature information of different scales includes N two-dimensional feature information output by the N ECA network layers and two-dimensional feature information output by the second CBAM network layer; wherein, the first CBAM network layer The input is the two-dimensional image, the input of the first ECA network layer is the output of the first CBAM network layer, and the input of the i-th ECA network layer is the i-1th ECA network layer , the input of the second CBAM network layer is the output of the last ECA network layer; N is a positive integer greater than 1, and i is any positive integer greater than 1 and less than or equal to N.
  • the dimension transformation network includes N+1 transformation sub-networks
  • the input of the j-th transformation sub-network is the output of the j-th ECA network layer, and the output of the j-th transformation sub-network
  • the input of the last said transformation sub-network is the output of the second CBAM network layer
  • the input of the last said transformation sub-network The output is the three-dimensional feature information converted from the two-dimensional feature information output by the second CBAM network layer
  • j is a positive integer less than or equal to N.
  • the image generation network includes N+1 feature generation network layers and target convolutional network layers; the input of the first feature generation network layer is the output of the first transformation sub-network, and the mth The input of the feature generation network layer includes the output of the mth transformation subnetwork and the output of the m-1th transformation subnetwork; the input of the target convolutional network layer is the last transformation subnetwork The output of the network, the output of the target convolutional network layer is the target three-dimensional reconstructed image; wherein, m is any positive integer greater than 1 and less than or equal to N+1.
  • the image three-dimensional reconstruction method is applied to the three-dimensional reconstruction of X-ray images, and the image three-dimensional reconstruction device further includes:
  • the training unit is used to obtain a preset number of CT sample images from a preset CT image data set; obtain a preset number of two-dimensional projection images according to the preset number of CT sample images and a DDR projection algorithm; according to the A preset number of two-dimensional projection images and a preset style transfer algorithm are used to generate a preset number of X-ray sample images; based on the preset number of X-ray sample images, the feature extraction network, dimension transformation network and image generation network are training to obtain the trained feature extraction network, the trained dimension transformation network and the trained image generation network.
  • Fig. 7 is a schematic diagram of an electronic device provided by an embodiment of the present application.
  • the electronic device 7 of this embodiment includes: a processor 70 , a memory 71 , and a computer program 72 stored in the memory 71 and operable on the processor 70 , such as an image three-dimensional reconstruction program.
  • the processor 70 executes the computer program 72, the steps in the above-mentioned embodiments of the three-dimensional image reconstruction method are implemented, such as steps S101 to S104 shown in FIG. 1 .
  • the processor 70 executes the computer program 72, it realizes the functions of the modules/units in the above-mentioned device embodiments, such as the functions of the acquisition unit 61 to the reconstruction unit 64 shown in FIG. 6 .
  • the computer program 72 can be divided into one or more modules/units, and the one or more modules/units are stored in the memory 71 and executed by the processor 70 to complete this application.
  • the one or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program 72 in the electronic device 7 .
  • the electronic device 7 may be a computing device such as a desktop computer, a notebook, a palmtop computer, or an X-ray machine.
  • the electronic device may include, but not limited to, a processor 70 and a memory 71 .
  • FIG. 7 is only an example of the electronic device 7, and does not constitute a limitation to the electronic device 7. It may include more or less components than shown in the figure, or combine certain components, or different components. , for example, the electronic device may also include an input and output device, a network access device, a bus, and the like.
  • the so-called processor 70 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the memory 71 may be an internal storage unit of the electronic device 7 , such as a hard disk or a memory of the electronic device 7 .
  • the memory 71 can also be an external storage device of the electronic device 7, such as a plug-in hard disk equipped on the electronic device 7, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc.
  • the memory 71 may also include both an internal storage unit of the electronic device 7 and an external storage device.
  • the memory 71 is used to store the computer program and other programs and data required by the electronic device.
  • the memory 71 can also be used to temporarily store data that has been output or will be output.
  • the disclosed device/electronic equipment and method can be implemented in other ways.
  • the device/electronic device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated module/unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments in the present application can also be completed by instructing related hardware through computer programs.
  • the computer programs can be stored in a computer-readable storage medium, and the computer When the program is executed by the processor, the steps in the above-mentioned various method embodiments can be realized.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form.
  • the computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, computer-readable media Excludes electrical carrier signals and telecommunication signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Software Systems (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

本申请适用于图像处理技术领域，提供了图像三维重建方法、装置、电子设备及存储介质，包括：获取二维图像；将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息；将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息；将所述三维特征信息输入已训练的图像生成网络进行处理，得到目标三维重建图像；其中，所述图像生成网络依次包括三维逐点卷积层和三维反卷积层。本申请实施例能够高效准确地实现图像的三维重建。

Description

图像三维重建方法、装置、电子设备及存储介质 技术领域
本申请属于图像处理技术领域，尤其涉及一种图像三维重建方法、装置、电子设备及存储介质。
 背景技术
图像三维重建，指的是从单个或者多个二维图像中推断出图像中物体的三维形状，以重建得到包含物体三维信息的图像。随着计算机视觉的发展，图像三维重建算法已被广泛应用于许多领域，例如机器人导航、三维建模、物体识别、场景识别、医学诊断等。然而，目前缺乏一种能够保证效率和准确性的图像三维重建方法。
 技术问题
有鉴于此，本申请实施例提供了图像三维重建方法、装置、电子设备及存储介质，以解决现有技术中如何高效准确地实现图像的三维重建的问题。
 技术解决方案
本申请实施例的第一方面提供了一种图像三维重建方法，包括：
获取二维图像；
将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息；
将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息；
将所述三维特征信息输入已训练的图像生成网络进行处理，得到目标三维重建图像；其中，所述图像生成网络依次包括三维逐点卷积层和三维反卷积层。
可选地，所述图像生成网络的所述三维反卷积层中包含了第一预设数目的不同尺度的三维卷积核。
可选地，所述将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息，包括：
将所述二维图像输入已训练的特征提取网络进行处理，得到至少两个不同尺度的二维特征信息；其中，所述特征提取网络包括依次连接的至少两个网络层，不同尺度的所述二维特征信息分别对应为不同的所述网络层输出的特征信息；
对应地，所述将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息，包括：
将各个所述二维特征信息输入已训练的维度变换网络进行处理，得到与各个所述二维特征信息分别对应的各个三维特征信息。
可选地，所述特征提取网络包括依次连接的第一CBAM网络层、N个ECA网络层、第二CBAM网络层，所述ECA网络层包含残差块；
所述不同尺度的二维特征信息包括所述N个ECA网络层输出的N个二维特征信息以及所述第二CBAM网络层输出的二维特征信息；其中，所述第一CBAM网络层的输入为所述二维图像，第一个所述ECA网络层的输入为所述第一CBAM网络层的输出，第i个所述ECA网络层的输入为第i-1个所述ECA网络层的输出，所述第二CBAM网络层的输入为最后一个所述ECA网络层的输出；N为大于1的正整数，i为大于1且小于或者等于N的任一正整数。
可选地，所述维度变换网络包括N+1个变换子网络，第j个所述变换子网络的输入为第j个所述ECA网络层的输出，第j个所述变换子网络的输出为第j个所述ECA网络层输出的二维特征信息转换成的三维特征信息；最后一个所述变换子网络的输入为所述第二CBAM网络层的输出，最后一个所述变换子网络的输出为所述第二CBAM网络层输出的二维特征信息转换成的三维特征信息；其中，j为小于或者等于N的正整数。
可选地，所述图像生成网络包括N+1个特征生成网络层和目标卷积网络层；第一个所述特征生成网络层的输入为第一个所述变换子网络的输出，第m个所述特征生成网络层的输入包括第m个所述变换子网络的输出以及第m-1个所述变换子网络的输出；所述目标卷积网络层的输入为最后一个所述变换子网络的输出，所述目标卷积网络层的输出为所述目标三维重建图像；其中，m为大于1，并且小于或者等于N+1的任一正整数。
可选地，所述图像三维重建方法应用于X光图像的三维重建，在所述获取二维图像之前，还包括：
从预设的CT图像数据集中获取预设数量的CT样本图像；
根据所述预设数量的CT样本图像和DDR投影算法，得到预设数量的二维投影图像；
根据所述预设数量的二维投影图像和预设的风格迁移算法，生成预设数量的X光样本图像；
基于所述预设数量的X光样本图像对特征提取网络、维度变换网络和图像生成网络进行训练，得到所述已训练的特征提取网络、所述已训练的维度变换网络和所述已训练的图像生成网络。
本申请实施例的第二方面提供了一种图像三维重建装置，包括：
获取单元，用于获取二维图像；
特征提取单元，用于将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息；
维度变换单元，用于将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息；
重建单元，用于将所述三维特征信息输入已训练的图像生成网络进行处理，得到目标三维重建图像；其中，所述图像生成网络依次包括三维逐点卷积层和三维反卷积层。
本申请实施例的第三方面提供了一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，当所述处理器执行所述计算机程序时，使得电子设备实现如所述图像三维重建方法的步骤。
本申请实施例的第四方面提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，当所述计算机程序被处理器执行时，使得电子设备实现如所述图像三维重建方法的步骤。
本申请实施例的第五方面提供了一种计算机程序产品，当计算机程序产品在电子设备上运行时，使得电子设备执行上述第一方面中任一项所述的图像三维重建方法。
 有益效果
本申请实施例与现有技术相比存在的有益效果是：本申请实施例中，在获取二维图像后，将该二维图像输入已训练的特征提取网络进行处理，得到二维图像对应该二维特征信息；再将该二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息；之后，将该三维特征信息输入已训练的图像生成网络进行处理，可以得到目标三维重建图像。由于该图像生成网络包括了三维反卷积层，通过该三维反卷积层可以实现准确的上采样，弥补原来在特征提取网络中由于卷积操作下采样过程造成的图像分辨率损失，提升图像重建效果；并且，由于该图像生成网络在该三维反卷积层之前，还包括了三维逐点卷积层，通过添加该三维逐点卷积层，能够在生成的图像中获得更多的细节，同时可以有效地减少输入后续三维反卷积层的冗余特征数量，减少后续三维反卷积过程的计算量，提高计算效率，进而提高图像重建效率。因此，本申请实施例的图像三维重建方法能够在保证图像重建效果的同时，提高图像重建效率，高效准确地实现图像三维重建。
 附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍。
图1是本申请实施例提供的一种图像三维重建方法的实现流程示意图；
图2是本申请实施例提供的一种特征生成网络层的结构示意图；
图3是本申请实施例提供的一种CBAM网络层的结构示意图；
图4是本申请实施例提供的一种ECA网络层的结构示意图；
图5是本申请实施例提供的图像三维重建网络的示例图；
图6是本申请实施例提供的一种图像三维重建装置的示意图；
图7是本申请实施例提供的电子设备的示意图。
 本发明的实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。
为了说明本申请所述的技术方案，下面通过具体实施例来进行说明。
应当理解，当在本说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解，在此本申请说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本申请。如在本申请说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。
还应当进一步理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
如在本说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。
另外，在本申请的描述中，术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
目前，图像三维重建通常是基于卷积神经网络（Convolutional Neural Networks, CNN）实现的。然而，由于卷积神经网络在网络采样过程会造成一定的分辨率损失，并且图像重建任务计算量较大，导致目前图像三维重建的准确性和效率较低。
为了解决上述的技术问题，本申请实施例提供了一种图像三维重建方法、装置、电子设备及存储介质，在获取二维图像后，将该二维图像输入已训练的特征提取网络进行处理，得到二维图像对应该二维特征信息；再将该二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息；之后，将该三维特征信息输入图像生成网络依次进行三维逐点卷积处理和三维反卷积处理，可以得到目标三维重建图像。
由于该图像生成网络包括了三维反卷积层，通过该三维反卷积层可以实现准确的上采样，弥补原来在特征提取网络中由于卷积操作下采样过程造成的图像分辨率损失，提升图像重建效果；并且，由于该图像生成网络在该三维反卷积层之前，还包括了三维逐点卷积层，通过添加该三维逐点卷积层，能够在生成的图像中获得更多的细节，同时可以有效地减少输入后续三维反卷积层的冗余特征数量，减少后续三维反卷积过程的计算量，提高计算效率，进而提高图像重建效率。因此，本申请实施例的图像三维重建方法能够在保证图像重建效果的同时，提高图像重建效率，高效准确地实现图像三维重建。
示例性地，本申请实施例的图像三维重建方法可以应用于医疗场景中。
在临床上，通过电子计算机断层扫描（Computed Tomography，CT）技术，能够获取患者组织的三维体积信息，然而这种方式在实践中，需要从大量不同角度位置进行投影，以保持可接受的分辨率并减轻断层扫描仪基于物理的伪影，可见，CT扫描技术具有高辐射、重建时间长且存在伪影的缺陷。除了CT扫描技术，新型的电子光学系统（Electro Optical System，EOS）为全身双平面X射线扫描和整个骨骼的三维重建提供了更好的选择，然而EOS成像的重建过程是基于统计形状模型（statistical shape model,SSM）实现的，因此获得的模型不是患者组织的完全反映，而是语义相似的虚拟模型。可见，通过CT技术或者EOS技术实现三维重建仍存在一定的缺陷。
为了解决该技术问题，可以通过拍摄患者的X光图像，基于该二维的X光图像，采用本申请实施例的图像三维重建方法重建得到目标三维重建图像，该目标三维重建图像可以为包含人体组织的空间信息的CT序列图像。其中，X光图像指的是通过X射线（也称为伦琴射线）照射人体部位，利用人体不同组织对X射线的吸收程度不同的原理，生成的能够呈现人体不同组织部位的影像；X光图像的采集过程相对于CT扫描过程，对患者的辐射较少，并且二维的X光图像实际包含了隐藏的空间信息，因此通过本申请实施例的图像三维重建方法对X光图像进行三维重建，能够在对人体影响较小的情况下，高效准确地重建得到包含人体组织三维空间信息的目标三维重建图像。该方法可以应用于人体脊柱、牙齿、肺部、胸部多种人体组织的X光图像的三维重建任务中，为医生的工作提供极大的便利。
实施例一：
图1示出了本申请实施例提供的一种图像三维重建方法的流程示意图，该图像三维重建方法应用于电子设备，该电子设备包括但不限于电脑、平板电脑、图像拍摄设备等。如图1所示的图像三维重建方法详述如下：
在S101中，获取二维图像。
本申请实施例中的二维图像能够通过二维的像素表现阴影、遮挡、相对大小等信息，体现被拍摄物体的相对空间。在一个实施例中，该二维图像为通过投影射线照相得到的图像，该二维图像中的像素隐含了对不透明表面以外的透明体积结构的信息。示例性地，该二维图像可以为X光图像。
在一个实施例中，本申请实施例的电子设备能够与图像拍摄设备进行通信，在图像拍摄设备拍摄物体生成二维图像后，接收该图像拍摄设备传输的二维图像。在另一个实施例中，本申请实施例的电子设备可以直接为图像拍摄设备，在接收到拍摄指令后，开始对指定拍摄区域的物体进行拍摄，生成该二维图像。示例性地，该图像拍摄设备可以为X光机。
在S102中，将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息。
本申请实施例中，电子设备包含了已训练的图像三维重建网络，该图像三维重建网络为神经网络，已训练的图像三维重建网络中，具体包含已训练的特征提取网络、已训练的维度变换网络和已训练的图像生成网络这三部分。
在获取到二维图像后，将该二维图像输入上述的已训练的特征提取网络进行特征提取处理，得到该二维图像对应的二维特征信息。该特征提取网络具体为包含卷积层的深度卷积网络，通过该卷积层的卷积操作，能够提取二维图像的特征信息。该特征提取网络通常包含多个网络层的卷积处理，在特征提取中包含了下采样过程，使得最终提取到的二维特征信息包含尺度小于输入尺度的深层特征信息。
在S103中，将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息。
本申请实施例中，维度变换网络具体为包含维度转换函数和激活函数的神经网络。示例性地，设经过特征提取网络提取得到的二维特征信息包含尺寸为（C,H,W）的二维特征信息（其中C代表通道信息，H代表图像高度信息，W代表图像宽度信息），将该二维特征信息通过维度转换函数的处理，可以得到尺寸为（C,1,H,W）的初始三维特征信息。该初始三维特征信息（C,1,H,W）进一步可以依次通过内核大小为D×1×1的反卷积层、激活函数、归一化函数（例如batch normalization函数）的处理，得到该二维特征信息对应的三维特征信息。其中，D为预设的深度信息，在一些实施例中，D的大小可以等于当前的通道数，或者可以根据当前的网络层数而确定。前述的激活函数可以为线性整流函数（Linear rectification function，ReLU）。
在S104中，将所述三维特征信息输入已训练的图像生成网络进行处理，得到目标三维重建图像；其中，所述图像生成网络依次包括三维逐点卷积层和三维反卷积层。
本申请实施例中，图像生成网络为能够基于图像的三维特征信息重建得到包含图像三维信息的目标图三维重建图像的深度卷积网络。该图像生成网络具体包括三维逐点卷积层和三维反卷积层。
在基于二维图像的二维特征信息得到对应的三维特征信息后，将该三维特征信息输入已训练的图像生成网络，先经过三维逐点卷积层进行单点的特征提取，在获取更多细节特征信息的同时，降低卷积维度；之后，再将三维逐点卷积层输出的特征信息输入三维反卷积层进行处理，该三维反卷积层能够实现上采样效果，将低尺度特征信息转换为高尺度特征信息进行输出。在得到高尺度特征信息后，再基于该高尺度特征信息和卷积操作，生成包含三维特征信息的目标三维重建图像。在一个实施例中，该目标三维重建图像为一张三维图像，在另一个实施例中，目标三维重建图像为多张包含物体的三维体积信息的图像序列。例如，设二维图像为X光图像，则该目标三维重建图像可以为包含人体组织体积信息的CT序列图像。
本申请实施例中，在获取二维图像后，将该二维图像输入已训练的特征提取网络进行处理，得到二维图像对应该二维特征信息；再将该二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息；之后，将该三维特征信息输入已训练的图像生成网络进行处理，可以得到目标三维重建图像。由于该图像生成网络包括了三维反卷积层，通过该三维反卷积层可以实现准确的上采样，弥补原来在特征提取网络中由于卷积操作下采样过程造成的图像分辨率损失，提升图像重建效果；并且，由于该图像生成网络在该三维反卷积层之前，还包括了三维逐点卷积层，通过添加该三维逐点卷积层，能够在生成的图像中获得更多的细节，同时可以有效地减少输入后续三维反卷积层的冗余特征数量，减少后续三维反卷积过程的计算量，提高计算效率，进而提高图像重建效率。因此，本申请实施例的图像三维重建方法能够在保证图像重建效果的同时，提高图像重建效率，高效准确地实现图像三维重建。
可选地，所述特征提取网络包含了残差块和/或注意力模块。
在一个实施例中，上述的特征提取网络可以包含残差块。示例性地，该特征提取网络的网络结构可以为ResNet结构（一种残差网络Residual Network结构），例如ResNet34结构。此时的特征提取网络的输入可以为大小为128×128的二维图像，网络的第一层由内核大小为7×7、步长为2的卷积层组成，第二层到第五层由4个残差块组成，并且其中包含两个卷积核为3×3的卷积层。在该网络结构中，每个残差块中卷积层的通道数保持相同，以确保快捷路径和残差路径在逐元素相加操作期间可以保持相同的大小。通过残差块的设置，使得中间特征信息能够通过长连接直接连接到最后的残差块，使得后面的网络层可以学习前面的中间特征信息，减少特征信息在经过各层卷积层时的信息丢失、损耗问题，提高特征提取的准确性，也解决了训练过程中的梯度消失问题。
在另一个实施例中，上述的特征提取网络还包含注意力模块。在将二维图像输入特征提取网络后，先通过注意力模块对该二维图像进行处理，得到该二维图像的注意力特征信息；之后基于该注意力特征信息对该二维图像进行卷积处理，得到该二维图像对应的二维特征信息。通过该注意力模块，能够强化二维图像中的有效特征的提取，减少冗余的无用信息的提取，提高图像重建效率，减少最终重建得到的图像的冗余特征。
可选地，所述图像生成网络的所述三维反卷积层中包含了第一预设数目的不同尺度的三维卷积核。
本申请实施例中，图像生成网络具体包含至少一个特征生成网络层，该特征生成网络依次包括一个三维逐点卷积层和一个三维反卷积层，该三维反卷积层包含了第一预设数目的不同尺度的三维卷积核。该第一预设数目大于1，可以为2、3、4等，根据实际需要而设定。示例性地，该特征生成网络层如图2所示。基于维度变换网络生成的三维特征信息输入尺寸为1×1×1的三维逐点卷积层、激活层ReLu、归一化层Batch Normalization的处理后，输入三维反卷积层，该三维反卷积层包含两个分支，其中，上面一个分支包含卷积核为5×5×5的反卷积，下面一个分支包含卷积核为3×3×3反卷积，这两个反卷积层后面也同样连接了激活层ReLu和归一化层Batch Normalization。
本申请实施例中，由于三维反卷积层包含了不同的分支结构，每个分支包含不同尺度大小的卷积核，从而能够在反卷积时生成不同尺度的信息，生成更丰富更准确的细节特征信息，提高三维图像重建的准确性。
可选地，所述将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息，包括：
将所述二维图像输入已训练的特征提取网络进行处理，得到至少两个不同尺度的二维特征信息；其中，所述特征提取网络包括依次连接的至少两个网络层，不同尺度的所述二维特征信息分别对应为不同的所述网络层输出的特征信息；
对应地，所述将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息，包括：
将各个所述二维特征信息输入已训练的维度变换网络进行处理，得到与各个所述二维特征信息分别对应的各个三维特征信息。
本申请实施例中，特征提取网络具体包含了至少两个网络层，每个网络层均存在卷积操作。二维图像在输入已训练的特征提取网络后，依次经过不同网络层的卷积操作及下采样操作，得到由大到小的不同尺度的二维特征信息，即，不同尺度的二维特征信息分别为不同的网络层输出的特征信息。在特征提取网络的网络层中，浅层网络输入出的二维特征信息尺度较大，语义信息表示能力较强，但缺乏空间几何细节信息；深度网络输出的二维特征信息尺度较小，具有较强的几何细节信息表示能力，但语义表示能力较弱。
在得到各个不同尺度的二维特征信息后，将各个二维特征信息输入已训练的维度变换网络分别进行维度转换处理，得到各个所述二维特征信息分别对应的各个三维特征信息。
本申请实施例中，由于能够通过获取特征提取网络不同网络层输出的特征信息，得到不同尺度的二维特征信息，并通过维度变换网络生成与这些二维特征信息分别对应的三维特征信息，使得之后将这些三维特征信息输入图像生成网络进行处理，能够融合图像不同尺度的三维特征，使得图像的语义信息和深层几何细节信息都能够准确地进行重构，因此能够使得最终得到的重构图像更加清晰准确。
可选地，所述特征提取网络包括依次连接的第一CBAM网络层、N个ECA网络层、第二CBAM网络层，所述ECA网络层包含残差块；
所述不同尺度的二维特征信息包括所述N个ECA网络层输出的N个二维特征信息以及所述第二CBAM网络层输出的二维特征信息；其中，所述第一CBAM网络层的输入为所述二维图像，第一个所述ECA网络层的输入为所述第一CBAM网络层的输出，第i个所述ECA网络层的输入为第i-1个所述ECA网络层的输出，所述第二CBAM网络层的输入为最后一个所述ECA网络层的输出；N为大于1的正整数，i为大于1且小于或者等于N的任一正整数。
本申请实施例中的特征提取网络具体包含了卷积块注意力模块（Convolutional Block Attention module，CBAM）网络层和高效通道注意力模块（Efficient channel Attention module）网络层，这两种注意力模块均为轻量级的注意力模块，使得在不增加网络复杂度的情况下，高效准确地实现二维图像的特征提取。
示例性地，本申请实施例的CBAM网络层的结构如图3所示，包含通道注意力模块和空间注意力模块，从而能够基于通道注意力和空间注意力更加准确有效地提取二维图像的特征信息。
示例性地，本申请实施例的ECA网络层的结构如图4所示，首尾为两个残差块，中间为两个卷积核为1×1×C的卷积层（其中C为通道数）。该ECA为一个不降低特征维数的局部跨通道交互模块，通过将每个通道与其相邻的K个通道结合，获得局部跨通道交互信息。其中K为正整数，具体可以通过关于通道的预设函数
Figure dest_path_image001
自适应确定。
本申请实施例中的特征提取网络的具体在ResNet34网络结构的基础上进行改进，该特征提取网络的第一层和最后一层均为CBAM网络层（为了以示区别，将其分别成为第一CBAM网络层和第二CBAM网络层）其能够在保证整体网络结构不受影响，输入特征和输出特征的尺度与原来的ResNet34网络的尺度保持一致的前提下，提高特征自适应提取能力。而中间的网络层可以包括第二预设数目个ECA网络层，在不降低特征维数和网络复杂度的前提下高效地获得局部跨通道交互信息。
本申请实施例的特征提取网络具体依次包含第一CBAM网络层、N个ECA网络层和第二CBAM网络层。其中，N为大于1的正整数，例如可以为2、3、4等。
在该特征提取网络中，输入特征提取网络的二维图像作为第一CBAM网络层的输入，该第一CBAM网络层的输出作为初始特征信息X0。由于该第一CBAM网络层为首层网络层，因此该第一CBAM网络层输出的特征信息为较为浅层的不足以有效表示二维图像特征的信息，此时不将其作为二维特征信息，而是将其继续传入下一网络层进行处理，即将其输入第一个ECA网络层（例如图5所示的ECA网络层1）进行处理。
对于N个ECA网络层来说，第一个ECA网络层的输入即为连接在该第一个ECA网络层之前的第一CBAM网络层的输出。而对于第一个ECA网络层之后的ECA网络层，即第i个ECA网络层，其输入为第i-1个ECA网络层的输出。对于N个ECA网络层中的每一个ECA网络层，其输出的一个分支输入至下一个网络层（即下一个ECA网络或者第二CBAM网络层），其输出的另一个分支的信息分别为一个二维特征信息。即，N个ECA网络层分别对应输出N各二维特征信息。
对于特征提取网络的最后一个网络层，即第二CBAM网络层，其输入为N个ECA网络层中的最后一个ECA网络层输出的特征信息，其输出为最后一个尺度最小的二维特征信息。
该特征提取网络最终输出的各个不同尺度的二维特征信息包括每个ECA网络层分别输出的二维特征信息，以及第二CBAM网络输出的二维特征信息，一共为N+1个二维特征信息。
本申请实施例中，特征提取网络由CBAM网络层和ECA网络层有序地组成，通过CBAM网络层和ECA网络层的处理，能够在轻量化网络，减少计算量的同时，基于注意力机制准确有效地实现二维图像的特征提取，从而提高图像三维重建的效率和准确性。
可选地，所述维度变换网络包括N+1个变换子网络，第j个所述变换子网络的输入为第j个所述ECA网络层的输出，第j个所述变换子网络的输出为第j个所述ECA网络层输出的二维特征信息转换成的三维特征信息；最后一个所述变换子网络的输入为所述第二CBAM网络层的输出，最后一个所述变换子网络的输出为所述第二CBAM网络层输出的二维特征信息转换成的三维特征信息；其中，j为小于或者等于N的正整数。
本申请实施例中，与特征提取网络输出的N+1个二维特征信息相对应，维度变换网络包括N+1个变换子网络，每个变换子网络分别对应对一个二维特征信息进行维度转换处理。示例性地，每个变换子网络均包含上述步骤S103所述的维度转换函数、内核大小为D×1×1的反卷积层、激活函数和归一化函数。
具体地，前N个变换子网络中的任意一个，即第j个变换子网络，其输入为第j个ECA网络层的输出，其输出为将其对应的第j个ECA网络层输出的二维特征信息进行维度转换得到的三维特征信息。而对于最后一个变换子网络，即第N+1个变换子网络，其输入为第二CBAM网络层的输出，其输出为将其对应的第二CBAM网络层输出的二维特征信息进行维度转换得到的三维特征信息。
本申请实施例中，通过维度变换网络中的各个变换子网络分别对各个二维特征信息进行处理，能够准确地生成与各个二维特征信息一一对应的各个三维特征信息，从而提高图像三维重建的准确性。
可选地，所述图像生成网络包括N+1个特征生成网络层和目标卷积网络层；第一个所述特征生成网络层的输入为最后一个所述变换子网络的输出，第m个所述特征生成网络层的输入包括倒数第m个所述变换子网络的输出以及第m-1个所述变换子网络的输出；所述目标卷积网络层的输入为最后一个所述变换子网络的输出，所述目标卷积网络层的输出为所述目标三维重建图像；其中，m为大于1，并且小于或者等于N+1的任一正整数。
本申请实施例中图像生成网络具体包含用于上采样和融合图像的三维特征信息的特征生成网络层，以及用于最终的目标三维特征信息重建得到目标三维重建图像的目标卷积网络层。其中，特征生成网络层的数量与三维特征信息的数量一致，为N+1个。
具体地，由于特征提取网络卷积过程中出现的下采样，特征提取网络输出的N+1个二维特征信息中，其尺度依次从到变小，对应地，变换子网络1~变换子网络N+1分别输出的N+1个三维特征信息Y 1~Y N+1中，其尺度大小的排列顺序为从大到小。在N+1个特征生成网络层中，第一个特征生成网络层的输入直接为尺度最小的三维特征信息Y N+1，其对应为最后一个变换子网络（即变换子网络N+1）的输出。而在对于第一个特征生成网络层后的每个特征生成网络层，即第m个特征生成网络层，其输入包含两个分支，一个输入为倒数第m个变换子网络（即正数第N+1-m个变换子网络）的输出，另一个输入为第m-1个特征生成网络层的输出。通过该网络结构，每次将上一尺度的三维特征信息上采样得到的三维特征信息与当前尺度的三维特征信息一同输入特征生成网络层进行三维逐点卷积和三维反卷积处理，实现两个尺度的三维特征信息的融合，最终得到目标三维特征信息。该目标三维特征信息即融合了各个不同尺度的所述三维特征信息。该目标三维特征信息能够融合图像的浅层语义信息和深层几何细节信息。
在生成目标三维特征信息后，将该目标三维特征信息输入目标卷积网络层进行处理，即可得到与原来的二维图像的尺度大小一致，并且细节清晰、边界清晰的目标三维重构图像。
本申请实施例中，由于能够基于包含三维逐点卷积层和三维反卷积层的特征生成网络，实现对不同尺度的三维特征信息的上采样和融合处理，因此能够准确地生成目标三维特征信息，使得基于该目标三维特征信息能够准确地得到清晰的目标三维重构图像。
可选地，所述图像三维重建方法应用于X光图像的三维重建，在所述获取二维图像之前，还包括：
从预设的CT图像数据集中获取预设数量的CT样本图像；
根据所述预设数量的CT样本图像和DDR投影算法，得到预设数量的二维投影图像；
根据所述预设数量的二维投影图像和预设的风格迁移算法，生成预设数量的X光样本图像；
基于所述预设数量的X光样本图像对特征提取网络、维度变换网络和图像生成网络进行训练，得到所述已训练的特征提取网络、所述已训练的维度变换网络和所述已训练的图像生成网络。
本申请实施例中，可以将由特征提取网络、维度变换网络和图像生成网络组合而成的神经网络称为图像三维重建网络。当本申请实施例的图像三维重建方法具体应用于X光图像的三维重建，即本申请实施例的二维图像为X光图像时，则对应地，该图像三维重建网络需要基于X光样本图像训练得到。然而，目前基本没有公开的数据量较大的X光图像数据集，并且人工构建X光图像数据集较为较为费力且成本较高，而现有公开的数据集中存在数据量足够的CT图像数据集，因此本申请实施例中，可以基于该CT图像数据集以及风格迁移算法，构建得到X光样本图像，再基于构建得到的X光样本图像准确地实现对图像三维重建网络的训练。
首先，可以从预设的CT图像数据集中获取预设数量的CT样本图像，该预设数量可以根据实际的训练需要而确定，例如可以为1000张。
在获取到预设数量的CT样本图像后，对于每张CT样本图像，均通过数字重建放射影像（Digitally Reconstructured Radiograph，DRR）投影算法进行处理，最终可以生成预设数量的二维投影图像。
在生成二维投影图像后，可以基于一张实际拍摄的原始X光图像，采用预设的风格迁移算法（例如通过提前训练得到的风格迁移神经网络），对每张二维投影图像分别进行风格迁移处理，从而使得每张二维投影图像都能够模仿该原始X光图像的特征，生成预设数量的X光样本图像。
在生成预设数量的X光样本图像之后，可以将这预设数量的X光样本图像输入图像三维重建网络中，对其中的特征提取网络、维度变换网络和图像生成网络进行端到端的训练，最终得到已训练的图像三维重建网络，即得到已训练的特征提取网络、已训练的维度变换网络和已训练的图像生成网络。
本申请实施例中，由于能够基于CT图像数据集、DRR投影算法和风格迁移算法，便捷有效地生成预设数量的X光样本图像，基于这些X光样本图像准确地对特征提取网络、维度变换网络和图像生成网络进行训练，最终准确地生成已训练的网络，使得后续可以基于这些网络高效准确地实现图像三维重建。
示例性地，设上述的N=3，本申请实施例的图像三维重建网络可以如图5所示，详述如下：
特征提取网络包含依次连接的第一CBAM网络层、3层ECA网络层和第二CBAM网络层。二维图像输入第一CBAM网络层进行处理后，得到初始特征信息X0。该初始特征信息X0作为第一个ECA网络层，即ECA网络层1的输入。该ECA网络层1对该初始特征信息X0进行处理后得到的二维特征信息X1分为两个分支，一个分支输入至维度变换网络中对应的变换子网络1进行维度变换处理，另一个分支输入至下一个ECA网络层，即ECA网络层2进行处理。同样地，ECA网络层2处理ECA网络层1输入的二维特征信息后，得到的二维特征信息X2分为两个分支，一个分支输入至维度变换网络中对应的变换子网络2进行维度变换处理，另一个分支输入至下一个ECA网络层，即ECA网络层3进行处理；ECA网络层3处理ECA网络层2输入的二维特征信息后，得到的二维特征信息X3分为两个分支，一个分支输入至维度变换网络中对应的变换子网络3进行维度变换处理，另一个分支输入至下一个网络层，即第二CBAM网络层进行处理；第二CBAM网络层对ECA网络层3输入的二维特征信息进行处理，得到二维特征信息X4，将其直接输入至维度变换网络中的变换子网络4进行处理。
在特征提取网络层输出的X1~X4四个二维特征信息中，尺度依次从大变小；对应地，维度变换网络输出的Y1~Y4四个三维特征信息的尺度也依次从大变小。
三维特征信息Y1~Y4输入图像生成网络后，最后一个变换子网络，即变换子网络4输出的尺度最小的三维特征信息Y4先输入第一个特征生成网络层，即特征生成网络层1进行三维逐点卷积处理及反卷积处理，得到尺度扩增三维特征信息Y4’；将该Y4’与三维特征信息Y3一同输入特征生成网络层2进行处理，得到尺度扩增三维特征信息Y3’；将该Y3’与三维特征信息Y2一同输入特征生成网络层3进行处理，得到尺度扩增三维特征信息Y2’；将该Y2’与三维特征信息Y1一同输入特征生成网络层4进行处理，即可得到尺度大小与第一CBAM网络层输出的二维特征信息X0的尺度大小相匹配（即长宽一致）的目标三维特征信息。将该目标三维特征信息输入目标卷积网络层进行处理，即可得到最终的目标三维重建图像。
通过图像三维重建网络，基于轻量化的注意力模块、残差块和多尺度特征融合，在大大减少所需的计算资源的同时，保证重构图像的精度，同时能够基于特征生成网络的三维逐点卷积层和三维反卷积层，准确高效地改善重建过程中的像素损失，从而能够高效准确地实现图像三维重建，得到清晰准确地目标三维重建图像。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
实施例二：
图3示出了本申请实施例提供的一种图像三维重建装置的结构示意图，为了便于说明，仅示出了与本申请实施例相关的部分：
该图像三维重建装置包括：获取单元61、特征提取单元62、维度变换单元63、重建单元64。其中：
获取单元61，用于获取二维图像。
特征提取单元62，用于将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息；所述特征提取网络包含卷积层。
维度变换单元63，用于将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息。
重建单元64，用于将所述三维特征信息输入已训练的图像生成网络进行处理，得到目标三维重建图像；其中，所述图像生成网络依次包括三维逐点卷积层和三维反卷积层。
可选地，所述图像生成网络的所述三维反卷积层中包含了第一预设数目的不同尺度的三维卷积核。
可选地，所述特征提取单元62，具体用于将所述二维图像输入已训练的特征提取网络进行处理，得到至少两个不同尺度的二维特征信息；其中，所述特征提取网络包括依次连接的至少两个网络层，不同尺度的所述二维特征信息分别对应为不同的所述网络层输出的特征信息；
对应地，所述维度变换单元63，具体用于将各个所述二维特征信息输入已训练的维度变换网络进行处理，得到与各个所述二维特征信息分别对应的各个三维特征信息。
可选地，所述特征提取网络包括依次连接的第一CBAM网络层、N个ECA网络层、第二CBAM网络层，所述ECA网络层包含残差块；
所述不同尺度的二维特征信息包括所述N个ECA网络层输出的N个二维特征信息以及所述第二CBAM网络层输出的二维特征信息；其中，所述第一CBAM网络层的输入为所述二维图像，第一个所述ECA网络层的输入为所述第一CBAM网络层的输出，第i个所述ECA网络层的输入为第i-1个所述ECA网络层的输出，所述第二CBAM网络层的输入为最后一个所述ECA网络层的输出；N为大于1的正整数，i为大于1且小于或者等于N的任一正整数。
可选地，所述维度变换网络包括N+1个变换子网络，第j个所述变换子网络的输入为第j个所述ECA网络层的输出，第j个所述变换子网络的输出为第j个所述ECA网络层输出的二维特征信息转换成的三维特征信息；最后一个所述变换子网络的输入为所述第二CBAM网络层的输出，最后一个所述变换子网络的输出为所述第二CBAM网络层输出的二维特征信息转换成的三维特征信息；其中，j为小于或者等于N的正整数。
可选地，所述图像生成网络包括N+1个特征生成网络层和目标卷积网络层；第一个所述特征生成网络层的输入为第一个所述变换子网络的输出，第m个所述特征生成网络层的输入包括第m个所述变换子网络的输出以及第m-1个所述变换子网络的输出；所述目标卷积网络层的输入为最后一个所述变换子网络的输出，所述目标卷积网络层的输出为所述目标三维重建图像；其中，m为大于1，并且小于或者等于N+1的任一正整数。
可选地，所述图像三维重建方法应用于X光图像的三维重建，所述图像三维重建装置，还包括：
训练单元，用于从预设的CT图像数据集中获取预设数量的CT样本图像；根据所述预设数量的CT样本图像和DDR投影算法，得到预设数量的二维投影图像；根据所述预设数量的二维投影图像和预设的风格迁移算法，生成预设数量的X光样本图像；基于所述预设数量的X光样本图像对特征提取网络、维度变换网络和图像生成网络进行训练，得到所述已训练的特征提取网络、所述已训练的维度变换网络和所述已训练的图像生成网络。
需要说明的是，上述装置/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其具体功能及带来的技术效果，具体可参见方法实施例部分，此处不再赘述。
实施例三：
图7是本申请一实施例提供的电子设备的示意图。如图7所示，该实施例的电子设备7包括：处理器70、存储器71以及存储在所述存储器71中并可在所述处理器70上运行的计算机程序72，例如图像三维重建程序。所述处理器70执行所述计算机程序72时实现上述各个图像三维重建方法实施例中的步骤，例如图1所示的步骤S101至S104。或者，所述处理器70执行所述计算机程序72时实现上述各装置实施例中各模块/单元的功能，例如图6所示获取单元61至重建单元64的功能。
示例性的，所述计算机程序72可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器71中，并由所述处理器70执行，以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序72在所述电子设备7中的执行过程。
所述电子设备7可以是桌上型计算机、笔记本、掌上电脑、X光机等计算设备。所述电子设备可包括，但不仅限于，处理器70、存储器71。本领域技术人员可以理解，图7仅仅是电子设备7的示例，并不构成对电子设备7的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述电子设备还可以包括输入输出设备、网络接入设备、总线等。
所称处理器70可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器 (Digital Signal Processor，DSP)、专用集成电路 (Application Specific Integrated Circuit，ASIC)、现场可编程门阵列 (Field-Programmable Gate Array，FPGA) 或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器71可以是所述电子设备7的内部存储单元，例如电子设备7的硬盘或内存。所述存储器71也可以是所述电子设备7的外部存储设备，例如所述电子设备7上配备的插接式硬盘，智能存储卡（Smart Media Card, SMC），安全数字（Secure Digital, SD）卡，闪存卡（Flash Card）等。进一步地，所述存储器71还可以既包括所述电子设备7的内部存储单元也包括外部存储设备。所述存储器71用于存储所述计算机程序以及所述电子设备所需的其他程序和数据。所述存储器71还可以用于暂时地存储已经输出或者将要输出的数据。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的装置/电子设备和方法，可以通过其它的方式实现。例如，以上所描述的装置/电子设备实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，Random Access Memory）、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (10)

  1. 一种图像三维重建方法，其特征在于，包括：
    获取二维图像；
    将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息；
    将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息；
    将所述三维特征信息输入已训练的图像生成网络进行处理，得到目标三维重建图像；其中，所述图像生成网络依次包括三维逐点卷积层和三维反卷积层。
  2. 如权利要求1所述的图像三维重建方法，其特征在于，所述图像生成网络的所述三维反卷积层中包含了第一预设数目的不同尺度的三维卷积核。
  3. 如权利要求1所述的图像三维重建方法，其特征在于，所述将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息，包括：
    将所述二维图像输入已训练的特征提取网络进行处理，得到至少两个不同尺度的二维特征信息；其中，所述特征提取网络包括依次连接的至少两个网络层，不同尺度的所述二维特征信息分别对应为不同的所述网络层输出的特征信息；
    对应地，所述将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息，包括：
    将各个所述二维特征信息输入已训练的维度变换网络进行处理，得到与各个所述二维特征信息分别对应的各个三维特征信息。
  4. 如权利要求3所述的图像三维重建方法，其特征在于，所述特征提取网络包括依次连接的第一CBAM网络层、N个ECA网络层、第二CBAM网络层，所述ECA网络层包含残差块；
    所述不同尺度的二维特征信息包括所述N个ECA网络层输出的N个二维特征信息以及所述第二CBAM网络层输出的二维特征信息；其中，所述第一CBAM网络层的输入为所述二维图像，第一个所述ECA网络层的输入为所述第一CBAM网络层的输出，第i个所述ECA网络层的输入为第i-1个所述ECA网络层的输出，所述第二CBAM网络层的输入为最后一个所述ECA网络层的输出；N为大于1的正整数，i为大于1且小于或者等于N的任一正整数。
  5. 如权利要求4所述的图像三维重建方法，其特征在于，所述维度变换网络包括N+1个变换子网络，第j个所述变换子网络的输入为第j个所述ECA网络层的输出，第j个所述变换子网络的输出为第j个所述ECA网络层输出的二维特征信息转换成的三维特征信息；最后一个所述变换子网络的输入为所述第二CBAM网络层的输出，最后一个所述变换子网络的输出为所述第二CBAM网络层输出的二维特征信息转换成的三维特征信息；其中，j为小于或者等于N的正整数。
  6. 如权利要求5所述的图像三维重建方法，其特征在于，所述图像生成网络包括N+1个特征生成网络层和目标卷积网络层；第一个所述特征生成网络层的输入为最后一个所述变换子网络的输出，第m个所述特征生成网络层的输入包括倒数第m个所述变换子网络的输出以及第m-1个所述变换子网络的输出；所述目标卷积网络层的输入为最后一个所述变换子网络的输出，所述目标卷积网络层的输出为所述目标三维重建图像；其中，m为大于1，并且小于或者等于N+1的任一正整数。
  7. 如权利要求1至6任意一项所述的图像三维重建方法，其特征在于，所述图像三维重建方法应用于X光图像的三维重建，在所述获取二维图像之前，还包括：
    从预设的CT图像数据集中获取预设数量的CT样本图像；
    根据所述预设数量的CT样本图像和DDR投影算法，得到预设数量的二维投影图像；
    根据所述预设数量的二维投影图像和预设的风格迁移算法，生成预设数量的X光样本图像；
    基于所述预设数量的X光样本图像对特征提取网络、维度变换网络和图像生成网络进行训练，得到所述已训练的特征提取网络、所述已训练的维度变换网络和所述已训练的图像生成网络。
  8. 一种图像三维重建装置，其特征在于，包括：
    获取单元，用于获取二维图像；
    特征提取单元，用于将所述二维图像输入已训练的特征提取网络进行处理，得到所述二维图像对应的二维特征信息；
    维度变换单元，用于将所述二维特征信息输入已训练的维度变换网络进行处理，得到三维特征信息；
    重建单元，用于将所述三维特征信息输入已训练的图像生成网络进行处理，得到目标三维重建图像；其中，所述图像生成网络依次包括三维逐点卷积层和三维反卷积层。
  9. 一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，当所述处理器执行所述计算机程序时，使得电子设备实现如权利要求1至7任一项所述方法的步骤。
  10. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，当所述计算机程序被处理器执行时，使得电子设备实现如权利要求1至7任一项所述方法的步骤。
PCT/CN2022/140102 2022-01-28 2022-12-19 图像三维重建方法、装置、电子设备及存储介质 WO2023142781A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210105985.9 2022-01-28
CN202210105985.9A CN114548238A (zh) 2022-01-28 2022-01-28 图像三维重建方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023142781A1 true WO2023142781A1 (zh) 2023-08-03

Family

ID=81673912

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/140102 WO2023142781A1 (zh) 2022-01-28 2022-12-19 图像三维重建方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN114548238A (zh)
WO (1) WO2023142781A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116805284A (zh) * 2023-08-28 2023-09-26 之江实验室 基于特征迁移的三维磁共振平面间超分辨重建方法和系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114548238A (zh) * 2022-01-28 2022-05-27 中国科学院深圳先进技术研究院 图像三维重建方法、装置、电子设备及存储介质
CN115937638B (zh) * 2022-12-30 2023-07-25 北京瑞莱智慧科技有限公司 模型训练方法、图像处理方法、相关装置及存储介质
CN116091704B (zh) * 2023-03-15 2023-06-13 广州思涵信息科技有限公司 一种远程人体三维图像重构方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190035117A1 (en) * 2017-07-25 2019-01-31 Nuctech Company Limited Method and device for reconstructing ct image and storage medium
CN112435341A (zh) * 2020-11-23 2021-03-02 推想医疗科技股份有限公司 三维重建网络的训练方法及装置、三维重建方法及装置
CN113554742A (zh) * 2020-04-26 2021-10-26 上海联影医疗科技股份有限公司 一种三维图像的重建方法、装置、设备及存储介质
CN113658326A (zh) * 2021-08-05 2021-11-16 北京奇艺世纪科技有限公司 三维毛发的重建方法及装置
CN114548238A (zh) * 2022-01-28 2022-05-27 中国科学院深圳先进技术研究院 图像三维重建方法、装置、电子设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190035117A1 (en) * 2017-07-25 2019-01-31 Nuctech Company Limited Method and device for reconstructing ct image and storage medium
CN113554742A (zh) * 2020-04-26 2021-10-26 上海联影医疗科技股份有限公司 一种三维图像的重建方法、装置、设备及存储介质
CN112435341A (zh) * 2020-11-23 2021-03-02 推想医疗科技股份有限公司 三维重建网络的训练方法及装置、三维重建方法及装置
CN113658326A (zh) * 2021-08-05 2021-11-16 北京奇艺世纪科技有限公司 三维毛发的重建方法及装置
CN114548238A (zh) * 2022-01-28 2022-05-27 中国科学院深圳先进技术研究院 图像三维重建方法、装置、电子设备及存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116805284A (zh) * 2023-08-28 2023-09-26 之江实验室 基于特征迁移的三维磁共振平面间超分辨重建方法和系统
CN116805284B (zh) * 2023-08-28 2023-12-19 之江实验室 基于特征迁移的三维磁共振平面间超分辨重建方法和系统

Also Published As

Publication number Publication date
CN114548238A (zh) 2022-05-27

Similar Documents

Publication Publication Date Title
WO2023142781A1 (zh) 图像三维重建方法、装置、电子设备及存储介质
US11120582B2 (en) Unified dual-domain network for medical image formation, recovery, and analysis
WO2020125498A1 (zh) 心脏磁共振图像分割方法、装置、终端设备及存储介质
CN111368849B (zh) 图像处理方法、装置、电子设备及存储介质
CN111815766B (zh) 基于2d-dsa图像重建血管三维模型处理方法及系统
CN115953494B (zh) 基于低剂量和超分辨率的多任务高质量ct图像重建方法
TWI791979B (zh) 三維醫學影像的建立方法
CN111612689B (zh) 医学图像处理方法、装置、计算机设备和可读存储介质
CN111105475B (zh) 一种基于正交角度x光的骨骼三维重建方法
CN109741254A (zh) 字典训练及图像超分辨重建方法、系统、设备及存储介质
CN116563533A (zh) 基于目标位置先验信息的医学图像分割方法及系统
Al Nassan et al. A customized convolutional neural network for dental bitewing images segmentation
CN111489406A (zh) 生成高能ct图像模型的训练及生成方法、设备、存储介质
US11455755B2 (en) Methods and apparatus for neural network based image reconstruction
KR102346199B1 (ko) 파노라믹 영상 생성 방법 및 이를 위한 영상 처리장치
Qiu et al. 3D segmentation of mandible from multisectional CT scans by convolutional neural networks
Wang et al. TRCT-GAN: CT reconstruction from biplane X-rays using transformer and generative adversarial networks
CN115908610A (zh) 一种基于单模态pet图像获取衰减校正系数图像的方法
WO2021189383A1 (zh) 生成高能ct图像模型的训练及生成方法、设备、存储介质
CN114730476A (zh) 有限角度重建的网络确定
CN116228903B (zh) 基于csa模块及深度学习模型的高清ct图像重建方法
Liang et al. High quality low-dose SPECT reconstruction using CGAN-based transformer network with geometric tight framelet
Cheng et al. Sdct-gan: reconstructing CT from biplanar x-rays with self-driven generative adversarial networks
WO2022116143A1 (zh) Pet成像方法、装置与设备
US20240104802A1 (en) Medical image processing method and apparatus and medical device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22923567

Country of ref document: EP

Kind code of ref document: A1