CN111192320A - Position information determining method, device, equipment and storage medium - Google Patents
Position information determining method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111192320A CN111192320A CN201911390823.9A CN201911390823A CN111192320A CN 111192320 A CN111192320 A CN 111192320A CN 201911390823 A CN201911390823 A CN 201911390823A CN 111192320 A CN111192320 A CN 111192320A
- Authority
- CN
- China
- Prior art keywords
- dimensional image
- position information
- neural network
- determining
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computer Graphics (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The embodiment of the invention discloses a method, a device, equipment and a storage medium for determining position information, wherein the method for determining the position information comprises the following steps: inputting the first three-dimensional image into a candidate position determination neural network to obtain initial candidate position information corresponding to a first part in the first three-dimensional image; extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information; and inputting the second three-dimensional image into the accurate position determination neural network to obtain accurate position information of the second part, wherein the first part comprises the second part. The technical scheme of the embodiment of the invention can quickly and accurately realize the position information determination of the target part.
Description
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to a method, a device, equipment and a storage medium for determining position information.
Background
In existing image processing, in particular medical image processing, the exact position determination of key points or key areas is crucial for subsequent operations.
By way of example, the determination of the location of critical points of aortic valve anatomy in medical images, transcatheter aortic valve implantation is a minimally invasive valve replacement procedure in which the distance from the coronary artery opening to the aortic annulus, and the diameter of the annulus are important clinical measurement parameters for selecting the appropriate implantation equipment and valve size. In order to calculate the above important clinical measurement parameters, it is necessary to accurately locate important aortic valve anatomical key points, one right coronary opening point, one left coronary opening point, three sinotubular junctions (junctions), and three sinus floor nadirs.
The existing method for positioning the anatomical key points of the aortic valve mainly comprises a projection space learning algorithm, a partial strategy reinforcement learning algorithm, a multitask full convolution network-based detection method, a Normalized Cut image analysis classical algorithm and the like, wherein the positioning results of the projection space learning algorithm and the Normalized Cut image analysis classical algorithm are poor in accuracy, the positions of all the anatomical key points of the aortic valve are independently detected by the partial strategy reinforcement learning algorithm and the multitask full convolution network-based detection method, the method for independently detecting the positions of the anatomical key points of the aortic valve ignores the relative positions of all the anatomical key points, and the detection results may generate geometric drift or inaccurate detection.
Disclosure of Invention
The invention provides a method, a device, equipment and a storage medium for determining position information, which can quickly and accurately determine the position information of a target part.
In a first aspect, an embodiment of the present invention provides a method for determining location information, where the method for determining location information includes:
inputting a first three-dimensional image into a candidate position determination neural network to obtain initial candidate position information corresponding to a first part in the first three-dimensional image;
extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information;
and inputting the second three-dimensional image into a precise position determination neural network to obtain precise position information of a second part, wherein the first part comprises the second part.
In a second aspect, an embodiment of the present invention further provides a device for determining location information, where the device for determining location information includes:
the initial candidate position information determining module is used for inputting the first three-dimensional image into a candidate position determining neural network to obtain initial candidate position information corresponding to a first part in the first three-dimensional image;
the second three-dimensional image extraction module is used for extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information;
and the accurate position information determining module is used for inputting the second three-dimensional image into an accurate position determining neural network to obtain accurate position information of a second part, wherein the first part comprises the second part.
In a third aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the position information determination method according to any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the location information determining method according to any embodiment of the present invention.
According to the embodiment of the invention, the first three-dimensional image is input into the candidate position determination neural network to obtain initial candidate position information corresponding to the first part in the first three-dimensional image; extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information; and inputting the second three-dimensional image into the accurate position determination neural network to obtain the accurate position information of the second part, wherein the first part comprises the second part, and the position information determination of the target part can be quickly and accurately realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description will be given below of the drawings required for the embodiments or the technical solutions in the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1a is a flowchart of a method for determining location information according to a first embodiment of the present invention;
FIG. 1b is a schematic diagram of a network structure of a candidate position-determining neural network according to a first embodiment of the present invention;
FIG. 1c is a schematic diagram of a network architecture of a precise position determination neural network according to a first embodiment of the present invention;
FIG. 1d is a schematic diagram of a U-Net network structure of a neural network for accurate position determination according to a first embodiment of the present invention;
fig. 1e is a schematic diagram of a network structure of a first-stage candidate position determination neural network according to a first embodiment of the present invention;
FIG. 1f is a schematic diagram of a network structure of a second-stage candidate position determination neural network according to a first embodiment of the present invention;
FIG. 1g is a schematic diagram of the positions of the anatomical key points of the aortic valve determined by a position information determination method according to a first embodiment of the present invention;
FIG. 1h is a schematic diagram of the aortic valve anatomy key positions determined by another method based on position information determination according to the first embodiment of the present invention;
FIG. 1i is a schematic diagram of the positions of the anatomical key points of the aortic valve determined by another method for determining position information according to an embodiment of the present invention;
fig. 1j is a schematic diagram of a peripheral blood vessel segmentation of a hepatic vein junction based on a location information determination method according to a first embodiment of the present invention;
FIG. 1k is a schematic diagram of another location information determination-based method for segmenting peripheral blood vessels where hepatic veins merge according to an embodiment of the present invention;
fig. 2 is a flowchart of a location information determining method according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a position information determining apparatus according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1a is a flowchart of a method for determining location information according to an embodiment of the present invention, where the present embodiment is applicable to a situation where specific location information of a target region needs to be determined, and the method may be executed by a location information determining apparatus, which may be implemented in a software and/or hardware manner, and the apparatus may be configured in a computer device. As shown in fig. 1a, the method of this embodiment may specifically include:
s110, inputting the first three-dimensional image into a candidate position determination neural network to obtain initial candidate position information corresponding to the first part in the first three-dimensional image.
For example, the first three-dimensional image may be a medical three-dimensional image, or may be a non-medical three-dimensional image (for example, any one of a medical two-dimensional image, a natural image, a remote sensing image, and a multispectral image, etc.), and preferably, the medical three-dimensional image may be a reconstructed three-dimensional image. Taking the first three-dimensional image as a medical three-dimensional image, it may include a CTA (Computed Tomography) image with a small slice, or may include a CT image with a large slice, for example, a CT image including a chest and an abdomen.
Preferably, the candidate location determining neural network may include at least one level of a coder type deep neural network, wherein the coder type deep neural network may preferably be a neural network capable of coding high-dimensional image data into low-dimensional feature vectors. For example, the candidate position determination neural network may include a one-stage depth (full) convolutional neural network, or may include two or more stages of depth (full) convolutional neural networks.
In this embodiment, the initial candidate position information may roughly reflect an approximate position of the first portion in the first three-dimensional image. Preferably, the initial candidate location information may include: initial frame candidate position displacement information corresponding to the first region (the initial frame candidate position displacement information may be one or a plurality of pieces), classification information corresponding to the first region, and position displacement information of the target region in the first region. Preferably, the position information of the initial candidate frame in the first three-dimensional image may be determined based on the initial candidate frame position offset information; whether the corresponding part is the first part can be determined according to the classification information corresponding to the first part; the positional offset information of the target site may be utilized to provide a coarse positional reference for subsequent determination of precise positional information of a second site.
Furthermore, the initial candidate position information may further include initial candidate coordinate position information corresponding to the first portion (the coordinate position information may be coordinate position information of the first portion key boundary point, or may be coordinate position information of the first portion feature point), and the position information of the first portion in the first three-dimensional image may be determined based on the initial candidate coordinate position information. It can be understood that the initial candidate coordinate position information output after the candidate position determination neural network processing may not be matched with the corresponding coordinate in the first three-dimensional image, and at this time, the conversion relationship between the initial candidate coordinate position information and the corresponding coordinate in the first three-dimensional image may be determined based on the image size conversion ratio corresponding to the candidate position determination neural network and the size of the first three-dimensional image, so that the position information of the first portion in the first three-dimensional image may be determined.
Before the first three-dimensional image is input into the candidate position determination neural network to obtain the initial candidate position information corresponding to the first portion in the first three-dimensional image, it is preferable that the candidate position determination neural network is trained in advance by using training set data corresponding to the first portion. The training process and the established candidate position determination neural network structure to be trained can be designed according to actual tasks. For example, the detection of the anatomical key points of the medical image is not large in data volume of a detection data set, so that the detection accuracy of the positions of the anatomical key points can be improved by adopting a multi-task learning method. Preferably, the parameter optimization of the candidate position determination neural network may employ a variant algorithm Adam of a stochastic gradient descent method, and coefficients for calculating a gradient moving average value and a square value of the average value may be set to 0.9 and 0.999, respectively. In the multi-task learning, the two classification tasks corresponding to the first part may use a cross entropy loss function, and both the initial candidate frame position offset information prediction task corresponding to the first part and the prediction task of the position offset information of the target part may use a euclidean distance loss function.
After the training of the candidate position determination neural network is completed, the candidate position determination neural network may preferably be further tested using the test set data. In the testing phase, the first-stage deep convolutional network outputs only the binary information corresponding to the first location (illustratively, the binary information is represented in the form of probability), the initial frame candidate position offset information corresponding to the first location, and the position offset information of the target location in the first location. If a multi-level candidate position determination neural network is trained, in a testing phase, a next multi-level candidate position determination neural network may receive a number of second three-dimensional images including the first portion and a number of second three-dimensional images not including the first portion, further exclude the second three-dimensional images not including the first portion, and may optimize the corresponding boundary positions of the second three-dimensional images until only one second three-dimensional image of the first portion remains.
And S120, extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information.
Based on the initial candidate position information, possible positions of the first portion in the first three-dimensional image may be determined, and preferably, the selected possible positions may be extracted from the first three-dimensional image, so that the possible positions can be verified purposefully in the following. Preferably, the number of the second three-dimensional images may be one or more.
And S130, inputting the second three-dimensional image into the accurate position determination neural network to obtain accurate position information of a second part, wherein the first part comprises the second part.
Preferably, the precise position determination neural network includes an encoder-decoder type symmetric deep neural network and an encoder-decoder type asymmetric deep neural network; wherein the encoder-decoder type symmetric deep neural network includes any one of a U-Net deep convolutional neural network, a V-Net deep convolutional neural network, and a HourglassNet deep convolutional neural network. Among them, the encoder-decoder type network may be a network in which high-dimensional image data is encoded into low-dimensional feature vectors using an encoder and then the low-dimensional feature vectors are decoded into high-dimensional output images having the same size as the original input images using a decoder.
Before inputting the second three-dimensional image into the accurate position determination neural network to obtain the accurate position information of the second portion, it is preferable that the accurate position determination neural network is trained in advance by using training set data corresponding to the second portion. The training process and the established accurate position determination neural network structure to be trained can be designed according to actual tasks. For example, the medical image anatomy key point detection is taken as an example, the parameter optimization can adopt a variant algorithm Adam of a random gradient descent method, and coefficients of calculating a gradient moving average value and a square value of the average value can be set to 0.9 and 0.999 respectively. The accurate position information prediction task of the second part may preferably use a euclidean distance loss function.
In the testing stage, the accurate position determination neural network receives the output result of the previous stage candidate position determination neural network, outputs a multi-channel output image with the same size as the input second three-dimensional image, and each channel outputs accurate position information of the second part. If the candidate position determination neural network includes multiple stages, the precise position determination neural network receives only the unique output result of the previous stage candidate position determination neural network, outputs a multi-channel output image having the same size as the input second three-dimensional image, and each channel outputs precise position information of the second portion.
Preferably, the precise position information of the second portion may be precise coordinate position information, or may be an image capable of determining the position information, for example, a gaussian thermonuclear image.
Preferably, the present embodiment can be applied to any one of the scenarios of aortic valve dissection key point detection, aortic root positioning and segmentation, aortic valve leaflet calcification point detection, peripheral blood vessel positioning and segmentation at the hepatic vein junction, peripheral blood vessel positioning and segmentation at the left and right branches of the hepatic portal vein, positioning and segmentation at the junction of the right inferior hepatic vein and the inferior vena cava, peripheral blood vessel positioning and segmentation at the splenomental portal, peripheral blood vessel positioning and segmentation at the renal portal, and peripheral blood vessel positioning and segmentation at the left and right branches of the common iliac artery. For example, if the application scenario is aortic valve dissection key point detection, the first part is an aortic valve root, and the second part is 8 aortic dissection key points, which are a right coronary artery opening point, a left coronary artery opening point, three sinotubular junctions (junctions), and three aortic sinus floor lowest points, respectively.
For example, the above process is specifically described by taking an application scenario as aortic valve anatomy key point detection as an example:
when the candidate position determination neural network includes only one stage, the network structure of the network may be set to include 8 convolution layers, 5 Batch Normalization layers (BN), 5 parameterized ReLU function (pralu) active layers, and 2 Max Pooling (MP) layers. The candidate position determination neural network is a full convolution network and is characterized in that a block image is used as a training sample during training, and a whole three-dimensional image is used as test data during testing, so that the target detection speed and efficiency are improved. The input of the candidate position determination neural network is 24 × 24 × 24 × 1, where 24 × 24 × 24 represents a three-dimensional medical data image block, and 1 represents that the number of color channels of the input image is 1, that is, the image block is a grayscale image.
Fig. 1b is a schematic diagram of a network structure of a candidate position determination neural network according to an embodiment of the present invention, and as shown in fig. 1b, the network structure of the candidate position determination neural network sequentially includes from left to right:
the first layer 1101, convolutional layer Conv, with a convolutional kernel size k of 3 × 3 × 3, input channel size 1, output channel size f of 16, shift step s of 1, is followed by a batch normalization layer and a PReLU function activation layer.
The second layer 1102, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 16, an output channel size f of 16, a shift step s of 1, followed by a batch normalization layer and a PReLU function activation layer.
The third layer 1103, the maximum pooling layer MP, has a pooling interval k of 3 × 3 × 3 and a moving step s of 2.
The fourth layer 1104, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 16, an output channel size f of 32, a shift step s of 1, followed by a batch normalization layer and a PReLU function activation layer.
The fifth layer 1105, convolutional layer Conv, with convolutional kernel size k of 3 × 3 × 3, input channel size 32, output channel size f of 32, shift step s of 1, followed by batch normalization layer and PReLU function activation layer.
The sixth layer 1106 is the maximum pooling layer MP, and the pooling interval k is 3 × 3 × 3, and the moving step s is 2.
The seventh layer 1107, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 32, an output channel size f of 64, a shift step s of 1, followed by a batch normalization layer and a PReLU function activation layer.
The eighth layer 1108 is formed by connecting 3 convolutional layers Conv1-Conv3 in series with the seventh layer, wherein the sizes k of convolution kernels are all 1 × 1 × 1, the input channel size is 64, the output channel size f is 1, 6 and 24 respectively from top to bottom, and the 1 st convolutional layer Conv1 is connected with a Sigmoid function activation layer. The used supervision information is the binary classification information of the aortic root and the non-aortic root, the position offset information of the candidate frame of the aortic root and the position information of the aortic valve dissection key points respectively, wherein the aortic dissection key points comprise a right coronary artery opening point, a left coronary artery opening point, three sinotubular connections (junction points) and three sinus floor lowest points of the aorta.
Inputting the first aortic valve three-dimensional image into a first-stage candidate position determination neural network, obtaining the classification information of the aortic root and the non-aortic root, the position offset information of an aortic root candidate frame and the position offset information of an aortic valve dissection key point, determining a plurality of second aortic valve root three-dimensional images according to the information, and respectively inputting the plurality of second aortic valve root three-dimensional images into a second-stage accurate position determination neural network.
Preferably, the network structure of the precise position determination neural network may be configured to include 42 convolutional layers (residual block occupies 13 × 3 ═ 39 convolutional layers), 40 Batch Normalization (BN) layers, 42 ReLU function (ReLU) active layers, 4 Max Pooling (MP) layers, 4 nearest upsampling layers, and 4 skip layers with residual block transform. The input of the accurate position determination neural network is 64 × 64 × 64 × 1, wherein 64 × 64 × 64 represents a three-dimensional medical data image block, and 1 represents that the number of color channels of the input image is 1, that is, the image block is a grayscale image.
Fig. 1c is a schematic diagram of a network structure of a precise position determining neural network according to an embodiment of the present invention, and as shown in fig. 1c, the network structure of the precise position determining neural network sequentially includes from left to right:
the first layer 1201, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 1, an output channel size f of 32, a shift step s of 1, a fill size p of 1, and is followed by a ReLU function active layer.
The second layer 1202, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 32, an output channel size f of 64, a shift step s of 1, a fill size p of 1, followed by a ReLU function active layer.
The third layer 1203, a residual module ResBlock, includes 3 layers of convolution layers Conv, the first layer has a convolution kernel size k of 1 × 1 × 1, the second layer has a convolution kernel size k of 3 × 3 × 3, the third layer has a convolution kernel size k of 1 × 1 × 1, and all the sublayers are preceded by a batch normalization layer and a ReLU function activation layer. In addition, the input channel size of the first sublayer is 64, the output channel size is 32, the input channel size of the second sublayer is 32, the output channel size is 32, the input channel size of the third sublayer is 32, and the output channel size is 64. The residual block ResBlock described later is the same as described above.
A fourth layer 1204, a sixth layer 1206, an eighth layer 1208, and a tenth layer 1210, and the maximum pooling layer MP has a pooling interval k of 2 × 2 × 2 and a moving step s of 2.
Fifth 1205, seventh 1207, ninth 1209, eleventh 1211, thirteenth 1213, fifteenth 1215, seventeenth 1217 and nineteenth 1219 layers, a residual module ResBlock.
The twelfth layer 1212, the fourteenth layer 1214, the sixteenth layer 1216, and the eighteenth layer 1218, the nearest neighbor upsampling layer, may enlarge the feature map size by a factor of 2.
The twentieth layer 1220, convolution layer Conv, has a convolution kernel size k of 1 × 1 × 1, an input channel size of 64, an output channel size of the number of detected anatomical key points, and 8 aortic valve anatomical key points need to be located. Illustratively, the supervised information used is positional information represented in the form of a gaussian nuclear heat map of aortic valve anatomical keypoints.
The precise position determination neural network in the present embodiment may be a U-Net network in addition to the encoder-decoder type deep network in the above example. The U-Net network structure of the fine position determination neural network may be configured to include 15 convolutional layers, 14 Batch Normalization (BN) layers, 14 ReLU function (ReLU) active layers, 3 max pooling layers, 3 transposed convolutional layers, and 3 jump layers without any transform. The input of the U-Net network of the accurate position determination neural network is 64 × 64 × 64 × 1, wherein 64 × 64 × 64 represents a three-dimensional medical data image block, and 1 represents that the number of color channels of the input image is 1, that is, the image block is a gray image.
Fig. 1d is a schematic diagram of a U-Net network structure of an accurate position determination neural network according to an embodiment of the present invention, and as shown in fig. 1d, the U-Net network structure of the accurate position determination neural network sequentially includes from left to right:
the first layer 1301, convolutional layer Conv, with a convolutional kernel size k of 3 × 3 × 3, input channel size 1, output channel size f of 8, shift step s of 1, fill size p of 1, is followed by batch normalization layer and ReLU function activation layer.
The second layer 1302, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 8, an output channel size f of 16, a shift step s of 1, a fill size p of 1, followed by a batch normalization layer and a ReLU function activation layer.
The third layer 1303 is a maximum pooling layer MP having a pooling interval k of 2 × 2 × 2 and a moving step s of 2.
The fourth layer 1304, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 16, an output channel size f of 16, a shift step s of 1, a fill size p of 1, followed by a batch normalization layer and a ReLU function activation layer.
The fifth layer 1305, convolutional layer Conv, with convolutional kernel size k of 3 × 3 × 3, input channel size 16, output channel size f of 32, shift step s of 1, fill size p of 1, is followed by batch normalization layer and ReLU function activation layer.
The sixth layer 1306, the maximum pooling layer MP, has a pooling interval k of 2 × 2 × 2 and a moving step s of 2.
The seventh layer 1307, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 32, an output channel size f of 32, a shift step s of 1, a fill size p of 1, followed by a batch normalization layer and a ReLU function activation layer.
The eighth layer 1308, convolutional layer Conv, with a convolutional kernel size k of 3 × 3 × 3, input channel size 32, output channel size f of 64, shift step s of 1, fill size p of 1, is followed by the batch normalization layer and the ReLU function activation layer.
The ninth layer 1309, the maximum pooling layer MP, has a pooling interval k of 2 × 2 × 2 and a moving step s of 2.
The tenth layer 1310, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 64, an output channel size f of 64, a shift step s of 1, a fill size p of 1, followed by a batch normalization layer and a ReLU function activation layer.
The eleventh layer 1311, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 64, an output channel size f of 128, a shift step s of 1, a fill size p of 1, followed by a batch normalization layer and a ReLU function activation layer.
The twelfth layer 1312, transposes the convolution layer TConv, has a convolution kernel size k of 2 × 2 × 2, and a shift step s of 2.
Thirteenth 1313 and fourteenth 1314 layers, convolutional layer Conv, with a convolutional kernel size k of 3 × 3 × 3, input channel size 64, output channel size f of 64, shift step s of 1, fill size p of 1, followed by batch normalization layer and ReLU function activation layer.
The fifteenth layer 1315, transposes the convolution layer TConv with a convolution kernel size k of 2 × 2 × 2 and a shift step s of 2.
The eighteenth layer 1318, transpose convolution layer TConv, has a convolution kernel size k of 2 × 2 × 2, and a shift step s of 2.
The twenty-first layer 1321, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 16, and an output channel size f, which is a type of segmented anatomical structure, for example, the category of peripheral blood vessels where hepatic veins merge to be segmented is 3, and is respectively background, hepatic veins, and inferior vena cava. The supervision information used is a manually labeled blood vessel mask image.
If the candidate position determination neural network includes two levels, the network structure of the first-level candidate position determination neural network may be set to include 6 convolutional layers, 3 batch normalization layers, 3 parameterized ReLU function (PReLU) active layers, and 2 max pooling layers. The input of the first-level candidate position determination neural network is 24 × 24 × 24 × 1, wherein 24 × 24 × 24 represents a three-dimensional medical data image block, and 1 represents that the number of color channels of the input image is 1, that is, the image block is a gray image.
Fig. 1e is a schematic diagram of a network structure of a first-stage candidate position determining neural network according to an embodiment of the present invention, and as shown in fig. 1e, the network structure of the first-stage candidate position determining neural network sequentially includes from left to right:
the first layer 1401, convolutional layer Conv, with a convolutional kernel size k of 3 × 3 × 3, input channel size 1, output channel size f of 16, shift step s of 1, is followed by the batch normalization layer and the PReLU function activation layer.
The second layer 1402, the maximum pooling layer MP, has a pooling interval k of 3 × 3 × 3 and a moving step s of 2.
The third layer 1403, convolutional layer Conv, has a convolutional kernel size k of 3 × 3 × 3, an input channel size of 16, an output channel size f of 32, a shift step s of 1, followed by a batch normalization layer and a PReLU function activation layer.
The fourth layer 1404, the maximum pooling layer MP, has a pooling interval k of 3 × 3 × 3 and a moving step s of 2.
The sixth layer 1406, 3 convolutional layers Conv1-Conv3, each sublayer is connected with the fifth layer in series, the convolutional kernel size k is 1 × 1 × 1, the input channel size is 64, the output channel size f is 1, 6 and 24 respectively from top to bottom, the 1 st convolutional layer Conv1 is followed by a Sigmoid function activation layer, and the used supervision information is the classification information of the aortic root and the non-aortic root, the position offset information of the candidate frame of the aortic root and the position offset information of the key point of the aortic valve anatomy respectively.
The second level candidate position determination neural network may be configured to include 4 convolutional layers, 4 batch normalization layers, 5 parameterized ReLU function (prellu) active layers, and 3 max pooling layers and 4 Fully Connected (FC) layers. The input of the second-level candidate position determination neural network is 48 × 48 × 48 × 1, wherein 48 × 48 × 48 represents a three-dimensional medical data image block, and 1 represents that the number of color channels of the input image is 1, that is, the image block is a grayscale image.
Fig. 1f is a schematic diagram of a network structure of a second-stage candidate position determination neural network according to an embodiment of the present invention, and as shown in fig. 1f, the network structure of the second-stage candidate position determination neural network sequentially includes from left to right:
the first layer 1501, convolutional layer Conv, with a convolutional kernel size k of 3 × 3 × 3, input channel size 1, output channel size f of 32, shift step s of 1, is followed by batch normalization layer and PReLU function activation layer.
The second layer 1502, the maximum pooling layer MP, has a pooling interval k of 3 × 3 × 3 and a moving step s of 2.
The fourth layer 1504, the maximum pooling layer MP, has a pooling interval k of 3 × 3 × 3 and a moving step s of 2.
The fifth layer 1505, convolutional layer Conv, with convolutional kernel size k of 3 × 3 × 3, input channel size 64, output channel size f of 64, shift step s of 1, followed by batch normalization layer and PReLU function activation layer.
The sixth layer 1506 is the maximum pooling layer MP, and has a pooling interval k of 2 × 2 × 2 and a moving step s of 2.
The eighth layer 1508, the full connectivity layer FC, has a number of input neurons 128 × 2 × 2 × 2 and a number of output neurons f 256, followed by the prilu function activation layer.
The ninth layer 1509 is connected with the eighth layer in series through 3 full-connection layers FC1-FC3, each sublayer is connected with the eighth layer in series, the number of input neurons is 256, the number of output neurons is 1, 6 and 24 respectively from top to bottom, the 1 st full-connection layer FC1 is connected with a Sigmoid function activation layer in back, and used supervision information respectively comprises binary classification information of an aorta root and a non-aorta root, position offset information of an aorta root candidate box and position offset information of an aortic valve anatomical key point.
Fig. 1g, 1h, and 1i are schematic diagrams of positions of aortic valve anatomical key points determined by a position information determination method according to an embodiment of the present invention. As shown in fig. 1g, fig. 1g-1 is a cross-sectional view of a first three-dimensional image related to an aortic valve, fig. 1g-2 to fig. 1g-7 are the framed area of an aortic root candidate frame obtained after the first three-dimensional image is input into a candidate position determination neural network and the position offset information of the aortic valve anatomical key points in the framed area, and fig. 1g-8 are specific positions of eight anatomical key points of the aortic valve output by the accurate position determination neural network. As shown in fig. 1h, fig. 1h-1 is a cross-sectional view of another first three-dimensional image related to the aortic valve, fig. 1h-2 to fig. 1h-7 are the framed area of the candidate frame of the aortic root and the position offset information of the anatomical key points of the aortic valve in the framed area obtained after the first three-dimensional image is input into the candidate position determination neural network, and fig. 1h-8 is a specific position of eight anatomical key points of the aortic valve output by the accurate position determination neural network. As shown in fig. 1i, fig. 1i-1 to 1i-3 are sectional views of another first three-dimensional image related to the aortic valve, fig. 1i-4 to 1i-9 are the framed region of the candidate frame of the aortic root and the position offset information of the anatomical key points of the aortic valve in the framed region obtained after the first three-dimensional image is input into the candidate position determination neural network, and fig. 1i-10 are specific positions of eight anatomical key points of the aortic valve output by the precise position determination neural network.
Table 1 shows the comparison result between the position information determination method in the present embodiment and the existing method for determining the accuracy of the position of the aortic valve anatomical key point. As shown in table 1:
TABLE 1
The method of the embodiment can be used for determining the aortic valve anatomy key point detection and can also be used for carrying out peripheral blood vessel segmentation at the hepatic vein confluence. Fig. 1j and 1k are schematic diagrams of a peripheral blood vessel segmentation based on a position information determination method according to an embodiment of the present invention. As shown in fig. 1j, fig. 1j-1 to 1j-3 are cross-sectional views of a first three-dimensional image related to a hepatic vein, fig. 1j-4 to 1j-9 are frame selection areas of a peripheral blood vessel candidate frame where the hepatic vein is merged and position offset information of peripheral blood vessels where the hepatic vein is merged, which are obtained after the first three-dimensional image is input into a candidate position determination neural network, and fig. 1j-10 are position offset information of peripheral blood vessels where the hepatic vein is merged, which are obtained in the frame selection areas, and fig. 1j-10 are specific position determination of peripheral blood vessels where the hepatic vein is merged, which are output by the neural network. As shown in fig. 1k, fig. 1k-1 to 1k-3 are respectively cross-sectional views of a first three-dimensional image related to hepatic veins, fig. 1k-4 to 1k-9 are frame selection areas of a peripheral blood vessel candidate frame where hepatic veins are merged and position offset information of peripheral blood vessels where hepatic veins are merged, which are obtained after the first three-dimensional image is input into a candidate position determination neural network, and fig. 1k-10 are specific positions of peripheral blood vessels where hepatic veins are merged, which are output by the neural network, for accurate position determination. In the position information determining method provided by this embodiment, a first three-dimensional image is input into a candidate position determining neural network, so as to obtain initial candidate position information corresponding to a first part in the first three-dimensional image; extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information; and inputting the second three-dimensional image into the accurate position determination neural network to obtain accurate position information of a second part, wherein the first part comprises the second part, the positioning area of the image is reduced by determining initial candidate position information, the accurate position information of the target part is determined on the basis of the initial candidate position information, and the position information of the target part can be quickly and accurately determined.
On the basis of the foregoing embodiments, further before inputting the first three-dimensional image into the candidate position determination neural network, the method further includes:
isotropically transforming the first three-dimensional image into a first three-dimensional image having a first voxel, the volume of the first voxel being a first volume;
after extracting the second three-dimensional image from the first three-dimensional image according to the candidate position information, the method further comprises the following steps:
isotropically transforming the second three-dimensional image into a second three-dimensional image having a second voxel whose volume is a second volume, wherein the first volume is larger than the second volume.
In fact, in the first three-dimensional image, the length of each original voxel of the first three-dimensional image in the X, Y and Z directions is not equal, and in order to ensure that the voxels are uniformly distributed in the first three-dimensional image, the original voxels in the first three-dimensional image may be preferably isotropic into the first three-dimensional image with the first voxel, wherein the length of the first voxel in the X, Y and Z directions is equal.
Illustratively, the original voxels are 1mm × 1.5mm × 2mm in length in the X, Y and Z directions, respectively, the original voxels may be isotropic to the first voxel of 2.3mm × 2.3mm × 2.3mm, and the number of total voxels may be reduced while ensuring that the voxels are uniformly distributed in the first three-dimensional image, so as to reduce the amount of computation of the first-level network.
Accordingly, since the second three-dimensional image is an image extracted from the first three-dimensional image and has a relatively small volume, in order to display more detailed information thereof, it is preferable that the first voxel in the second three-dimensional image is isotropic to the second voxel, wherein the volume of the second voxel is smaller than that of the first voxel. Illustratively, the first voxel is 2.3mm × 2.3mm × 2.3mm and the second voxel is 1.0mm × 1.0mm × 1.0 mm.
On the basis of the foregoing embodiments, further, inputting the second three-dimensional image into the precise location determining neural network to obtain precise location information of the second portion, including:
inputting the second three-dimensional image into the accurate position determination neural network to obtain a Gaussian thermonuclear map corresponding to the second part;
and determining the accurate position information of the second part according to the Gaussian thermal kernel map.
Wherein the size of the Gaussian thermal kernel map is the same as the size of the second three-dimensional image. One precise position corresponds to one gaussian thermonuclear map, and each aortic anatomical key point corresponds to one gaussian thermonuclear map by taking 8 aortic anatomical key points as an example.
Each gaussian thermonuclear map includes a predicted gaussian kernel, each predicted gaussian kernel being an approximate location of the second site. Preferably, the position with the largest value in each gaussian thermonuclear map can be used as the accurate position information of the second part.
Example two
Fig. 2 is a flowchart of a method for determining location information according to a second embodiment of the present invention. In this embodiment, on the basis of the foregoing embodiments, selecting that the initial candidate position information includes initial candidate frame position information corresponding to the first portion, and extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information includes: determining the conversion ratio of the neural network to the image size by using the image size of the first three-dimensional image and the candidate position, and converting the initial candidate frame position information into final candidate frame position information; and extracting a second three-dimensional image from the first three-dimensional image according to the final candidate frame position information. And extracting a second three-dimensional image from the first three-dimensional image according to the final candidate frame position information, wherein the method comprises the following steps: expanding the frame selection range of the candidate frame in the first three-dimensional image based on the final candidate frame position information; and extracting a second three-dimensional image from the first three-dimensional image according to the expanded frame selection range. As shown in fig. 2, the method of this embodiment may specifically include:
s210, inputting the first three-dimensional image into a candidate position determination neural network to obtain initial candidate frame position information corresponding to the first part in the first three-dimensional image.
S220, determining the conversion ratio of the neural network to the image size by using the image size and the candidate position of the first three-dimensional image, and converting the initial candidate frame position information into final candidate frame position information.
Preferably, 1-3 initial candidate box position information may be output. The initial candidate box position information may preferably be initial candidate box position offset information. It should be noted that, when the first three-dimensional image passes through the candidate position determination neural network, the coordinates of the output initial candidate frame cannot be matched with the corresponding coordinates in the first three-dimensional image, and at this time, the initial candidate frame position offset information needs to be processed to obtain the coordinates of the initial candidate frame in the first three-dimensional image.
Preferably, the initial frame candidate position information may be converted into final frame candidate position information based on the initial frame candidate position offset information in combination with the image size of the first three-dimensional image and the conversion ratio of the candidate position determination neural network to the image size to determine the position information of the initial frame candidate in the first three-dimensional image.
S230, based on the final candidate frame position information, expanding the frame selection range of the candidate frame in the first three-dimensional image;
and S240, extracting a second three-dimensional image from the first three-dimensional image according to the expanded frame selection range.
In practical applications, the candidate frame corresponding to the final candidate frame position information may only include a part of the first portion, and in order to ensure that the first portion can maximally fall within the candidate frame, it is preferable that the frame selection range of the candidate frame in the first three-dimensional image is expanded to obtain an expanded frame selection range, and the second three-dimensional image is extracted from the first three-dimensional image based on the expanded frame selection range.
Illustratively, on the basis of the final candidate frame position information, each face corresponding to every four points in the candidate frame is vertically moved by 8.0mm in the direction away from the center of the candidate frame, and each face is extended to be vertically tangent to obtain a closed cuboid or cube, wherein the cuboid or cube is the expanded frame selection range.
And S250, inputting the second three-dimensional image into the accurate position determination neural network to obtain accurate position information of a second part, wherein the first part comprises the second part.
In the position information determining method provided by this embodiment, a first three-dimensional image is input into a candidate position determining neural network, so as to obtain initial candidate position information corresponding to a first part in the first three-dimensional image; extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information; and inputting the second three-dimensional image into the accurate position determination neural network to obtain accurate position information of a second part, wherein the first part comprises the second part, the positioning area of the image is reduced by determining initial candidate position information, the accurate position information of the target part is determined on the basis of the initial candidate position information, and the position information of the target part can be quickly and accurately determined.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a position information determining apparatus according to a third embodiment of the present invention. As shown in fig. 3, the apparatus of the present embodiment includes:
an initial candidate position information determining module 310, configured to input the first three-dimensional image into a candidate position determining neural network, so as to obtain initial candidate position information corresponding to a first portion in the first three-dimensional image;
a second three-dimensional image extraction module 320, configured to extract a second three-dimensional image from the first three-dimensional image based on the initial candidate position information;
and the precise position information determining module 330 is configured to input the second three-dimensional image into a precise position determining neural network to obtain precise position information of a second portion, where the first portion includes the second portion.
In the position information determining apparatus provided in this embodiment, an initial candidate position information determining module is used to input a first three-dimensional image into a candidate position determining neural network, so as to obtain initial candidate position information corresponding to a first portion in the first three-dimensional image; extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information by using a second three-dimensional image extraction module; and inputting the second three-dimensional image into the accurate position determination neural network by using the accurate position information determination module to obtain the accurate position information of the second part, wherein the first part comprises the second part, the positioning area of the image is reduced by determining the initial candidate position information, the accurate position information of the target part is determined on the basis of the initial candidate position information, and the position information determination of the target part can be quickly and accurately realized.
On the basis of the foregoing technical solutions, optionally, the initial candidate position information includes initial candidate frame position information corresponding to the first portion, and the second three-dimensional image extraction module 320 may specifically include:
a final candidate frame position information determining unit, configured to determine a conversion ratio of the neural network to the image size using the image size of the first three-dimensional image and the candidate position, and convert the initial candidate frame position information into final candidate frame position information;
and the second three-dimensional image extraction unit is used for extracting a second three-dimensional image from the first three-dimensional image according to the final candidate frame position information.
On the basis of the foregoing technical solutions, optionally, the second three-dimensional image extraction unit may specifically include:
a frame selection range expanding subunit, configured to expand a frame selection range of the candidate frame in the first three-dimensional image based on the final candidate frame position information;
and the second three-dimensional image extraction subunit is used for extracting a second three-dimensional image from the first three-dimensional image according to the expanded frame selection range.
On the basis of the foregoing technical solutions, optionally, the position information determining apparatus may further include a first isotropization module, configured to isotropically convert the first three-dimensional image into a first three-dimensional image having a first voxel before inputting the first three-dimensional image into the candidate position determination neural network, where a volume of the first voxel is a first volume;
the position information determining apparatus may further include a second isotropization module configured to, after the second three-dimensional image is extracted from the first three-dimensional image according to the candidate position information, isotropize the second three-dimensional image into a second three-dimensional image having a second voxel whose volume is a second volume, where the first volume is larger than the second volume.
On the basis of the above technical solutions, optionally, the accurate location information determining module 330 may specifically include:
the Gaussian thermal kernel map determining unit is used for inputting the second three-dimensional image into the accurate position determining neural network to obtain a Gaussian thermal kernel map corresponding to the second part;
and the accurate position information determining unit is used for determining the accurate position information of the second part according to the Gaussian thermal kernel map.
On the basis of the above technical solutions, optionally, the candidate position determination neural network includes at least one stage of encoder type deep neural network, and the accurate position determination neural network includes an encoder-decoder type symmetric deep neural network and an encoder-decoder type asymmetric deep neural network; wherein the encoder-decoder type symmetric deep neural network includes any one of a U-Net deep convolutional neural network, a V-Net deep convolutional neural network, and a HourglassNet deep convolutional neural network.
On the basis of the above technical solutions, optionally, the position information determining device may be specifically applied to any one of scenarios of aortic valve dissection key point detection, aortic root positioning and segmentation, aortic valve leaflet calcification detection, peripheral blood vessel positioning and segmentation at a hepatic vein junction, peripheral blood vessel positioning and segmentation at a hepatic portal vein left and right branch bifurcation, positioning and segmentation at a hepatic right inferior vein and inferior vena cava junction, peripheral blood vessel positioning and segmentation at a splenomegaly, peripheral blood vessel positioning and segmentation at a renal portal and peripheral blood vessel positioning and segmentation at a common iliac artery left and right bifurcation.
The position information determining device provided by the embodiment of the invention can execute the position information determining method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the executing method.
Example four
Fig. 4 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention. FIG. 4 illustrates a block diagram of an exemplary computer device 412 suitable for use in implementing embodiments of the present invention. The computer device 412 shown in FIG. 4 is only one example and should not impose any limitations on the functionality or scope of use of embodiments of the present invention.
As shown in FIG. 4, computer device 412 is in the form of a general purpose computing device. Components of computer device 412 may include, but are not limited to: one or more processors 416, a memory 428, and a bus 418 that couples the various system components (including the memory 428 and the processors 416).
A program/utility 440 having a set (at least one) of program modules 442 may be stored, for instance, in memory 428, such program modules 442 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. The program modules 442 generally perform the functions and/or methodologies of the described embodiments of the invention.
The computer device 412 may also communicate with one or more external devices 414 (e.g., keyboard, pointing device, display 424, etc., where the display 424 may be configurable or not as desired), one or more devices that enable a user to interact with the computer device 412, and/or any devices (e.g., network card, modem, etc.) that enable the computer device 412 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 422. Also, computer device 412 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet) through network adapter 420. As shown, network adapter 420 communicates with the other modules of computer device 412 over bus 418. It should be appreciated that although not shown in FIG. 4, other hardware and/or software modules may be used in conjunction with the computer device 412, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage, among others.
The processor 416 executes various functional applications and data processing, such as implementing a location information determination method provided by an embodiment of the present invention, by executing programs stored in the memory 428.
EXAMPLE five
An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a method for determining location information according to an embodiment of the present invention, where the method includes:
inputting the first three-dimensional image into a candidate position determination neural network to obtain initial candidate position information corresponding to a first part in the first three-dimensional image;
extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information;
and inputting the second three-dimensional image into the accurate position determination neural network to obtain accurate position information of the second part, wherein the first part comprises the second part.
Of course, the computer-readable storage medium provided in the embodiments of the present invention, on which the computer program is stored, is not limited to performing the method operations described above, and may also perform related operations in the method for determining location information based on a computer device provided in any embodiment of the present invention.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A method for determining location information, comprising:
inputting a first three-dimensional image into a candidate position determination neural network to obtain initial candidate position information corresponding to a first part in the first three-dimensional image;
extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information;
and inputting the second three-dimensional image into a precise position determination neural network to obtain precise position information of a second part, wherein the first part comprises the second part.
2. The method according to claim 1, wherein the initial candidate position information includes initial candidate frame position information corresponding to the first portion, and wherein extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information includes:
determining the conversion ratio of the neural network to the image size by using the image size of the first three-dimensional image and the candidate position, and converting the initial candidate frame position information into final candidate frame position information;
and extracting a second three-dimensional image from the first three-dimensional image according to the final candidate frame position information.
3. The method according to claim 2, wherein extracting a second three-dimensional image from the first three-dimensional image according to the final frame candidate position information comprises:
expanding the frame selection range of the candidate frame in the first three-dimensional image based on the final candidate frame position information;
and extracting a second three-dimensional image from the first three-dimensional image according to the expanded frame selection range.
4. The method of claim 1, further comprising, prior to inputting the first three-dimensional image into the candidate location-determining neural network:
isotropically transforming the first three-dimensional image into a first three-dimensional image having a first voxel, the first voxel having a volume that is a first volume;
after extracting a second three-dimensional image from the first three-dimensional image according to the candidate position information, the method further includes:
isotropically transforming the second three-dimensional image into a second three-dimensional image having a second voxel whose volume is a second volume, wherein the first volume is greater than the second volume.
5. The method according to any one of claims 1-4, wherein inputting the second three-dimensional image into a precise location-determining neural network to obtain precise location information of the second region comprises:
inputting the second three-dimensional image into a precise position determination neural network to obtain a Gaussian thermonuclear map corresponding to a second part;
and determining accurate position information of the second part according to the Gaussian thermal kernel map.
6. The method according to any one of claims 1-4, wherein the candidate position-determining neural networks comprise at least one stage of an encoder-type deep neural network, and the precise position-determining neural network comprises an encoder-decoder type symmetric deep neural network and an encoder-decoder type asymmetric deep neural network; wherein the encoder-decoder type symmetric deep neural network includes any one of a U-Net deep convolutional neural network, a V-Net deep convolutional neural network, and a HourglassNet deep convolutional neural network.
7. The method according to any one of claims 1-4, wherein the method is applied to any one of aortic valve anatomy keypoint detection, aortic root location and segmentation, aortic valve leaflet calcification detection, peripheral blood vessel location and segmentation at hepatic vein junction, peripheral blood vessel location and segmentation at hepatic portal left and right branch bifurcation, location and segmentation at hepatic right inferior right vein and inferior vena cava junction, peripheral blood vessel location and segmentation at splenomesal, peripheral blood vessel location and segmentation at renal portal and peripheral blood vessel location and segmentation at common iliac left and right bifurcation.
8. A position information determination apparatus, characterized by comprising:
the initial candidate position information determining module is used for inputting the first three-dimensional image into a candidate position determining neural network to obtain initial candidate position information corresponding to a first part in the first three-dimensional image;
the second three-dimensional image extraction module is used for extracting a second three-dimensional image from the first three-dimensional image based on the initial candidate position information;
and the accurate position information determining module is used for inputting the second three-dimensional image into an accurate position determining neural network to obtain accurate position information of a second part, wherein the first part comprises the second part.
9. A computer device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the location information determination method of any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of determining location information according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911390823.9A CN111192320B (en) | 2019-12-30 | 2019-12-30 | Position information determining method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911390823.9A CN111192320B (en) | 2019-12-30 | 2019-12-30 | Position information determining method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111192320A true CN111192320A (en) | 2020-05-22 |
| CN111192320B CN111192320B (en) | 2023-07-25 |
Family
ID=70707893
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911390823.9A Active CN111192320B (en) | 2019-12-30 | 2019-12-30 | Position information determining method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111192320B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696089A (en) * | 2020-06-05 | 2020-09-22 | 上海联影医疗科技有限公司 | Arteriovenous determining method, device, equipment and storage medium |
| CN113469258A (en) * | 2021-07-08 | 2021-10-01 | 中国科学院自动化研究所 | X-ray angiography image matching method and system based on two-stage CNN |
| CN113963241A (en) * | 2021-12-22 | 2022-01-21 | 苏州浪潮智能科技有限公司 | FPGA hardware architecture, data processing method thereof and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050232482A1 (en) * | 2004-01-14 | 2005-10-20 | Konica Minolta Photo Imaging, Inc. | Image processing method, image processing apparatus and image processing program |
| WO2018039368A1 (en) * | 2016-08-26 | 2018-03-01 | Elekta, Inc. | Image segmentation using neural network method |
| US20180157938A1 (en) * | 2016-12-07 | 2018-06-07 | Samsung Electronics Co., Ltd. | Target detection method and apparatus |
| CN108717693A (en) * | 2018-04-24 | 2018-10-30 | 浙江工业大学 | A kind of optic disk localization method based on RPN |
| CN108875833A (en) * | 2018-06-22 | 2018-11-23 | 北京智能管家科技有限公司 | Training method, face identification method and the device of neural network |
| US20190030371A1 (en) * | 2017-07-28 | 2019-01-31 | Elekta, Inc. | Automated image segmentation using dcnn such as for radiation therapy |
| WO2019030410A1 (en) * | 2017-08-10 | 2019-02-14 | Aidence B.V | Computer-aided diagnostics using deep neural networks |
| US20190080456A1 (en) * | 2017-09-12 | 2019-03-14 | Shenzhen Keya Medical Technology Corporation | Method and system for performing segmentation of image having a sparsely distributed object |
| CN109493317A (en) * | 2018-09-25 | 2019-03-19 | 哈尔滨理工大学 | The more vertebra dividing methods of 3D based on concatenated convolutional neural network |
| CN109598728A (en) * | 2018-11-30 | 2019-04-09 | 腾讯科技（深圳）有限公司 | Image partition method, device, diagnostic system and storage medium |
| CN109636846A (en) * | 2018-12-06 | 2019-04-16 | 重庆邮电大学 | Object localization method based on circulation attention convolutional neural networks |
| CN109785306A (en) * | 2019-01-09 | 2019-05-21 | 上海联影医疗科技有限公司 | Organ delineation method, device, computer equipment and storage medium |
-
2019
- 2019-12-30 CN CN201911390823.9A patent/CN111192320B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050232482A1 (en) * | 2004-01-14 | 2005-10-20 | Konica Minolta Photo Imaging, Inc. | Image processing method, image processing apparatus and image processing program |
| WO2018039368A1 (en) * | 2016-08-26 | 2018-03-01 | Elekta, Inc. | Image segmentation using neural network method |
| US20180157938A1 (en) * | 2016-12-07 | 2018-06-07 | Samsung Electronics Co., Ltd. | Target detection method and apparatus |
| US20190030371A1 (en) * | 2017-07-28 | 2019-01-31 | Elekta, Inc. | Automated image segmentation using dcnn such as for radiation therapy |
| WO2019030410A1 (en) * | 2017-08-10 | 2019-02-14 | Aidence B.V | Computer-aided diagnostics using deep neural networks |
| US20190080456A1 (en) * | 2017-09-12 | 2019-03-14 | Shenzhen Keya Medical Technology Corporation | Method and system for performing segmentation of image having a sparsely distributed object |
| CN108717693A (en) * | 2018-04-24 | 2018-10-30 | 浙江工业大学 | A kind of optic disk localization method based on RPN |
| CN108875833A (en) * | 2018-06-22 | 2018-11-23 | 北京智能管家科技有限公司 | Training method, face identification method and the device of neural network |
| CN109493317A (en) * | 2018-09-25 | 2019-03-19 | 哈尔滨理工大学 | The more vertebra dividing methods of 3D based on concatenated convolutional neural network |
| CN109598728A (en) * | 2018-11-30 | 2019-04-09 | 腾讯科技（深圳）有限公司 | Image partition method, device, diagnostic system and storage medium |
| CN109636846A (en) * | 2018-12-06 | 2019-04-16 | 重庆邮电大学 | Object localization method based on circulation attention convolutional neural networks |
| CN109785306A (en) * | 2019-01-09 | 2019-05-21 | 上海联影医疗科技有限公司 | Organ delineation method, device, computer equipment and storage medium |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696089A (en) * | 2020-06-05 | 2020-09-22 | 上海联影医疗科技有限公司 | Arteriovenous determining method, device, equipment and storage medium |
| CN113469258A (en) * | 2021-07-08 | 2021-10-01 | 中国科学院自动化研究所 | X-ray angiography image matching method and system based on two-stage CNN |
| CN113469258B (en) * | 2021-07-08 | 2022-03-11 | 中国科学院自动化研究所 | X-ray angiography image matching method and system based on two-stage CNN |
| CN113963241A (en) * | 2021-12-22 | 2022-01-21 | 苏州浪潮智能科技有限公司 | FPGA hardware architecture, data processing method thereof and storage medium |
| CN113963241B (en) * | 2021-12-22 | 2022-03-08 | 苏州浪潮智能科技有限公司 | FPGA hardware architecture, data processing method thereof and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111192320B (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10366491B2 (en) | Deep image-to-image recurrent network with shape basis for automatic vertebra labeling in large-scale 3D CT volumes | |
| Nie et al. | Automatic detection of melanoma with yolo deep convolutional neural networks | |
| JP7540127B2 (en) | Artificial intelligence-based image processing method, image processing device, computer program, and computer device | |
| US20230104173A1 (en) | Method and system for determining blood vessel information in an image | |
| CN109949276B (en) | Lymph node detection method for improving SegNet segmentation network | |
| CN118334070A (en) | System and method for anatomical segmentation in image analysis | |
| CN111192320B (en) | Position information determining method, device, equipment and storage medium | |
| CN113159056B (en) | Image segmentation method, device, equipment and storage medium | |
| CN112085714B (en) | Pulmonary nodule detection method, model training method, device, equipment and medium | |
| CN113326851B (en) | Image feature extraction method and device, electronic equipment and storage medium | |
| WO2023151237A1 (en) | Face pose estimation method and apparatus, electronic device, and storage medium | |
| CN112396605A (en) | Network training method and device, image recognition method and electronic equipment | |
| CN111798424A (en) | Medical image-based nodule detection method and device and electronic equipment | |
| CN110874591A (en) | Image positioning method, device, equipment and storage medium | |
| CN115409859A (en) | Coronary artery blood vessel image segmentation method and device, storage medium and terminal | |
| CN115100185A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN115082389A (en) | Method, apparatus and medium for rib detection of medical image | |
| CN114066905A (en) | Medical image segmentation method, system and device based on deep learning | |
| CN112488178B (en) | Network model training method and device, image processing method and device, and equipment | |
| CN111882532B (en) | Method for extracting key points in lower limb X-ray image | |
| CN113822846A (en) | Method, apparatus, device and medium for determining region of interest in medical image | |
| CN114972361B (en) | Blood flow segmentation method, device, equipment and storage medium | |
| CN113610856B (en) | Method and device for training image segmentation model and image segmentation | |
| CN113222989B (en) | Image grading method and device, storage medium and electronic equipment | |
| CN113792740B (en) | Artery and vein segmentation method, system, equipment and medium for fundus color illumination |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 201807 Shanghai City, north of the city of Jiading District Road No. 2258 Applicant after: Shanghai Lianying Medical Technology Co.,Ltd. Address before: 201807 Shanghai City, north of the city of Jiading District Road No. 2258 Applicant before: SHANGHAI UNITED IMAGING HEALTHCARE Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |