CN116965839A - Multi-mode heart target chart generation method, system, equipment and storage medium - Google Patents

Multi-mode heart target chart generation method, system, equipment and storage medium Download PDF

Info

Publication number
CN116965839A
CN116965839A CN202310948076.6A CN202310948076A CN116965839A CN 116965839 A CN116965839 A CN 116965839A CN 202310948076 A CN202310948076 A CN 202310948076A CN 116965839 A CN116965839 A CN 116965839A
Authority
CN
China
Prior art keywords
heart
pet image
cardiac
dimensional
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310948076.6A
Other languages
Chinese (zh)
Inventor
张敏
乔萧雅
黄秋
李彪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ruinjin Hospital Affiliated to Shanghai Jiaotong University School of Medicine Co Ltd
Original Assignee
Ruinjin Hospital Affiliated to Shanghai Jiaotong University School of Medicine Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ruinjin Hospital Affiliated to Shanghai Jiaotong University School of Medicine Co Ltd filed Critical Ruinjin Hospital Affiliated to Shanghai Jiaotong University School of Medicine Co Ltd
Priority to CN202310948076.6A priority Critical patent/CN116965839A/en
Publication of CN116965839A publication Critical patent/CN116965839A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/02—Devices for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/03—Computerised tomographs
    • A61B6/037—Emission tomography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/50—Clinical applications
    • A61B6/503—Clinical applications involving diagnosis of heart
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5223—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data generating planar views from image data, e.g. extracting a coronal view from a 3D image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/809—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of classification results, e.g. where the classifiers operate on the same input data
    • G06V10/811—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of classification results, e.g. where the classifiers operate on the same input data the classifiers operating on different input data, e.g. multi-modal recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing

Abstract

The applicationThe application provides a method, a system, equipment and a storage medium for generating a multi-mode heart bullseye chart, belonging to the technical field of medical image recognition. The heart target chart generation method comprises the following steps: acquiring a three-dimensional heart PET image; wherein the three-dimensional heart PET image is a passing by 68 Ga-FAPI marked heart transection position PET image; affine transformation of the three-dimensional heart PET image into a heart short axis position PET image; registering and fusing the cardiac short axis MRI images, and identifying and extracting the left ventricular myocardial contours of the cardiac short axis PET images; and projecting the short axis position PET image of the heart into a heart target heart chart according to the left ventricle myocardial contour of the short axis position PET image of the heart. The application fuses the multi-mode PET/MRI heart image information, effectively reduces the participation of manual correction in the process of generating the target chart, ensures the accuracy of the short axis positioning of the heart and the accuracy of the segmentation of the left ventricle and the myocardium, and realizes 68 Automatic generation of Ga-FAPI labeled PET image heart target graph.

Description

Multi-mode heart target chart generation method, system, equipment and storage medium
Technical Field
The application relates to the technical field of medical image recognition, in particular to a method, a system, equipment and a storage medium for generating a multi-mode heart bullseye chart.
Background
The cardiac muscle is subjected to reparative fibrosis after the acute myocardial infarction so as to stabilize the heart structure and prevent heart rupture. The activated myocardial fibroblasts act on myocardial infarction parts to form reparative scar tissues, but excessive myocardial fibrosis can lead to long-term left ventricular poor reconstruction, cardiac insufficiency and even heart failure, and is closely related to poor prognosis of patients. Early detection of ventricular remodeling due to myocardial fibrosis is critical in preventing the development of heart failure. Fibroblast Activation Protein (FAP) is a membrane-bound serine protease that is highly expressed in activated fibroblasts of damaged myocardium. 68 Ga-FAPI is 68 A Ga-labeled Fibroblast Activation Protein (FAP) inhibitor that specifically binds to FAP that is highly expressed on the surface of activated cardiomyocytes. Gallium 68 marked FAP inhibitor as the latest molecular imaging technology 68 Ga-FAPI) PET/MR imaging enables visual quantitative monitoring of activated myocardial fibroblasts.
Cardiac targeting is an important tool for quantitative analysis of PET myocardial perfusion imaging, and provides a method for combining or comparing different myocardial images in a two-dimensional coordinate system. In particular, the segmentation and nomenclature of the left ventricular myocardium according to the American Heart Association (AHA), 17-segmented heartsAnd (3) segmenting the model, and projecting the three-dimensional PET left myocardial short axis image to the corresponding position of the two-dimensional planar target graph. In this process, the rotation of the transection position PET heart image to the short axis position and the completion of the segmentation of the cardiac muscle are all important steps in the generation of the heart target map. There are some methods and commercial software available to complete the automatic bullseye generation of myocardial perfusion images, but these methods are mostly suitable for PET/SPECT images where myocardial imaging is more complete, such as FDG PET imaging. However, due to 68 Ga-FAPIPET is a focal function image, cannot contain complete heart anatomy, has incomplete left ventricular myocardial contours, cannot complete accurate short-axis positioning of the heart and segmentation of the left ventricular myocardium by the existing method, still requires a large amount of manual correction participation, and is very time-consuming and cannot guarantee accuracy.
Accordingly, there is a need to provide a multi-modal based cardiac bullseye generation method, system, apparatus, and storage medium to address the above-described issues.
Disclosure of Invention
In view of the above drawbacks of the prior art, an object of the present application is to provide a method, a system, an apparatus and a storage medium for generating a multi-modal heart target map, so as to solve the technical problem that the input three-dimensional heart PET image cannot contain a complete heart anatomy, and the three-dimensional heart PET image cannot be accurately subjected to heart short axis positioning and left ventricle myocardial segmentation in the prior art.
To achieve the above and other related objects, the present application provides a method for generating a multi-modal-based cardiac bullseye chart, comprising the following steps:
acquiring a three-dimensional heart PET image; wherein the three-dimensional heart PET image is a passing by 68 Ga-FAPI marked heart transection position PET image;
affine transformation of the three-dimensional heart PET image into a heart short axis position PET image;
identifying and extracting left ventricular myocardial contours of the short-axis position PET image of the heart;
and projecting the short axis position PET image of the heart into a heart target heart chart according to the left ventricle myocardial contour of the short axis position PET image of the heart.
In an embodiment of the present application, the present application further provides a multi-modal-based cardiac bulls-eye chart generating system, the system comprising:
the data acquisition unit is used for acquiring a three-dimensional heart PET image; wherein the three-dimensional heart PET image is a passing by 68 Ga-FAPI marked heart transection position PET image;
affine transformation unit, affine transformation of said three-dimensional heart PET image into heart short axis position PET image;
the contour sketching unit is used for identifying and extracting the left ventricular myocardial contour of the heart short axis PET image;
and the target heart map generation unit is used for projecting the short-axis position PET image of the heart into a target heart map according to the left ventricle myocardial contour of the short-axis position PET image of the heart.
In an embodiment of the present application, there is also provided a computer device including a processor, the processor being coupled to a memory, the memory storing program instructions that when executed by the processor implement the multi-modal cardiac bullseye chart generation model training method and the multi-modal cardiac bullseye chart generation method of any one of the embodiments described above.
In an embodiment of the present application, there is also provided a computer readable storage medium including a program, which when run on a computer, causes the computer to perform the multi-modal cardiac bullseye chart generation model training method and the multi-modal cardiac bullseye chart generation method according to any one of the embodiments described above.
According to the method, the system, the equipment and the storage medium for generating the multi-mode heart target chart, under the condition that the heart anatomy shape of the three-dimensional heart PET image is incomplete, the spatial relationship from the heart transverse position to the short axis position of the three-dimensional heart PET image can be deduced based on the spatial position of the heart image of the input three-dimensional heart PET image, the three-dimensional heart PET image is affine transformed to the heart short axis position PET image, and meanwhile, the current heart muscle outline in the heart short axis position PET image is extracted through a convolution neural network and is based onThe current ventricular myocardial profile delineates and complements the incomplete left ventricular myocardial profile, and finally, the short-axis cardiac PET image is projected into a cardiac target chart based on the complete left ventricular myocardial profile. The multi-mode heart-based bullseye chart generation method effectively reduces the participation of manual correction in the bullseye chart generation process, ensures the accuracy of heart short axis positioning and the accuracy of left ventricular myocardial segmentation, and realizes 68 Automatic generation of Ga-FAPI labeled PET image heart target graph.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a process for generating a heart target map based on a multi-modal heart target map method in accordance with an embodiment of the present application;
FIG. 2 is a flow chart of a method for generating a multi-modal-based cardiac bullseye chart according to an embodiment of the application;
FIG. 3 is a flowchart illustrating a step S2 in a multi-mode-based cardiac bullseye chart generation method according to an embodiment of the application;
FIG. 4 is a flowchart illustrating a step S21 in a multi-mode-based cardiac bullseye chart generation method according to an embodiment of the application;
FIG. 5 is a flowchart illustrating a step S3 in a multi-mode-based cardiac bullseye chart generation method according to an embodiment of the application;
FIG. 6 is a flowchart illustrating a step S3 in a multi-modality-based cardiac bullseye chart generation method according to another embodiment of the application;
FIG. 7 is a block diagram illustrating a multi-modal cardiac bullseye generation model training system in accordance with an embodiment of the application;
fig. 8 is a schematic diagram of a computer device according to an embodiment of the application.
Description of element numbers:
10. a multi-modal based cardiac bullseye chart generation system; 11. a data acquisition unit; 12. an affine transformation unit; 13. a contouring unit; 14. and a bullseye chart generation unit.
Detailed Description
Other advantages and effects of the present application will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present application with reference to specific examples. The application may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present application. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict. It is also to be understood that the terminology used in the examples of the application is for the purpose of describing particular embodiments only, and is not intended to limit the scope of the application. The test methods in the following examples, in which specific conditions are not noted, are generally conducted under conventional conditions or under conditions recommended by the respective manufacturers.
Please refer to fig. 1 to 8. It should be understood that the structures, proportions, sizes, etc. shown in the drawings are for illustration purposes only and should not be construed as limiting the application to the extent that it can be practiced, since modifications, changes in the proportions, or adjustments of the sizes, which are otherwise, used in the practice of the application, are included in the spirit and scope of the application which is otherwise, without departing from the spirit or scope thereof. Also, the terms such as "upper," "lower," "left," "right," "middle," and "a" and the like recited in the present specification are merely for descriptive purposes and are not intended to limit the scope of the application, but are intended to provide relative positional changes or modifications without materially altering the technical context in which the application may be practiced.
Where numerical ranges are provided in the examples, it is understood that unless otherwise stated herein, both endpoints of each numerical range and any number between the two endpoints are significant both in the numerical range. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs and to which this application belongs, and any method, apparatus, or material of the prior art similar or equivalent to the methods, apparatus, or materials described in the examples of this application may be used to practice the application.
In this specification, PET images are positron emission computed tomography images (Positron Emission Computed Tomography), and PET images are nuclear imaging techniques (also known as molecular imaging) that can display metabolic processes in vivo. PET imaging is based on the technology of detecting gamma-ray pairs emitted indirectly by positron-emitting radionuclides (also known as radiopharmaceuticals, radionuclides or radiotracers). The tracer is injected into the veins of the bioactive molecule, typically a sugar for cellular energy. The PET system sensitive detector captures gamma ray radiation inside the body and uses software to map the triangulated emissions source, creating a three-dimensional computed tomography image of the in vivo tracer concentration.
However, PET imaging can only scan images of the transverse position of the heart when applied to cardiac imaging, and since PET imaging requires injecting radionuclides into the heart to label designated cardiac myocytes, PET images developed for focal functions of the heart with specific radionuclides cannot contain complete heart anatomy, which is detrimental to further processing and analysis of three-dimensional cardiac PET images by current medical imaging software. For example, focus imaging of myocardial reparative fibrosis following acute myocardial infarction using PET imaging is commonly used 68 Ga-FAPI marks activated myocardial fibroblasts in the heart, but the three-dimensional heart PET image of the radionuclide-labeled scanning imaging cannot contain complete left ventricular myocardial contours, and pre-medical image software has difficulty in short axis positioning and left ventricular myocardial segmentation in the three-dimensional heart PET image.
Therefore, as shown in fig. 1, the method, the system, the equipment and the storage medium for generating the multi-mode heart target pattern can be used for completing the space transformation from the three-dimensional heart PET image to the heart short-axis PET image through a convolution neural network based on the space position of the heart in a transverse position and the current heart chamber myocardial contour recorded in the three-dimensional heart PET image, and completing the sketch of the incomplete left heart chamber myocardial contour based on the current heart chamber myocardial contour in the heart short-axis PET image, and finally projecting the left heart chamber myocardial region of the short-axis PET image to the corresponding position of the two-dimensional plane target pattern according to the method for establishing the AHA-17 segment heart segmentation model to generate the heart target pattern.
Referring to fig. 2, in an embodiment of the present application, the method for generating a multi-modal-based cardiac bullseye chart includes the following steps:
s1, acquiring a three-dimensional heart PET image; wherein the three-dimensional heart PET image is a radioactive tracer 68 Ga-FAPI marked heart transection position PET image;
step S1 comprises the steps of converting a DICOM-format three-dimensional heart PET image into a nii-format file; only a heart region is intercepted in the three-dimensional heart PET image, the intercepting region range is not limited, the heart position is ensured to be in the center of the image as far as possible, the initial position information of the image is stored in an image header file, and meanwhile, the shearing and scaling characteristic parameters of affine transformation of the image are updated.
S2, affine transformation is carried out on the three-dimensional heart PET image to obtain a heart short axis PET image;
in step S2, a spatial relationship from a heart transverse position to a heart short axis position of the three-dimensional heart PET image is obtained through downsampling of a 3D convolution layer of the first convolution neural network, affine transformation characteristics from the transverse position to the short axis position of the three-dimensional heart PET image are obtained based on the spatial relationship, and finally the three-dimensional heart PET image is transformed from the transverse position to the short axis position according to the affine transformation characteristics by utilizing the spatial transformation layer of the first convolution neural network, so that a heart short axis position PET image of the three-dimensional heart PET image is obtained.
Specifically, as shown in fig. 3, in step S2, the affine transformation of the three-dimensional cardiac PET image into a cardiac short axis PET image includes the steps of:
s21, convolving the three-dimensional heart PET image to obtain affine transformation characteristics of the three-dimensional heart PET image relative to the short axis of the heart;
specifically, as shown in fig. 1, through a 3D convolution layer and a full connection layer which are connected in series in a first convolution neural network, based on the spatial position of a recorded heart image in a three-dimensional heart PET image, affine transformation characteristics from a transverse position to a short axis position of the three-dimensional heart PET image are extracted;
the affine transformation feature comprises rotation feature parameters and translation feature parameters, the rotation feature parameters and translation feature parameters of the affine transformation feature are stored in a 1×6 matrix parameter, specifically expressed as pt= [ Prx, pry, prz, ptx, pty, ptz ], the first three column parameters of the matrix parameter are rotation angles Prx, pry, prz in the x, y and z directions when the heart short axis positioning network estimates that the three-dimensional heart PET image rotates in the short axis direction, and the last three column parameters are offset amounts Ptx, pty and Ptz in the x, y and z directions required after the heart short axis positioning network estimates that the three-dimensional heart PET image rotates in advance.
Further, as shown in fig. 4, in some embodiments, step S21 includes the steps of:
s211, performing convolution downsampling processing on a three-dimensional heart PET image through a plurality of cascade 3D convolution layers to obtain a space transformation characteristic image of the three-dimensional heart PET image relative to a heart short axis position; the space transformation characteristic image comprises rotation characteristics and translation characteristics from transverse position to short axis position of the three-dimensional heart PET image;
s212, processing the space transformation characteristic image by using a cascade 3D convolution layer and a full connection layer to obtain a rotation characteristic and a translation characteristic of the three-dimensional heart PET image relative to the short axis position of the heart;
s213, obtaining affine transformation characteristics according to the rotation characteristics and the translation characteristics;
in some embodiments, step S213 includes constructing an affine transformation matrix according to the rotation feature, the translation feature, and preset scaling parameters and shearing parameters, and taking the affine transformation matrix as an affine transformation feature;
specifically, matrix parameters R based on rotation characteristicsotation matrix And a preset scaling parameter Zoom matrix And Shear parameter Shear matrix Constructing a first transformation matrix in which the Shear parameter Shear is not involved due to rotation of the short axis bits of the heart matrix Take a value of 1, zoom parameter Zoom matrix The scaling parameters of the original three-dimensional cardiac PET image are preserved; the first transformation matrix is expressed as:
then obtaining Affine transformation characteristics based on matrix parameters of the first transformation matrix and the translation characteristics, wherein the Affine transformation characteristics are obtained in the form of a matrix Affine matrix Formally expressed, while translating matrix parameters of the feature [ Ptx, pty, ptz ]]In matrix Affine matrix Expressed as matrix parameters d, h, l; specifically, matrix Affine of Affine transformation features matrix The formula is:
s22, according to the affine transformation characteristics, the three-dimensional heart PET image is spatially transformed into a heart short axis PET image through a spatial transformation layer.
S3, identifying and extracting the left ventricle myocardial contours of the heart short axis PET image;
in step S3, the current ventricular myocardial contour feature recorded in the cardiac short axis PET image is first extracted by the convolution downsampling layer in the second convolution neural network, then the current ventricular myocardial contour feature is subjected to convolution upsampling by the convolution upsampling layer in the second convolution neural network, and the convolution upsampling layer complements the complete left ventricular myocardial contour in the restored cardiac short axis PET image based on the current ventricular myocardial contour feature in the convolution upsampling process, so that the complete left ventricular myocardial contour is obtained by segmentation from the output short axis PET image.
The second convolutional neural network comprises a plurality of convolutional upsampling layers and convolutional downsampling layers which are connected in series, each convolutional upsampling layer and each convolutional downsampling layer comprise a 3D convolutional layer, a normalization layer and a pooling layer, the number of the convolutional downsampling layers is consistent with that of the convolutional upsampling layers, the number of the convolutional downsampling layers and the number of the convolutional upsampling layers is at least 4, and the maximum number is not limited.
Specifically, as shown in fig. 5, in step S3, the identifying and extracting the left ventricular myocardial contours of the short-axis PET image of the heart includes the following steps:
s31, acquiring the outline characteristics of the heart muscle of the current ventricle in the short-axis position PET image of the heart according to the short-axis position PET image of the heart;
s32, complementing the left ventricular myocardial profile in the short-axis PET image according to the current ventricular myocardial profile characteristics so as to predict and obtain the complete left ventricular myocardial profile in the short-axis cardiac PET image.
Wherein in step S3, the second convolutional neural network predicts and complements the complete left ventricular myocardial contour based on the incomplete ventricular myocardial image in the input short-axis PET image by utilizing contour recognition capability obtained by training the three-dimensional image with the complete cardiac contour in advance to predict and obtain the complete left ventricular myocardial contour in the short-axis PET image of the heart.
In other embodiments, the second convolutional neural network is set as a dual channel, and in step S3, the cardiac short axis PET image and the cardiac short axis MRI image of the three-dimensional cardiac MRI image are input to the second convolutional neural network through the dual channel by calling the corresponding three-dimensional cardiac MRI image, the second convolutional neural network synchronously identifies and extracts the ventricular myocardium contour in the cardiac short axis PET image and the cardiac short axis MRI image, and in the process of extracting the ventricular myocardium contour feature by convolutional downsampling and segmenting the contour in the image by convolutional upsampling, the incomplete left ventricular myocardium contour in the cardiac short axis PET image is complemented according to the complete ventricular myocardium contour in the cardiac short axis MRI image so as to predict and obtain the complete left ventricular myocardium contour in the cardiac short axis PET image.
The MRI image is a magnetic resonance imaging image (Magnetic Resonance Imaging), and the MRI image is a structural image of the interior of the object by detecting electromagnetic waves emitted by the external gradient magnetic field according to different attenuations of the released energy in different structural environments of the interior of the object by using the principle of nuclear magnetic resonance, so that the positions and types of nuclei constituting the object can be obtained. MRI imaging technology has been commonly used in clinical diagnosis of cardiovascular disease, a non-invasive medical imaging technology that evaluates the function and structure of the cardiovascular system, and MRI imaging can scan the heart from multiple spatial orientations, such as the transverse, short and long axes, and clearly and completely display the ventricular myocardial contours of the heart in multiple modes in three-dimensional images.
Specifically, as shown in fig. 6, in the above embodiment, in step S3, the identifying extracts a left ventricular myocardial contour of the cardiac short axis PET image, including the steps of:
s31', calling a three-dimensional heart MRI image corresponding to the three-dimensional heart PET image;
step S31' specifically includes reading a three-dimensional cardiac cine sequence MRI image against a three-dimensional cardiac PET image, converting the format of the three-dimensional cardiac cine sequence MRI image into the same format as the three-dimensional cardiac PET image, for example, converting the DICOM format of the three-dimensional cardiac cine sequence MRI image into a nii format file; then three-dimensional heart MRI images of the end diastole of the heart in the three-dimensional heart film sequence are extracted and stored independently; and finally, carrying out normalization processing on the saved three-dimensional heart MRI image based on the three-dimensional heart PET image input into the first neural network, so that the three-dimensional heart MRI image has the same size and resolution as the three-dimensional heart PET image.
The three-dimensional cardiac cine sequence MRI image in step S31' may be a corresponding MRI image scanned together with the three-dimensional cardiac PET image, or may be a corresponding MRI image having a high similarity with the three-dimensional cardiac PET image, which is retrieved from the database.
S32', registering the three-dimensional cardiac MRI image based on the cardiac short axis PET image to obtain a cardiac short axis MRI image of the three-dimensional cardiac MRI image;
step S32' specifically comprises interpolating the three-dimensional cardiac MRI image in the direction of the long axis of the heart based on the short axis cardiac PET image, so that the interpolated three-dimensional cardiac MRI image has the same resolution as the short axis cardiac PET image; registering the interpolated three-dimensional cardiac MRI image with the short axis cardiac PET image to obtain a short axis cardiac MRI image.
S33', inputting the short-axis cardiac PET image and the short-axis cardiac MRI image into a convolutional neural network to respectively obtain the current ventricular myocardial profile of the short-axis cardiac PET image and the complete ventricular myocardial profile of the short-axis cardiac MRI image;
s34', comparing the complete ventricular myocardial contours of the short-axis MRI image of the heart, and completing the current ventricular myocardial contours so as to predict and obtain the complete left ventricular myocardial contours in the short-axis PET image of the heart.
In an example, in step S34', the multi-mode PET/MRI cardiac image information is fused by means of wavelet fusion, and the left ventricular myocardial contours that are incomplete in the identified short-axis PET image of the heart are fused with the complete left ventricular myocardial contours in the short-axis MRI image of the heart, so as to segment the complete left ventricular myocardial contours in the short-axis PET image of the heart.
And S4, projecting the short axis position PET image of the heart into a heart target chart according to the left ventricle myocardial contour of the short axis position PET image of the heart.
Specifically, in step S4, according to the method for establishing the AHA-17 segment heart segmentation model, the left ventricular myocardial region of the short axis PET image is projected to the corresponding position of the two-dimensional planar target map, and a heart target map is generated.
In summary, in the method for generating a cardiac target map, based on spatial information of a three-dimensional cardiac PET image, matrix parameters of affine transformation characteristics of a short axis of the three-dimensional cardiac PET image, specifically rotation and translation transformation parameters, are predicted, and the affine transformation characteristics are utilized to perform spatial transformation on the three-dimensional cardiac PET image to obtain a cardiac short axis PET image; after obtaining a heart short axis PET image, extracting the current ventricular myocardial contour in the heart short axis PET image through a convolutional neural network, and carrying out delineating and supplementing on the incomplete left ventricular myocardial contour based on the current ventricular myocardial contour; finally, according to the method for establishing the AHA-17 segment heart segmentation model, a three-dimensional PET left myocardial short axis image is projected to a corresponding position of a two-dimensional plane target heart map based on the complete left ventricular myocardial contour, and a heart target heart map is generated.
In an embodiment of the present application, a multi-mode-based cardiac bullseye chart generating system 10 is provided, which corresponds to the multi-mode-based cardiac bullseye chart generating method in the above embodiment one by one. As shown in fig. 7, the multi-modality-based cardiac bullseye chart generation system 10 includes a data acquisition unit 11, an affine transformation unit 12, a contour sketching unit 13, and a bullseye chart generation unit 14. The functional modules are described in detail as follows:
a data acquisition unit 11 for acquiring a three-dimensional cardiac PET image; wherein the three-dimensional heart PET image is a passing by 68 Ga-FAPI marked heart transection position PET image;
an affine transformation unit 12 affine transforming the three-dimensional heart PET image into a heart short axis position PET image;
a contouring unit 13 for identifying a left ventricular myocardial contour from which the short-axis PET image of the heart is extracted;
a target map generation unit 14, configured to project the short axis PET image of the heart as a target map of the heart according to the outline of the left ventricle myocardium of the short axis PET image of the heart.
In an embodiment, affine transformation unit 12 is specifically configured to:
convolving the three-dimensional heart PET image to obtain affine transformation characteristics of the three-dimensional heart PET image relative to the short axis of the heart;
and according to the affine transformation characteristics, spatially transforming the three-dimensional heart PET image into a heart short axis PET image.
In an embodiment, affine transformation unit 12 is specifically configured to:
convolving the three-dimensional heart PET image to obtain a space transformation characteristic image of the three-dimensional heart PET image relative to the short axis of the heart;
convolving the space transformation characteristic image to obtain a rotation characteristic and a translation characteristic of the three-dimensional heart PET image relative to the short axis of the heart;
from the rotation feature and the translation feature, affine transformation features are obtained.
In an embodiment, affine transformation unit 12 is specifically configured to:
and constructing an affine transformation matrix based on the rotation characteristic, the translation characteristic and preset scaling parameters and shearing parameters, and taking the affine transformation matrix as affine transformation characteristic.
In an embodiment, the contouring unit 13 is specifically configured to:
acquiring the current ventricular myocardial contour characteristic in the short-axis position PET image of the heart according to the short-axis position PET image of the heart;
and according to the current ventricular myocardial profile characteristics, the left ventricular myocardial profile in the short-axis PET image is complemented so as to predict and obtain the complete left ventricular myocardial profile in the short-axis cardiac PET image.
In an embodiment, the contouring unit 13 is specifically configured to:
calling a three-dimensional heart MRI image corresponding to the three-dimensional heart PET image;
registering the three-dimensional cardiac MRI image based on the cardiac short axis PET image to obtain a cardiac short axis MRI image of the three-dimensional cardiac MRI image;
inputting the short-axis cardiac PET image and the short-axis cardiac MRI image into a convolutional neural network to respectively obtain the current ventricular myocardial profile of the short-axis cardiac PET image and the complete ventricular myocardial profile of the short-axis cardiac MRI image;
and comparing the complete ventricular myocardial profile of the short-axis MRI image of the heart, and complementing the current ventricular myocardial profile to predict and obtain the complete left ventricular myocardial profile in the short-axis PET image of the heart.
In an embodiment, the bullseye chart generating unit 14 is specifically configured to:
and according to an AHA-17 segment heart segmentation method, projecting the left ventricular myocardial contours in the short-axis position PET image of the heart to the corresponding positions of the two-dimensional planar target heart map, and generating the heart target heart map.
In one embodiment, a computer device is provided, the internal structure of which may be as shown in FIG. 8. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is for communicating with an external server via a network connection. The computer program, when executed by a processor, performs the functions or steps of a dialog mode preference method.
In one embodiment, a computer device is provided comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of when executing the computer program:
acquiring a three-dimensional heart PET image; wherein the three-dimensional heart PET image is a passing by 68 Ga-FAPI marked heart transection position PET image;
affine transformation of the three-dimensional heart PET image into a heart short axis position PET image;
identifying and extracting left ventricular myocardial contours of the short-axis position PET image of the heart;
and projecting the short axis position PET image of the heart into a heart target heart chart according to the left ventricle myocardial contour of the short axis position PET image of the heart.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a three-dimensional heart PET image; wherein the three-dimensional heart PET image is a passing by 68 Ga-FAPI marked heart transection position PET image;
affine transformation of the three-dimensional heart PET image into a heart short axis position PET image;
identifying and extracting left ventricular myocardial contours of the short-axis position PET image of the heart;
and projecting the short axis position PET image of the heart into a heart target heart chart according to the left ventricle myocardial contour of the short axis position PET image of the heart.
It should be noted that, the functions or steps implemented by the computer readable storage medium or the computer device may correspond to the relevant descriptions of the server side and the client side in the foregoing method embodiments, and are not described herein for avoiding repetition.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
In the application, the method, the system, the equipment and the storage medium for generating the multi-mode heart target chart can be used for three purposesUnder the condition of incomplete heart anatomy in the dimensional heart PET image, based on the spatial position of a heart image in the input three-dimensional heart PET image, deducing the spatial relation from the transverse position to the short axis position of the heart in the three-dimensional heart PET image, affine transforming the three-dimensional heart PET image to the short axis position PET image of the heart, extracting the current ventricular myocardial contour in the short axis position PET image of the heart through a convolution neural network, and carrying out delineating and supplementing on the incomplete left ventricular myocardial contour based on the current ventricular myocardial contour, and finally projecting the short axis position heart PET image into a heart target graph based on the complete left ventricular myocardial contour. The multi-mode heart-based bullseye chart generation method effectively reduces the participation of manual correction in the bullseye chart generation process, ensures the accuracy of heart short axis positioning and the accuracy of left ventricular myocardial segmentation, and realizes 68 Automatic generation of Ga-FAPI labeled PET image heart target graph.
The above embodiments are merely illustrative of the principles of the present application and its effectiveness, and are not intended to limit the application. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the application. Accordingly, it is intended that all equivalent modifications and variations of the application be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.

Claims (10)

1. A method for generating a heart target map based on multiple modes, comprising:
acquiring a three-dimensional heart PET image; wherein the three-dimensional heart PET image is a passing by 68 Ga-FAPI marked heart transection position PET image;
affine transformation of the three-dimensional heart PET image into a heart short axis position PET image;
identifying and extracting left ventricular myocardial contours of the short-axis position PET image of the heart;
and projecting the short axis position PET image of the heart into a heart target heart chart according to the left ventricle myocardial contour of the short axis position PET image of the heart.
2. The multi-modality based cardiac bullseye chart generation method of claim 1, wherein affine transforming the three-dimensional cardiac PET image to a cardiac short axis PET image comprises:
convolving the three-dimensional heart PET image to obtain affine transformation characteristics of the three-dimensional heart PET image relative to the short axis of the heart;
and according to the affine transformation characteristics, spatially transforming the three-dimensional heart PET image into a heart short axis PET image.
3. The multi-modality based cardiac bullseye chart generation method of claim 2 wherein the convolving the three-dimensional cardiac PET image to obtain affine transformation characteristics of the three-dimensional cardiac PET image relative to a short axis of the heart comprises:
convolving the three-dimensional heart PET image to obtain a space transformation characteristic image of the three-dimensional heart PET image relative to the short axis of the heart;
convolving the space transformation characteristic image to obtain a rotation characteristic and a translation characteristic of the three-dimensional heart PET image relative to the short axis of the heart;
from the rotation feature and the translation feature, affine transformation features are obtained.
4. A method of generating a multi-modal based cardiac bullseye chart according to claim 3, wherein said obtaining affine transformation features from rotation features and translation features comprises:
and constructing an affine transformation matrix based on the rotation characteristic, the translation characteristic and preset scaling parameters and shearing parameters, and taking the affine transformation matrix as affine transformation characteristic.
5. The multi-modality based cardiac bullseye chart generation method of claim 1, wherein said identifying extracts left ventricular myocardium contours of the cardiac short axis PET image comprises:
acquiring the current ventricular myocardial contour characteristic in the short-axis position PET image of the heart according to the short-axis position PET image of the heart;
and according to the current ventricular myocardial profile characteristics, the left ventricular myocardial profile in the short-axis PET image is complemented so as to predict and obtain the complete left ventricular myocardial profile in the short-axis cardiac PET image.
6. The multi-modality based cardiac bullseye chart generation method of claim 5, wherein said identifying extracts left ventricular myocardium contours of the short axis PET image of the heart comprises:
calling a three-dimensional heart MRI image corresponding to the three-dimensional heart PET image;
registering the three-dimensional cardiac MRI image based on the cardiac short axis PET image to obtain a cardiac short axis MRI image of the three-dimensional cardiac MRI image;
inputting the short-axis cardiac PET image and the short-axis cardiac MRI image into a convolutional neural network to respectively obtain the current ventricular myocardial profile of the short-axis cardiac PET image and the complete ventricular myocardial profile of the short-axis cardiac MRI image;
and comparing the complete ventricular myocardial profile of the short-axis MRI image of the heart, and complementing the current ventricular myocardial profile to predict and obtain the complete left ventricular myocardial profile in the short-axis PET image of the heart.
7. The method for generating a multi-modality-based cardiac target map according to claim 1, wherein the outputting the cardiac target map of the short axis PET image of the heart from the left ventricular myocardium contour of the short axis PET image of the heart comprises:
and according to an AHA-17 segment heart segmentation method, projecting the left ventricular myocardial contours in the short-axis position PET image of the heart to the corresponding positions of the two-dimensional planar target heart map, and generating the heart target heart map.
8. A multi-modality-based cardiac bulls-eye map generation system, comprising:
a data acquisition unit for acquiringA three-dimensional cardiac PET image; wherein the three-dimensional heart PET image is a passing by 68 Ga-FAPI marked heart transection position PET image;
affine transformation unit, affine transformation of said three-dimensional heart PET image into heart short axis position PET image;
the contour sketching unit is used for identifying and extracting the left ventricular myocardial contour of the heart short axis PET image;
and the target heart map generation unit is used for projecting the short-axis position PET image of the heart into a target heart map according to the left ventricle myocardial contour of the short-axis position PET image of the heart.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 7 when the computer program is executed by the processor.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
CN202310948076.6A 2023-07-28 2023-07-28 Multi-mode heart target chart generation method, system, equipment and storage medium Pending CN116965839A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310948076.6A CN116965839A (en) 2023-07-28 2023-07-28 Multi-mode heart target chart generation method, system, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310948076.6A CN116965839A (en) 2023-07-28 2023-07-28 Multi-mode heart target chart generation method, system, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116965839A true CN116965839A (en) 2023-10-31

Family

ID=88472741

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310948076.6A Pending CN116965839A (en) 2023-07-28 2023-07-28 Multi-mode heart target chart generation method, system, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116965839A (en)

Similar Documents

Publication Publication Date Title
US10842445B2 (en) System and method for unsupervised deep learning for deformable image registration
JP4918048B2 (en) Image processing apparatus and method
Penney et al. Registration-based interpolation
CN111080573B (en) Rib image detection method, computer device and storage medium
US7689021B2 (en) Segmentation of regions in measurements of a body based on a deformable model
US7117026B2 (en) Physiological model based non-rigid image registration
US8064673B2 (en) Combined segmentation and registration framework for parametric shapes
Lin et al. Deep learning based automatic segmentation of metastasis hotspots in thorax bone SPECT images
US9098912B2 (en) Method, system and computer readable medium for automatic segmentation of a medical image
Khalil et al. 2D to 3D fusion of echocardiography and cardiac CT for TAVR and TAVI image guidance
CN110570483A (en) scanning method, scanning device, computer equipment and storage medium
Kim et al. Automatic segmentation of the left ventricle in echocardiographic images using convolutional neural networks
CN115100067A (en) Image reconstruction method and device
Lu et al. Automatic delineation of left and right ventricles in cardiac MRI sequences using a joint ventricular model
Chen et al. Real-time and multimodal brain slice-to-volume registration using CNN
US10945709B2 (en) Systems, methods and computer readable storage media storing instructions for image-guided interventions based on patient-specific models
CN112150485B (en) Image segmentation method, device, computer equipment and storage medium
Barba-J et al. A 3D Hermite-based multiscale local active contour method with elliptical shape constraints for segmentation of cardiac MR and CT volumes
JP2017189338A (en) Image positioning apparatus and method, and program
Al‐Kofahi et al. Algorithms for accurate 3D registration of neuronal images acquired by confocal scanning laser microscopy
CN116965839A (en) Multi-mode heart target chart generation method, system, equipment and storage medium
Peressutti et al. Registration of multiview echocardiography sequences using a subspace error metric
Noble et al. Myocardial delineation via registration in a polar coordinate system
Puyol-Anton et al. Towards a multimodal cardiac motion atlas
CN116977823A (en) Bullseye chart generation model training and generating method, system, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination