WO2022077858A1

WO2022077858A1 - Full-modal medical image sequence grouping method based on deep learning sign structure

Info

Publication number: WO2022077858A1
Application number: PCT/CN2021/081413
Authority: WO
Inventors: 石洪成; 陈曙光; 胡鹏程; 刘国兵; 顾宇参; 余浩军
Original assignee: 复旦大学附属中山医院
Priority date: 2020-10-15
Filing date: 2021-03-18
Publication date: 2022-04-21
Also published as: CN112308888A

Abstract

A full-modal medical image sequence grouping method based on a deep learning sign structure. The method comprises: acquiring medical image information; performing information extraction on the acquired medical image information; establishing a full-modal deep learning AI sequence matching system; performing sequence matching processing; transmitting processed medical image sequences to a display unit in groups; and the display unit displaying full-modal medical image sequences in groups. A deep learning neural network is used, human skeletons are precisely identified and segmented into relatively fixed local regions according to precise CT and MR anatomy information of a human body, precise human body position segmentation is performed using specified skeleton parts, precise positioning is performed according to a CT or MR image in dual modalities of a molecular image, the CT or MR image is converted to a corresponding layer of a full-modal image, and automatic and precise registration and display is performed, such that diagnosis errors caused by a technical level difference are reduced, and the working efficiency of physicians is also improved.

Description

A full-modality medical image sequence grouping method based on deep learning sign structure

technical field

The present technology relates to the technical field of medical image information, in particular to a full-modality medical image sequence grouping method based on a deep learning sign structure.

Background technique

In the molecular imaging multimodal medical imaging system, according to the clinical or scientific research needs, the scanning method often adopts multiphase or multimodal equipment to collect multiple sets of sequence images. This process often involves relatively complex scanning procedures and multi-modal image comparisons on the same or different computer, as well as special molecular imaging sequences, such as whole-body PET dynamic image display, SPECT quantitative reconstruction image, SPECT-RAWDATA display, etc. In conventional medical imaging systems, this kind of full-modality multi-temporal multi-sequence image reading evaluation is very complicated.

In addition, the existing multi-modal browsing and registration technology of medical images cannot meet the multi-modal registration display of special modalities. During the full-modal comparison browsing, the doctor's interested parts cannot be accurately selected for detailed reading, and senior doctors are still relied on. Subjective and objective factors have a great influence on the professional experience of selecting the corresponding level for registration reading. For example, in the mixed reading process of many images such as PET/CT and PET/MR whole-body imaging, the accuracy is affected by the degree of fatigue and technical ability of the physician, and the special sequence of molecular images cannot be displayed and compared on one reading interface, let alone accurate matching. .

To this end, it is necessary to design a full-modality medical image sequence grouping method, system and storage medium based on deep learning sign structure that combines artificial intelligence and can group and register medical images for display.

SUMMARY OF THE INVENTION

The invention breaks through the difficulties of the prior art, and designs a full-modal medical image sequence grouping method, system and storage medium based on deep learning sign structure that can group and register medical images and display them in combination with artificial intelligence.

In order to achieve the above purpose, the present invention designs a full-modality medical image sequence grouping method based on deep learning sign structure, which is characterized in that: it includes:

S1 obtains medical image information;

S2 performs information extraction on the acquired medical image information;

S3 establishes a full-modal deep learning AI sequence matching system;

S4 performs sequence matching processing;

S5 transmits the processed medical image sequence groupings to the display unit;

The S6 display unit performs grouped display of full-modality medical image sequences.

Further, the medical image information is one or more of a normal modal whole body skeleton image, a PET/CT and PET/MR fusion image on the same machine, and a full modal image heterogeneous fusion image.

Further, the information extraction is one or more of extracting a segmented image of a fixed bone part, and extracting a segmented image of a PET corresponding bone area on the same machine.

Further, the specific method of the sequence matching processing is as follows:

S41 Labeling datasets for medical images;

S42 performs bone image segmentation and AI algorithm recognition;

S43 performs spine bone segmentation and identification test data;

S44 full-modal AI fusion matching.

Further, when the medical image information is a normal state whole-body skeleton image, the information extraction method is to extract the image of the segmented region of the fixed skeletal part.

Further, when the medical image information is one or both of the PET/CT and PET/MR fusion graphics on the same machine, and the full-modality image fusion image on the same machine, the information extraction method is to extract the same machine of the corresponding bone region of the PET. Split the image.

The present invention also designs a system for a full-modality medical image sequence grouping method based on a deep learning sign structure, which is characterized by: comprising: an input unit for inputting medical image information;

a first processing unit, configured to perform information extraction on the input medical image information;

a second processing unit, configured to perform sequence matching processing on the extracted information;

A display unit, used to display full-modality medical image sequences in groups;

The signal input terminal of the input unit is connected to an external information input device, the signal output terminal of the input unit is connected to the signal input terminal of the first processing unit, and the signal output terminal of the first processing unit is connected to the signal input terminal of the second processing unit. The signal input end is connected, and the signal output end of the second processing unit is connected with the signal input end of the display unit.

Further, the second processing unit is provided with an all-modal deep learning AI sequence matching system.

The present invention also designs a computer storage medium on which computer program instructions are stored, and when the computer program instructions are executed by the processor, the following methods are realized:

S1 obtains medical image information;

S2 performs information extraction on the acquired medical image information;

S3 establishes a full-modal deep learning AI sequence matching system;

S4 for medical image annotation dataset;

S5 performs bone image segmentation and AI algorithm recognition;

S6 performs spine bone segmentation and identification test data;

S7 completes full-modal AI fusion matching;

S8 transmits the processed medical image sequence groupings to the display unit;

The S9 display unit performs grouped display of full-modality medical image sequences.

Compared with the prior art, the present invention uses a deep learning neural network, and based on the accurate anatomical information of human CT and MR, the human skeleton is accurately identified and segmented into relatively fixed local areas, and the specific bone parts are used to perform accurate body position segmentation, with a partial Organ structure information, according to the inspection requirements, identify the part of interest, and accurately locate the CT or MR image in the dual-modality molecular imaging, convert it to the corresponding level of the full-modality image, and perform automatic and accurate registration and display, which greatly reduces the manual operation of doctors. Matching layers can reduce diagnostic errors caused by differences in technical levels, and at the same time improve the work efficiency of doctors, and the invention has the function of recognition and registration of special modal images of molecular images, such as full-body PET dynamic fusion image display, SPECT quantitative reconstruction image, SPECT- RAWDATA truly realizes full-modality image registration and browsing. Data from different modalities can be complemented to prevent omission of effective information, so as to provide data support for subsequent judgments more accurately, without relying on the doctor's ability, and has a wider range of applications.

The present invention can perform grouped display of full-modal medical image sequences, and the grouping method is realized based on self-defined rules, such as the order of key parts, time order, etc., the display is more intuitive, and the efficiency is improved.

Description of drawings

FIG. 1 is a schematic flowchart of a method for grouping a full-modality medical image sequence based on a deep learning sign structure in a specific embodiment.

2 is a schematic diagram of a sequence matching process performed in a full-modal deep learning AI sequence matching system in a full-modality medical image sequence grouping method based on a deep learning sign structure in a specific embodiment.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings, but it is not intended to limit the present invention.

Referring to Fig. 1, the present invention designs a full-modality medical image sequence grouping method based on deep learning sign structure, including:

S1 obtains medical image information, that is, normal modal whole-body skeleton image or PET/CT and PET/MR fused image on the same machine or full modal image heterogeneous fusion image;

S2 performs information extraction on the acquired medical image information. When the medical image information is a normal state whole-body skeleton image, the information extraction method is to extract the segmented image of the fixed bone part; when the medical image information is PET/CT and PET/MR on the same machine When one or both of the fusion graphics and the all-modal image heterogeneous fusion images are used, the information extraction method is to extract the same-machine segmented images of the PET corresponding bone region;

S3 establishes a full-modal deep learning AI sequence matching system;

S4 performs sequence matching processing. First, the medical image data set is labeled, skeletal image segmentation and AI algorithm recognition are performed, and the segmented spine image is identified and tested. Finally, full-modal AI fusion matching is performed to form medical image sequence grouping;

The present invention also designs a system for a full-modality medical image sequence grouping method based on the deep learning sign structure, including: an input unit for inputting medical image information;

The present invention also designs a computer storage medium on which computer program instructions are stored, and when the computer program instructions are executed by the processor, the following methods are implemented:

S1 obtains medical image information;

S2 performs information extraction on the acquired medical image information;

S3 establishes a full-modal deep learning AI sequence matching system;

S4 for medical image annotation dataset;

S5 performs bone image segmentation and AI algorithm recognition;

S6 performs spine bone segmentation and identification test data;

S7 completes full-modal AI fusion matching;

Example:

Import the PET/CT and PET/MR images of the same patient at the same time into the input unit of the present invention, extract 50 corresponding data sets through the first processing unit, and transmit the data sets to the second processing unit. The whole body position CT and MR parts are used to extract the image of the fixed bone part segmentation area. The fixed bone parts taken are the spine and the skull. AI is used to identify the images of these two parts and match them, and then use the full-modal AI intelligent matching to import the PET image. , matching PET/CT/MR images, according to the calling requirements, such as the focus of the lesion is in the abdomen, the calling range is the thoracic spine or the lumbar spine, or it can be located in a single spine range, which can quickly display the PET/CT/MR of the corresponding part, modal The combination can be adjusted arbitrarily, which greatly improves the diagnostic efficiency and avoids the mistakes of image mismatch.

The present invention utilizes a deep learning neural network, and based on the accurate anatomical information of human CT and MR, the human skeleton is accurately identified and segmented into relatively fixed local areas, and the precise body position segmentation is performed with specific bone parts, and the structural information of some organs is attached. According to the requirements, identify the part of interest, accurately locate the CT or MR image in the dual-modality molecular imaging, convert it to the corresponding level of the full-modality image, and perform automatic and accurate registration and display, which greatly reduces the doctor's manual matching of layers and reduces the technical level. Diagnosis errors caused by differences, while improving the efficiency of physicians. Moreover, the present invention adopts the special modal image recognition and registration function of molecular images, such as whole body PET dynamic fusion image display, SPECT quantitative reconstruction image, SPECT-RAWDATA, and truly realizes full modal image registration and browsing.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware by a computer program, and the program can be stored in a computer-readable storage medium, such as an embodiment of the present invention , the program can be stored in a storage medium of a computer system and executed by at least one processor in the computer system to implement the processes including the embodiments of the above-mentioned methods. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity, all possible combinations of the technical features in the above-described embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, all It should be considered as the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present invention, and the descriptions thereof are specific and detailed, but should not be construed as limiting the scope of the present invention. It should be pointed out that for those of ordinary skill in the art, , without departing from the concept of the present invention, several modifications and improvements can be made, which all belong to the protection scope of the present invention.

Claims

The full-modality medical image sequence grouping method based on deep learning sign structure is characterized by: including:

S1 obtains medical image information;

S2 performs information extraction on the acquired medical image information;

S3 establishes a full-modal deep learning AI sequence matching system;

S4 performs sequence matching processing;

S5 transmits the processed medical image sequence groupings to the display unit;

The S6 display unit performs grouped display of full-modality medical image sequences.
The full-modality medical image sequence grouping method based on deep learning sign structure according to claim 1, wherein the medical image information is a normal modal whole-body skeleton image, a PET/CT and PET/MR fusion graph on the same machine , One or more of the all-modal image heterogenous fusion images.
The full-modality medical image sequence grouping method based on deep learning sign structure according to claim 1, characterized in that: the information extraction is to extract a segmented image of a fixed skeletal part, and to extract a segmented image of a PET corresponding skeletal region on the same machine. one or more of them.
The full-modality medical image sequence grouping method based on deep learning sign structure according to claim 1, characterized in that: the specific method of the sequence matching processing is as follows:

S41 Labeling datasets for medical images;

S42 performs bone image segmentation and AI algorithm recognition;

S43 performs spine bone segmentation and identification test data;

S44 full-modal AI fusion matching.
The full-modality medical image sequence grouping method based on deep learning sign structure according to claim 2, characterized in that: when the medical image information is a normal modal whole-body skeleton image, the information extraction method is to extract an image of a segmented area of a fixed bone part .
The full-modality medical image sequence grouping method based on deep learning sign structure according to claim 2, characterized in that: when the medical image information is PET/CT and PET/MR fusion graphics on the same machine, and full-modality image fusion on the same machine When one or two of the images are used, the information extraction method is to extract the same-machine segmented images of the PET-corresponding bone region.
The system for a full-modality medical image sequence grouping method based on a deep learning sign structure according to any one of claims 1-6, characterized in that: comprising: an input unit for inputting medical image information;

a first processing unit, configured to perform information extraction on the input medical image information;

a second processing unit, configured to perform sequence matching processing on the extracted information;

A display unit, used to display full-modality medical image sequences in groups;

The signal input terminal of the input unit is connected to an external information input device, the signal output terminal of the input unit is connected to the signal input terminal of the first processing unit, and the signal output terminal of the first processing unit is connected to the signal input terminal of the second processing unit. The signal input end is connected, and the signal output end of the second processing unit is connected with the signal input end of the display unit.
The system for a full-modality medical image sequence grouping method based on a deep learning sign structure according to claim 7, wherein the second processing unit is provided with a full-modality deep learning AI sequence matching system.
A computer storage medium having computer program instructions stored thereon, characterized in that: when the computer program instructions are executed by a processor, the method of any one of claims 1 to 6 is implemented.