WO2021115313A1

WO2021115313A1 - System for automatically sketching contour line of normal organ in medical image

Info

Publication number: WO2021115313A1
Application number: PCT/CN2020/134841
Authority: WO
Inventors: 魏军; 沈烁; 谢培梁; 郑少逵; 吕丽云; 田孟秋
Original assignee: 广州柏视医疗科技有限公司
Priority date: 2019-12-10
Filing date: 2020-12-09
Publication date: 2021-06-17
Also published as: CN111008984A; WO2021115312A1; CN112508965B; CN111008984B; CN112508965A

Abstract

A system for automatically sketching a contour line of a normal organ in a medical image. The system for automatically sketching a contour line comprises: a patient image preprocessing module for acquiring a patient image collected before a medical image, and preprocessing the patient image; a normal organ grouping and target normal organ sub-partition positioning module for grouping, grade by grade, all normal organs to be segmented, and gradually positioning target normal organ sub-partitions using an iterative method; and a normal organ segmentation module for performing grading according to the segmentation difficulty of each normal organ in the normal organ sub-partitions, and automatically sketching a contour line of a target normal organ on an image corresponding to the normal organ sub-partitions by using an iterative constraint type normal organ segmentation model until all the normal organs in all the normal organ sub-partitions have been segmented.

Description

Automatic outline drawing system of normal organs in medical imaging

Technical field

The invention relates to the technical field of medical image processing, in particular to a system for automatically drawing contour lines of normal organs in medical images.

Background technique

Radiotherapy is currently one of the three important methods of clinical tumor treatment. In radiotherapy, the delineation of target areas and organs at risk has a crucial influence on the accuracy of radiotherapy. At present, the contours of target areas and organs at risk are mainly obtained by doctors manually delineating the contours of the target areas and organs at risk in clinic. Manual sketching by doctors has the following drawbacks: 1. The sketching efficiency is low; 2. It is heavily dependent on the doctor's clinical experience; 3. The reproducibility is poor, and the results of the sketching by different doctors at different times and different conditions are inconsistent. Therefore, there is an urgent need for accurate and fast automatic medical image segmentation algorithms to reduce the burden on doctors and improve the accuracy and automation of normal organ segmentation in medical images.

The atlas-based segmentation method is a popular method for automatic delineation of normal organs in medical imaging, especially in the medical imaging of head and neck tumors. Because the head and neck structure has a relatively fixed positional relationship, atlas-based segmentation The method has a good performance in the segmentation of normal organs of the head and neck. Atlas-based segmentation methods are generally divided into single atlas and multiple atlas methods. However, the single atlas segmentation method is very sensitive to the choice of atlas and the difference in the anatomical structure of the patient. When the target image is significantly different from the atlas, the single atlas method may fail to segment. The multi-atlas segmentation method can reduce the sensitivity to the difference between atlas and patients, and has higher segmentation accuracy than single atlas, but the segmentation efficiency is lower. In addition, the atlas-based segmentation method relies on image registration algorithms, which may introduce additional registration errors. The atlas-based normal organ segmentation method has many defects and cannot meet clinical needs. In recent years, machine learning and deep learning, especially Convolutional Neural Network (CNN), have achieved great success in the fields of image classification, computer vision, and target extraction. Many researchers also apply it to the segmentation of medical images. For example, in 2017, Ibragimov, B and others proposed a head and neck normal organ segmentation method based on convolutional neural networks [Ibragimov B and Xing L 2017 Segmentation of organs-at-risks in head and neck CT images using convolutional neural networks ,Medical physics 44 547-57], and applied it to the automatic segmentation of 9 normal organs in head and neck medical imaging. The method consists of the following steps: (1) Roughly determine the region of interest of the target normal organ according to the relative positional relationship between the normal brain organs and the center coordinates of the brain; (2) Based on the target pixel points and background in the target normal organ region of interest The image patch where the pixel is located is trained on a classification model based on a convolutional neural network; (3) Then, all pixels are classified on the region of interest of the target normal organ on the image to be segmented, so as to realize the normal organ in the image (4) Finally, use Markov random field to post-process the segmentation results to remove some over-segmented pixels. This method uses the fixed position relationship of the normal organs of the brain to roughly determine the region of interest, and uses the results drawn by the doctor to train the normal organ segmentation model to realize the automatic segmentation of multiple normal organs in the head and neck.

Compared with the traditional normal organ segmentation method based on atlas, this method has higher segmentation accuracy on most normal organs, but the image segmentation method based on convolutional neural network is based on the characteristics of the data drawn by the doctor to learn the target. Therefore, the target area can be better recognized and segmented from the image, while the contrast of the image such as the optic nerve and the optic chiasm is low, and the smaller normal organs have less effective information on the image. Therefore, the conventional patch-based method still has low accuracy in segmentation on such normal organs. Moreover, the image contrast is low, which makes the image segmentation of small normal organs more dependent on the three-dimensional image environment, but the current hardware level is difficult to support the training of the convolutional neural network model under the large three-dimensional image matrix, so It is still a very challenging problem to segment the target normal organs of different sizes and different gray levels from clinical medical images.

Summary of the invention

In view of the problems existing in the prior art, embodiments of the present invention provide a method and system for automatically delineating the contour lines of normal organs in medical images.

In the first aspect, an embodiment of the present invention provides a method for automatically delineating a contour line of a normal organ in a medical image, which includes the following steps:

Step S1: Obtain the patient image collected before the medical image, and preprocess it;

Step S2: Group all the normal organs to be segmented step by step, and use an iterative method to gradually locate the target normal organ sub-regions;

Step S3: Perform classification according to the segmentation difficulty of each normal organ in the determined normal organ sub-region; and use an iterative constrained normal organ segmentation model on the image corresponding to the determined normal organ sub-region to automatically perform the contour line of the target normal organ Delineate until all normal organs in all normal organ subdivisions are segmented.

Further, the preprocessing of the patient image includes: resampling and normalization of the image gray level.

Further, step S2 specifically includes the following steps:

S21: All When normal organ to be segmented as a target, various dimensions of the patient images preprocessed in the step S1 is obtained 2 ⁿ times down-sampling, a convolutional neural network model based on the 2 ⁿ times downsampling Perform target area recognition on the resulting image to obtain the rough positions of all target normal organs, and then crop the preprocessed image obtained in step S1 according to the prior information of the center coordinates of the target area and the size of the target normal organ , Remove most of the background area in the image;

S22: The same partition when the normal organs as a target, the respective dimensions of the trimmed image obtained in step S21 is 2 ^n-1 times down-sampling, a convolutional neural network model based on 2 ^n-1 times the drop Perform normal organ partition recognition on the sampled image, and perform region cropping on the preprocessed image obtained in step S1 according to different normal organ partition recognition results to obtain images of each normal organ partition;

S23: Iterate step by step until the positions of all normal organ sub-divisions are located, and the images corresponding to each normal organ sub-division are cut out.

Further, step S3 specifically includes the following steps:

S31: Taking the image corresponding to the normal organ sub-division determined in step S2 as input, and segmenting the first-level normal organ based on the convolutional neural network model to obtain the first-level normal organ segmentation result;

S32: Taking the first-level normal organ segmentation result obtained in step S31 and the image corresponding to the normal organ sub-region determined in step S2 as input, and restricting the second-level normal organ segmentation based on the volume The product neural network model segmented the second-level normal organs to obtain the second-level normal organ segmentation results;

S33: Iterate step by step, taking the segmentation results of all normal organs at the segmented level and the images corresponding to the normal organ sub-regions determined in step S2 as input, and constrain the segmentation of normal organs at the current level, and based on convolution The neural network model segments the normal organs at the current segmentation level, and obtains the segmentation results of the normal organs at the current level until all normal organs are segmented.

Further, the convolutional neural network model specifically includes the following steps:

Establish a convolutional neural network model. The convolutional neural network model takes the segmentation results of patient images and other known normal organs as input, and the segmentation results as output;

Collect patient images collected before medical imaging and outlines of normal organs drawn by experienced doctors; preprocess the collected patient images, and then convert the outlines of each normal organ drawn by doctors into mask images;

The preprocessed patient image is used as the input of the convolutional neural network model, and the loss function of the current segmentation model is calculated according to the current output of the convolutional neural network model and the collected mask images of the normal organs outlined by the doctor, using back propagation The method updates the parameters of the convolutional neural network model; iterates repeatedly, and when the preset number of model training iterations is reached or the loss function reaches the preset threshold, the convolutional neural network model training is completed and the model parameters are saved.

Further, the automatic drawing of the contour line of the target normal organ in step S3 includes the following steps:

Import the corresponding trained convolutional neural network model;

Input the corresponding image and the segmentation results of other known normal organs into the trained convolutional neural network model to obtain the mask image of the normal organ;

According to the obtained mask image of normal organs, it is transformed into contour lines.

Further, the mask image is a binary mask image.

In a second aspect, an embodiment of the present invention provides a system for automatically delineating the contours of normal organs in medical images, including:

Patient image preprocessing module: used to obtain and preprocess patient images collected before medical images;

Normal organ grouping and target normal organ sub-region module: used to group all normal organs to be segmented step by step, and use an iterative method to gradually locate the target normal organ sub-region;

Normal organ segmentation module: used to classify the segmentation difficulty of each normal organ in the normal organ sub-division; and use the iterative constrained normal organ segmentation model on the image corresponding to the normal organ sub-division to automatically perform the contour line of the target normal organ Delineate until all normal organs in all normal organ subdivisions are segmented.

In a third aspect, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, the implementation is as described in the first aspect. Provides the steps of the method for automatically delineating the contour lines of normal organs in medical images.

In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the outline of a normal organ in a medical image as provided in the first aspect is realized. The steps of the automatic line drawing method.

The method and system for automatically delineating the contours of normal organs in medical images provided by the embodiments of the present invention are aimed at the problem of high hardware requirements for segmentation under large three-dimensional images, and adopts an iterative method to gradually reduce background areas and reduce volume-based The computational complexity of the segmentation model of the product network greatly reduces its requirements for hardware devices. In addition, for the problem of low image contrast and low segmentation accuracy of normal organs with small volumes, in the iterative segmentation framework, normal organs are segmented from easy to difficult, and the normal organ segmentation results of the previous iteration are used to constrain The segmentation of normal organs in the next iteration improves the accuracy of segmentation.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are For some of the embodiments of the present invention, those of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

FIG. 1 is a flowchart of a method for automatically delineating contour lines of normal organs in medical images according to an embodiment of the present invention;

2 is a flowchart of step S2 in the method provided by an embodiment of the present invention;

FIG. 3 is a flowchart of step S3 in the method provided by an embodiment of the present invention;

4 is a flowchart of a convolutional neural network model in a method provided by an embodiment of the present invention;

FIG. 5 is a flowchart of automatically delineating the contour line of the target normal organ in step S3 of the method provided by the embodiment of the present invention;

Fig. 6 is a schematic diagram of a system for automatically delineating the contours of normal organs in medical images according to an embodiment of the present invention;

FIG. 7 is a framework diagram of iterative segmentation in the method provided by an embodiment of the present invention;

FIG. 8 is a block diagram of an iterative constrained normal organ segmentation model in the method provided by an embodiment of the present invention;

FIG. 9 is a physical structure diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

Fig. 1 is a flow chart of a method for automatically delineating the contours of normal organs in medical images according to an embodiment of the present invention. As shown in Fig. 1, the method includes:

Step S1: Obtain patient images collected by medical images and preprocess them;

Patient images include: CT (Computed Tomography), MR (magnetic resonance) or PET (Positron Emission Tomography), etc. Among them, CT stands for electronic computer tomography, which uses precisely collimated X-ray beams, gamma rays, ultrasonic waves, etc., together with extremely sensitive detectors to scan a certain part of the human body one by one. MR is a method of medical examination and a revolution in medical imaging. Biological tissues can be penetrated by short-wave components in the electromagnetic spectrum such as X-rays, but can block mid-wave components such as ultraviolet, infrared and long waves. Human tissue allows long-wave components such as radio waves generated by magnetic resonance to pass through, which is one of the basic conditions for magnetic resonance in clinical applications. PET is a relatively advanced clinical examination imaging technology in the field of nuclear medicine. Normal range PET is especially suitable for early diagnosis of disease, discovery of subclinical lesions and evaluation of treatment effect before there is no morphological change. At present, PET has shown particularly important value in the diagnosis and treatment of the three major types of diseases: tumor, coronary heart disease and brain disease.

In step S1 of the embodiment of the present invention, preprocessing the patient image includes: resampling and image gray normalization.

Step S2: Divide all normal organs to be segmented into groups (for example: level 1: all normal organs; level 2: normal organs partition; level 3: normal organ sub-partitions; level 4: normal organs), and adopt an iterative method Gradually locate the target normal organ sub-region;

As shown in FIG. 2, step S2 in the embodiment of the present invention specifically includes the following steps:

S21: Regard all normal organs to be segmented as a target, perform 2 ⁿ times downsampling on each dimension of the preprocessed patient image obtained in step S1, based on the convolutional neural network model after 2 ⁿ times downsampling Recognize the target area on the image to obtain the rough positions of all target normal organs, and then crop the preprocessed image obtained in step S1 according to the prior information of the center coordinates of the target area and the size of the target normal organ to remove the image Most of the background area;

After normal organs when the same partition as a target, various dimensions of the cropped image obtained in step S21 is 2 ^n-1 times down-sampling, a convolutional neural network model based on the 2 ^n-1 times downsampling: S22 Perform normal organ partition recognition on the image of, and perform region cropping on the preprocessed image obtained in step S1 according to different normal organ partition recognition results to obtain images of each normal organ partition;

Step S3: Perform classification according to the division difficulty of each normal organ in the normal organ sub-division determined in step S2 (for example, level I: easy, level II: general, and level III: difficult). On the image corresponding to the normal organ sub-division determined in step S2, the iterative constrained normal organ segmentation model as shown in Fig. 8 is used to automatically delineate the contour line of the target normal organ until all normal organs in all normal organ sub-divisions are The organ segmentation is complete.

As shown in Fig. 3, step S3 specifically includes the following steps:

S31: Taking the image corresponding to the normal organ sub-region determined in step S2 as input, and segmenting the normal organs of level I (level 1) based on the convolutional neural network model to obtain normal organs of level I (level 1) Segmentation result

S32: The segmentation result of the normal organ of level I (level 1) obtained in step S31 and the image corresponding to the normal organ sub-region determined in step S2 are used as input, and the segmentation of the normal organ of level II (level 2) is constrained , Based on the convolutional neural network model, segment the normal organs of level II (level 2), and obtain the segmentation results of normal organs of level II (level 2);

S33: Iterate step by step, taking the segmentation results of all normal organs at the segmented level and the images corresponding to the normal organ sub-regions determined in step S2 as input, and constraining the segmentation of normal organs at the current level, based on the convolutional neural network model , To segment the normal organs at the current segmentation level to obtain the segmentation results of the normal organs at the current level until all normal organs are segmented.

Figure 7 shows the frame diagram of iterative segmentation. The convolutional neural network model used in step S2 and step S3 adopts a supervised learning method, based on pre-collected patient image data and sketched by experienced doctors. The normal organ contour data and the known segmentation results of other normal organs (if any) are trained to obtain a stable normal organ detection model, normal organ sub-division detection model, and normal organ segmentation model corresponding to the sub-division, as shown in Figure 7. The dashed part is shown. As shown in Figure 4, the convolutional neural network model specifically includes three steps:

(A) Establish a convolutional neural network model. The convolutional neural network model takes the patient image and the segmentation results of other known normal organs (if any) as input, and the segmentation results as output;

(B) Collect collected patient images (CT, MR or PET), and contour lines of normal organs drawn by experienced doctors. And pre-process the collected patient image resampling and image gray normalization, and then convert the contour line of each normal organ drawn by the doctor into a binary mask image with a target area of 1 and a background area of 0;

(C) The patient image preprocessed in step (B) is used as the input of the convolutional neural network model, based on the current output of the convolutional neural network model and the masks of the normal organs drawn by the doctors collected in step (B) Model image, calculate the loss function of the current segmentation model, and use the back propagation method to update the parameters of the convolutional neural network model. Iterate repeatedly. When the preset number of model training iterations is reached or the loss function reaches the preset threshold, the convolutional neural network model training is completed and the model parameters are saved.

Wherein, as shown in FIG. 5, the automatic drawing of the contour line of the target normal organ in step S3 includes the following steps:

(A) Import the corresponding trained convolutional neural network model;

(B) Input the corresponding image and the segmentation results of other known normal organs (if they exist) into the trained convolutional neural network model to obtain the binary mask image of the normal organs (that is, the target area is 1, the background area is 0);

(C) According to step (B), the mask image of the normal organ is converted into contour lines.

Based on any of the foregoing embodiments, FIG. 6 is a schematic structural diagram of a system for automatically delineating the contours of normal organs in medical imaging according to an embodiment of the present invention, and the system includes:

In summary, the method and system for automatically delineating the contours of normal organs in medical images provided by the embodiments of the present invention addresses the problem of high hardware requirements for segmentation under large three-dimensional images, and adopts an iterative method to gradually reduce the background area. Reduce the computational complexity of the segmentation model based on the convolutional network, so that its requirements for hardware devices are greatly reduced. In addition, for the problem of low image contrast and low segmentation accuracy of normal organs with small volumes, in the iterative segmentation framework, normal organs are segmented from easy to difficult, and the normal organ segmentation results of the previous iteration are used to constrain The segmentation of normal organs in the next iteration improves the accuracy of segmentation.

FIG. 9 is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. As shown in FIG. 9, the electronic device may include: a processor 301, a communications interface 302, and a memory 303 And the communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304. The processor 301 can call a computer program stored in the memory 303 and running on the processor 301 to execute the methods provided in the foregoing embodiments, for example, including:

Acquire patient images collected before medical imaging and preprocess them;

All normal organs to be segmented are grouped step by step, and an iterative method is used to gradually locate the target normal organ sub-regions;

The classification is based on the difficulty of segmentation of each normal organ in the normal organ sub-division; and the iterative constrained normal organ segmentation model is used on the image corresponding to the normal organ sub-division to automatically delineate the contour line of the target normal organ until all normal organs are sub-division. All normal organs in the sub-area have been segmented.

In addition, the above-mentioned logical instructions in the memory 303 can be implemented in the form of a software functional unit and when sold or used as an independent product, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention can be embodied in the form of software products in essence or parts that contribute to the prior art or parts of the technical solutions, and the computer software products are stored in a storage medium. , Including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .

The embodiment of the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. The computer program is implemented when executed by a processor to perform the methods provided in the foregoing embodiments, for example, including:

Obtain collected patient images and preprocess them;

The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement it without creative work.

Through the description of the above implementation manners, those skilled in the art can clearly understand that each implementation manner can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic A disc, an optical disc, etc., include several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiment.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments are modified, or some of the technical features thereof are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

A system for automatically delineating the contours of normal organs in medical imaging, which is characterized in that it comprises:

Patient image preprocessing module: used to obtain and preprocess patient images collected before medical images;

Normal organ grouping and target normal organ sub-region module: used to group all normal organs to be segmented step by step, and use an iterative method to gradually locate the target normal organ sub-region;

Normal organ segmentation module: It is used to classify the segmentation difficulty of each normal organ in the normal organ sub-division; and on the image corresponding to the normal organ sub-division, the iterative constrained normal organ segmentation model is used to automatically perform the contour line of the target normal organ Delineate until all normal organs in all normal organ sub-divisions are segmented; the iterative constrained normal organ segmentation model specifically includes the following steps:

S31: Taking the image corresponding to the normal organ sub-region determined by the normal organ grouping and positioning target normal organ sub-region module as input, and segmenting the normal organs of the first level based on the convolutional neural network model to obtain the first level Segmentation results of normal organs;

S32: Use the segmentation result of the first-level normal organs obtained in the step S31 and the images corresponding to the normal organ sub-regions determined by the normal organ grouping and positioning target normal organ sub-region module as input, and compare the second-level normal organs Constrain the segmentation, and segment the second-level normal organs based on the convolutional neural network model to obtain the second-level normal organ segmentation results;

S33: Iterate step by step, taking the segmentation results of all normal organs of the segmented level and the images corresponding to the normal organ sub-division determined by the normal organ grouping and positioning target normal organ sub-division module as input, and the current level of normal organs The segmentation is constrained, and the normal organs of the current segmentation level are segmented based on the convolutional neural network model to obtain the segmentation results of the normal organs at the current level, until all normal organs are segmented.