WO2022198554A1

WO2022198554A1 - Method and system for three-dimensional image guided positioning of organs in motion, and storage medium

Info

Publication number: WO2022198554A1
Application number: PCT/CN2021/082939
Authority: WO
Inventors: 申国盛; 李强; 刘新国; 戴中颖; 金晓东; 贺鹏博
Original assignee: 中国科学院近代物理研究所
Priority date: 2021-03-25
Filing date: 2021-03-25
Publication date: 2022-09-29

Abstract

The present invention relates to a method and system for three-dimensional image guided positioning of organs in motion, and a storage medium. The method comprises: on the basis of real-time DR images of a patient, generating a virtual 3D-CT image set of the patient by using an artificial intelligence network algorithm; applying an automatic tissue and organ segmentation algorithm to a 3D-CT image set of the patient so as to automatically obtain tissues and organs by means of segmentation, and by means of a 3D tissue and organ model reconstruction algorithm, reconstructing a 3D model of the tissues and organs of the patient that are in motion; applying the automatic tissue and organ segmentation algorithm to the virtual 3D-CT image set of the patient so as to automatically obtain virtual tissues and organs by means of segmentation, and by means of the 3D tissue and organ model reconstruction algorithm, reconstructing a 3D model of the virtual tissues and organs of the patient that are in motion; and performing a registration calculation on the 3D model of the tissues and organs of the patient that are in motion and the 3D model of the virtual tissues and organs thereof that are in motion, and outputting 3D model shapes and motion offsets of specified tissues and organs that are in motion. By means of the present invention, 3D shape and position change information of in-vivo tissues and organs which are in motion can be obtained.

Description

Three-dimensional image-guided motion organ localization method, system and storage medium

technical field

The invention relates to a three-dimensional (3D) image-guided moving organ positioning method, system and storage medium based on artificial intelligence technology and digital X-ray imaging (DR) system in radiotherapy, and relates to the technical field of radiotherapy.

Background technique

The purpose of radiation therapy is to use radioactive rays to kill tumor cells while maximizing the protection of the patient's normal tissues and organs. However, in radiotherapy, some tissues and organs and tumor target areas in the patient will be displaced due to physiological movements such as the patient's breathing movement, resulting in poor treatment effect for the patient. Therefore, the protection and treatment of moving organs and target areas are the difficulties of radiotherapy.

The current image-guided radiotherapy technology for guiding moving organs and tumor target areas mainly includes: respiratory gating system, which collects the patient's breathing signal to predict the displacement of the patient's tissues and organs with the breathing movement to guide radiotherapy; in the patient's moving organs The gold standard is implanted in the target area, and two intersecting DR imaging devices are used to detect and track the position of the implanted gold standard for the treatment of moving tissues and organs; cone beam CT (CBCT) and rail CT (CT-on-rail) The image guidance system registers the reconstructed CBCT or orbital CT image with the CT image of the treatment plan, obtains the state and position of the patient's functional organs and target volume to guide the patient's treatment, etc.

However, in the prior art, the respiratory gating system can only predict the approximate displacement of a part of the tissues and organs in the patient with the breathing movement based on the patient's breathing signal information. This technology cannot directly detect the movement of the tissues and organs in the patient, and does not It is suitable for the movement of tissues and organs produced by other physiological movements in the patient's body. Implanting a gold label in a patient will cause secondary damage to the patient, and is not suitable for patients with weak constitutions and young and old patients. The DR imaging system can only track the position of the implanted gold label, but cannot detect tissue, organs and tumor target areas. The 3D shape and coordinates do not realize 3D guidance in the true sense, and the other two intersecting DR imaging devices will add a large additional radiation dose to the patient, and the system is expensive. CBCT and orbital CT imaging systems add additional high radiation doses to the patient, increasing the risk of patient complications, and the CBCT image density resolution is low, the system is expensive, and the accuracy and speed of patient planning CT registration is not high.

To sum up, it is necessary to study the method and system to realize 3D high-precision guidance of moving organs and target areas while reducing the price of equipment.

SUMMARY OF THE INVENTION

In view of the above problems, the purpose of the present invention is to provide a kind of artificial intelligence technology and digital X-ray imaging that can accurately obtain the 3D shape and position change information of moving tissues and organs and tumor target areas in vivo, and guide the radiotherapy of moving organs and target areas. The system's three-dimensional image-guided moving organ positioning method, system and storage medium.

To achieve the above object, the present invention adopts the following technical solutions:

In a first aspect, the present invention provides a three-dimensional image-guided moving organ positioning method, comprising:

Based on the patient's real-time DR images, the artificial intelligence network algorithm is used to generate the patient's virtual 3D-CT image set;

The patient's 3D-CT image set is automatically segmented into tissues and organs by the tissue and organ automatic segmentation algorithm, and the 3D model of the patient's moving tissues and organs is reconstructed through the 3D tissue and organ model reconstruction algorithm;

The patient's virtual 3D-CT image set is automatically segmented into virtual tissues and organs by the tissue and organ automatic segmentation algorithm, and the 3D model of the patient's virtual moving tissues and organs is reconstructed through the 3D tissue and organ model reconstruction algorithm;

The 3D model of the patient's moving tissues and organs and the 3D model of the virtual moving tissues and organs are registered and calculated, and the 3D model shape and motion offset of the specified moving tissues and organs are output.

Further, the real-time DR image of the patient is acquired by using a DR imaging device.

Further, the DR imaging device includes a set of X-ray sources and a corresponding imaging flat panel;

The X-ray source is installed on the top of the treatment room, and the imaging plate is installed on the floor portion of the treatment room, and each uses a small-angle track to move; or,

The X-ray source and the imaging plate are connected by a C-arm for small-angle movement.

Further, the artificial intelligence network algorithm is obtained through training and verification, including: using the DR imaging equipment to take the DR image of the patient, and using the CT system to take the 3D-CT image of the same part of the same patient, so that the DR image and the 3D-CT image of the patient are the same. One correspondence, establish the DR image and the corresponding 3D-CT image data set; take part of the data in the established data set as the training data set, and the other part as the verification data set, build a neural network model for training and verification, and iterate continuously through operations Obtain the weight parameters of the artificial intelligence network, and then obtain the trained artificial intelligence network model.

Further, the automatic segmentation algorithm of tissues and organs adopts a deep learning-based convolutional neural network model, which can automatically segment tissues and organs according to the input CT images.

Further, the 3D tissue and organ reconstruction algorithm can reconstruct 3D models of all or specified tissues and organs, and can render and display different tissues and organs in different colors and modes, which is convenient for users to observe and distinguish operations.

Further, the registration calculation employs a 3D tissue-organ registration algorithm for manual and/or automatic 3D model registration.

In a second aspect, the present invention also provides a three-dimensional image-guided motion organ positioning system, the system comprising:

The virtual image generation unit is configured to generate a virtual 3D-CT image set of the patient based on the real-time DR image of the patient using an artificial intelligence network algorithm;

The organ reconstruction unit is configured to automatically segment the tissues and organs from the patient's 3D-CT image set using the tissue-organ automatic segmentation algorithm, and reconstruct the 3D model of the patient's moving tissues and organs through the 3D tissue-organ model reconstruction algorithm;

The virtual organ reconstruction unit is configured to automatically segment the virtual tissue and organ from the patient's virtual 3D-CT image set using the tissue-organ automatic segmentation algorithm, and reconstruct the 3D model of the patient's virtual moving tissue and organ through the 3D tissue-organ model reconstruction algorithm;

The offset calculation unit is configured to perform registration calculation on the 3D model of the patient's moving tissue and organ and the 3D model of the virtual moving tissue and organ, and output the shape of the 3D model and the movement offset of the selected moving tissue and organ.

In a third aspect, the present invention further provides a processing device, the processing device includes at least a processor and a memory, and a computer program is stored on the memory, characterized in that the processor executes the computer program to realize The three-dimensional image-guided moving organ positioning method described in the first aspect of the present invention.

In a fourth aspect, the present invention further provides a computer storage medium having computer-readable instructions stored thereon, and the computer-readable instructions can be executed by a processor to implement the three-dimensional image-guided movement organ positioning method according to the first aspect of the present invention .

The present invention has the following advantages due to taking the above technical solutions:

1. The artificial intelligence algorithm provided by the present invention generates a virtual 3D-CT image of the patient according to a small number of DR images, and manually and/or uses the artificial intelligence algorithm to automatically segment the virtual 3D-CT image and the planned CT image to segment the moving tissues, organs and tumor targets in the body. Perform 3D reconstruction and registration to obtain 3D shape and position change information of moving tissues and organs and tumor target areas in vivo, guide the radiotherapy of moving organs and target areas, and solve the defects and deficiencies in conventional DR image and CBCT image guidance;

2. The equipment cost required for the implementation of the present invention is low, only a single DR imaging equipment is required, and the 3D state and position information of the tissues and organs in the patient can be obtained without using the implanted gold standard method. The additional radiation dose is lower and the equipment is inexpensive;

In conclusion, the present invention can be widely used in radiotherapy.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be considered limiting of the invention. The same reference numerals are used to refer to the same parts throughout the drawings. In the attached image:

FIG. 1 is a schematic flowchart of a three-dimensional image-guided radiotherapy method for moving organs according to an embodiment of the present invention;

2 is a schematic diagram of a DR imaging device according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an artificial intelligence network algorithm for generating virtual 3D-CT images according to the present invention.

Detailed ways

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present invention will be more thoroughly understood, and will fully convey the scope of the present invention to those skilled in the art.

It is to be understood that the terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" can also be intended to include the plural forms unless the context clearly dictates otherwise. The terms "comprising", "comprising", "containing" and "having" are inclusive and thus indicate the presence of stated features, steps, operations, elements and/or components, but do not preclude the presence or addition of one or Various other features, steps, operations, elements, components, and/or combinations thereof. Method steps, procedures, and operations described herein are not to be construed as requiring that they be performed in the particular order described or illustrated, unless an order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

For ease of description, spatially relative terms may be used herein to describe the relationship of one element or feature to another element or feature as shown in the figures, such as "inner", "outer", "inner" ", "outside", "below", "above", etc. This spatially relative term is intended to include different orientations of the device in use or operation other than the orientation depicted in the figures.

Computer technology, especially artificial intelligence technology, has shown excellent performance in computer vision and medical image processing segmentation and multi-modal image generation. More and more multi-modal image generation and automatic segmentation technologies are realized.

Example 1

Based on artificial intelligence technology, as shown in FIG. 1 , the method for 3D-DR image-guided positioning of moving tissues and organs provided in this embodiment includes the following contents:

S1: A DR imaging device is provided.

Specifically, according to the schematic diagram of the DR imaging device shown in FIG. 2 , the DR imaging device in this embodiment includes a set of X-ray sources and an imaging flat panel corresponding thereto, which are used to acquire real-time DR images of patients. The system equipment can install the X-ray source 1 on the top of the treatment room, and the imaging plate is installed on the ground part of the treatment room, each of which uses a small-angle track to move, and the movement mode is controlled by the corresponding control system to ensure the consistency of the movement direction and the accuracy of the position. Of course, the X-ray source and the imaging plate can also be connected together using a C-arm according to needs, and they can move at a small angle as a whole, which is not limited here, and is selected according to the actual situation.

In some implementations, the DR imaging device of this embodiment can take the center point of the treatment room as the origin to perform small-angle rotation imaging, wherein, in the treatment room coordinate axis XYZ, the coordinate origin is the beam isocenter of the treatment room, The X-axis is parallel to the floor of the treatment room and points to the zero-degree direction of the treatment couch, the Y-axis is parallel to the floor of the treatment room and points to the 90° direction of the treatment couch, and the Z-axis is perpendicular to the floor of the treatment room and points to the top of the treatment room.

In other implementations, the small angle in this embodiment is defined between -15 degrees and +15 degrees.

S2. Obtain the weight parameters of the artificial intelligence network, and train to obtain the artificial intelligence network algorithm model, including:

S21: Establish a patient's DR image and 3D-CT image data set for training and verification of the artificial intelligence network algorithm model.

Specifically, a DR imaging device is used to capture a DR image of a patient, and a CT system is used to capture a 3D-CT image of the same part of the same patient, so that the DR image of the patient and the 3D-CT image are in one-to-one correspondence, and a DR image and its corresponding 3D-CT image dataset. Take 80% of the data in the established dataset as the training dataset and 20% as the validation dataset, first build the model and then train and validate.

S22: Build an artificial intelligence network algorithm model.

According to the schematic diagram of the artificial intelligence network algorithm generated from the virtual 3D-CT image shown in Figure 3, the algorithm can realize the function of inputting a small number of DR images, such as 1 to 8, and outputting a virtual 3D-CT data set.

For the artificial intelligence network algorithm model, during training and verification, the input N DR images and the corresponding M-layer 3D-CT images, the value range of N is greater than or equal to 1, and the shooting angle of each DR image is different , although in theory the larger the value of N, the better, but with the increase of the value of N, more and more DR images need to be taken, and more and more additional radiation doses are added to the patient, resulting in economic costs. is also larger, so the N value should not exceed 8. The number of M layers is determined with reference to the number of CT layers in the treatment plan. Generally, the number of layers in M is close to or equal to the CT in the treatment plan. CT for registration.

S23: Use the DR image set established in S21 and the corresponding 3D-CT image to train and verify the artificial intelligence network algorithm model in S22, and obtain the weights and parameters of the artificial intelligence network through continuous iteration through operations. The parameters include each neural network of the network model. Element weights and neuron parameters.

S3: Obtain the real-time DR image of the patient, use the DR image generated in real time, and use the trained artificial intelligence network algorithm to generate the virtual 3D-CT image of the current patient;

Specifically, the real-time DR image refers to the DR image captured by the patient during the current fractional treatment, and the DR image is used to obtain the current state of the patient's tissues and organs.

S4: Construct an automatic tissue and organ segmentation algorithm based on deep learning convolutional neural network. After training and verification by using CT images and corresponding doctors to manually segment tissues and organs, the algorithm can automatically and accurately segment the CT images according to the CT images. Tissue organs and tumor target areas.

Specifically, an automatic segmentation algorithm for tissues and organs based on deep learning, which uses a deep learning convolutional neural network model to automatically segment tissues and organs according to the input CT images. The quality of the delineated tissues and organs and the corresponding CT, training set and validation data sets must be guaranteed. The algorithm can output specific moving tissues and organs and target areas for 3D reconstruction and registration.

S5: Construct a conventional 3D tissue and organ model reconstruction algorithm to realize input 3D-CT images and/or through automatically generated and/or manually generated tissue and organ contour sets, capable of reconstructing and outputting 3D models of moving tissues, organs and target areas.

Specifically, the above 3D tissue and organ reconstruction algorithm can reconstruct 3D models of all or specified tissues and organs, and render and display different tissues and organs in different colors and modes, which is convenient for users to observe and distinguish operations.

S6: Use the 3D tissue and organ reconstruction algorithm to perform three-dimensional reconstruction according to the current patient's planned CT data and/or tissue and organ contour sets and the generated patient virtual CT data and/or tissue and organ contour sets, respectively, to obtain the current patient's treatment plan and Virtual 3D specific motion tissue organs and target areas;

S7: Build a 3D model registration algorithm for registering multiple input 3D models, and output the 3D model shape and motion offset of specific moving tissues, organs and target areas;

Specifically, the conventional 3D tissue and organ registration algorithm can perform manual and/or automatic 3D model registration according to the reconstructed 3D tissue and organ model, and accurately output the 3D state and position change offset of the moving tissue and organ and the target area. The displacement parameter is obtained by registering the planned CT data of the current patient and/or the contour set of the tissue and organ and the generated virtual CT data of the patient and/or the 3D tissue and organ generated by the contour set of the tissue and organ.

S8: Use the 3D model registration algorithm to perform automatic or/and manual registration calculations using the 3D moving tissues, organs and target areas of the patient treatment plan 3D-CT and virtual 3D-CT as input, and output the specific moving tissues, organs and target areas. 3D model shape and motion offsets;

S9: According to the current 3D shape and motion offset of the 3D moving tissues and organs and the tumor target, it is applied to guide the implementation of radiotherapy.

In some implementations, the criteria for compliance with treatment requirements are determined by physicians in conjunction with research and engineering personnel in accordance with radiotherapy laws and regulations and industry standards.

To sum up, in this embodiment, artificial intelligence technology is used to reconstruct 2D-DR images in 3D to obtain a virtual 3D-CT image of the patient, and the reconstructed virtual 3D-CT and the 3D-CT image of the patient's treatment plan are automatically segmented into tissues and organs. And 3D reconstruction and registration are performed to obtain the 3D state and position change offset of the patient's moving tissues, organs and target areas, so as to guide the patient to perform precise radiotherapy of the moving target area.

Example 2

The foregoing Embodiment 1 provides a three-dimensional image-guided moving organ positioning method, and correspondingly, this embodiment provides a three-dimensional image-guided moving organ positioning system. The positioning system provided in this embodiment may implement the three-dimensional image-guided moving organ positioning method of Embodiment 1, and the positioning system may be implemented by software, hardware, or a combination of software and hardware. For example, the positioning system may include integrated or separate functional modules or functional units to perform corresponding steps in each method of Embodiment 1. Since the positioning system of this embodiment is basically similar to the method embodiment, the description process of this embodiment is relatively simple, and the relevant part may refer to the partial description of Embodiment 1, and the embodiment of the positioning system of this embodiment is only schematic of.

A three-dimensional image-guided motion organ positioning system provided in this embodiment includes:

Example 3

This embodiment provides a processing device corresponding to the three-dimensional image-guided moving organ positioning method provided in Embodiment 1, and the processing device may be an electronic device used for a client, such as a mobile phone, a notebook computer, a tablet computer, a desktop computer, etc. , to execute the positioning method of Embodiment 1.

The processing device includes a processor, a memory, a communication interface and a bus, and the processor, the memory and the communication interface are connected through the bus to complete mutual communication. The bus can be an industry standard architecture (ISA, Industry Standard Architecture) bus, a peripheral device interconnect (PCI, Peripheral Component) bus or an extended industry standard architecture (EISA, Extended Industry Standard Component) bus and so on. A computer program that can be run on the processor is stored in the memory, and the processor executes the three-dimensional image-guided movement organ positioning method provided in Embodiment 1 when the processor runs the computer program.

In some implementations, the memory may be a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

In other implementations, the processor may be various types of general-purpose processors such as a central processing unit (CPU) and a digital signal processor (DSP), which are not limited herein.

Example 4

The three-dimensional image-guided moving organ positioning method in this embodiment 1 may be embodied as a computer program product, and the computer program product may include a computer-readable storage medium on which a computer for executing the positioning method described in this embodiment 1 is loaded. Readable program instructions.

A computer-readable storage medium may be a tangible device that retains and stores instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination of the above.

It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function(s). Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the present invention can still be Modifications or equivalent replacements are made to the specific embodiments of the present invention, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention shall be included within the protection scope of the claims of the present invention. The above contents are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the technical scope disclosed in the present application can easily think of changes or replacements, which should cover within the scope of protection of this application. Therefore, the protection scope of the present application should be the protection scope of the claims.

Claims

A three-dimensional image-guided moving organ positioning method, characterized in that it comprises:

Based on the patient's real-time DR images, the artificial intelligence network algorithm is used to generate the patient's virtual 3D-CT image set;

The patient's 3D-CT image set is automatically segmented into tissues and organs by the tissue and organ automatic segmentation algorithm, and the 3D model of the moving tissues and organs of the patient's treatment plan is reconstructed through the 3D tissue and organ model reconstruction algorithm;

The patient's virtual 3D-CT image set is automatically segmented into virtual tissues and organs by the tissue and organ automatic segmentation algorithm, and the 3D model of the patient's virtual moving tissues and organs is reconstructed through the 3D tissue and organ model reconstruction algorithm;

The 3D model of the moving tissue and organ of the patient's treatment plan and the 3D model of the virtual moving tissue and organ are registered and calculated, and the 3D model shape and motion offset of the specified moving tissue and organ are output.
The three-dimensional image-guided moving organ positioning method according to claim 1, wherein the real-time DR image of the patient is acquired by using a DR imaging device.
The three-dimensional image-guided moving organ positioning method according to claim 2, wherein the DR imaging device comprises a set of X-ray sources and an imaging flat panel corresponding thereto;

The X-ray source is installed on the top of the treatment room, and the imaging plate is installed on the floor portion of the treatment room, and each uses a small-angle track to move; or,

The X-ray source and the imaging plate are connected by a C-arm for small-angle movement.
The three-dimensional image-guided moving organ positioning method according to claim 1, wherein the artificial intelligence network algorithm is obtained through training and verification, comprising:

Use the DR imaging equipment to capture the DR image of the patient, and use the CT system to capture the 3D-CT image of the same part of the same patient, so that the DR image of the patient and the 3D-CT image are in one-to-one correspondence, and the DR image and the corresponding 3D-CT image are established. CT image data set; take part of the data in the established data set as the training data set and the other part as the verification data set, build a neural network model for training and verification, and continuously iterate through operations to obtain the weight parameters of the artificial intelligence network, and then obtain the trained Artificial intelligence network model.
The three-dimensional image-guided moving organ localization method according to claim 1, wherein the automatic tissue and organ segmentation algorithm adopts a deep learning convolutional neural network model, which can automatically segment the tissue and organs according to the input CT image.
The three-dimensional image-guided moving organ positioning method according to claim 1, wherein the 3D tissue and organ reconstruction algorithm can reconstruct the 3D models of all or specified tissues and organs, and can perform different colors and modalities for different tissues and organs. Rendered display, easy for users to observe and distinguish operations.
The three-dimensional image-guided moving organ positioning method according to claim 1, wherein the registration calculation adopts a 3D tissue-organ registration algorithm to perform manual and/or automatic 3D model registration.
A three-dimensional image-guided motion organ positioning system, characterized in that the system includes:

The virtual image generation unit is configured to generate a virtual 3D-CT image set of the patient based on the real-time DR image of the patient using an artificial intelligence network algorithm;

The organ reconstruction unit is configured to automatically segment the tissue and organs from the patient's 3D-CT image set using the tissue-organ automatic segmentation algorithm, and reconstruct the 3D model of the moving tissue and organ of the patient's treatment plan through the 3D tissue-organ model reconstruction algorithm;

The virtual organ reconstruction unit is configured to automatically segment the virtual tissue and organ from the patient's virtual 3D-CT image set using the tissue-organ automatic segmentation algorithm, and reconstruct the 3D model of the patient's virtual moving tissue and organ through the 3D tissue-organ model reconstruction algorithm;

The offset calculation unit is configured to perform a registration calculation on the 3D model of the moving tissue and organ of the patient's treatment plan and the 3D model of the virtual moving tissue and organ, and output the shape of the 3D model and the motion offset of the selected moving tissue and organ.
A processing device, the processing device includes at least a processor and a memory, and a computer program is stored on the memory, wherein the processor executes the computer program when running the computer program to realize any one of claims 1 to 7 The three-dimensional image-guided motion organ localization method described in item 1.
A computer storage medium, characterized in that computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the three-dimensional image-guided movement organ positioning method according to any one of claims 1 to 7.