CN114681813B - Automatic radiation therapy planning system, automatic radiation therapy planning method, and storage medium - Google Patents
Automatic radiation therapy planning system, automatic radiation therapy planning method, and storage medium Download PDFInfo
- Publication number
- CN114681813B CN114681813B CN202011575203.5A CN202011575203A CN114681813B CN 114681813 B CN114681813 B CN 114681813B CN 202011575203 A CN202011575203 A CN 202011575203A CN 114681813 B CN114681813 B CN 114681813B
- Authority
- CN
- China
- Prior art keywords
- information
- tps
- medical image
- dose
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N5/00—Radiation therapy
- A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
- A61N5/103—Treatment planning systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N5/00—Radiation therapy
- A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N5/00—Radiation therapy
- A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
- A61N5/1001—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy using radiation sources introduced into or applied onto the body; brachytherapy
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N5/00—Radiation therapy
- A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
- A61N5/103—Treatment planning systems
- A61N5/1039—Treatment planning systems using functional images, e.g. PET or MRI
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N5/00—Radiation therapy
- A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
- A61N5/1048—Monitoring, verifying, controlling systems and methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/40—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N5/00—Radiation therapy
- A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
- A61N2005/1092—Details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Public Health (AREA)
- Radiology & Medical Imaging (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Pathology (AREA)
- Veterinary Medicine (AREA)
- Medical Informatics (AREA)
- Animal Behavior & Ethology (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Urology & Nephrology (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Surgery (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Radiation-Therapy Devices (AREA)
Abstract
The invention provides an automatic radiotherapy planning system, which comprises an information input unit, a parameter generation unit, a dose prediction unit, an evaluation unit and an output unit, wherein the information input unit is used for inputting patient information comprising medical image information, contour sketching information and prescription dose information; the parameter generating unit generates initialization parameters based on the first neural network model according to the medical image information, the contour sketching information and the prescription dose information, and outputs the initialization parameters to the TPS, so that the TPS can determine a radiation treatment plan by using the initialization parameters; the dose prediction unit predicts the dose distribution of the radiotherapy plan generated by TPS based on the second neural network model according to the medical image information and the contour sketching information; the evaluation unit compares the dose distribution of the radiotherapy plan determined by the TPS with the predicted dose distribution to generate an evaluation result; and the output unit outputs the radiotherapy plan generated by TPS according to the evaluation result. The invention can improve the efficiency of designing radiation treatment plans.
Description
Technical Field
The present invention relates to the field of radiation therapy, and in particular, to an automatic radiation therapy planning system, an automatic radiation therapy planning method, and a computer-readable storage medium.
Background
Radiotherapy utilizes radiation to treat diseases, is one of important means for tumor treatment, and has great significance for improving human health and prolonging human life.
Generally, the dose distribution of a radiation treatment plan depends on a dose target defined by a physical engineer. Traditional forward treatment planning designs, highly dependent on the experience of the physicist, have gradually been replaced by inverse planning. The physical practitioner submits the dose prescription to a computer, which uses dose calculation and optimization techniques to meet the set target volume and organ-at-risk dose requirements. To ensure safety and reliability of treatment, a treatment planning system (Treatment Planning System, TPS) follows. In TPS, a radiation plan is created for a patient by modeling the radiation source and the patient, and the plan is simulated for implementation, and when the radiation plan meets clinical requirements, clinical treatment is performed again. In particular, the radiation treatment planning process includes acquiring patient target and normal tissue delineation results, designing beams and radiation types, dose calculations, planning evaluations, selecting to continue optimizing the irradiation technique based on the evaluation results, or completing planning designs into a planning verification step.
In order to reversely calculate the beam-emitting mode of the therapeutic machine from the wanted dose distribution, a physical engineer is required to design the number of the radiation fields, the objective function and the related weights thereof and give the radiation fields to the TPS for optimization, and the TPS obtains the optimal beam intensity distribution of each radiation field through repeated iterative calculation of a computer according to the factors such as the position of a tumor in the body, the tissue non-uniformity, the position of a key tissue, the number of the radiation fields and the like, so that the spatial dose distribution formed in the body in practice is closest to the prescription dose of a doctor. After calculating the dose distribution in the patient from the TPS, the treatment plan is evaluated, which is also usually entirely dependent on the experience of the physicist. If the requirements are not met, the corresponding parameters in the treatment plan design process need to be adjusted to design the treatment process again, and the plan and plan evaluation are simulated.
How to design parameters such as proper radiation fields, objective functions, weights and the like for different target positions, shapes and dosage requirements is highly dependent on the experience of a physical engineer. Repeating until the treatment requirement is met. In order to reduce the irradiated dose of normal tissues as much as possible while ensuring the dose of the target area, the physical engineer needs to continuously adjust and optimize the given dose target, which requires a lot of manpower and time, and it is difficult to ensure that the optimal solution is achieved. While the physicist may save some templates for fine tuning based on previous successful planning experience, the ultimate success of achieving a more satisfactory result is dependent on the physicist's experience. The primary physicist is likely to need to repeat the design optimization 5 to 7 times, ranging from half an hour to several hours, in order to get a planned design to the prescription requirements, and may not even be able to complete the planned design at all.
In addition, due to unavoidable experience and level differences, there is a great difference in quality between hospitals and different physicists in the radiation treatment plans designed for the same case, thereby possibly reducing the tumor control probability of part of patients or increasing the probability of unnecessary normal tissue complications.
The prior art still has shortcomings in terms of complexity and time-consuming design of radiation treatment plans.
Disclosure of Invention
It is an object of the present invention to provide an automated radiation treatment planning system to improve the efficiency of planning radiation treatment plans.
According to a first aspect of the present invention, there is provided a radiation therapy automatic planning system, wherein the radiation therapy automatic planning system comprises an information input unit, a parameter generation unit, a dose prediction unit, an evaluation unit and an output unit, wherein the information input unit is arranged at least for inputting patient information, the patient information comprising medical image information, contour delineation information and prescription dose information; the parameter generating unit is configured to be capable of generating initialization parameters for the TPS based on the first neural network model from the medical image information, the contour delineation information and the prescription dose information, and outputting the initialization parameters to the TPS so that the TPS can determine the radiation treatment plan using the initialization parameters; the dose prediction unit is arranged to be able to predict a dose distribution of TPS for a radiation treatment plan generated by the patient from medical image information and contouring information based on the second neural network model; the evaluation unit is configured to be able to compare the dose distribution of the radiation treatment plan determined by the TPS with the dose distribution predicted by the dose prediction unit and to generate an evaluation result; and an output unit arranged to be able to output the TPS generated radiation treatment plan as a final radiation treatment plan based on the evaluation result.
According to an exemplary embodiment of the invention, the parameter generating unit is arranged to be able to determine an objective function for the TPS, the initialization parameter comprising objective function information representing said objective function.
According to an exemplary embodiment of the present invention, the objective function information includes a vector formed by combining a plurality of objective function items with weights, the objective function items being objective function items that TPS can provide; and/or the objective function information comprises a vector composed of all objective function terms with weights for each region to be calculated in the medical image, the objective function terms being the objective function terms that TPS can provide.
According to an exemplary embodiment of the present invention, the first neural network model performs feature extraction with medical image information and contouring information including image data of a plurality of consecutive medical image planes as inputs; and/or the first neural network model is arranged to be able to extract at least one of the following features: features relating to density information displayed by the medical image, features relating to the size of the target volume contour and the organ-at-risk contour, features relating to the positional relationship between the target volume contour and the organ-at-risk contour.
According to an exemplary embodiment of the present invention, the second neural network model takes as input medical image information and contouring information comprising image data of a plurality of consecutive medical image planes, and the predicted dose distribution map of each medical image plane as output; and/or in the second neural network model, in order to distinguish all target regions and organ-at-risk regions, the input contouring information needs to be preprocessed in the following manner: the target region in the medical image and the pixels within each organ-at-risk are assigned unique tag values, and/or the pixels of the portion of the medical image where the target region overlaps the organ-at-risk use the sum of the tag values of the overlapping target region and the organ-at-risk as the tag value.
According to an exemplary embodiment of the present invention, the information input unit is further arranged for inputting incident field information, wherein: the field information comprises field quantity information and/or field direction information; and/or the parameter generating unit is configured to be capable of inputting the portal information into the first neural network model and for generating initialization parameters for TPS; and/or the dose prediction unit is arranged to be able to input portal information into the second neural network model and for predicting a dose distribution of the TPS for a radiation treatment plan generated for said patient.
According to an exemplary embodiment of the invention, the output unit is arranged to be capable of: outputting a radiation treatment plan generated by TPS as a final radiation treatment plan when the evaluation result reaches a preset requirement; and outputting prompt information when the evaluation result does not reach the preset requirement, wherein the prompt information indicates that the initialization parameters for TPS need to be adjusted to generate a new radiation treatment plan.
According to an exemplary embodiment of the invention, the first neural network model and/or the second neural network model is a convolutional neural network deep learning model; and/or the medical image information comprises CT image information of the patient; and/or the delineating information includes delineating information of the target region and the organ at risk.
According to a second aspect of the present invention there is provided a radiation therapy automatic planning method using the radiation therapy automatic planning system of the present invention, wherein the method comprises the steps of: inputting patient information, the patient information including medical image information, contour delineation information, and prescription dose information; based on the first neural network model, generating initialization parameters for the TPS according to the medical image information, the contour sketching information and the prescription dose information, and outputting the initialization parameters to the TPS, so that the TPS can determine a radiation treatment plan by using the initialization parameters; based on a second neural network model, predicting a dose distribution of TPS for the patient-generated radiation treatment plan from the medical image information and the delineation information; comparing the dose distribution of the radiation treatment plan determined by the TPS with the dose distribution predicted by the dose prediction unit, and generating an evaluation result; and outputting the radiation treatment plan generated by the TPS as a final radiation treatment plan when the evaluation result reaches the preset requirement.
According to a third aspect of the present invention there is provided a computer program product comprising computer program instructions, wherein the computer program instructions, when executed by one or more processors, cause the processors to perform the automatic radiation therapy planning method according to the present invention.
The invention has the positive effects that: by means of the radiation therapy automatic planning system and the corresponding automatic planning method and computer program product according to the invention, initialization parameters for TPS can be automatically selected for generating a radiation therapy plan, which radiation therapy plan can also be automatically evaluated. Thus, the time for the physicist to invoke the template and repeatedly modify the initialisation parameters for the TPS may be reduced, possibly even eliminating this step, thereby speeding up and optimising the process of generating a clinically usable radiation treatment plan.
Automation of the radiation treatment planning process can improve the planning quality while reducing the time spent, and reduce the planning quality differences caused by human factors. In addition, the dose distribution generated by the automatic plan can be used as the starting point of the manual plan, and the optimization and adjustment can be performed according to the specific requirements of doctors on the basis of the dose distribution.
Drawings
The principles, features and advantages of the present invention may be better understood by describing the present invention in more detail with reference to the drawings. The drawings include:
FIG. 1 schematically illustrates a radiation therapy automatic planning system according to an exemplary embodiment of the present invention; and
fig. 2 schematically illustrates a radiation therapy automatic planning method according to an exemplary embodiment of the invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous technical effects to be solved by the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and a plurality of exemplary embodiments. It should be understood that the detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the invention.
The present invention will be described in detail below by taking a radiation therapy plan for brain tumors as an example. However, it will be appreciated by those skilled in the art that the present application is applicable not only to the treatment of brain tumors, but also to radiation therapy planning for other conditions such as lung tumors, prostate tumors, esophageal tumors, mesothelioma, head and neck tumors, central nervous system tumors, gynaecological and obstetrical tumors, and gastrointestinal tumors.
Brain tumor is also called intracranial tumor, which is a brain disease with slow onset and gradual aggravation. Brain tumors include primary brain tumors and brain metastases, and the treatment means comprise complete excision by surgery, but the complete excision of a large scale by surgery is extremely difficult due to the unresectability of normal brain tissues and the characteristic of extensive invasive growth of malignant tumors into the cranium. With the application of stereotactic radiotherapy (stereotactic body radiotherapy, SBRT), the survival time of brain tumor patients is significantly prolonged. SBRT is a general term of radiation therapy technology which utilizes a three-dimensional positioning technology and a special ray device to focus multi-source, multi-beam or multi-field three-dimensional space focused high-energy rays on a certain target area in a body, precisely causes damage to the volume of well-defined focal tissues, and causes the focal tissues to be irradiated with high dose and the surrounding normal tissues to be reduced, thereby obtaining high clinical curative effect and less adverse reaction. SBRT has the characteristics of high accuracy, high split dosage, high conformality and less treatment times. SBRT requires very accurate target volume, very high demands on doctors and physicists, and the manual planning process is very time consuming. Although the invention is described herein by way of example with respect to SBRT, those skilled in the art will appreciate that the invention is applicable not only to SBRT, but to other radiotherapy techniques such as IMRT, IGRT, and the like.
Currently, radiation treatment plans may be generated by the TPS 20. By the physical engineer giving the number of shots, the objective function and their associated weights, the TPS 20 uses a specific algorithm to give a radiation treatment plan that meets the requirements of the dose distribution, and is then available for clinical treatment after evaluation by the radiation therapist and physical engineer, and verification of the plan. The radiation treatment planning process is very complex and lengthy, requiring first the clinician to delineate the tumor target area and the organs at risk on the patient's medical image, such as CT image, in combination with other diagnostic results, and then to pass the planning to the physical engineer after the superior clinician confirms the treatment target area and organ at risk contours and dose prescriptions. In order to reversely calculate the beam-emitting mode of the therapeutic machine from the desired dose distribution by the TPS 20, a physical engineer is required to create an auxiliary structure in advance, arrange the therapeutic mode and the radiation field, design an objective function and its related weight to form an equation set, repeatedly optimize the TPS 20, and adjust the radiation field and the objective function according to the optimization result until the minimum value of the equation set is sought, and each parameter value of the radiation field corresponding to the minimum value is the final required solution, so that a corresponding radiation therapy plan can be made. The radiation treatment plan is evaluated by radiation therapists and physicists and validated for clinical treatment.
Fig. 1 schematically illustrates a radiation therapy automatic planning system according to an exemplary embodiment of the present invention. The radiation therapy automatic planning system comprises an information input unit 11, a parameter generation unit 12, a dose prediction unit 13, an evaluation unit 14 and an output unit 15, wherein the information input unit 11 is arranged at least for inputting patient information, which comprises medical image information, contour delineation information and prescription dose information; the parameter generation unit 12 is arranged to be able to generate initialization parameters for the TPS 20 based on the first neural network model from the medical image information, the delineation information and the prescription dose information and to output the initialization parameters to the TPS 20 such that the TPS 20 is able to determine a radiation treatment plan using the initialization parameters; the dose prediction unit 13 is arranged to be able to predict a dose distribution of the TPS 20 for the patient generated radiation treatment plan from the medical image information and the delineation information based on the second neural network model; the evaluation unit 14 is arranged to be able to compare the dose distribution of the radiation treatment plan determined by the TPS 20 with the dose distribution predicted by the dose prediction unit 13 and to generate an evaluation result; and the output unit 15 is arranged to be able to output the radiation treatment plan generated by the TPS 20 as a final radiation treatment plan, based on the evaluation result.
With the automatic radiation therapy planning system according to the present invention, the initialization parameters for the TPS 20 can be automatically selected for generating a radiation therapy plan based on the medical image information, the delineation information and the prescription dose information, and the radiation therapy plan can also be automatically evaluated. Thus, the efficiency of designing a radiation treatment plan can be improved.
The radiation therapy automatic planning system may reduce the time for a physical operator to invoke the template and repeatedly modify the initialization parameters for the TPS 20, possibly even eliminating this step, thereby speeding up and optimizing the process of generating a clinically usable radiation therapy plan.
By automating the radiation treatment plan optimization process, the plan quality can be improved while reducing the time spent, reducing the plan quality variance due to artifacts. In addition, the dose distribution generated by the automatic plan can be used as the starting point of the manual plan, and the optimization and adjustment can be performed according to the specific requirements of doctors on the basis of the dose distribution.
Because of unavoidable experience and level differences, there is a great difference in quality between hospitals and different physical engineers in the radiation treatment plans designed for the same case, thereby potentially reducing the probability of tumor control in some patients or increasing the probability of unnecessary normal tissue complications. This variability problem can be addressed with an automated radiation therapy planning system. The radiation therapy automatic planning system can improve the work efficiency of a physical engineer, and help the physical engineer to find and improve suboptimal plans in time. In addition, the automated planning model developed by the high-level hospitals trained by using the high-quality plans of the high-level hospitals can be transplanted into the same type of planning systems of other hospitals, and patient individualization optimization parameter benchmark values are provided so as to assist in improving the planning quality of the high-level hospitals, so that the planning design quality difference between the hospitals and physical operators is reduced finally, and patients benefit from the high-level hospitals.
The first neural network model may be implemented using a deep learning convolutional neural network, for example. The first neural network model may be selected from VGG, resnet, densenet, etc.
By means of the first neural network model, at least one, in particular all, of the following features can be extracted from the input CT image: features relating to density information displayed by the CT image, features relating to the size of the target region contour and the organ-at-risk contour, features relating to the positional relationship between the target region contour and the organ-at-risk contour. In view of the spatial positional relationship of the target region and the organ at risk, the first neural network model can be configured in particular as a 3D or 2.5D network model and features extraction takes as input medical image information and contouring information comprising image data of a plurality of successive medical image planes. Successive medical image slices may be spaced apart by a predetermined spacing. Thus, the first neural network model may acquire three-dimensional information contained in the medical image information. Specifically, the 3D network model takes as input the feature extraction of the whole 3D CT image, i.e. the medical image information and the target volume delineation results of all medical image planes containing the whole scan result. Better initialization parameters can be obtained through the 3D first neural network model. In order to reduce the computational effort, a 2.5D network model can also be used, which likewise has as input medical image information and contouring information comprising image data of a plurality of successive medical image planes, but not entirely of all medical image planes of the entire scan result. The first neural network model can correspond to the initialization parameters for the TPS 20 based on the learning of these features.
To train the first neural network model, brain tumor SBRT radiation therapy planning data, such as SBRT radiation therapy planning data for 300 brain tumor patients, may be collected and a radiation therapy planning database constructed. The radiation treatment plan data may include patient data, patient medical images, physician prescriptions, diagnostic data, DVH (dose volume histogram), isodose curve preset dose map, objective functions and related parameters, target volume and organ-at-risk profile, and the like. To ensure the quality of the data, the collected data may be screened and expert validated. The expert group may consist of no less than 5 radiation therapists and physicists with radiation therapy experiences over 15 years. Optionally, each case of data is carefully checked and verified by an expert group, so that all information of the data is ensured to be accurate and can completely meet clinical use requirements.
Optionally, based on the collected data, small sample learning is adopted, and the training strategy adjusts the convolutional neural network in a transfer learning mode.
Optionally, data enhancement is performed on the collected data, samples are expanded, and the training network learns the universality characteristics under small data.
In an exemplary embodiment, the parameter generating unit 12 is for example arranged to be able to determine an objective function for the TPS 20, the initialization parameter comprising objective function information representing said objective function. Thus, the parameter generation unit 12 may assist the physicist in skipping the steps of calling templates, iteratively modifying the objective functions and weights, thereby expediting and optimizing the process of generating a clinically useful radiation treatment plan.
Alternatively, the objective function information may comprise a vector of all objective function terms with weights for each region to be calculated in the medical image combined, said objective function terms being the objective function terms that the TPS 20 is able to provide. In the objective function information, all the areas to be calculated need to be arranged in a fixed order, and all the objective functions of each area to be calculated are also arranged in a fixed order. The vector can be expressed as: (w) 11 f 11 (u 11 ),…w 1n f 1n (u 1n ),w 21 f 21 (u 21 ),…w 2n f 2n (u 2n ),…w m1 f m1 (u m1 ),…w mn f mn (u mn ) Where w represents the weight, f (u) represents the objective function term, u represents the private function of the objective function term, n represents the number of objective function terms provided by the TPS 20M represents the number of regions to be calculated, w x1 f x1 (u x1 ),……w xn f xn (u xn ) And representing each objective function term with weight determined for the x-th area to be calculated.
For example, a respective objective function term is set for each region (target region or organ) to be calculated, each objective function term may have a corresponding weight value and dose value. All selectable objective function items with weights of each area to be calculated are combined into a vector, wherein the weight value of the objective function which is not selected is set to be 0. All vectors of the regions to be calculated are combined into an objective function vector as an output of the parameter generating unit 12.
The output layer of the first neural network model is set to be a full-connection layer with the same length as the vector of the objective function, so that the selection of the objective function and the setting of parameter values such as corresponding weights can be realized.
The second neural network model is also implemented using, for example, a deep learning convolutional neural network. The second neural network model may employ some typical segmentation network model, such as a U-net model.
With the second neural network model, a predicted dose distribution of the TPS 20 for the patient generated radiation treatment plan may be output from the input CT image and the contouring information. To this end, the second neural network model may be trained using a radiation treatment plan database.
The predicted dose distribution may be represented by a dose distribution map. The dose profile may be predicted for each pixel value.
Alternatively, the second neural network model may be provided as a 3D or 2.5D network model, for example, with CT images including a plurality of consecutive medical image planes, the delineation information of the target region and the organ at risk as input, and the predicted dose distribution map of each medical image plane as output. In order to distinguish all target regions from organ-at-risk regions, the input contouring information needs to be preprocessed, and the processing modes can include: the target region and each pixel within the organ at risk are assigned unique tag values, and the sum of the tag values of the target region and the organ at risk overlap is used as the tag, which is also unique among all tags.
The accuracy of the second neural network model of the dose prediction unit 13 may be estimated using the mean absolute error.
After the evaluation unit 14 compares the dose distribution of the radiation treatment plan determined by the TPS 20 with the dose distribution predicted by the dose prediction unit 13 and generates an evaluation result, the output unit 15 is arranged to be able to: outputting the radiation treatment plan generated by the TPS 20 as a final radiation treatment plan when the evaluation result reaches the preset requirement; and outputting prompt information when the evaluation result does not meet the preset requirement, wherein the prompt information indicates that the initialization parameters for the TPS 20 need to be adjusted to generate a new radiation treatment plan.
The prompt message may be displayed, for example, through a screen. Based on the prompt, the physicist may further adjust the initialization parameters for the TPS 20 to generate a new radiation treatment plan. The evaluation unit 14 may then compare the new radiation treatment plan with the dose distribution predicted by the dose prediction unit 13 and generate an evaluation result. This process may be repeated until the evaluation results meet the predetermined requirements.
Optionally, the information input unit 11 is further arranged for inputting shot information, said shot information comprising shot number information and/or shot direction information set by a physicist. The parameter generation unit 12 may be arranged to be able to input the portal information into the first neural network model and for generating initialization parameters for the TPS 20. Alternatively or additionally, the dose prediction unit 13 may be arranged to be able to input portal information into the second neural network model and for predicting a dose distribution of the TPS 20 for the patient generated radiation treatment plan.
In an alternative embodiment according to the invention, the radiation therapy automatic planning system may comprise only the information input unit 11, the parameter generation unit 12 and the optional output unit 15, and not the dose prediction unit 13 and the evaluation unit 14. The information input unit 11 is arranged at least for inputting patient information including medical image information, contour delineation information and prescription dose information; the parameter generation unit 12 is arranged to be able to generate initialization parameters for the TPS 20 based on the first neural network model from the medical image information, the delineation information and the prescription dose information and to output the initialization parameters to the TPS 20 such that the TPS 20 is able to determine a radiation treatment plan using the initialization parameters; the output unit 15 is arranged to be able to output the final radiation treatment plan. Without the dose prediction unit 13 and the evaluation unit 14, the evaluation and adjustment of the radiation treatment plan can be achieved manually. In this case, the efficiency of designing the radiation treatment plan can still be improved by the parameter generation unit 12.
Fig. 2 schematically illustrates a radiation therapy automatic planning method utilizing a radiation therapy automatic planning system according to an exemplary embodiment of the present invention. The method comprises the following steps:
step one: inputting patient information, the patient information including medical image information, contour delineation information, and prescription dose information;
step two: based on the first neural network model, generating initialization parameters for the TPS 20 according to the medical image information, the contour sketching information and the prescription dose information, and outputting the initialization parameters to the TPS 20 so that the TPS 20 can determine a radiation treatment plan using the initialization parameters;
step three: predicting a dose distribution of the TPS 20 for the patient generated radiation treatment plan based on the second neural network model from the medical image information and the contouring information;
step four: comparing the dose distribution of the radiation treatment plan determined by the TPS 20 with the dose distribution predicted by the dose prediction unit 13 and generating an evaluation result; and
step five: and outputting the radiation treatment plan generated by the TPS 20 as a final radiation treatment plan according to the evaluation result.
In the embodiment shown in fig. 2, the first neural network model and the second neural network model are input accordingly with the CT image, the RT structure storing the sketching information, and the number of shots and direction information set by the physicist as inputs. The initialization parameters output from the first neural network model may include weights of the objective function terms and corresponding private parameter values, and the predicted optimal dose distribution map is output from the second neural network model. The initialization parameters determined by the first neural network model are input into the Monaco planning system to generate a corresponding radiation treatment plan. In step four, the dose profile of the radiation treatment plan is compared to the predicted optimal dose profile. If the dose profile of the radiation treatment plan is very close to the optimal dose profile, the current radiation treatment plan is output, otherwise parameters for the TPS 20 may be adjusted to regenerate the radiation treatment plan until the evaluation results meet the predetermined requirements.
In addition, the present invention also provides a computer program product comprising computer program instructions which, when executed by one or more processors, cause the processors to perform the automatic radiation therapy planning method according to the present invention. The computer program instructions may be stored in a computer readable storage medium, which may include, for example, any electronic, magnetic, optical, or other physical storage device. For example, the computer readable storage medium may be: RAM, volatile memory, non-volatile memory, flash memory, a storage drive (e.g., a hard drive), a solid state disk, any type of storage disk (e.g., an optical disk), or similar storage medium, or a combination thereof.
In order to improve the efficiency of radiation treatment planning and reduce the man-made planning quality difference, the invention provides an automatic radiation treatment planning system, and realizes the following advantages:
the correlation from the image to the objective function is obtained by utilizing the strong characteristic learning capability of the first neural network model in extracting medical image information such as the volume, overlapping relation, relative distance and the like of the target area and the organs at risk, so as to realize the automatic initialization of the radiotherapy plan;
providing a solution for vectorization output of the radiation treatment plan initialization parameters, constructing a convolutional neural network model, and realizing end-to-end learning training;
the second neural network model is used to predict the optimal dose distribution, compared to the results provided by the TPS 20, for evaluating the set planning parameters, and guiding the physicist to further adjust the parameters.
Although specific embodiments of the invention have been described in detail herein, they are presented for purposes of illustration only and are not to be construed as limiting the scope of the invention. Various substitutions, alterations, and modifications can be made without departing from the spirit and scope of the invention.
Claims (9)
1. A radiation therapy automatic planning system, wherein the radiation therapy automatic planning system comprises an information input unit (11), a parameter generation unit (12), a dose prediction unit (13), an evaluation unit (14) and an output unit (15), wherein,
an information input unit (11) is arranged at least for inputting patient information, the patient information comprising medical image information, contour delineation information and prescription dose information;
a parameter generating unit (12) arranged to be able to generate initialization parameters for the TPS (20) based on the first neural network model from the medical image information, the contouring information and the prescription dose information and to output said initialization parameters to the TPS (20) such that the TPS (20) is able to determine a radiation treatment plan using the initialization parameters, wherein the parameter generating unit (12) is arranged to be able to determine an objective function for the TPS (20), the initialization parameters comprising objective function information representing said objective function;
a dose prediction unit (13) is arranged to be able to predict a dose distribution of a radiation treatment plan generated by the TPS (20) for said patient from medical image information and contouring information based on a second neural network model, wherein the second neural network model is input with medical image information and contouring information comprising image data of a plurality of consecutive medical image planes, the predicted dose distribution map of each medical image plane being output;
an evaluation unit (14) is arranged to be able to compare the dose distribution of the radiation treatment plan determined by the TPS (20) with the dose distribution predicted by the dose prediction unit (13) and to generate an evaluation result; and
the output unit (15) is arranged to be able to output the radiation treatment plan generated by the TPS (20) as a final radiation treatment plan based on the evaluation result.
2. The automatic radiation therapy planning system of claim 1, wherein the objective function information comprises a vector of a combination of weighted objective function terms, the objective function terms being objective function terms that the TPS (20) is capable of providing; and/or
The objective function information comprises a vector of all objective function terms with weights for each region to be calculated in the medical image combined, said objective function terms being the objective function terms that the TPS (20) is able to provide.
3. The radiation therapy automatic planning system of claim 1 or 2, wherein the first neural network model performs feature extraction with medical image information and contouring information including image data of a plurality of consecutive medical image planes as inputs; and/or
The first neural network model is configured to be capable of extracting at least one of the following features: features relating to density information displayed by the medical image, features relating to the size of the target volume contour and the organ-at-risk contour, features relating to the positional relationship between the target volume contour and the organ-at-risk contour.
4. An automated radiation therapy planning system according to claim 1 or claim 2 wherein,
in the second neural network model, to distinguish all target regions from organ-at-risk regions, the following preprocessing is performed on the input contouring information: the target region in the medical image and the pixels within each organ-at-risk are assigned unique tag values, and/or the pixels of the portion of the medical image where the target region overlaps the organ-at-risk use the sum of the tag values of the overlapping target region and the organ-at-risk as the tag value.
5. The radiation therapy automatic planning system according to claim 1 or 2, wherein the information input unit (11) is further arranged for inputting field information, wherein:
the field information comprises field quantity information and/or field direction information; and/or
A parameter generation unit (12) is arranged to be able to input portal information into the first neural network model and for generating initialization parameters for the TPS (20); and/or
A dose prediction unit (13) is arranged to be able to input portal information into the second neural network model and for predicting a dose distribution of the TPS (20) for the patient generated radiation treatment plan.
6. The radiation therapy automatic planning system according to claim 1 or 2, wherein the output unit (15) is arranged to be capable of:
outputting the radiation treatment plan generated by the TPS (20) as a final radiation treatment plan when the evaluation result reaches the preset requirement; and
when the evaluation result does not meet the predetermined requirement, a prompt is output indicating that the initialization parameters for the TPS (20) need to be adjusted to generate a new radiation treatment plan.
7. An automated radiation therapy planning system according to claim 1 or claim 2 wherein,
the first neural network model and/or the second neural network model are convolutional neural network deep learning models; and/or
The medical image information includes CT image information of the patient; and/or
The delineating information includes delineating information of the target region and the organ at risk.
8. A radiation therapy automatic planning method using a radiation therapy automatic planning system according to any one of claims 1-7, wherein the method comprises the steps of:
inputting patient information, the patient information including medical image information, contour delineation information, and prescription dose information;
generating initialization parameters for the TPS (20) based on the first neural network model from the medical image information, the delineation information and the prescription dose information and outputting the initialization parameters to the TPS (20) such that the TPS (20) can determine a radiation treatment plan using the initialization parameters;
predicting a dose distribution of a TPS (20) for the patient generated radiation therapy plan based on the second neural network model from the medical image information and the contouring information;
comparing the dose distribution of the radiation treatment plan determined by the TPS (20) with the dose distribution predicted by the dose prediction unit (13) and generating an evaluation result; and
when the evaluation result reaches a predetermined requirement, outputting the radiation treatment plan generated by the TPS (20) as a final radiation treatment plan.
9. A computer-readable storage medium storing computer program instructions, wherein the computer program instructions, when executed by one or more processors, cause the processors to perform the radiation therapy automatic planning method of claim 8.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011575203.5A CN114681813B (en) | 2020-12-28 | 2020-12-28 | Automatic radiation therapy planning system, automatic radiation therapy planning method, and storage medium |
| PCT/CN2021/130360 WO2022142770A1 (en) | 2020-12-28 | 2021-11-12 | Automatic radiation treatment planning system and method, and computer program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011575203.5A CN114681813B (en) | 2020-12-28 | 2020-12-28 | Automatic radiation therapy planning system, automatic radiation therapy planning method, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114681813A CN114681813A (en) | 2022-07-01 |
| CN114681813B true CN114681813B (en) | 2023-07-14 |
Family
ID=82130416
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011575203.5A Active CN114681813B (en) | 2020-12-28 | 2020-12-28 | Automatic radiation therapy planning system, automatic radiation therapy planning method, and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114681813B (en) |
| WO (1) | WO2022142770A1 (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115100185A (en) * | 2022-07-22 | 2022-09-23 | 深圳市联影高端医疗装备创新研究院 | Image processing method, image processing device, computer equipment and storage medium |
| CN115300811B (en) * | 2022-08-08 | 2024-01-05 | 中山大学肿瘤防治中心（中山大学附属肿瘤医院、中山大学肿瘤研究所） | Dose distribution determining method and device based on machine learning |
| CN115938591B (en) * | 2023-02-23 | 2023-05-09 | 福建自贸试验区厦门片区Manteia数据科技有限公司 | Dose distribution interval determining device based on radiotherapy and electronic equipment |
| CN116072263B (en) * | 2023-03-06 | 2023-06-09 | 福建自贸试验区厦门片区Manteia数据科技有限公司 | Planning parameter prediction device based on radiotherapy |
| CN116344001B (en) * | 2023-03-10 | 2023-10-24 | 中南大学湘雅三医院 | Medical information visual management system and method based on artificial intelligence |
| CN117011245B (en) * | 2023-07-11 | 2024-03-26 | 北京医智影科技有限公司 | Automatic sketching method and device for rectal cancer tumor area fusing MR information to guide CT |
| CN116612853B (en) * | 2023-07-17 | 2023-09-26 | 中国医学科学院肿瘤医院 | Radiotherapy verification plan dose generation method, radiotherapy verification plan dose generation system, electronic equipment and storage medium |
| CN116779173B (en) * | 2023-08-24 | 2023-11-24 | 北京大学第三医院（北京大学第三临床医学院） | Radiation therapy dose prediction system and method based on artificial intelligence |
| CN116844734B (en) * | 2023-09-01 | 2024-01-16 | 福建自贸试验区厦门片区Manteia数据科技有限公司 | Method and device for generating dose prediction model, electronic equipment and storage medium |
| CN117323584B (en) * | 2023-10-18 | 2024-03-29 | 迈胜医疗设备有限公司 | Electronic device for radiation treatment plan adjustment, radiation treatment system and related apparatus |
| CN117618802A (en) * | 2023-12-11 | 2024-03-01 | 迈胜医疗设备有限公司 | Tissue segmentation proton dose calculation method based on sketched area and related equipment |
| CN117717723B (en) * | 2024-02-08 | 2024-06-11 | 福建自贸试验区厦门片区Manteia数据科技有限公司 | Portal information determining device, processor and electronic equipment |
| CN118217006B (en) * | 2024-04-03 | 2024-09-17 | 武汉长江激光科技有限公司 | Laser-based surgical reference scheme generation method and device |
| CN118366614B (en) * | 2024-06-19 | 2024-09-13 | 华硼中子科技(杭州)有限公司 | Boron neutron capture treatment plan curative effect prediction model construction method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108717866A (en) * | 2018-04-03 | 2018-10-30 | 陈辛元 | A kind of method, apparatus, equipment and the storage medium of the distribution of prediction radiotherapy planning dosage |
| CN109843377A (en) * | 2016-09-07 | 2019-06-04 | 医科达有限公司 | System and method for predicting the learning model of the radiotherapeutic treatment plan of radiation therapy dose distribution |
| CN109979564A (en) * | 2017-12-28 | 2019-07-05 | 北京连心医疗科技有限公司 | A kind of intelligence radiotherapy planning method, equipment and storage medium |
| CN110197709A (en) * | 2019-05-29 | 2019-09-03 | 广州瑞多思医疗科技有限公司 | A kind of 3-dimensional dose prediction technique based on deep learning Yu priori plan |
| CN111128340A (en) * | 2019-12-25 | 2020-05-08 | 上海联影医疗科技有限公司 | Radiotherapy plan generation device, radiotherapy plan generation apparatus, and storage medium |
| CN111432879A (en) * | 2017-12-08 | 2020-07-17 | 医科达有限公司 | Determining bundle model parameters using deep convolutional neural networks |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10806949B2 (en) * | 2016-11-15 | 2020-10-20 | Our United Corporation | Method and system of generating radiation treatment plan |
| KR101968671B1 (en) * | 2018-01-17 | 2019-04-12 | 사회복지법인 삼성생명공익재단 | Intelligent automated method and system for setting radiation therapy plan |
| US11615873B2 (en) * | 2018-08-31 | 2023-03-28 | The Board Of Regents Of The University Of Texas System | Deep learning based dosed prediction for treatment planning and quality assurance in radiation therapy |
-
2020
- 2020-12-28 CN CN202011575203.5A patent/CN114681813B/en active Active
-
2021
- 2021-11-12 WO PCT/CN2021/130360 patent/WO2022142770A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109843377A (en) * | 2016-09-07 | 2019-06-04 | 医科达有限公司 | System and method for predicting the learning model of the radiotherapeutic treatment plan of radiation therapy dose distribution |
| CN111432879A (en) * | 2017-12-08 | 2020-07-17 | 医科达有限公司 | Determining bundle model parameters using deep convolutional neural networks |
| CN109979564A (en) * | 2017-12-28 | 2019-07-05 | 北京连心医疗科技有限公司 | A kind of intelligence radiotherapy planning method, equipment and storage medium |
| CN108717866A (en) * | 2018-04-03 | 2018-10-30 | 陈辛元 | A kind of method, apparatus, equipment and the storage medium of the distribution of prediction radiotherapy planning dosage |
| CN110197709A (en) * | 2019-05-29 | 2019-09-03 | 广州瑞多思医疗科技有限公司 | A kind of 3-dimensional dose prediction technique based on deep learning Yu priori plan |
| CN111128340A (en) * | 2019-12-25 | 2020-05-08 | 上海联影医疗科技有限公司 | Radiotherapy plan generation device, radiotherapy plan generation apparatus, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022142770A1 (en) | 2022-07-07 |
| CN114681813A (en) | 2022-07-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114681813B (en) | Automatic radiation therapy planning system, automatic radiation therapy planning method, and storage medium | |
| US20240075315A1 (en) | System and method for learning models of radiotherapy treatment plans to predict radiotherapy dose distributions | |
| JP6934559B2 (en) | Radiation therapy planning using deep convolutional neural networks | |
| EP3436148B1 (en) | Adaptive radiation therapy planning | |
| US8121252B2 (en) | Use of planning atlas in radiation therapy | |
| CN110211664B (en) | System for automatically designing radiotherapy scheme based on machine learning | |
| CN114206438B (en) | Predicting radiation therapy control points using projection images | |
| US9782607B2 (en) | Systems and methods for radiation treatment planning using combined imaging and treatment dose | |
| CN116391234A (en) | Machine learning optimization of fluence maps for radiation therapy | |
| CN112770811A (en) | Method and system for radiation therapy treatment planning using a deep learning engine | |
| US11291859B2 (en) | Radiation treatment planning for delivering high dose rates to spots in a target | |
| US11544854B2 (en) | Automatic contour adaptation using three sub-networks of a neural network | |
| EP4126214B1 (en) | Automatically-planned radiation-based treatment | |
| US10342994B2 (en) | Methods and systems for generating dose estimation models for radiotherapy treatment planning | |
| CN110706779B (en) | Automatic generation method of accurate target function of radiotherapy plan | |
| US12083358B2 (en) | Method and system for robust radiotherapy treatment planning for dose mapping uncertainties | |
| CN115666717A (en) | Method and apparatus for deriving and using virtual volume structures to predict potential collisions in delivering therapeutic radiation | |
| CN113941100A (en) | Method and apparatus for generating deliverable radiotherapy plan according to three-dimensional spatial dose distribution | |
| JAMPA-NGERN | Study on Quick Prediction of Dose Volume Statistics in Proton Beam Therapy using Deep Learning | |
| Foster | Machine learning applications for personalised automated radiotherapy planning | |
| Thongphiew | ON-LINE RE-OPTIMIZATION OF PROSTATE IMRT PLAN FOR ADAPTIVE RADIATION THERAPY–A FEASIBILITY STUDY AND IMPLEMENTATION |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |