CN116503672A - Liver tumor classification method, system and storage medium - Google Patents
Liver tumor classification method, system and storage medium Download PDFInfo
- Publication number
- CN116503672A CN116503672A CN202310755532.5A CN202310755532A CN116503672A CN 116503672 A CN116503672 A CN 116503672A CN 202310755532 A CN202310755532 A CN 202310755532A CN 116503672 A CN116503672 A CN 116503672A
- Authority
- CN
- China
- Prior art keywords
- image data
- liver tumor
- liver
- tumor
- edge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 206010019695 Hepatic neoplasm Diseases 0.000 title claims abstract description 195
- 208000014018 liver neoplasm Diseases 0.000 title claims abstract description 189
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000005748 tumor development Effects 0.000 claims abstract description 88
- 238000001514 detection method Methods 0.000 claims abstract description 68
- 206010028980 Neoplasm Diseases 0.000 claims abstract description 46
- 238000011161 development Methods 0.000 claims abstract description 35
- 230000018109 developmental process Effects 0.000 claims abstract description 35
- 210000004185 liver Anatomy 0.000 claims abstract description 30
- 238000004458 analytical method Methods 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 15
- 230000003902 lesion Effects 0.000 claims description 33
- 238000012549 training Methods 0.000 claims description 17
- 238000012216 screening Methods 0.000 claims description 13
- 238000002372 labelling Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 8
- 230000007246 mechanism Effects 0.000 claims description 8
- 238000002591 computed tomography Methods 0.000 abstract description 17
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 3
- 230000005540 biological transmission Effects 0.000 abstract description 2
- 201000011510 cancer Diseases 0.000 description 16
- 210000004027 cell Anatomy 0.000 description 14
- 210000004881 tumor cell Anatomy 0.000 description 10
- 230000004069 differentiation Effects 0.000 description 8
- 230000036210 malignancy Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 206010027476 Metastases Diseases 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000003211 malignant effect Effects 0.000 description 4
- 230000009401 metastasis Effects 0.000 description 4
- 210000000056 organ Anatomy 0.000 description 4
- 238000003745 diagnosis Methods 0.000 description 3
- INZOTETZQBPBCE-NYLDSJSYSA-N 3-sialyl lewis Chemical compound O[C@H]1[C@H](O)[C@H](O)[C@H](C)O[C@H]1O[C@H]([C@H](O)CO)[C@@H]([C@@H](NC(C)=O)C=O)O[C@H]1[C@H](O)[C@@H](O[C@]2(O[C@H]([C@H](NC(C)=O)[C@@H](O)C2)[C@H](O)[C@H](O)CO)C(O)=O)[C@@H](O)[C@@H](CO)O1 INZOTETZQBPBCE-NYLDSJSYSA-N 0.000 description 2
- 208000020202 Abnormality of the digestive system Diseases 0.000 description 2
- 206010003445 Ascites Diseases 0.000 description 2
- 102000012406 Carcinoembryonic Antigen Human genes 0.000 description 2
- 108010022366 Carcinoembryonic Antigen Proteins 0.000 description 2
- WQZGKKKJIJFFOK-GASJEMHNSA-N Glucose Natural products OC[C@H]1OC(O)[C@H](O)[C@@H](O)[C@@H]1O WQZGKKKJIJFFOK-GASJEMHNSA-N 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 102000013529 alpha-Fetoproteins Human genes 0.000 description 2
- 108010026331 alpha-Fetoproteins Proteins 0.000 description 2
- 239000008280 blood Substances 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 238000009534 blood test Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000024245 cell differentiation Effects 0.000 description 2
- 230000015271 coagulation Effects 0.000 description 2
- 238000005345 coagulation Methods 0.000 description 2
- 238000004195 computer-aided diagnosis Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000008103 glucose Substances 0.000 description 2
- 201000011066 hemangioma Diseases 0.000 description 2
- 206010073071 hepatocellular carcinoma Diseases 0.000 description 2
- 231100000844 hepatocellular carcinoma Toxicity 0.000 description 2
- 230000001394 metastastic effect Effects 0.000 description 2
- 206010061289 metastatic neoplasm Diseases 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/52—Scale-space analysis, e.g. wavelet analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
- G06V2201/031—Recognition of patterns in medical or anatomical images of internal organs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
- G06V2201/032—Recognition of patterns in medical or anatomical images of protuberances, polyps nodules, etc.
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Databases & Information Systems (AREA)
- Epidemiology (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Primary Health Care (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Data Mining & Analysis (AREA)
- Pathology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The invention discloses a liver tumor classification method, a liver tumor classification system and a liver tumor classification storage medium, which belong to the technical field of data processing and data transmission, wherein the liver tumor classification method comprises the following steps: acquiring detection image data; performing edge extraction on the detected image data to obtain edge image data; respectively inputting the detection image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information; and sending the liver tumor type information and the liver tumor development prediction information to a preset terminal for display. According to the invention, the CT scan image of the liver is analyzed through the image features of the region of interest and the edge features of the edge image, the type of liver tumor and the development condition of the tumor are predicted, and a doctor is assisted in judging liver tumor diseases.
Description
Technical Field
The present application relates to the field of data processing and data transmission, and more particularly, to a liver tumor classification method, system and storage medium.
Background
Along with the development of computer technology, in the medical field, medical images are analyzed through the computer technology, so that doctors can be assisted in judging the illness state of patients. In the traditional computer aided diagnosis system, the diagnosis effect is very good in the aspects of focus identification, tumor classification and the like, but the traditional computer aided diagnosis system mainly defines and selects focus characteristics through subjective factors of people, and the final diagnosis result is influenced by human factors, so that the focus characteristics cannot be accurately judged under some conditions, errors occur in the diagnosis result, and misdiagnosis of doctors is caused.
Therefore, the prior art has defects, and improvement is needed.
Disclosure of Invention
In view of the above problems, the present invention aims to provide a liver tumor classification method, a liver tumor classification system and a storage medium, which can obtain liver tumor type information more effectively and more rapidly.
The first aspect of the invention provides a liver tumor classification method, comprising the following steps:
acquiring detection image data;
performing edge extraction on the detection image data to obtain edge image data;
respectively inputting the detection image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information;
and sending the liver tumor type information and the liver tumor development prediction information to a preset terminal for display.
In this scheme, still include:
acquiring sample image data;
screening the sample image data to obtain sample image data meeting training requirements;
adjusting the image size and the image format of the sample image data meeting the training requirements to obtain preprocessed image data;
and performing analysis training according to the preprocessed image data, and establishing a preset liver tumor classification and development prediction model.
In this scheme, will detect image data with the marginal image data is input respectively and is predetermine liver tumour classification and development prediction model and carry out the analysis, obtains liver tumour kind information and liver tumour development prediction information, includes:
analyzing the detected image data through an attention mechanism to obtain region-of-interest data, and labeling the region-of-interest data through a labeling frame;
extracting features of the region-of-interest data in a multi-scale feature fusion mode to obtain image features;
analyzing according to the edge image data to obtain edge characteristics of liver tumors;
analyzing according to the image characteristics, and combining the edge characteristics of the liver tumor to obtain liver tumor type information;
screening sample image data which are the same as the edge characteristics of the liver tumor according to the edge characteristics of the liver tumor, establishing a tumor development prediction curve according to the tumor development process of the sample image data,
and predicting the tumor development speed of the patient according to the tumor development prediction curve to obtain tumor development prediction information.
In this scheme, still include:
acquiring historical detection image data of a patient;
and dynamically adjusting a tumor development prediction curve of the patient according to the historical detection image data and the current detection image data.
In this scheme, still include:
analyzing according to the liver tumor type information;
if a plurality of classification information exists, calculating the matching degree of each tumor type according to the image characteristics;
and arranging the liver tumor type information in a descending order according to the calculated matching degree, and selecting and displaying the liver tumor type information with the matching degree larger than a preset threshold value.
In this scheme, still include:
acquiring case information of a patient;
and analyzing according to the case information, and dynamically adjusting the matching degree of the tumor types.
The second aspect of the present invention provides a liver tumor classification system, comprising a memory and a processor, wherein the memory comprises a liver tumor classification method program, and the liver tumor classification method program when executed by the processor realizes the following steps:
acquiring detection image data;
performing edge extraction on the detection image data to obtain edge image data;
respectively inputting the detection image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information;
and sending the liver tumor type information and the liver tumor development prediction information to a preset terminal for display.
In this scheme, still include:
acquiring sample image data;
screening the sample image data to obtain sample image data meeting training requirements;
adjusting the image size and the image format of the sample image data meeting the training requirements to obtain preprocessed image data;
and performing analysis training according to the preprocessed image data, and establishing a preset liver tumor classification and development prediction model.
In this scheme, will detect image data with the marginal image data is input respectively and is predetermine liver tumour classification and development prediction model and carry out the analysis, obtains liver tumour kind information and liver tumour development prediction information, includes:
analyzing the detected image data through an attention mechanism to obtain region-of-interest data, and labeling the region-of-interest data through a labeling frame;
extracting features of the region-of-interest data in a multi-scale feature fusion mode to obtain image features;
analyzing according to the edge image data to obtain edge characteristics of liver tumors;
analyzing according to the image characteristics, and combining the edge characteristics of the liver tumor to obtain liver tumor type information;
screening sample image data which are the same as the edge characteristics of the liver tumor according to the edge characteristics of the liver tumor, establishing a tumor development prediction curve according to the tumor development process of the sample image data,
and predicting the tumor development speed of the patient according to the tumor development prediction curve to obtain tumor development prediction information.
In this scheme, still include:
acquiring historical detection image data of a patient;
and dynamically adjusting a tumor development prediction curve of the patient according to the historical detection image data and the current detection image data.
In this scheme, still include:
analyzing according to the liver tumor type information;
if a plurality of classification information exists, calculating the matching degree of each tumor type according to the image characteristics;
and arranging the liver tumor type information in a descending order according to the calculated matching degree, and selecting and displaying the liver tumor type information with the matching degree larger than a preset threshold value.
In this scheme, still include:
acquiring case information of a patient;
and analyzing according to the case information, and dynamically adjusting the matching degree of the tumor types.
A third aspect of the present invention provides a computer readable storage medium comprising a liver lesion classification method program which, when executed by a processor, implements the steps of a liver lesion classification method as described in any one of the preceding claims.
The invention discloses a liver tumor classification method, a liver tumor classification system and a liver tumor classification storage medium, wherein the liver tumor classification method comprises the following steps: acquiring detection image data; performing edge extraction on the detected image data to obtain edge image data; respectively inputting the detection image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information; and sending the liver tumor type information and the liver tumor development prediction information to a preset terminal for display. According to the invention, the CT scan image of the liver is analyzed through the image features of the region of interest and the edge features of the edge image, the type of liver tumor and the development condition of the tumor are predicted, and a doctor is assisted in judging liver tumor diseases.
Drawings
FIG. 1 shows a flow chart of a liver tumor classification method of the present invention;
FIG. 2 is a flow chart of a method of establishing a predictive model for classifying and developing a preset liver tumor in accordance with the present invention;
FIG. 3 is a flow chart showing a method of analyzing liver tumor type information and liver tumor development prediction information according to the present invention;
fig. 4 shows a block diagram of a liver lesion classification system according to the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, in the case of no conflict, the embodiments of the present application and the features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those described herein, and therefore the scope of the present invention is not limited to the specific embodiments disclosed below.
Fig. 1 shows a flow chart of a liver tumor classification method according to the invention.
As shown in fig. 1, the invention discloses a liver tumor classification method, which comprises the following steps:
s102, acquiring detection image data;
s104, carrying out edge extraction on the detection image data to obtain edge image data;
s106, inputting the detection image data and the edge image data into a preset liver tumor classification and development prediction model respectively for analysis to obtain liver tumor type information and liver tumor development prediction information;
s108, the liver tumor type information and the liver tumor development prediction information are sent to a preset terminal to be displayed.
According to the embodiment of the invention, firstly, the data of the region of interest (focal lesion region) is obtained through analysis of the detected image data, the data is marked through a marking frame, and then the edge extraction is carried out based on the region of interest, so that the edge image data is obtained. And then, respectively carrying out feature extraction on the data of the region of interest and the edge image data through a model to obtain image features and edge features, analyzing and judging according to the obtained image features and the edge features to obtain liver tumor type information, screening in a system database based on the edge features, selecting sample image data which is the same as the edge features, establishing a tumor development prediction curve, and predicting the liver tumor development condition of the patient through the tumor development prediction curve to obtain liver tumor development prediction information. And finally, sending the liver tumor type information and the liver tumor development prediction information to a preset terminal, namely a display terminal, to display.
Fig. 2 shows a flow chart of a method for establishing a preset liver tumor classification and development prediction model according to the present invention.
As shown in fig. 2, according to an embodiment of the present invention, further includes:
s202, acquiring sample image data;
s204, screening the sample image data to obtain sample image data meeting training requirements;
s206, adjusting the image size and the image format of the sample image data meeting the training requirement to obtain preprocessed image data;
and S208, performing analysis training according to the preprocessed image data, and establishing a preset liver tumor classification and development prediction model.
It should be noted that, the sample image data is obtained through a certain hospital, and is a liver CT scan image of many years in the hospital, and a liver CT scan image suitable for the preset liver tumor classification and development prediction model in the scheme is selected from the liver CT scan images, including a local lesion image (such as hepatocellular carcinoma, hemangioma, etc.) and a healthy liver Patch image, other irrelevant data are deleted, and then the image size of the liver CT scan image is adjusted, and is fixed to 64×64. In addition, the original image format of a liver CT scan image obtained from a hospital is typically DICOM, the pixel value is usually (0, 4096), and when the model is trained, the pixel value of the image needs to be converted into a CT value, the conversion can be performed by the conversion formula hu=pv×rs+ri, where HU represents the CT value, PV represents the pixel value of the image, RS represents the scaling slope, and RI represents the scaling intercept, in this case, RS takes 1, RI takes-1024, so that the image pixel value (gray value) is converted into the CT value. In addition, different organs in the human body have different relative densities, and the corresponding CT values are different, so that in order to reduce the interference of other organs of the human body on CT scanning of the liver, the obtained CT values can be fixed between [ -100,400], namely, the numerical value larger than 400 is fixed to 400, and the numerical value smaller than-100 is fixed to-100, so that the liver part is clearer in the CT image. Then extracting tumor images and edge images from the adjusted CT images, and training the tumor images and the edge images respectively to obtain a preset liver tumor classification and development prediction model.
Fig. 3 shows a flow chart of a method for analyzing liver tumor type information and liver tumor development prediction information according to the present invention.
As shown in fig. 3, according to an embodiment of the present invention, the inputting the detected image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information includes:
s302, analyzing the detected image data through an attention mechanism to obtain region-of-interest data, and labeling the region-of-interest data through a labeling frame;
s304, extracting features of the region of interest data in a multi-scale feature fusion mode to obtain image features;
s306, analyzing according to the edge image data to obtain edge characteristics of the liver tumor;
s308, analyzing according to the image characteristics, and combining the edge characteristics of the liver tumor to obtain liver tumor type information;
s310, screening sample image data which are the same as the edge characteristics of the liver tumor according to the edge characteristics of the liver tumor, establishing a tumor development prediction curve according to the tumor development process of the sample image data,
s312, predicting the tumor development speed of the patient according to the tumor development prediction curve to obtain tumor development prediction information.
It should be noted that, the region of interest data refers to a focal lesion region existing in a liver, firstly, the focal lesion region in a CT image is identified and marked through a model, then, a minimum external rectangle, namely a marking frame, is produced according to the marked focal lesion region, in this process, the image can be calculated through an area rect function, then, the image is cut according to the marking frame, feature extraction is performed on the image data obtained after cutting, the image features of different scales are extracted through a multi-scale fusion module through different layers, and the image features of different scales are integrated to obtain the image features of the focal lesion region.
The edge features of tumors in different states are different, for example, the boundary of benign tumors is clear, the envelope is complete, the boundary of malignant tumors is fuzzy, burrs exist on the boundary surface, and the envelope is incomplete, so that the types of tumors can be predicted by analyzing the image features of focal lesion areas in combination with the edge features. Meanwhile, the development speed of the tumor can be predicted through the edge features of the edge images of the tumor, the edge features in the detection images are compared with the edge features of the tumor images in the database through the model, sample images with the same features are selected, all detection images of people corresponding to the sample images are derived, and a tumor development prediction curve is drawn according to the edge features of the detection images and the detection time. If a plurality of sample images with the same characteristics exist in the database, a plurality of tumor development prediction curves are drawn according to the used sample images with the same characteristics, then an average trend line is calculated according to the plurality of tumor development prediction curves, and the obtained average trend line is used as a tumor development prediction curve of the detection image. The tumor development speed (including tumor size, predicted development time, etc.) of the patient is then predicted by a tumor development prediction curve.
In addition, in the process of identifying the focal lesion area, the focal lesion area is identified through an attention mechanism, more energy is used for acquiring the focal lesion area, and the attention degree of other areas is reduced, so that the identification efficiency is improved.
According to an embodiment of the present invention, further comprising:
acquiring historical detection image data of a patient;
and dynamically adjusting a tumor development prediction curve of the patient according to the historical detection image data and the current detection image data.
It should be noted that, the history detection image data of the patient refers to all CT scan images recorded in the system by the current patient, after the tumor development prediction curve is drawn, the history detection image data of the current patient is detected, if the history detection image data exists, the edge feature and the detection time of the history detection image data are obtained, the edge feature and the detection time are input into the tumor development prediction curve, the data of the corresponding time are replaced, and the tumor development prediction curve is dynamically adjusted according to the replaced data.
According to an embodiment of the present invention, further comprising:
analyzing according to the liver tumor type information;
if a plurality of classification information exists, calculating the matching degree of each tumor type according to the image characteristics;
and arranging the liver tumor type information in a descending order according to the calculated matching degree, and selecting and displaying the liver tumor type information with the matching degree larger than a preset threshold value.
It should be noted that, under some circumstances, since features between different lesions are similar, the variability is not large, the model cannot accurately predict liver tumor type information according to the CT detection image, and may obtain prediction results of multiple liver tumor types, in this case, matching degree calculation may be performed according to image features of the detection image and the obtained lesion features of different types of liver tumors, then the obtained liver tumor type information may be sorted in descending order according to the matching degree, and compared with a preset threshold, the liver tumor type information with matching degree smaller than the preset threshold may be filtered, and liver tumor type information with similarity greater than the preset threshold may be retained, and the liver tumor type information and its corresponding matching degree may be displayed.
According to an embodiment of the present invention, further comprising:
acquiring case information of a patient;
and analyzing according to the case information, and dynamically adjusting the matching degree of the tumor types.
The case information of the patient includes paper cases, electronic medical records, test reports, and the like of the patient, from which a plurality of data related to tumor type prediction are extracted, for example, a blood test report (alpha fetoprotein content, carbohydrate antigen 19-9 content, carcinoembryonic antigen content, coagulation function, blood glucose level), the presence of ascites, abnormality of the digestive system, and the like. If the above situation is abnormal, the possibility of the occurrence of malignant tumor is high.
According to an embodiment of the present invention, further comprising:
obtaining information of the differentiation degree of tumor cells;
liver tumors were classified according to the tumor cell differentiation degree information.
It should be noted that, the differentiation degree of tumor cells is judged according to the differentiation degree of tumor cells under a microscope, and then liver tumors can be classified into three grades of low malignancy, moderate malignancy and high malignancy according to the differentiation degree of tumor cells. Wherein, the low malignancy refers to highly differentiated cancer cells, which are similar to normal cells, the cells are well differentiated, the growth speed is slow, and the possibility of metastasis is low; moderately malignant refers to moderately differentiated cancer cells, cells differentiate moderately, grow faster than normal cells, and the likelihood of metastasis is general; highly malignant means poorly differentiated cells, with a faster growth rate, with extremely aggressive and easily metastatic.
Fig. 4 shows a block diagram of a liver lesion classification system according to the present invention.
As shown in fig. 4, a second aspect of the present invention provides a liver tumor classification system 4, comprising a memory 41 and a processor 42, wherein the memory comprises a liver tumor classification method program, and the liver tumor classification method program when executed by the processor implements the following steps:
acquiring detection image data;
performing edge extraction on the detection image data to obtain edge image data;
respectively inputting the detection image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information;
and sending the liver tumor type information and the liver tumor development prediction information to a preset terminal for display.
According to the embodiment of the invention, firstly, the data of the region of interest (focal lesion region) is obtained through analysis of the detected image data, the data is marked through a marking frame, and then the edge extraction is carried out based on the region of interest, so that the edge image data is obtained. And then, respectively carrying out feature extraction on the data of the region of interest and the edge image data through a model to obtain image features and edge features, analyzing and judging according to the obtained image features and the edge features to obtain liver tumor type information, screening in a system database based on the edge features, selecting sample image data which is the same as the edge features, establishing a tumor development prediction curve, and predicting the liver tumor development condition of the patient through the tumor development prediction curve to obtain liver tumor development prediction information. And finally, sending the liver tumor type information and the liver tumor development prediction information to a preset terminal, namely a display terminal, to display.
According to an embodiment of the present invention, further comprising:
acquiring sample image data;
screening the sample image data to obtain sample image data meeting training requirements;
adjusting the image size and the image format of the sample image data meeting the training requirements to obtain preprocessed image data;
and performing analysis training according to the preprocessed image data, and establishing a preset liver tumor classification and development prediction model.
It should be noted that, the sample image data is obtained through a certain hospital, and is a liver CT scan image of many years in the hospital, and a liver CT scan image suitable for the preset liver tumor classification and development prediction model in the scheme is selected from the liver CT scan images, including a local lesion image (such as hepatocellular carcinoma, hemangioma, etc.) and a healthy liver Patch image, other irrelevant data are deleted, and then the image size of the liver CT scan image is adjusted, and is fixed to 64×64. In addition, the original image format of a liver CT scan image obtained from a hospital is typically DICOM, the pixel value is usually (0, 4096), and when the model is trained, the pixel value of the image needs to be converted into a CT value, the conversion can be performed by the conversion formula hu=pv×rs+ri, where HU represents the CT value, PV represents the pixel value of the image, RS represents the scaling slope, and RI represents the scaling intercept, in this case, RS takes 1, RI takes-1024, so that the image pixel value (gray value) is converted into the CT value. In addition, different organs in the human body have different relative densities, and the corresponding CT values are different, so that in order to reduce the interference of other organs of the human body on CT scanning of the liver, the obtained CT values can be fixed between [ -100,400], namely, the numerical value larger than 400 is fixed to 400, and the numerical value smaller than-100 is fixed to-100, so that the liver part is clearer in the CT image. Then extracting tumor images and edge images from the adjusted CT images, and training the tumor images and the edge images respectively to obtain a preset liver tumor classification and development prediction model.
According to an embodiment of the present invention, the inputting the detected image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information includes:
analyzing the detected image data through an attention mechanism to obtain region-of-interest data, and labeling the region-of-interest data through a labeling frame;
extracting features of the region-of-interest data in a multi-scale feature fusion mode to obtain image features;
analyzing according to the edge image data to obtain edge characteristics of liver tumors;
analyzing according to the image characteristics, and combining the edge characteristics of the liver tumor to obtain liver tumor type information;
screening sample image data which are the same as the edge characteristics of the liver tumor according to the edge characteristics of the liver tumor, establishing a tumor development prediction curve according to the tumor development process of the sample image data,
and predicting the tumor development speed of the patient according to the tumor development prediction curve to obtain tumor development prediction information.
It should be noted that, the region of interest data refers to a focal lesion region existing in a liver, firstly, the focal lesion region in a CT image is identified and marked through a model, then, a minimum external rectangle, namely a marking frame, is produced according to the marked focal lesion region, in this process, the image can be calculated through an area rect function, then, the image is cut according to the marking frame, feature extraction is performed on the image data obtained after cutting, the image features of different scales are extracted through a multi-scale fusion module through different layers, and the image features of different scales are integrated to obtain the image features of the focal lesion region.
The edge features of tumors in different states are different, for example, the boundary of benign tumors is clear, the envelope is complete, the boundary of malignant tumors is fuzzy, burrs exist on the boundary surface, and the envelope is incomplete, so that the types of tumors can be predicted by analyzing the image features of focal lesion areas in combination with the edge features. Meanwhile, the development speed of the tumor can be predicted through the edge features of the edge images of the tumor, the edge features in the detection images are compared with the edge features of the tumor images in the database through the model, sample images with the same features are selected, all detection images of people corresponding to the sample images are derived, and a tumor development prediction curve is drawn according to the edge features of the detection images and the detection time. If a plurality of sample images with the same characteristics exist in the database, a plurality of tumor development prediction curves are drawn according to the used sample images with the same characteristics, then an average trend line is calculated according to the plurality of tumor development prediction curves, and the obtained average trend line is used as a tumor development prediction curve of the detection image. The tumor development speed (including tumor size, predicted development time, etc.) of the patient is then predicted by a tumor development prediction curve.
In addition, in the process of identifying the focal lesion area, the focal lesion area is identified through an attention mechanism, more energy is used for acquiring the focal lesion area, and the attention degree of other areas is reduced, so that the identification efficiency is improved.
According to an embodiment of the present invention, further comprising:
acquiring historical detection image data of a patient;
and dynamically adjusting a tumor development prediction curve of the patient according to the historical detection image data and the current detection image data.
It should be noted that, the history detection image data of the patient refers to all CT scan images recorded in the system by the current patient, after the tumor development prediction curve is drawn, the history detection image data of the current patient is detected, if the history detection image data exists, the edge feature and the detection time of the history detection image data are obtained, the edge feature and the detection time are input into the tumor development prediction curve, the data of the corresponding time are replaced, and the tumor development prediction curve is dynamically adjusted according to the replaced data.
According to an embodiment of the present invention, further comprising:
analyzing according to the liver tumor type information;
if a plurality of classification information exists, calculating the matching degree of each tumor type according to the image characteristics;
and arranging the liver tumor type information in a descending order according to the calculated matching degree, and selecting and displaying the liver tumor type information with the matching degree larger than a preset threshold value.
It should be noted that, under some circumstances, since features between different lesions are similar, the variability is not large, the model cannot accurately predict liver tumor type information according to the CT detection image, and may obtain prediction results of multiple liver tumor types, in this case, matching degree calculation may be performed according to image features of the detection image and the obtained lesion features of different types of liver tumors, then the obtained liver tumor type information may be sorted in descending order according to the matching degree, and compared with a preset threshold, the liver tumor type information with matching degree smaller than the preset threshold may be filtered, and liver tumor type information with similarity greater than the preset threshold may be retained, and the liver tumor type information and its corresponding matching degree may be displayed.
According to an embodiment of the present invention, further comprising:
acquiring case information of a patient;
and analyzing according to the case information, and dynamically adjusting the matching degree of the tumor types.
The case information of the patient includes paper cases, electronic medical records, test reports, and the like of the patient, from which a plurality of data related to tumor type prediction are extracted, for example, a blood test report (alpha fetoprotein content, carbohydrate antigen 19-9 content, carcinoembryonic antigen content, coagulation function, blood glucose level), the presence of ascites, abnormality of the digestive system, and the like. If the above situation is abnormal, the possibility of the occurrence of malignant tumor is high.
According to an embodiment of the present invention, further comprising:
obtaining information of the differentiation degree of tumor cells;
liver tumors were classified according to the tumor cell differentiation degree information.
It should be noted that, the differentiation degree of tumor cells is judged according to the differentiation degree of tumor cells under a microscope, and then liver tumors can be classified into three grades of low malignancy, moderate malignancy and high malignancy according to the differentiation degree of tumor cells. Wherein, the low malignancy refers to highly differentiated cancer cells, which are similar to normal cells, the cells are well differentiated, the growth speed is slow, and the possibility of metastasis is low; moderately malignant refers to moderately differentiated cancer cells, cells differentiate moderately, grow faster than normal cells, and the likelihood of metastasis is general; highly malignant means poorly differentiated cells, with a faster growth rate, with extremely aggressive and easily metastatic.
A third aspect of the present invention provides a computer readable storage medium comprising a liver lesion classification method program which, when executed by a processor, implements the steps of a liver lesion classification method as described in any one of the preceding claims.
The invention discloses a liver tumor classification method, a liver tumor classification system and a liver tumor classification storage medium, wherein the liver tumor classification method comprises the following steps: acquiring detection image data; performing edge extraction on the detected image data to obtain edge image data; respectively inputting the detection image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information; and sending the liver tumor type information and the liver tumor development prediction information to a preset terminal for display. According to the invention, the CT scan image of the liver is analyzed through the image features of the region of interest and the edge features of the edge image, the type of liver tumor and the development condition of the tumor are predicted, and a doctor is assisted in judging liver tumor diseases.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units; can be located in one place or distributed to a plurality of network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present invention may be integrated in one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated in one unit; the integrated units may be implemented in hardware or in hardware plus software functional units.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware related to program instructions, and the foregoing program may be stored in a computer readable storage medium, where the program, when executed, performs steps including the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, or the like, which can store program codes.
Alternatively, the above-described integrated units of the present invention may be stored in a computer-readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in essence or a part contributing to the prior art in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
Claims (10)
1. A method of classifying liver tumors, comprising:
acquiring detection image data;
performing edge extraction on the detection image data to obtain edge image data;
respectively inputting the detection image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information;
and sending the liver tumor type information and the liver tumor development prediction information to a preset terminal for display.
2. The liver tumor classification method according to claim 1, further comprising:
acquiring sample image data;
screening the sample image data to obtain sample image data meeting training requirements;
adjusting the image size and the image format of the sample image data meeting the training requirements to obtain preprocessed image data;
and performing analysis training according to the preprocessed image data, and establishing a preset liver tumor classification and development prediction model.
3. The liver tumor classification method according to claim 1, wherein the inputting the detected image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information, respectively, comprises:
analyzing the detected image data through an attention mechanism to obtain region-of-interest data, and labeling the region-of-interest data through a labeling frame;
extracting features of the region-of-interest data in a multi-scale feature fusion mode to obtain image features;
analyzing according to the edge image data to obtain edge characteristics of liver tumors;
analyzing according to the image characteristics, and combining the edge characteristics of the liver tumor to obtain liver tumor type information;
screening sample image data which are the same as the edge characteristics of the liver tumor according to the edge characteristics of the liver tumor, and establishing a tumor development prediction curve according to the tumor development process of the sample image data;
and predicting the tumor development speed of the patient according to the tumor development prediction curve to obtain tumor development prediction information.
4. A method of classifying liver tumors as claimed in claim 3, further comprising:
acquiring historical detection image data of a patient;
and dynamically adjusting a tumor development prediction curve of the patient according to the historical detection image data and the current detection image data.
5. A method of classifying liver tumors as claimed in claim 3, further comprising:
analyzing according to the liver tumor type information;
if a plurality of classification information exists, calculating the matching degree of each tumor type according to the image characteristics;
and arranging the liver tumor type information in a descending order according to the calculated matching degree, and selecting and displaying the liver tumor type information with the matching degree larger than a preset threshold value.
6. The liver tumor classification method according to claim 1, further comprising:
acquiring case information of a patient;
and analyzing according to the case information, and dynamically adjusting the matching degree of the tumor types.
7. A liver lesion classification system comprising a memory and a processor, the memory including a liver lesion classification method program therein, the liver lesion classification method program when executed by the processor performing the steps of:
acquiring detection image data;
performing edge extraction on the detection image data to obtain edge image data;
respectively inputting the detection image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information;
and sending the liver tumor type information and the liver tumor development prediction information to a preset terminal for display.
8. The liver tumor classification system according to claim 7, wherein the inputting the detected image data and the edge image data into a preset liver tumor classification and development prediction model for analysis to obtain liver tumor type information and liver tumor development prediction information, respectively, comprises:
analyzing the detected image data through an attention mechanism to obtain region-of-interest data, and labeling the region-of-interest data through a labeling frame;
extracting features of the region-of-interest data in a multi-scale feature fusion mode to obtain image features;
analyzing according to the edge image data to obtain edge characteristics of liver tumors;
analyzing according to the image characteristics, and combining the edge characteristics of the liver tumor to obtain liver tumor type information;
screening sample image data which are the same as the edge characteristics of the liver tumor according to the edge characteristics of the liver tumor, establishing a tumor development prediction curve according to the tumor development process of the sample image data,
and predicting the tumor development speed of the patient according to the tumor development prediction curve to obtain tumor development prediction information.
9. The liver tumor classification system of claim 8, further comprising:
acquiring historical detection image data of a patient;
and dynamically adjusting a tumor development prediction curve of the patient according to the historical detection image data and the current detection image data.
10. A computer-readable storage medium, characterized in that it comprises a liver tumor classification method program, which, when executed by a processor, implements the steps of a liver tumor classification method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310755532.5A CN116503672B (en) | 2023-06-26 | 2023-06-26 | Liver tumor classification method, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310755532.5A CN116503672B (en) | 2023-06-26 | 2023-06-26 | Liver tumor classification method, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116503672A true CN116503672A (en) | 2023-07-28 |
| CN116503672B CN116503672B (en) | 2023-08-25 |
Family
ID=87325139
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310755532.5A Active CN116503672B (en) | 2023-06-26 | 2023-06-26 | Liver tumor classification method, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116503672B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111178449A (en) * | 2019-12-31 | 2020-05-19 | 浙江大学 | Liver cancer image classification method and device combining computer vision characteristics and imaging omics characteristics |
| US20210153838A1 (en) * | 2019-11-21 | 2021-05-27 | Hsiao-Ching Nien | Method and Apparatus of Intelligent Analysis for Liver Tumor |
| CN114494199A (en) * | 2022-01-27 | 2022-05-13 | 杭州职业技术学院 | Liver CT tumor segmentation and classification method based on deep learning |
| CN114974567A (en) * | 2022-05-25 | 2022-08-30 | 浙江大学 | Liver tumor interventional response post-evaluation method and system based on antagonistic generation network |
| CN115601299A (en) * | 2022-09-06 | 2023-01-13 | 嘉兴市第一医院（Cn） | Intelligent liver cirrhosis state evaluation system and method based on images |
| CN115841476A (en) * | 2022-12-14 | 2023-03-24 | 杭州类脑科技有限公司 | Method, device, equipment and medium for predicting life cycle of liver cancer patient |
-
2023
- 2023-06-26 CN CN202310755532.5A patent/CN116503672B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210153838A1 (en) * | 2019-11-21 | 2021-05-27 | Hsiao-Ching Nien | Method and Apparatus of Intelligent Analysis for Liver Tumor |
| CN111178449A (en) * | 2019-12-31 | 2020-05-19 | 浙江大学 | Liver cancer image classification method and device combining computer vision characteristics and imaging omics characteristics |
| CN114494199A (en) * | 2022-01-27 | 2022-05-13 | 杭州职业技术学院 | Liver CT tumor segmentation and classification method based on deep learning |
| CN114974567A (en) * | 2022-05-25 | 2022-08-30 | 浙江大学 | Liver tumor interventional response post-evaluation method and system based on antagonistic generation network |
| CN115601299A (en) * | 2022-09-06 | 2023-01-13 | 嘉兴市第一医院（Cn） | Intelligent liver cirrhosis state evaluation system and method based on images |
| CN115841476A (en) * | 2022-12-14 | 2023-03-24 | 杭州类脑科技有限公司 | Method, device, equipment and medium for predicting life cycle of liver cancer patient |
Non-Patent Citations (1)
| Title |
|---|
| 马金林;邓媛媛;马自萍;: "肝脏肿瘤CT图像深度学习分割方法综述", 中国图象图形学报, no. 10 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116503672B (en) | 2023-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP0757544B1 (en) | Computerized detection of masses and parenchymal distortions | |
| Yap et al. | A novel algorithm for initial lesion detection in ultrasound breast images | |
| US8144963B2 (en) | Method for processing biomedical images | |
| US9305204B2 (en) | Method and system to detect the microcalcifications in X-ray images using nonlinear energy operator | |
| Abbas et al. | Computer‐aided pattern classification system for dermoscopy images | |
| CN111986150A (en) | Interactive marking refinement method for digital pathological image | |
| Székely et al. | A hybrid system for detecting masses in mammographic images | |
| Xu et al. | Computerized classification of prostate cancer gleason scores from whole slide images | |
| US9122897B2 (en) | Method for the detection and classification of microcalcification clusters in digital mammograms | |
| Torrents-Barrena et al. | Computer-aided diagnosis of breast cancer via Gabor wavelet bank and binary-class SVM in mammographic images | |
| CN114529505A (en) | Breast lesion risk assessment system based on deep learning | |
| CN117036343B (en) | FFOCT image analysis method and device for identifying axillary lymph node metastasis | |
| Ramadijanti et al. | Automatic breast tumor segmentation using hierarchical k-means on mammogram | |
| CN116503672B (en) | Liver tumor classification method, system and storage medium | |
| Oliver et al. | Influence of using manual or automatic breast density information in a mass detection cad system | |
| Zhou et al. | Improved breast lesion detection in mammogram images using a deep neural network | |
| Li et al. | A novel approach of computer-aided detection of focal ground-glass opacity in 2D lung CT images | |
| Vaidehi et al. | Automatic classification and retrieval of mammographic tissue density using texture features | |
| Elguebaly et al. | A bayesian approach for the classification of mammographic masses | |
| Cazzolato et al. | dp-breath: Heat maps and probabilistic classification assisting the analysis of abnormal lung regions | |
| Leichter et al. | The use of an interactive software program for quantitative characterization of microcalcifications on digitized film-screen mammograms | |
| Astley | Evaluation of computer-aided detection (CAD) prompting techniques for mammography | |
| CN113920100A (en) | Knowledge distillation-based weak supervision bone scanning image hot spot segmentation method and system | |
| WO2022153100A1 (en) | A method for detecting breast cancer using artificial neural network | |
| Munasinghe et al. | Yuwathi: early detection of breast cancer and classification of mammography images using machine learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |