CN111870224A - Tumor blood vessel normalization detection system and detection method - Google Patents

Tumor blood vessel normalization detection system and detection method Download PDF

Info

Publication number
CN111870224A
CN111870224A CN202010689075.0A CN202010689075A CN111870224A CN 111870224 A CN111870224 A CN 111870224A CN 202010689075 A CN202010689075 A CN 202010689075A CN 111870224 A CN111870224 A CN 111870224A
Authority
CN
China
Prior art keywords
blood vessel
tumor
image
blood
normalization
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010689075.0A
Other languages
Chinese (zh)
Other versions
CN111870224B (en
Inventor
汪洋
刘立龙
陈亚昕
陈雪寒
袁茜
杨静
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN202010689075.0A priority Critical patent/CN111870224B/en
Publication of CN111870224A publication Critical patent/CN111870224A/en
Application granted granted Critical
Publication of CN111870224B publication Critical patent/CN111870224B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/026Measuring blood flow
    • A61B5/0261Measuring blood flow using optical means, e.g. infrared light
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/145Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
    • A61B5/1455Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue using optical sensors, e.g. spectral photometrical oximeters
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10056Microscopic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20024Filtering details
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30024Cell structures in vitro; Tissue sections in vitro
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30096Tumor; Lesion
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30101Blood vessel; Artery; Vein; Vascular

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physiology (AREA)
  • Cardiology (AREA)
  • Hematology (AREA)
  • Quality & Reliability (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Optics & Photonics (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a tumor blood vessel normalization detection system and a detection method, wherein the system comprises: the blood flow detection and analysis system comprises a laser light source module, an imaging module, an image processor, a blood flow detection and analysis module, a blood vessel morphology analysis module, an artificial intelligence analysis module and a display module; the blood flow detection and analysis module is used for acquiring a blood flow velocity image and processing the speckle image into the blood flow velocity image through an algorithm; the blood vessel morphology analysis module is used for extracting a tumor blood vessel morphological structure and measuring the tumor blood vessel density and the blood vessel distortion degree; and the artificial intelligence analysis module is used for integrating the blood flow speed, the blood vessel density and the distortion degree of the tumor, judging whether the tumor blood vessels are normalized or not and giving a prediction result of the normalization window period of the tumor blood vessels. The invention can evaluate whether the tumor blood vessel is in a normalized window stage or not by imaging the structure and the function of the tumor blood vessel, improves the clinical status of the anti-tumor angiogenesis therapy and provides a basis for formulating a more reasonable anti-tumor combined treatment scheme.

Description

Tumor blood vessel normalization detection system and detection method
Technical Field
The invention relates to the technical field of artificial intelligent detection for tumor vessel normalization, in particular to a tumor vessel normalization detection system and a tumor vessel normalization detection method.
Background
With the change of life style and the pollution of living environment, the incidence of tumor diseases is higher and higher. Tumors have become a common disease that seriously jeopardizes human health and life. According to the report of '2018 global cancer statistical data', it is estimated that 1810 ten thousand cancer cases and 970 ten thousand cancer death cases will be newly added in 2018 all over the world. By the end of this century, cancer will become the world's first killer', the largest "road barrage" that hinders the extension of human life expectancy. According to the latest statistical data, Chinese malignant tumor patients account for nearly 20% of the world, the number of deaths accounts for nearly 24% of the world, and the morbidity and the mortality are the first global. At present, malignant tumors become the first leading cause of death of residents in cities in China and the second leading cause of death of residents in rural areas. The incidence of Chinese malignant tumor has been increased by about 3.9% each year, the death rate has been increased by 2.5% each year, and the medical cost caused by malignant tumor exceeds 2200 hundred million each year.
Tumor blood vessels play an important role in tumor cell proliferation, invasion and metastasis. In 1971, professor Folkman leads to the theory of anti-angiogenesis therapy of malignant tumor, namely, the aim of inhibiting the growth of tumor is achieved by cutting off the nutrition supply of tumor by inhibiting the new blood vessel of tumor. Anti-angiogenesis therapy, one of the important methods for cancer treatment, can significantly inhibit tumor growth and metastasis. However, the application of long-term anti-vascular drugs can cause serious degeneration of tumor blood vessels to block the transmission of drugs and oxygen, resulting in the reduction of the drug resistance of tumor cells to chemotherapy and the sensitivity to radiotherapy.
Research shows that the reasonable application of the anti-angiogenesis medicine can make the tumor blood vessels with abnormal distortion tend to be normal, so that oxygen and the medicine can be more effectively delivered to the tumor tissues, and the curative effects of chemotherapy, radiotherapy and immunotherapy can be improved. However, the normalization process is transient and reversible, resulting in a specific "time window" for normalization. This "time window" is typically one to several days, during which the tumor microvascular structure and function changes significantly, gradually repairing the abnormality. The anti-tumor effect can be more effectively exerted by combining radiotherapy and chemotherapy in the time window. The tumor blood vessel normalization theory improves the clinical status of the anti-tumor angiogenesis therapy and provides a theoretical basis for formulating a more reasonable anti-tumor combined treatment scheme.
However, the time window for normalization of tumor vessels is related to the tumor type, location and anti-angiogenic drugs. Therefore, how to determine the specific time and range of the "window" is the key to further optimize the combination of radiotherapy, chemotherapy and immunotherapy with anti-tumor angiogenesis therapy mode and to exert the optimal synergistic effect.
Compared with normal blood vessels, the vascular system in the tumor tissue has disordered structure, great density of micro blood vessels, and tortuous and expanded shape. Normal blood vessels appear as two branches, while tumor vessels can appear as three and more branches. The vessel wall is squeezed by tumor cells and stromal cells, resulting in uneven vessel diameter. In addition, abnormalities in the vascular structure of tumors can cause disordered local stagnant blood flow, leading to blood flow disturbances, exhibiting temporal and spatial heterogeneity, and increased ineffective circulation.
To date, the "gold standard" for assessing tumor angiogenesis status and normalization of tumor blood vessels remains a histopathological index such as microvessel density, basal membrane thickness, pericyte coverage, and the like. However, the above indexes have the disadvantages of invasiveness, excessive dependence on precise materials, one-sidedness, and the like. Non-invasive methods include imaging techniques such as dynamic MRI, CT, PET, etc. High resolution imaging techniques, which can measure temporal and spatial changes in blood flow and other parameters, are useful in evaluating the efficacy of anti-angiogenic therapy on the vascular function of tumors in different locations. At present, the methods have high cost, complicated operation and difficult guarantee of accuracy, so the application of the methods in research is limited to a great extent. Therefore, the detection method for accurately judging the normalization window period of the tumor blood vessels is simple and convenient to operate and intelligent, and has good application prospect.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a tumor blood vessel normalization detection system and a detection method aiming at the defects in the prior art, the system has the advantages of high spatial and temporal resolution, no radiation and intelligent analysis, can automatically and quickly analyze the tumor blood vessel normalization window period according to the tumor blood flow perfusion, the blood vessel density, the blood vessel branch node and the blood vessel distortion degree, improves the clinical status of an anti-tumor angiogenesis therapy, and provides a basis for formulating a more reasonable anti-tumor combined treatment scheme.
The technical scheme adopted by the invention for solving the technical problems is as follows:
the invention provides a tumor blood vessel normalization detection system, which comprises:
a laser light source module comprising: a laser for emitting laser light to illuminate the biological tissue; the variable attenuator is used for adjusting the light intensity of the laser; the beam expander is used for changing the beam diameter and the divergence angle of the laser; the reflector is used for reflecting the adjusted laser to the biological tissue to be detected;
an imaging module, comprising: the stereoscopic microscope is used for amplifying the biological tissue image irradiated by the laser; the camera is connected with the stereoscopic microscope and used for receiving the amplified image signal;
the image processor is connected with the imaging module and used for carrying out image processing on the image acquired by the imaging module, and the image processing comprises the following steps: demosaicing, denoising, defogging and color enhancement;
the blood flow detection and analysis module is connected with the imaging module and used for acquiring a blood flow velocity image, and the blood flow detection and analysis module processes the image and processes the speckle image into the blood flow velocity image through an algorithm;
the blood vessel morphology analysis module is connected with the blood flow detection analysis module and is used for extracting the tumor blood vessel morphology structure and measuring the tumor blood vessel density and the blood vessel distortion degree;
the artificial intelligence analysis module is connected with the blood vessel morphology analysis module and used for integrating the blood flow speed, the blood vessel density and the distortion degree of the tumor, judging whether the tumor blood vessel is normalized or not and giving a prediction result of the normalization window period of the tumor blood vessel;
and the display module is connected with the output ends of the image processor and the artificial intelligence analysis module and is used for displaying images and prompting the judgment result of whether the tumor blood vessels are normalized.
Furthermore, the wavelength range of the laser is 600 nm-1300 nm.
Furthermore, the hardware of the blood flow detection and analysis module is composed of a Field Programmable Gate Array (FPGA), a proprietary Asic chip or a high-performance display card; the hardware of the artificial intelligence analysis module is formed into a proprietary Asic chip or a high-performance display card.
Further, the display module of the invention comprises a display and a voice prompter.
The invention provides a tumor blood vessel normalization detection method, which comprises the following steps:
step 1: irradiating the biological tissue to be detected by the adjusted laser, collecting a tumor speckle image of the irradiated biological tissue, and reconstructing a tumor blood flow image from the speckle image;
step 2: detecting tumor vascular structures from the tumor blood flow images based on image morphological processing;
and step 3: extracting blood vessel density, blood vessel diameter and blood vessel branch nodes;
and 4, step 4: calculating the distortion degree of the blood vessels, comparing the distortion degree with the preset normal tissue blood vessels with the same diameter, and grading the distortion degree of the tumor blood vessels;
and 5: the artificial intelligence analysis module is used for integrating the blood flow speed, the blood vessel density and the distortion degree of the tumor, judging whether the tumor blood vessel is normalized or not, and giving a voice prompt of a tumor blood vessel normalization window period prediction result.
Further, the specific method for acquiring the tumor blood flow image in the step 1 of the present invention is as follows:
with Ns*NsThe window filter processes the original speckle image to obtain a normalized variance image Vns(x, y), the processing formula is:
Figure BDA0002588660600000041
wherein N issThe size of the window filter is 5-11, Is(i, j) is the gray value of the pixel in the window filter, i and j are respectively the horizontal and vertical coordinates of the pixel in the window filter,
Figure BDA0002588660600000043
the mean value of the pixel gray level in the window filter is used, and x and y are the horizontal and vertical coordinates of the newly generated pixel points of the normalized variance image;
calculating a two-dimensional blood flow image of the tumor, wherein the formula is as follows:
Figure BDA0002588660600000042
wherein, v (x, y) is the blood flow image.
Further, the specific method for detecting and detecting the tumor vascular structure in step 2 of the present invention is as follows:
(a) respectively filtering the blood flow image by using 12 two-dimensional Gabor filters in different directions to obtain a filtered blood flow image Bλ,,θ(x, y), the processing formula is:
Bλ,,θ(x,y)=∫∫f(u,v)gλ,,θ(x-u,y-v)dudv
wherein f (u, v) is a blood flow image, gλ,,θ(x-u, y-v) is a two-dimensional Gabor filter, the parameter lambda is the reciprocal of the spatial frequency, the parameter is the variance characteristic of the Gabor filter, the value range of the lambda is 0.1-10, the value interval is 0.1, and is lambda/2; the parameter theta being Gabor filterThe filtering direction of the wave filter realizes the detection of blood vessels in different directions by changing a parameter theta, wherein the value of theta is N pi/12, and N is 0,1, … and 12;
(b) extraction of Bλ,,θMaximum response values max (B) in 12 different directions for each pixel in (x, y)λ,,θ(x, y)) are combined into a new image, i.e. the image of the vessel structure H (x, y).
Further, the specific method of step 3 of the present invention is:
the specific method for extracting the blood vessel density comprises the following steps:
(a) carrying out binarization threshold operation on the blood vessel structure image H (x, y), resetting a pixel value with a pixel gray value larger than 0 in the blood vessel structure image to be 1, and resetting a pixel gray value smaller than or equal to 0 to be 0; a new blood vessel structure image H' (x, y) is obtained again;
(b) counting all pixels with pixel value 1 in H' (x, y), and dividing by the total pixels in the blood vessel structure image to obtain blood vessel density;
the method for calculating the number of the blood vessel branch nodes comprises the following steps:
(a) extracting a new blood vessel central skeleton image of the blood vessel structure image H' (x, y) through a conventional window filter function such as a two-dimensional Gabor filter or a Gaussian filter;
(b) analyzing the communication relationship between each pixel point and a neighborhood pixel point on the blood vessel central skeleton image, and recording a certain blood vessel central skeleton pixel point as a blood vessel branch node when the communication number of the certain blood vessel central skeleton pixel point and the neighborhood blood vessel central skeleton pixel point is more than 3;
(c) traversing the whole blood vessel central skeleton image, obtaining all blood vessel branch nodes in the image, and calculating the total number of the blood vessel branch nodes.
Further, the specific method for calculating the degree of blood vessel distortion in step 4 of the present invention is as follows:
(a) dividing the blood vessel skeleton image into a plurality of sections according to branch nodes; calculating the curvature of each point of each section of the vascular skeleton by using the following formula;
Figure BDA0002588660600000051
wherein, K (x, y) represents curvature, and x, y represent horizontal and vertical coordinates of each point respectively;
(b) and taking an absolute value of the curvature of each point of a certain section of the blood vessel skeleton, and then carrying out superposition averaging to obtain the distortion degree of the section of the blood vessel.
Further, the method for determining whether tumor blood vessels are normalized and the prediction result of the window period of normalization of blood vessels in step 5 of the present invention comprises:
(a) collecting 1000 tumor speckle images by using a tumor blood vessel normalization detection system, and extracting blood vessel morphological function information of blood flow velocity, blood vessel density, blood vessel diameter and blood vessel distortion degree;
(b) collecting animal tumor tissue samples, and judging whether tumor blood vessels are normalized or not by an immunofluorescence or immunohistochemical method;
(c) establishing a corresponding relation database of blood flow velocity, blood vessel density, blood vessel diameter, blood vessel distortion degree and tumor blood vessel normalization, wherein the training data are blood flow velocity, blood vessel density, blood vessel diameter and blood vessel distortion degree, and the training label is tumor blood vessel normalization or tumor blood vessel non-normalization;
(d) training a deep neural network model by using the database, wherein the deep neural network model comprises VGG16, GooleNet and ResNet deep neural network models;
(e) and (4) predicting a new tumor blood flow image by using the training result, and giving a prompt for whether tumor blood vessels are normalized.
The invention has the following beneficial effects: the invention is based on the laser speckle blood flow imaging principle and an artificial intelligence analysis system, and can be used for carrying out functional and structural imaging on blood vessels in tumor tissues and further automatically judging whether the tumor blood vessels are normalized. Compared with other existing tumor blood vessel imaging methods, the tumor blood vessel normalization detection system provided by the invention has the advantages that:
(1) the adopted blood flow image acquisition method is based on the laser speckle optical imaging principle, can obtain the blood flow perfusion image with high spatial resolution, and is convenient for extracting the characteristic indexes of tumor blood flow velocity, blood vessel density, blood vessel distortion degree and the like related to tumor blood vessel normalization. Compared with color Doppler ultrasound, the color Doppler ultrasound has obvious technical advantages.
(2) The invention introduces a convolutional neural network into a tumor blood vessel normalization analysis system, can extract high-dimensional characteristic information which is difficult to obtain by human eyes, and helps to judge and predict the tumor blood vessel normalization window period more accurately.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
FIG. 1 is a system schematic of an embodiment of the invention;
FIG. 2 is a flow chart of a processing method of an embodiment of the invention;
FIG. 3 is a flowchart of a method for determining normalization of tumor vasculature according to an embodiment of the invention;
FIG. 4 is a graph illustrating survival rates for an embodiment of the present invention;
FIG. 5 is a schematic representation of a tumor mass according to an embodiment of the present invention;
FIG. 6 is a schematic representation of immunofluorescence detection in accordance with embodiments of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[ example 1 ] tumor blood vessel normalization detection system
The invention discloses a tumor blood vessel normalization detection system and method. Referring to fig. 1, the system is composed of a laser light source module, an imaging module, an image processor, a blood flow detection and analysis module, a blood vessel morphology analysis module, an artificial intelligence analysis module and a display module.
The laser emitted by the He-Ne laser firstly attenuates the light intensity through the variable attenuator, then the beam diameter and the divergence angle of the laser are changed through the beam expander, and finally the laser is uniformly irradiated on the surface of the tumor tissue through the reflector. The light reflected by the tumor tissue is amplified by a stereoscopic microscope and then collected by a CCD camera. The CCD camera transmits the collected speckle images to a blood flow detection analysis module for blood flow image reconstruction processing. Then, the blood vessel morphology analysis module extracts characteristic parameters of the blood flow image, wherein the extracted parameters comprise blood flow speed, blood vessel diameter, blood vessel density, the number of blood vessel branch nodes, blood vessel distortion degree and the like. And finally, the artificial intelligence analysis module synthesizes the characteristic parameters to carry out comprehensive decision, judges whether the tumor blood vessels are normalized or not and provides a prediction result of the normalization window period of the blood vessels. The display module is used for displaying and prompting the video image and the prediction result of the blood vessel normalization.
The invention discloses a tumor blood vessel normalization detection method, which comprises the following specific steps as shown in figure 2:
(1) collecting speckle images of the tumor, and reconstructing a blood flow image of the tumor from the speckle images;
(2) detecting tumor vascular structures from the tumor blood flow images based on image morphological processing;
(3) extracting blood vessel density, blood vessel diameter and blood vessel branch nodes;
(4) calculating the distortion degree of the blood vessels, comparing the distortion degree with the preset normal tissue blood vessels with the same diameter, and grading the distortion degree of the tumor blood vessels;
(5) and (3) integrating the tumor blood flow velocity, the blood vessel density and the distortion degree through an artificial intelligence analysis module, judging whether the tumor blood vessels are normalized or not, and giving a voice prompt of a tumor blood vessel normalization window period prediction result.
The specific method for acquiring the blood flow image in the step (1) comprises the following steps:
with Ns*NsThe window filter processes the original speckle image to obtain a normalized variance image Vns(x, y), the processing method is shown as formula (I):
Figure BDA0002588660600000071
wherein N issThe size of the window filter is generally 5-11, Is(i, j) is a window filterThe gray value of the pixel in the wave filter, i and j are respectively the horizontal and vertical coordinates of the pixel point in the window filter,
Figure BDA0002588660600000082
the mean value of the pixel gray levels in the window filter is used, and x and y are the horizontal and vertical coordinates of the newly generated pixel points of the normalized variance image.
A two-dimensional blood flow image of the tumor is calculated using formula (II), where v (x, y) is the blood flow image.
Figure BDA0002588660600000081
The specific method for detecting the vascular structure in the step (2) comprises the following steps:
(a) respectively filtering the blood flow image by using 12 two-dimensional Gabor filters in different directions to obtain a filtered blood flow image Bλ,,θ(x, y), the treatment method is shown in formula (III):
Bλ,,θ(x,y)=∫∫f(u,v)gλ,,θ(x-u,y-v)dudv (Ⅲ)
wherein f (u, v) is a blood flow image, gλ,,θ(x-u, y-v) is a two-dimensional Gabor filter, the parameter λ is the reciprocal of the spatial frequency, the parameter is the variance characteristic of the Gabor filter, and the value range of λ adopted in this embodiment is 0.1 to 10, and the value interval is 0.1, which is λ/2. The parameter θ is the filtering direction of the Gabor filter, and blood vessel detection in different directions can be achieved by changing the parameter θ, in this embodiment, the value of θ is N pi/12, where N is 0,1, …, and 12.
(b) Extraction of Bλ,,θMaximum response values max (B) in 12 different directions for each pixel in (x, y)λ,,θ(x, y)) are combined into a new image, i.e. the image of the vessel structure H (x, y).
The specific method for extracting the blood vessel density in the step (3) comprises the following steps:
(a) carrying out binarization threshold operation on the blood vessel structure image H (x, y), resetting a pixel value with a pixel gray value larger than 0 in the blood vessel structure image to be 1, and resetting a pixel gray value smaller than or equal to 0 to be 0; a new blood vessel structure image H' (x, y) is obtained again;
(b) and counting all pixels with the pixel value of 1 in H' (x, y), and dividing by the total pixels in the blood vessel structure image to obtain the blood vessel density.
The method for calculating the number of the blood vessel branch nodes in the step (3) comprises the following steps:
(a) extracting a new blood vessel central skeleton image of the blood vessel structure image H' (x, y) through a conventional window filter function such as a two-dimensional Gabor filter or a Gaussian filter;
(b) analyzing the communication relation between each pixel point and the neighborhood pixel point on the blood vessel central skeleton image, and when the communication number between a certain blood vessel central skeleton pixel point and the neighborhood blood vessel central skeleton pixel point is more than 3, marking the certain blood vessel central skeleton pixel point as a blood vessel branch node.
(c) Traversing the whole blood vessel central skeleton image, obtaining all blood vessel branch nodes in the image, and calculating the total number of the blood vessel branch nodes.
The method for calculating the blood vessel distortion degree in the step (4) comprises the following steps:
(a) dividing the blood vessel skeleton image into a plurality of sections according to branch nodes; calculating the curvature of each point of each section of the blood vessel skeleton by using a formula (IV);
Figure BDA0002588660600000091
(b) and taking an absolute value of the curvature of each point of a certain section of the blood vessel skeleton, and then carrying out superposition averaging to obtain the distortion degree of the section of the blood vessel.
The method for determining whether tumor blood vessels are normalized and the prediction result of the window period of normalization of blood vessels in step (5) is shown in FIG. 3:
(a) the system provided by the invention is used for collecting 1000 tumor speckle images and extracting blood vessel morphological and functional information such as blood flow velocity, blood vessel density, blood vessel diameter, blood vessel distortion degree and the like.
(b) Animal tumor tissue samples are collected, and whether tumor blood vessels are normalized or not is judged by an immunofluorescence or immunohistochemistry method.
(c) Establishing a corresponding relation database of blood flow velocity, blood vessel density, blood vessel diameter, blood vessel distortion degree and tumor blood vessel normalization, wherein the training data are blood flow velocity, blood vessel density, blood vessel diameter and blood vessel distortion degree, and the training label is tumor blood vessel normalization or tumor blood vessel non-normalization.
(d) The database is used for training a deep neural network model, and the deep neural network model can be a conventional deep neural network model such as VGG16, GooleNet, ResNet and the like.
(e) And (4) predicting a new tumor blood flow image by using the training result, and giving a prompt for whether tumor blood vessels are normalized.
Example 2 in Lung cancer transplantable tumor model, the vascular normalization detection device directs Melafolone to enhance the therapeutic effect of anti-PD-1 therapy
(ii) animal grouping and handling
Mouse lung cancer cells LLC and CMT167(5 × 10)6) The C57BL/6 mice were inoculated subcutaneously, respectively. Tumor-bearing mice were randomly divided into a blank control group, a PD-1 monoclonal antibody group, a Melafolone (Mel, which induces normalization of tumor blood vessels), and a PD-1 monoclonal antibody and Mel combination group.
(II) test and detection
1. Recording the survival rate: mice were observed daily for survival and survival curves were generated.
2. Determination of tumor weight: and (3) killing the animals 24h after the last administration, completely stripping the tumor, observing the color and texture of the tumor tissue target body, weighing the tumor weight by using an electronic balance, and calculating the tumor inhibition rate.
3. Monitoring by a tumor blood vessel normalization detection system: the mice were anesthetized with 0.6% sodium pentobarbital and placed on a 37 ℃ hot plate and the skin opened for detection. The method mainly comprises the following steps:
opening a software acquisition interface of an industrial control computer;
adjusting the stereoscopic microscope until the surface morphology of the tumor can be clearly observed;
thirdly, turning on the laser to uniformly illuminate the tumor tissue;
starting a blood flow analysis mode in a software interface to acquire the blood vessel morphological information and the blood flow information of the tumor;
clicking a 'blood vessel normalization' button, and analyzing and acquiring information whether the tumor blood vessels are normalized or not through an artificial intelligence algorithm.
4. Immunofluorescence assay of pericytes and endothelial cells in tumors: tumor tissues are taken to prepare pathological sections, and the pathological sections are stained by an anti-CD 31 antibody and an anti-alpha-SMA antibody, and then are stained by a fluorescent secondary antibody and observed under a fluorescent microscope.
(III) results of the experiment
1. As shown in fig. 4, in a, compared with the blank group, the single use of PD-1 inhibitor has no significant effect on the survival rate of LLC-bearing mice, and the single use of Mel can significantly improve the survival rate of LLC-bearing mice; compared with the PD-1 inhibitor group, the combination of the PD-1 inhibitor and Mel can significantly increase the survival rate of LLC-bearing mice.
2. As shown in fig. 4, B, the use of PD-1 inhibitor alone and Mel alone significantly improved survival in CMT 167-bearing mice compared to the blank group; compared with the PD-1 inhibitor group, the combination of the PD-1 inhibitor and Mel can significantly increase the survival rate of LLC-bearing mice.
2. As in fig. 5, in a, the PD-1 inhibitor and Mel alone had no significant effect on tumor mass in LLC-bearing mice compared to the blank control group; the combination of PD-1 inhibitor and Mel significantly reduced tumor weight in LLC-bearing mice compared to the PD-1 inhibitor group.
As shown in fig. 5, B, the use of PD-1 inhibitor alone significantly reduced tumor weight in CMT 167-loaded mice compared to the blank control group; the combination of PD-1 inhibitor and Mel reduced tumor weight significantly more in LLC-bearing mice than in the PD-1 inhibitor group.
3. As shown in FIG. 6, the combination of Mel and PD-1 inhibitor significantly reduced CD31 in tumor tissues of LLC-and CMT 167-bearing mice, as compared to the group with PD-1 inhibitor alone+Blood vessel number, and can significantly increase CD31+α-SMA+The number of vessels.
(IV) conclusion
The above experimental results show that the use of the blood vessel normalization detection system can enhance the therapeutic effect of the anti-PD-1 inhibitor.
It will be understood that modifications and variations can be made by persons skilled in the art in light of the above teachings and all such modifications and variations are intended to be included within the scope of the invention as defined in the appended claims.

Claims (10)

1. A tumor-normalization detection system, comprising:
a laser light source module comprising: a laser for emitting laser light to illuminate the biological tissue; the variable attenuator is used for adjusting the light intensity of the laser; the beam expander is used for changing the beam diameter and the divergence angle of the laser; the reflector is used for reflecting the adjusted laser to the biological tissue to be detected;
an imaging module, comprising: the stereoscopic microscope is used for amplifying the biological tissue image irradiated by the laser; the camera is connected with the stereoscopic microscope and used for receiving the amplified image signal;
the image processor is connected with the imaging module and used for carrying out image processing on the image acquired by the imaging module, and the image processing comprises the following steps: demosaicing, denoising, defogging and color enhancement;
the blood flow detection and analysis module is connected with the imaging module and used for acquiring a blood flow velocity image, and the blood flow detection and analysis module processes the image and processes the speckle image into the blood flow velocity image through an algorithm;
the blood vessel morphology analysis module is connected with the blood flow detection analysis module and is used for extracting the tumor blood vessel morphology structure and measuring the tumor blood vessel density and the blood vessel distortion degree;
the artificial intelligence analysis module is connected with the blood vessel morphology analysis module and used for integrating the blood flow speed, the blood vessel density and the distortion degree of the tumor, judging whether the tumor blood vessel is normalized or not and giving a prediction result of the normalization window period of the tumor blood vessel;
and the display module is connected with the output ends of the image processor and the artificial intelligence analysis module and is used for displaying images and prompting the judgment result of whether the tumor blood vessels are normalized.
2. The system for detecting normalization of tumor blood vessels according to claim 1, wherein the laser has a wavelength ranging from 600nm to 1300 nm.
3. The system for detecting the normalization of tumor blood vessels according to claim 1, wherein the hardware of the blood flow detection and analysis module is configured as a Field Programmable Gate Array (FPGA), a proprietary Asic chip or a high-performance display card; the hardware of the artificial intelligence analysis module is formed into a proprietary Asic chip or a high-performance display card.
4. The system for normalizing tumor blood vessels of claim 1, wherein the display module comprises a display and a voice prompt.
5. A method for detecting normalization of tumor blood vessels using the system for detecting normalization of tumor blood vessels according to claim 1, comprising the steps of:
step 1: irradiating the biological tissue to be detected by the adjusted laser, collecting a tumor speckle image of the irradiated biological tissue, and reconstructing a tumor blood flow image from the speckle image;
step 2: detecting tumor vascular structures from the tumor blood flow images based on image morphological processing;
and step 3: extracting blood vessel density, blood vessel diameter and blood vessel branch nodes;
and 4, step 4: calculating the distortion degree of the blood vessels, comparing the distortion degree with the preset normal tissue blood vessels with the same diameter, and grading the distortion degree of the tumor blood vessels;
and 5: the artificial intelligence analysis module is used for integrating the blood flow speed, the blood vessel density and the distortion degree of the tumor, judging whether the tumor blood vessel is normalized or not, and giving a voice prompt of a tumor blood vessel normalization window period prediction result.
6. The method for detecting normalization of tumor blood vessels according to claim 5, wherein the step 1 of the method for obtaining the blood flow image of tumor comprises the following steps:
with Ns*NsThe window filter processes the original speckle image to obtain a normalized variance image Vns(x, y), the processing formula is:
Figure FDA0002588660590000021
wherein N issThe size of the window filter is 5-11, Is(I, j) is the gray value of the pixel in the window filter, I and j are respectively the horizontal and vertical coordinates of the pixel in the window filter, IsThe mean value of the pixel gray level in the window filter is used, and x and y are the horizontal and vertical coordinates of the newly generated pixel points of the normalized variance image;
calculating a two-dimensional blood flow image of the tumor, wherein the formula is as follows:
Figure FDA0002588660590000022
wherein, v (x, y) is the blood flow image.
7. The method for detecting the normalization of tumor blood vessels according to claim 5, wherein the step 2 of the method for detecting the tumor blood vessel structure comprises the following steps:
(a) respectively filtering the blood flow image by using 12 two-dimensional Gabor filters in different directions to obtain a filtered blood flow image Bλ,,θ(x, y), the processing formula is:
Bλ,,θ(x,y)=∫∫f(u,v)gλ,,θ(x-u,y-v)dudv
wherein f (u, v) is a blood flow image, gλ,,θ(x-u, y-v) is a two-dimensional Gabor filter, the parameter lambda is the reciprocal of the spatial frequency, the parameter is the variance characteristic of the Gabor filter, the value range of the lambda is 0.1-10, the value interval is 0.1, and is lambda/2; the parameter theta is the filtering direction of the Gabor filter, the blood vessel detection in different directions is realized by changing the parameter theta, and the value of theta is N pi/12, wherein N is 0,1, … and 12;
(b) extraction of Bλ,,θMaximum response values max (B) in 12 different directions for each pixel in (x, y)λ,,θ(x, y)) are combined into a new image, i.e. the image of the vessel structure H (x, y).
8. The method for detecting the normalization of tumor blood vessels according to claim 5, wherein the specific method in step 3 of the method comprises:
the specific method for extracting the blood vessel density comprises the following steps:
(a) carrying out binarization threshold operation on the blood vessel structure image H (x, y), resetting a pixel value with a pixel gray value larger than 0 in the blood vessel structure image to be 1, and resetting a pixel gray value smaller than or equal to 0 to be 0; a new blood vessel structure image H' (x, y) is obtained again;
(b) counting all pixels with pixel value 1 in H' (x, y), and dividing by the total pixels in the blood vessel structure image to obtain blood vessel density;
the method for calculating the number of the blood vessel branch nodes comprises the following steps:
(a) extracting a new blood vessel central skeleton image of the blood vessel structure image H' (x, y) through a conventional window filter function such as a two-dimensional Gabor filter or a Gaussian filter;
(b) analyzing the communication relationship between each pixel point and a neighborhood pixel point on the blood vessel central skeleton image, and recording a certain blood vessel central skeleton pixel point as a blood vessel branch node when the communication number of the certain blood vessel central skeleton pixel point and the neighborhood blood vessel central skeleton pixel point is more than 3;
(c) traversing the whole blood vessel central skeleton image, obtaining all blood vessel branch nodes in the image, and calculating the total number of the blood vessel branch nodes.
9. The method for detecting normalization of tumor blood vessels according to claim 5, wherein the specific method for calculating the degree of blood vessel distortion in step 4 of the method is:
(a) dividing the blood vessel skeleton image into a plurality of sections according to branch nodes; calculating the curvature of each point of each section of the vascular skeleton by using the following formula;
Figure FDA0002588660590000031
wherein, K (x, y) represents curvature, and x, y represent horizontal and vertical coordinates of each point respectively;
(b) and taking an absolute value of the curvature of each point of a certain section of the blood vessel skeleton, and then carrying out superposition averaging to obtain the distortion degree of the section of the blood vessel.
10. The method for detecting normalization of tumor blood vessels according to claim 5, wherein the method for determining whether tumor blood vessels are normalized and the prediction result of the window period of normalization of blood vessels in step 5 comprises:
(a) collecting 1000 tumor speckle images by using a tumor blood vessel normalization detection system, and extracting blood vessel morphological function information of blood flow velocity, blood vessel density, blood vessel diameter and blood vessel distortion degree;
(b) collecting animal tumor tissue samples, and judging whether tumor blood vessels are normalized or not by an immunofluorescence or immunohistochemical method;
(c) establishing a corresponding relation database of blood flow velocity, blood vessel density, blood vessel diameter, blood vessel distortion degree and tumor blood vessel normalization, wherein the training data are blood flow velocity, blood vessel density, blood vessel diameter and blood vessel distortion degree, and the training label is tumor blood vessel normalization or tumor blood vessel non-normalization;
(d) training a deep neural network model by using the database, wherein the deep neural network model comprises VGG16, GooleNet and ResNet deep neural network models;
(e) and (4) predicting a new tumor blood flow image by using the training result, and giving a prompt for whether tumor blood vessels are normalized.
CN202010689075.0A 2020-07-16 2020-07-16 Tumor blood vessel normalization detection system and detection method Expired - Fee Related CN111870224B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010689075.0A CN111870224B (en) 2020-07-16 2020-07-16 Tumor blood vessel normalization detection system and detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010689075.0A CN111870224B (en) 2020-07-16 2020-07-16 Tumor blood vessel normalization detection system and detection method

Publications (2)

Publication Number Publication Date
CN111870224A true CN111870224A (en) 2020-11-03
CN111870224B CN111870224B (en) 2022-05-20

Family

ID=73155593

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010689075.0A Expired - Fee Related CN111870224B (en) 2020-07-16 2020-07-16 Tumor blood vessel normalization detection system and detection method

Country Status (1)

Country Link
CN (1) CN111870224B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113393425A (en) * 2021-05-19 2021-09-14 武汉大学 Microvessel distribution symmetry quantification method for gastric mucosa staining amplification imaging
CN114287880A (en) * 2021-12-08 2022-04-08 四川大学华西医院 Early stage tumor formation monitoring method for animal experiment tumor based on infrared image processing
CN117495951A (en) * 2023-12-29 2024-02-02 苏州国科康成医疗科技有限公司 Intracranial aneurysm positioning method, device, computer equipment and storage medium
WO2024091180A1 (en) * 2022-10-26 2024-05-02 National University Of Singapore Light sheet imaging apparatus and method

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101283910A (en) * 2008-06-05 2008-10-15 华北电力大学 Method for obtaining the coronary artery vasomotion information
CN101697871A (en) * 2009-11-16 2010-04-28 华中科技大学 Laser imaging method and device for automatically cutting artery blood vessel and vein blood vessel
CN101763641A (en) * 2009-12-29 2010-06-30 电子科技大学 Method for detecting contour of image target object by simulated vision mechanism
CN102429650A (en) * 2011-11-10 2012-05-02 华中科技大学 Laser speckle blood flow imaging contrast analytical method
CN102722882A (en) * 2012-04-12 2012-10-10 华北电力大学(保定) Elastic registration method of CAG image sequence
CN102842136A (en) * 2012-07-19 2012-12-26 湘潭大学 Optic disc projection location method synthesizing vascular distribution with video disc appearance characteristics
CN107411707A (en) * 2017-05-08 2017-12-01 武汉大学 A kind of tumor-microvessel imager and tumor-microvessel imaging method
CN108403082A (en) * 2018-01-24 2018-08-17 苏州中科先进技术研究院有限公司 A kind of imaging in biological tissues system and imaging method
CN108430306A (en) * 2015-10-09 2018-08-21 瓦索普蒂奇医疗公司 System and method for using laser speckle contrast Imaging fast to check vascular system and particle stream
CN109124615A (en) * 2018-09-06 2019-01-04 佛山科学技术学院 One kind can constituency high dynamic laser speckle blood current imaging device and method
CN109893102A (en) * 2019-01-15 2019-06-18 温州医科大学 A kind of parser of macular area choroidal capillaries density

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101283910A (en) * 2008-06-05 2008-10-15 华北电力大学 Method for obtaining the coronary artery vasomotion information
CN101697871A (en) * 2009-11-16 2010-04-28 华中科技大学 Laser imaging method and device for automatically cutting artery blood vessel and vein blood vessel
CN101763641A (en) * 2009-12-29 2010-06-30 电子科技大学 Method for detecting contour of image target object by simulated vision mechanism
CN102429650A (en) * 2011-11-10 2012-05-02 华中科技大学 Laser speckle blood flow imaging contrast analytical method
CN102722882A (en) * 2012-04-12 2012-10-10 华北电力大学(保定) Elastic registration method of CAG image sequence
CN102842136A (en) * 2012-07-19 2012-12-26 湘潭大学 Optic disc projection location method synthesizing vascular distribution with video disc appearance characteristics
CN108430306A (en) * 2015-10-09 2018-08-21 瓦索普蒂奇医疗公司 System and method for using laser speckle contrast Imaging fast to check vascular system and particle stream
CN107411707A (en) * 2017-05-08 2017-12-01 武汉大学 A kind of tumor-microvessel imager and tumor-microvessel imaging method
CN108403082A (en) * 2018-01-24 2018-08-17 苏州中科先进技术研究院有限公司 A kind of imaging in biological tissues system and imaging method
CN109124615A (en) * 2018-09-06 2019-01-04 佛山科学技术学院 One kind can constituency high dynamic laser speckle blood current imaging device and method
CN109893102A (en) * 2019-01-15 2019-06-18 温州医科大学 A kind of parser of macular area choroidal capillaries density

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
沈娜: "抗血管生成药物时间窗的研究进展", 《肿瘤预防与治疗》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113393425A (en) * 2021-05-19 2021-09-14 武汉大学 Microvessel distribution symmetry quantification method for gastric mucosa staining amplification imaging
CN113393425B (en) * 2021-05-19 2022-04-26 武汉大学 Microvessel distribution symmetry quantification method for gastric mucosa staining amplification imaging
CN114287880A (en) * 2021-12-08 2022-04-08 四川大学华西医院 Early stage tumor formation monitoring method for animal experiment tumor based on infrared image processing
WO2024091180A1 (en) * 2022-10-26 2024-05-02 National University Of Singapore Light sheet imaging apparatus and method
CN117495951A (en) * 2023-12-29 2024-02-02 苏州国科康成医疗科技有限公司 Intracranial aneurysm positioning method, device, computer equipment and storage medium
CN117495951B (en) * 2023-12-29 2024-03-29 苏州国科康成医疗科技有限公司 Intracranial aneurysm positioning method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN111870224B (en) 2022-05-20

Similar Documents

Publication Publication Date Title
CN111870224B (en) Tumor blood vessel normalization detection system and detection method
Li et al. Deep learning based early stage diabetic retinopathy detection using optical coherence tomography
Jitpakdee et al. A survey on hemorrhage detection in diabetic retinopathy retinal images
CN111870230A (en) Multi-parameter tumor blood vessel normalization detection system and detection method
Reynaud et al. Automated quantification of optic nerve axons in primate glaucomatous and normal eyes—method and comparison to semi-automated manual quantification
CN111870231B (en) Endoscopic tumor blood vessel normalization detection system and detection method
TW201935403A (en) Prediction model for grouping hepatocellular carcinoma, prediction system thereof, and method for determining hepatocellular carcinoma group
CN103325128A (en) Method and device intelligently identifying characteristics of images collected by colposcope
EP3695378A1 (en) Tracking in ultrasound imaging
US20170140533A1 (en) System and method for automated detection and monitoring of dysplasia and administration of immunotherapy and chemotherapy
Li et al. From deep learning towards finding skin lesion biomarkers
CN113243887B (en) Intelligent diagnosis and treatment instrument for macular degeneration of old people
CN112274110B (en) Pore detection system, device and method based on skin fluorescence image
Soliman et al. Automatic breast cancer detection using digital thermal images
Li et al. An Improved Approach for Accurate and Efficient Measurement of Common Carotid Artery Intima‐Media Thickness in Ultrasound Images
Kanemura et al. Assessment of skin inflammation using near-infrared Raman spectroscopy combined with artificial intelligence analysis in an animal model
CN109003659A (en) Stomach Helicobacter pylori infects pathological diagnosis and supports system and method
CA3009813A1 (en) System and method for automated detection and monitoring of dysplasia and administration of chemoprevention
Fraz et al. Retinal vasculature segmentation by morphological curvature, reconstruction and adapted hysteresis thresholding
Kalaiarasan et al. Deep Learning-based Transfer Learning for Classification of Skin Cancer
Song et al. Application of convolutional neural network in signal classification for in vivo photoacoustic flow cytometry
US10062164B2 (en) Method for the analysis of image data representing a three-dimensional volume of biological tissue
CN116678835A (en) Method for rapidly screening COVID-19 virus for detecting blood glucose concentration based on hyperspectral imaging technology
Vijayalakshmi et al. Development of prognosis tool for type-II diabetics using tongue image analysis
Iroshan et al. Detection of diabetes by macrovascular tortuosity of superior bulbar conjunctiva

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20220520