CN110428896B - Occupational pneumoconiosis auxiliary screening intelligent model - Google Patents
Occupational pneumoconiosis auxiliary screening intelligent model Download PDFInfo
- Publication number
- CN110428896B CN110428896B CN201910325006.9A CN201910325006A CN110428896B CN 110428896 B CN110428896 B CN 110428896B CN 201910325006 A CN201910325006 A CN 201910325006A CN 110428896 B CN110428896 B CN 110428896B
- Authority
- CN
- China
- Prior art keywords
- model
- unit
- pneumoconiosis
- result
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Epidemiology (AREA)
- Pathology (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Primary Health Care (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the application relates to an occupational pneumoconiosis auxiliary screening intelligent model, which comprises a plurality of model units, wherein the model units comprise a residual part and a compression activation part, and the occupational pneumoconiosis auxiliary screening intelligent model comprises the following components: the residual error part comprises a preset number of parallel convolution network channels, a splicing unit and an adding unit; the compression activation part comprises an average pooling unit, a first full-connection layer unit, a linear processing unit, a second full-connection layer unit, a nonlinear processing unit and a result integration unit which are sequentially connected. The technical scheme provided by the application can automatically identify the chest radiography, thereby improving the identification efficiency and accuracy.
Description
Technical Field
The application relates to the technical field of data processing, in particular to an intelligent auxiliary screening model for occupational pneumoconiosis.
Background
The main way of pneumoconiosis examination screening at present is to read, grade and diagnose the chest film of a patient by a doctor comparing with the standard pneumoconiosis diagnosis chest film, but the workload of the doctor does not exceed 200 chest films at most every day; long-time film reading can lead to fatigue of doctors and reduced accuracy; moreover, many basic doctors have low level, do not diagnose pneumoconiosis qualification, and are easy to have the condition of missed diagnosis and misdiagnosis.
Disclosure of Invention
The application aims to provide an intelligent professional pneumoconiosis auxiliary screening model which can automatically identify chest films, thereby improving identification efficiency and accuracy.
To achieve the above object, the present application provides an occupational pneumoconiosis auxiliary screening intelligent model comprising a plurality of model units including a residual part and a compression activation part, wherein:
the residual error part comprises a preset number of parallel convolution network channels, a splicing unit and an adding unit, wherein after the image pictures are input into the preset number of parallel convolution network channels, the results of the convolution network channels are spliced together through the splicing unit, and the output result of the splicing unit and the image pictures are added through the adding unit to obtain residual error results;
the compression activating part comprises an average pooling unit, a first full-connection layer unit, a linear processing unit, a second full-connection layer unit, a nonlinear processing unit and a result integrating unit which are sequentially connected, wherein the average pooling unit is used for receiving the residual error result, the nonlinear processing unit is used for outputting the result of the compression activating part, and the result integrating unit is used for adding the result of the compression activating part and the residual error result to obtain the pneumoconiosis characteristic.
Further, the preset number is 32, and the convolution network channels have channel attributes, where the channel attributes include the number of channels of the input data, the convolution kernel size, and the number of channels of the output data.
Further, the number of channels of the input data and the number of channels of the residual result are 256.
Further, after the average pooling unit averages the width and height of the residual error result to 1, inputting the homogenized result into the first full connection layer, so as to scale the channel number of the residual error result according to a preset scaling parameter through the first full connection layer.
Further, the preset scaling parameter is 16, and accordingly, the number of channels after scaling is 16.
Further, the model further comprises a final full-connection layer and a softmax layer, wherein the final full-connection layer is used for receiving the pneumoconiosis characteristics output by the result integration unit, and the softmax layer is used for calculating the pneumoconiosis probability corresponding to the image picture according to the output result of the final full-connection layer.
Further, the model also comprises a data processing unit, wherein the data processing unit is used for carrying out histogram equalization processing on the input image picture and adjusting the size of the input image picture according to the system input requirement.
Further, the model further comprises a migration learning unit, wherein the migration learning unit is used for performing migration learning on the professional pneumoconiosis auxiliary screening intelligent model in advance so as to initialize the characteristics of the professional pneumoconiosis auxiliary screening intelligent model; wherein the data set of the transfer learning includes: imageNet or Chest X-ray 14.
Further, the professional pneumoconiosis auxiliary screening intelligent model is trained on a Chest X-ray 14 data set after an expected result is obtained through training of an ImageNet data set, so that good medical Chest characteristics are obtained.
Further, the model further comprises a data enhancement unit, wherein the data enhancement unit is used for carrying out data enhancement processing on the data input into the occupational pneumoconiosis auxiliary screening intelligent model in advance, and inputting the data subjected to the data enhancement processing and the label thereof into the occupational pneumoconiosis auxiliary screening intelligent model after transfer learning for training; and during training, optimizing convergence of the occupational pneumoconiosis auxiliary screening intelligent model by using an Adam optimization algorithm.
From the above, the application has advanced accuracy in the industry and is accepted by applying the deep convolutional neural network to the field of screening of pneumoconiosis patients. Through the intelligent screening model, the working pressure of doctors can be reduced, and the screening efficiency and quality are improved.
Drawings
FIG. 1 is a schematic diagram of a professional pneumoconiosis auxiliary screening intelligent model in an embodiment of the application;
fig. 2 is a thermodynamic diagram of a pneumoconiosis lesion in an embodiment of the application.
Detailed Description
In order to make the technical solution of the present application better understood by those skilled in the art, the technical solution of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments of the present application. All other embodiments, based on the embodiments of the application, which would be apparent to one of ordinary skill in the art without undue burden, are intended to be within the scope of the application.
Referring to fig. 1, the present application provides an occupational pneumoconiosis auxiliary screening intelligent model comprising a plurality of model units including a residual portion and a compression activation portion, wherein:
the residual error part comprises a preset number of parallel convolution network channels, a splicing unit and an adding unit, wherein after the image pictures are input into the preset number of parallel convolution network channels, the results of the convolution network channels are spliced together through the splicing unit, and the output result of the splicing unit and the image pictures are added through the adding unit to obtain residual error results;
the compression activating part comprises an average pooling unit, a first full-connection layer unit, a linear processing unit, a second full-connection layer unit, a nonlinear processing unit and a result integrating unit which are sequentially connected, wherein the average pooling unit is used for receiving the residual error result, the nonlinear processing unit is used for outputting the result of the compression activating part, and the result integrating unit is used for adding the result of the compression activating part and the residual error result to obtain the pneumoconiosis characteristic.
In one embodiment, the preset number is 32, and the convolution network channels are provided with channel attributes including the number of channels of the input data, the convolution kernel size, and the number of channels of the output data.
In one embodiment, the number of channels of the input data and the number of channels of the residual result are both 256.
In one embodiment, after the averaging and pooling unit averages the width and height dimensions of the residual result to 1, the averaged result is input to the first full-connection layer, so as to scale the number of channels of the residual result according to a preset scaling parameter through the first full-connection layer.
In one embodiment, the preset scaling parameter is 16, and accordingly, the number of channels after scaling is 16.
In one embodiment, the model further includes a final fully connected layer for receiving the characteristics of pneumoconiosis output by the result integrating unit and a softmax layer for calculating the probability of pneumoconiosis corresponding to the image picture according to the output result of the final fully connected layer.
In one embodiment, the model further includes a data processing unit, where the data processing unit is configured to perform histogram equalization processing on an input image picture, and adjust a size of the input image picture according to a system input requirement.
In one embodiment, the model further includes a migration learning unit, where the migration learning unit is configured to perform migration learning on the professional pneumoconiosis auxiliary screening intelligent model in advance, so as to initialize characteristics of the professional pneumoconiosis auxiliary screening intelligent model; wherein the data set of the transfer learning includes: imageNet or Chest X-ray 14.
In one embodiment, the professional pneumoconiosis auxiliary screening intelligent model is trained on the ImageNet data set to obtain expected results, and then trained on the Chest X-ray 14 data set to obtain good medical Chest characteristics.
In one embodiment, the model further comprises a data enhancement unit, wherein the data enhancement unit is used for performing data enhancement processing on the data input into the occupational pneumoconiosis auxiliary screening intelligent model in advance, and inputting the data subjected to the data enhancement processing and the label thereof into the occupational pneumoconiosis auxiliary screening intelligent model subjected to transfer learning for training; and during training, optimizing convergence of the occupational pneumoconiosis auxiliary screening intelligent model by using an Adam optimization algorithm.
In practical application, the technical scheme of the application can be realized in the following way:
CFNet model structure
1.1. Model unit
The model unit is the most basic component module of the CFNet model and consists of a residual part and a compression activation part.
1.1.1. Unit-residual part:
specifically, a 256 x 256 picture input, entering the model unit, will first enter 32 parallel convolutional network channels. After passing through the parallel channels, all channels are spliced together, and the obtained result is added with the input original image at the tail part of the unit and enters the subsequent links as output.
1.1.2. Unit-compression activation portion:
if the data input from the residual portion is X, the size is h×w×c (high×wide×channel number), one branch enters the compression activating portion. First Global Average Pooling (global average pooling) the size becomes 1 x c. Then, the full connection FC (Fully Connected) layer (1×1×c/r, r is a preset scaling parameter) is entered, the ReLU (Rectified Linear Unit, linear rectification function) is processed linearly, and then the full connection layer FC is connected, and the Sigmoid function is processed non-linearly. Finally, the residual error part results are added and combined, and the output size is still H, W and C.
CFNet model integral structure
The overall result of the model may be formed by a plurality of model units connected end to end, wherein the assignment of the parameters may be as follows:
the number of parallel channels of the residual part is 32, r=16, c=256.
2. Acquiring training data
Tens of thousands of training data and yin-yang labels of the data can be obtained from existing coal hospitals and collaborative occupational disease prevention and treatment hospitals.
3. Training data processing
Before the data enter the model training process, a series of image processing processes are carried out, and the sequence is as follows:
equalizing the histogram;
and adjusting the image size according to the model input requirement.
4. Model training
4.1 transfer learning
Before training with pneumoconiosis data, the model is subjected to transfer learning, and features of the model are initialized. The data set of the transfer learning is: imageNet, chest X-ray 14.CFNet is trained on the ImageNet data set to obtain a good result, and then is trained on the Chest X-ray 14 data set to obtain good medical Chest characteristics.
4.2CFNet training
Before entering the model, the data undergoes a data enhancement Data Augmentation process: affine transformation, horizontal flipping, rotation, scaling, translation.
And feeding the enhanced data and the labels thereof to the CFNet after transfer learning for training. During training, the model convergence is optimized by using an Adam optimization algorithm. After a certain training period, the performance and accuracy of the model are optimized, and then the pneumoconiosis screening model is obtained.
4.3CFNet working principle and application
4.3.1 focal thermodynamic diagrams
The last layer of the convolution layer of the model, we call the feature layer. After passing through the entire convolutional layer of CFNet, the model learns all features of pneumoconiosis and generalizes to 512 channels. Through the last full connection layer and softmax layer we obtained weights for 512 feature channels. Assuming fk is the kth characteristic channel, wk is the weight of the channel, and M is the focal thermodynamic diagram, then:
an example of lesion location as shown in fig. 2 may ultimately be obtained.
4.3.2 pneumoconiosis prediction
At the last layer of the model, the model calculates the probability (0% -100%) that the chest radiography is a pneumoconiosis patient by integrating the model characteristics and the calculated weights.
5. Model assembly
Model combining is a technique of combining a plurality of models. We used the input data 2048 x 2048 and 1024 x 1024 in two dimensions, model results CFNet-50 and CFNet-101, with 4 models trained. And carrying out simple voting on the prediction results generated by the 4 models to obtain a final prediction result.
6. Model features
The screening accuracy of pneumoconiosis is improved by 10%, and the effect is most obvious.
The last pooling layer in the model structure is pooled by global maximization and is modified into global average pooling, and the accuracy is improved by 2%.
The target loss function is optimized, the general loss function is mean square error, and the model uses the cross entropy error degree, so that the accuracy of the model is improved by 2%.
The model accuracy is further improved by 5% through the combination technology of the model.
From the above, the application has advanced accuracy in the industry and is accepted by applying the deep convolutional neural network to the field of screening of pneumoconiosis patients. Through the intelligent screening model, the working pressure of doctors can be reduced, and the screening efficiency and quality are improved.
The foregoing description of various embodiments of the application has been presented to those skilled in the art for the purposes of illustration. It is not intended to be exhaustive or to limit the application to the precise embodiments disclosed. As described above, various alternatives and variations of the present application will be apparent to those skilled in the art. Thus, while some alternative embodiments have been specifically discussed, other embodiments will be apparent or relatively readily available to those skilled in the art. The present application is intended to embrace all alternatives, modifications, and variations of the present application that have been discussed herein and other embodiments that fall within the spirit and scope of the above-described application.
Claims (6)
1. A pneumoconiosis screening model, comprising a plurality of model units, the model units comprising a residual part and a compression activation part, wherein:
the residual error part comprises a preset number of parallel convolution network channels, a splicing unit and an adding unit, wherein after the image pictures are input into the preset number of parallel convolution network channels, the results of the convolution network channels are spliced together through the splicing unit, and the output result of the splicing unit and the image pictures are added through the adding unit to obtain residual error results;
the compression activating part comprises an average pooling unit, a first full-connection layer unit, a linear processing unit, a second full-connection layer unit, a nonlinear processing unit and a result integrating unit which are sequentially connected, wherein the average pooling unit is used for receiving the residual error result, the nonlinear processing unit is used for outputting the result of the compression activating part, and the result integrating unit is used for adding the result of the compression activating part and the residual error result to obtain the pneumoconiosis characteristic;
the model also comprises a data processing unit, wherein the data processing unit is used for carrying out histogram equalization processing on the input image picture and adjusting the size of the input image picture according to the system input requirement;
the model also comprises a transfer learning unit, wherein the transfer learning unit is used for performing transfer learning on the pneumoconiosis screening model in advance so as to initialize the characteristics of the pneumoconiosis screening model; wherein the data set of the transfer learning includes: imageNet or Chest X-ray 14;
the pneumoconiosis screening model is trained on a Chest X-ray 14 data set after an expected result is obtained through training of an ImageNet data set, so that good medical Chest characteristics are obtained;
the model also comprises a data enhancement unit, wherein the data enhancement unit is used for carrying out data enhancement processing on the data input into the pneumoconiosis screening model in advance, and inputting the data subjected to the data enhancement processing and the label thereof into the pneumoconiosis screening model subjected to transfer learning for training; during training, optimizing convergence of the pneumoconiosis screening model by using an Adam optimization algorithm;
the last layer of the convolution layer of the model is a characteristic layer; after passing through the whole convolution layer of CFNet, the model learns all features of pneumoconiosis and generalizes to 512 channels; weights of 512 characteristic channels are obtained through the final full-connection layer and the softmax layer; assuming fk is the kth characteristic channel, wk is the weight of the channel, and M is the focal thermodynamic diagram, then:
finally, the focus position can be obtained;
at the last layer of the model, the model calculates the probability of chest radiography as pneumoconiosis patient by integrating the model characteristics and the calculated weight;
the input data are 2048 x 2048 and 1024 x 1024 under two sizes, the model results are CFNet-50 and CFNet-101, and the trained 4 models are used in combination; and carrying out simple voting on the prediction results generated by the 4 models to obtain a final prediction result.
2. The model of claim 1, wherein the predetermined number is 32 and the convolutional network channel has channel properties including a number of channels of input data, a convolutional kernel size, and a number of channels of output data.
3. The model of claim 2, wherein the number of channels of the input data and the number of channels of the residual result are each 256.
4. A model according to claim 3, characterized in that the averaging pooling unit, after averaging the width and height dimensions of the residual result to 1, inputs the averaged result into the first full connection layer to scale the number of channels of the residual result by the first full connection layer according to a preset scaling parameter.
5. The model of claim 4, wherein the predetermined scaling parameter is 16, and the number of channels scaled is 16 accordingly.
6. The model of claim 1, further comprising a final fully connected layer for receiving characteristics of pneumoconiosis output by the result integration unit and a softmax layer for calculating a pneumoconiosis disease probability corresponding to the image picture according to an output result of the final fully connected layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910325006.9A CN110428896B (en) | 2019-04-22 | 2019-04-22 | Occupational pneumoconiosis auxiliary screening intelligent model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910325006.9A CN110428896B (en) | 2019-04-22 | 2019-04-22 | Occupational pneumoconiosis auxiliary screening intelligent model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110428896A CN110428896A (en) | 2019-11-08 |
| CN110428896B true CN110428896B (en) | 2023-09-08 |
Family
ID=68408394
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910325006.9A Active CN110428896B (en) | 2019-04-22 | 2019-04-22 | Occupational pneumoconiosis auxiliary screening intelligent model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110428896B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113610785A (en) * | 2021-07-26 | 2021-11-05 | 安徽理工大学 | Pneumoconiosis early warning method and device based on intelligent image and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106971198A (en) * | 2017-03-03 | 2017-07-21 | 北京市计算中心 | A kind of pneumoconiosis grade decision method and system based on deep learning |
| CN108510489A (en) * | 2018-03-30 | 2018-09-07 | 四川元匠科技有限公司 | A kind of pneumoconiosis detection method and system based on deep learning |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10528846B2 (en) * | 2016-11-14 | 2020-01-07 | Samsung Electronics Co., Ltd. | Method and apparatus for analyzing facial image |
-
2019
- 2019-04-22 CN CN201910325006.9A patent/CN110428896B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106971198A (en) * | 2017-03-03 | 2017-07-21 | 北京市计算中心 | A kind of pneumoconiosis grade decision method and system based on deep learning |
| CN108510489A (en) * | 2018-03-30 | 2018-09-07 | 四川元匠科技有限公司 | A kind of pneumoconiosis detection method and system based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 基于特征图切分的轻量级卷积神经网络;张雨丰等;《模式识别与人工智能》;20190331;第32卷(第03期);第237-246页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110428896A (en) | 2019-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110428475B (en) | Medical image classification method, model training method and server | |
| He et al. | Fakd: Feature-affinity based knowledge distillation for efficient image super-resolution | |
| CN108510194B (en) | Wind control model training method, risk identification method, device, equipment and medium | |
| US8433121B2 (en) | Method for brightness level calculation in the area of interest of the digital X-ray image for medical applications | |
| CN109816742B (en) | Cone beam CT geometric artifact removing method based on fully-connected convolutional neural network | |
| CN110276749B (en) | Quality control artificial intelligence system and quality control method for children radiation shooting and diagnosis | |
| CN111932550A (en) | 3D ventricle nuclear magnetic resonance video segmentation system based on deep learning | |
| CN111179235B (en) | Image detection model generation method and device, and application method and device | |
| US20210090289A1 (en) | Methods and systems for providing guidance for adjusting an object based on similarity | |
| CN110428896B (en) | Occupational pneumoconiosis auxiliary screening intelligent model | |
| CN117115045A (en) | Method for improving medical image data quality based on Internet generation type artificial intelligence | |
| CN113539402A (en) | Multi-mode image automatic sketching model migration method | |
| CN116342516A (en) | Model integration-based method and system for assessing bone age of X-ray images of hand bones of children | |
| Wang et al. | RGA-CNNs: Convolutional neural networks based on reduced geometric algebra | |
| CN112258537A (en) | Method for monitoring dark vision image edge detection based on convolutional neural network | |
| KR102036052B1 (en) | Artificial intelligence-based apparatus that discriminates and converts medical image conformity of non-standardized skin image | |
| CN112767273A (en) | Low-dose CT image restoration method and system applying feature decoupling | |
| CN116091446A (en) | Method, system, medium and equipment for detecting abnormality of esophageal endoscope image | |
| CN112785559B (en) | Bone age prediction method based on deep learning and formed by mutually combining multiple heterogeneous models | |
| CN112233030B (en) | CT projection geometric correction method based on deep learning | |
| CN115272176A (en) | Fracture identification model and method based on hip joint DR image | |
| Faure et al. | A convolutional neural network for dental panoramic radiograph classification | |
| CN112613521A (en) | Multilevel data analysis system and method based on data conversion | |
| CN110647926A (en) | Medical image stream identification method and device, electronic equipment and storage medium | |
| CN111274953A (en) | Method and system for judging pain according to expressions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |