CN109697425A - A kind of highway threat information recognition methods - Google Patents
A kind of highway threat information recognition methods Download PDFInfo
- Publication number
- CN109697425A CN109697425A CN201811568503.3A CN201811568503A CN109697425A CN 109697425 A CN109697425 A CN 109697425A CN 201811568503 A CN201811568503 A CN 201811568503A CN 109697425 A CN109697425 A CN 109697425A
- Authority
- CN
- China
- Prior art keywords
- sample
- threat
- error
- formula
- tally
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of highway threat information recognition methods, the steps include: 1) to select representative region to establish sample set and tally set according to photo or aerophotograph building ground object sample collection；2) sample data is standardized, then integrates red, green, blue wave band standardization pixel value collection, sample set is divided into training set and test set, constructs threat identification model, calculates error rate；3) picture of information to be extracted or aerophotograph are digitized, and are standardized, then bring standardization result into threat identification model, realize that threat information is extracted.The present invention, which is allowed for, identifies problem using threat information of the threat identification model to photochrome or aerophotograph, its numerical results shows, the threat information of offer is enough assessment Disaster degree, determines that rescue work preparation and formulation emergency plan HSE etc. provide guidance, to the advantageous supplement in the case of the data deficiencies such as remote sensing image, the efficiency that can be improved threat identification plays a significant role in emergency management and rescue decision Rapid Implementation.
Description
Technical field
The invention belongs to threaten source identification and contingency management field, it is related to using high-resolution remote sensing image, aviation phase
The information such as piece or high-resolution picture identify that a kind of highway threat information identification in place occur in landslide scene or barrier lake
Method.
Background technique
Up to the present, it is carried out both at home and abroad in the remote Sensing Interpretation of the threat identifications such as landslide mainly by visual interpretation, this needs
Interpretation mark is established according to contents such as the geometrical characteristic of image, textural characteristics and context relations, and by veteran profession
Personnel carry out interpretation, and still without being formed completely and the knowledge base of the threat identification of system, and the foundation for interpreting mark has area
Domain characteristic, generalization are limited；How independent of professional interpretation personnel, the migration of interpretation knowledge is realized, without each area
Interpretation re-establish interpretation mark, and realize the threat based on the amateur image data such as universal color photo quickly identify
It is a problem to be solved, no matter the solving of this problem to the method for threat identification or instructs the progress of emergency management and rescue all to have
Extremely important application value.
Summary of the invention
The object of the present invention is to provide it is a kind of can overcome drawbacks described above, using sampling camera photochrome or aerophotograph can know
Not Hua Po, the highway threat information recognition methods that threatens of the highways such as barrier lake.Its technical solution is using following steps:
1) training, test set are established:
(1) it takes pictures to highway, atural object sample point is acquired on the photo or aerophotograph taken, construct ground object sample collection:
Wherein, in formula (1) n be each subset in air route sum, S indicates total sample set, S1、S2、Si、SnRespectively
For the 1st, 2, i and n sample set；
(2) it selects representative region to establish sample set: selecting typical region for each sample set, and in representative region
Sample point acquisition is carried out, mathematical description is as follows:
Si={ xj|xj∈(z1, z2... zm), xj={ xjr, xjg, xjb} (2)
xjIndicate sample set SiJ-th of sampled point, m is the typical sampling area number selected, z1、z2、zmRespectively
1st, 2 and m typical sampling area, xjr、xjg、xjbRespectively indicate the picture of the corresponding red, green, blue wave band acquisition of j-th of sampled point
Element value；
(3) the tally set LB assignment of sample set S, mathematical description are as follows:
Wherein, LB is the corresponding tally set of sample set, LB1、LB2、LBnRespectively sample set S1, sample set S2, sample
Subset SnThe 1st, 2 and n sub-set of tags；
2) typical feature information extraction, comprising the following steps:
(1) standardization of sample set S data:
Zx in formula (4), (5) and (6)jr、zxjg、zxjbThe picture of red, green, blue wave band acquisition after respectively indicating standardization
Minimum value and max function are sought in element value, min () and max () expression；
(2) synthesis of red, green, blue wave band standardization pixel value collection and its tally set:
Sj=[zxjr, zxjg, zxjb] (7)
LB in formula (8)jr, LBjgAnd LBjbRespectively indicate zxjr、zxjgAnd zxjbCorresponding label value, bracket [] table
Show matrix, the comma in matrix is for segmentation between the column and the column；
(3) sample set S is divided for training set StrWith test set Ste, tally set LB is divided for training set LBtrAnd test set
LBte:
NStr≥NSTe,NLBtr≥NLBte, NStr+NSte=n, NLBtr+NLBte=n (11)
NS in formula (9)~(11)tr、NSteRespectively indicate the sample number of training set, the sample number of test set, NLBtr、
NLBteRespectively indicate the tally set sample number of training set and the tally set sample number of test set；
(4) threat identification model is constructed:
LModel={ Weight, error, ThrV } (12)
Weight=log10(1-error)/error) (13)
ThrV=min (sum (LBtr-LBtrf)) (14)
In formula (12), (13) and (14), Weight be forecast sample weight, error be forecast sample error and,
ThrV is the threshold value of label assignment, and sum () is summing function, LBtrfFor the training sample label value of prediction；
Classifier number range 1~20, according to the threat identification model of building, to step since first classifier
(1)~(3) training set, test set and its tally set obtained is trained, and the classification of training set is calculated according to formula (15)
Device error rate, until classifier error rate is not more than ThrV, deconditioning:
Etri=(NStr-Rtr)/NStr (15)
E in formula (15)triFor the error in classification rate that i-th of classifier of training set obtains, RtrFor the correct number of classification；
(5) test data set and its tally set are substituted into threat identification model, and computation model error rate:
Etei=(NSte-Rte)/NSte (16)
E in formula (16)teiFor the error in classification rate that i-th of classifier of test set obtains, RteFor the correct number of classification；
3) highway threat information identifies:
(1) the picture aerophotograph of information to be extracted is digitized；
(2) the picture aerophotograph after digitlization is standardized；
(3) standardization result of photo or aerophotograph is substituted into threat identification model formation (12)~(14), realizes and threatens letter
Breath identification.
Compared with prior art, the present invention advantage are as follows:
A. this method has the ability of the highways threat information identification such as processing landslide and barrier lake, for similar scene occurs
Emergency relief provides accident treatment template and guidance；Meanwhile by different photos or the pixel data of the red, green, blue color of aerophotograph
The threat identification of standardization building can be improved the recognition efficiency of threat, provide one kind effectively to solve the highway threat identification
Method；
B. position and the range of threat information can be quickly identified by aerophotograph or photochrome, this method is to remote sensing shadow
As the advantageous supplement in the case of data deficiencies, the efficiency of threat identification can be improved, have in emergency management and rescue decision Rapid Implementation
It plays an important role.
Specific embodiment
Technical solution of the present invention is described further below with reference to embodiment.Specific steps are as follows:
1) training, test set are established:
(1) photochrome comprising road is selected, constructs ground object sample collection S, sample set S includes road sample set
S1With non-rice habitats sample set S2, and
It (2) is sample set S1, sample set S2Respectively 5 typical region z of selection1~z5, and in sample set S1, sample
Subset S2Representative region in respectively acquire 100 sample points, xjr、xjg、xjbRespectively indicate j-th of sampled point it is corresponding it is red, green,
The pixel value of blue wave band acquisition:
x1r=[134 114 147 111 112 108 104 122 112 134 132 114 117 115 115 129
112 138 113 120 142 151 124 130 130 130 121 123 121 125 122 119 161 147 126
123 131 140 153 113 127 121 129 122 133 125 130 131 126 119 108 152 136 118
129 127 129 130 138 152 120 151 147 150 150 141 145 148 147 145 154 154 165
158 162 163 162 159 165 153 147 148 149 136 149 147 138 106 126 144 88 93 100
102 94 96 97 81 98 104 99 89 100 92 122 95 98 95 84 87 104 95 91 99 88 103 96
80 89 97 94 104 85 88 104 95 91 99 88 103 95 81 88 98 94 104 86 88]
x1g=[96 76 77 69 72 76 69 81 77 90 85 70 71 74 81 88 73 88 68 87 80
94 84 89 89 82 81 89 77 94 81 79 100 84 86 85 86 85 86 69 75 82 80 78 84 80
77 81 93 77 69 83 92 77 88 75 88 82 75 8681 81 126 131 129 119 125 126 130
127 134 133 148 143 141 142 137 137 146 131 125 131 129 117 126 126 117 92
108 116 78 81 90 86 82 86 87 70 84 90 89 78 86 82 103 83 82 84 84 79 92 89 81
85 77 91 86 74 77 82 84 92 79 76 92 89 81 85 77 91 87 74 78 82 84 93 79 76]
x1b=[87 73 67 70 64 77 67 75 71 81 77 67 73 72 79 84 68 81 65 78 69
87 82 85 85 82 79 80 74 92 75 77 81 79 84 84 83 78 78 69 77 75 83 75 80 77 69
80 86 78 64 76 89 73 82 77 84 82 66 70 76 71 107 114 108 106 100 103 104 103
109 112 122 122 114 115 115 114 131 108 102 105 105 103 110 109 100 92 104
105 77 83 89 86 84 87 86 76 84 89 90 86 86 81 99 83 79 82 85 77 92 89 82 85
83 95 87 74 79 79 83 92 79 78 92 89 82 85 83 95 88 75 79 79 84 92 77 78]
x2r=[85 83 80 78 79 56 49 76 56 51 73 65 67 67 61 48 65 57 66 54 65
23 67 66 59 45 60 49 37 57 57 63 69 58 53 65 50 62 43 62 75 73 66 64 64 64 75
52 59 55 60 56 41 49 72 45 70 65 58 53 43 30]
x2g=[72 68 68 66 68 52 44 68 57 50 70 61 59 57 58 43 57 51 58 47 62
23 57 59 56 39 59 43 36 51 47 64 63 53 49 64 49 56 41 59 64 63 58 64 64 66 61
48 55 46 55 50 40 45 61 42 71 62 51 52 41 30]
x2b=[66 63 68 66 66 51 38 66 49 45 63 58 56 56 53 39 54 51 56 39 57
23 55 53 49 39 54 45 32 51 46 59 63 49 48 60 45 56 44 54 60 61 56 62 62 65 60
45 52 39 52 50 35 42 59 37 65 57 52 48 44 28]
(3) to sample set S1Tally set LB1, sample set S2Tally set LB2Assignment, mathematical description are as follows:
LB1=1, LB2=2 (3)
2) typical feature information identifies, the specific steps are as follows:
(1) standardization of sample set S data:
Zx in formula (4), (5) and (6)jr、zxjg、zxjbThe picture of red, green, blue wave band acquisition after respectively indicating standardization
Minimum value and max function are sought in element value, min () and max () expression；
(2) synthesis of red, green, blue wave band standardization pixel value collection and its tally set:
Sj=[zxjr, zxjg, zxjb] (7)
LB in formula (8)jr、LBjgAnd LBjbRespectively indicate zxjr、zxjgAnd zxjbCorresponding label value, bracket [] table
Show matrix, the comma in matrix is for segmentation between the column and the column；
(3) sample set S is divided for training set StrWith test set Ste, tally set LB is divided for training set LBtrAnd test set
LBte, n value is 200, NStr、NSteRespectively indicate the sample number of training set, the sample number of test set, NLBtr、NLBteIt respectively indicates
The tally set sample number of training set and the tally set sample number of test set, are 100:
(4) threat identification model is constructed, classifier number 20 is set, training set, the test set that step (1)~(3) are obtained
And its tally set substitutes into threat identification model, and calculates 20 training set classifier error rates:
LModel={ Weight, error, ThrV } (12)
Weight=log10(1-error)/error) (13)
ThrV=min (sum (LBtr-LBtrf)) (14)
Etri=(100-Rtr)/100 (15)
(5) experiment scene of embodiment is 4: first experiment scene is the high-resolution remote sensing image of residential quarters,
Second environment is the landslide image picture downloaded on network, and third environment is similarly the high-resolution weir downloaded on network
Lake picture is filled in, the scene of the 4th rescue is the mud-rock flow photo downloaded on network.By the test of 4 experiment scene photochromes
Data set and its tally set substitute into threat identification model respectively, calculate its error rate:
Etei=(100-Rte)/100 (16)
3) highway threat information identifies:
(1) digitlization of the picture aerophotograph of information to be extracted is realized；
(2) digitized picture aerophotograph is standardized；
(3) standardization result of photo is substituted into threat identification model formation (12)~(14), realizes threat information identification.
The recognition result of embodiment is as shown in table 1:
1 threat information of table extracts result and evaluation
Table 1 shows that the scene of 4 experiments, first experiment scene are affected since pavement small in cell is more
The precision 93.75 of identification, second experiment scene are impended identification using the threat identification model of first experiment, when planning
Between also reduce nearly half, recognition correct rate 100%, it is also complete that by the model barrier lake region occurs for third experiment scene
It is extracted；The mud-rock flow occurrence scope of 4th experiment scene equally extracts, by being tested above it can be seen that logical
The identification threatened can be completed by crossing migration computation model, the threat identification time also much smaller than testing an experiment scene, it can thus be concluded that
This method provides safety guarantee for emergency episode rescue.
Claims (1)
1. a kind of highway threat information recognition methods, feature include following implemented step:
1) training, test set are established:
(1) it takes pictures to highway, atural object sample point is acquired on the photo or aerophotograph taken, construct ground object sample collection:
Wherein, in formula (1) n be each subset in air route sum, S indicates total sample set, S1、S2、Si、SnRespectively the 1st,
2, i and n sample set；
(2) it selects representative region to establish sample set: selecting typical region for each sample set, and carried out in representative region
Sample point acquisition, mathematical description are as follows:
Si={ xj|xj∈(z1, z2... zm), xj={ xjr, xjg, xjb} (2)
xjIndicate sample set SiJ-th of sampled point, m is the typical sampling area number selected, z1、z2、zmRespectively the 1st, 2
With m typical sampling area, xjr、xjg、xjbRespectively indicate the pixel value of the corresponding red, green, blue wave band acquisition of j-th of sampled point；
(3) the tally set LB assignment of sample set S, mathematical description are as follows:
Wherein, LB is the corresponding tally set of sample set, LB1、LB2、LBnRespectively sample set S1, sample set S2, sample set
SnThe 1st, 2 and n sub-set of tags；
2) typical feature information extraction, comprising the following steps:
(1) standardization of sample set S data:
Zx in formula (4), (5) and (6)jr、zxjg、zxjbThe pixel value of red, green, blue wave band acquisition after respectively indicating standardization,
Minimum value and max function are sought in min () and max () expression；
(2) synthesis of red, green, blue wave band standardization pixel value collection and its tally set:
Sj=[zxjr, zxjg, zxjb] (7)
LB in formula (8)jr, LBjgAnd LBjbRespectively indicate zxjr、zxjgAnd zxjbCorresponding label value, bracket [] indicate square
Gust, the comma in matrix is for segmentation between the column and the column；
(3) sample set S is divided for training set StrWith test set Ste, tally set LB is divided for training set LBtrWith test set LBte:
NStr≥NSte, NLBtr≥NLBte, NStr+NSte=n, NLBtr+NLBte=n (11)
NS in formula (9)~(11)tr、NSteRespectively indicate the sample number of training set, the sample number of test set, NLBtr、NLBtePoint
It Biao Shi not the tally set sample number of training set and the tally set sample number of test set；
(4) threat identification model is constructed:
LModel={ Weight, error, ThrV } (12)
Weight=log10(1-error)/error) (13)
ThrV=min (sum (LRtr-LBtrf) (14)
In formula (12), (13) and (14), Weight is the weight of forecast sample, and error is the error and ThrV of forecast sample
For the threshold value of label assignment, sum () is summing function, LBtrfFor the training sample label value of prediction；
Classifier number range 1~20, according to the threat identification model of building, since first classifier to step (1)~
(3) training set, test set and its tally set obtained is trained, and the classifier error of training set is calculated according to formula (15)
Rate, until classifier error rate is not more than ThrV, deconditioning:
Etri=(NStr-Rtr)/NStr (15)
E in formula (15)triFor the error in classification rate that i-th of classifier of training set obtains, RtrFor the correct number of classification；
(5) test data set and its tally set are substituted into threat identification model, and computation model error rate:
Etei=(NSte-Rte)/NSte (16)
E in formula (16)teiFor the error in classification rate that i-th of classifier of test set obtains, RteFor the correct number of classification；
3) highway threat information identifies:
(1) the picture aerophotograph of information to be extracted is digitized；
(2) the picture aerophotograph after digitlization is standardized；
(3) standardization result of photo or aerophotograph is substituted into threat identification model formation (12)~(14), realizes that threat information is known
Not.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811568503.3A CN109697425A (en) | 2018-12-21 | 2018-12-21 | A kind of highway threat information recognition methods |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811568503.3A CN109697425A (en) | 2018-12-21 | 2018-12-21 | A kind of highway threat information recognition methods |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109697425A true CN109697425A (en) | 2019-04-30 |
Family
ID=66231902
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811568503.3A Pending CN109697425A (en) | 2018-12-21 | 2018-12-21 | A kind of highway threat information recognition methods |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109697425A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750804A (en) * | 2011-04-22 | 2012-10-24 | 江西西菱轨道交通技术发展有限公司 | Early warning system and early warning method for railway landslide hazard |
| CN106250635A (en) * | 2016-08-02 | 2016-12-21 | 中国科学院水利部成都山地灾害与环境研究所 | The prevention and controls of a kind of ice-lake breach type mud-rock flow and application thereof |
| US20170184601A1 (en) * | 2006-07-14 | 2017-06-29 | The Regents Of The University Of California | Cancer biomarkers and methods of use thereof |
| CN107097810A (en) * | 2017-04-30 | 2017-08-29 | 中南大学 | A kind of Along Railway foreign body intrusion UAV Intelligent identification and method for early warning and system |
| CN107749145A (en) * | 2017-11-20 | 2018-03-02 | 重庆交通职业学院 | Railway periphery high gradient slope crag Rolling Stone warning device |
-
2018
- 2018-12-21 CN CN201811568503.3A patent/CN109697425A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170184601A1 (en) * | 2006-07-14 | 2017-06-29 | The Regents Of The University Of California | Cancer biomarkers and methods of use thereof |
| CN102750804A (en) * | 2011-04-22 | 2012-10-24 | 江西西菱轨道交通技术发展有限公司 | Early warning system and early warning method for railway landslide hazard |
| CN106250635A (en) * | 2016-08-02 | 2016-12-21 | 中国科学院水利部成都山地灾害与环境研究所 | The prevention and controls of a kind of ice-lake breach type mud-rock flow and application thereof |
| CN107097810A (en) * | 2017-04-30 | 2017-08-29 | 中南大学 | A kind of Along Railway foreign body intrusion UAV Intelligent identification and method for early warning and system |
| CN107749145A (en) * | 2017-11-20 | 2018-03-02 | 重庆交通职业学院 | Railway periphery high gradient slope crag Rolling Stone warning device |
Non-Patent Citations (2)
| Title |
|---|
| 苗荣凡: "一种基于相干运算和降噪自动编码机的3D掌纹验证方法", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 郭碧: "基于单目视觉的列车前方障碍物检测识别算法研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zeng et al. | A fast approach for large-scale Sky View Factor estimation using street view images | |
| CN112435207B (en) | Forest fire monitoring and early warning method based on sky-ground integration | |
| Li et al. | Quantifying the shade provision of street trees in urban landscape: A case study in Boston, USA, using Google Street View | |
| Li et al. | RSI-CB: A large scale remote sensing image classification benchmark via crowdsource data | |
| US8724900B2 (en) | Method and apparatus for direct detection, location, analysis, identification, and reporting of vegetation clearance violations technical field | |
| CN109117749A (en) | A kind of abnormal object monitoring and managing method and system based on unmanned plane inspection image | |
| Xia et al. | Sky view factor estimation from street view images based on semantic segmentation | |
| CN104239885B (en) | A kind of earthquake disaster damage degree appraisal procedure based on unmanned plane | |
| CN106600574B (en) | A kind of landslide extracting method based on remote sensing image and altitude data | |
| CN109858450A (en) | Ten meter level spatial resolution remote sensing image cities and towns extracting methods of one kind and system | |
| Sunarta et al. | Coastal Tourism: Impact For Built-Up Area Growth And Correlation To Vegetation And Water Indices Derived From Sentinel-2 Remote Sensing Imagery | |
| Sritarapipat et al. | Building classification in Yangon City, Myanmar using Stereo GeoEye images, Landsat image and night-time light data | |
| Nadoushan et al. | Predicting urban expansion in Arak Metropolitan Area using two land change models | |
| CN106960027B (en) | The UAV Video big data multidate association analysis method of spatial information auxiliary | |
| CN114332634A (en) | Method and device for determining position of electric power tower at risk, electronic equipment and storage medium | |
| CN109697425A (en) | A kind of highway threat information recognition methods | |
| Xiong et al. | Detecting and Mapping Individual Fruit Trees in Complex Natural Environments via UAV Remote Sensing and Optimized YOLOv5 | |
| CN112066998A (en) | Rendering method and system for airline map | |
| Kourtz | An application of Landsat digital technology to forest fire fuel type mapping | |
| Kinga | The spatio-temporal analysis of impervious surfaces in Cluj-Napoca, Romania | |
| CN114140703A (en) | Intelligent recognition method and system for forest pine wood nematode diseases | |
| EP3534338B1 (en) | Creation of a simulation scene from a specified view point | |
| Shen et al. | Individual tree location detection by high-resolution RGB satellite imagery in urban area | |
| Singh et al. | Green indexing of Hisar Municipal Corporation using geospatial techniques | |
| Munawar et al. | Application of Deep Learning on UAV-Based Aerial Images for Flood Detection. Smart Cities 2021, 4, 1220–1242 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |