CN110969125B - Detection spectrum band optimization method for target identification task - Google Patents
Detection spectrum band optimization method for target identification task Download PDFInfo
- Publication number
- CN110969125B CN110969125B CN201911214745.7A CN201911214745A CN110969125B CN 110969125 B CN110969125 B CN 110969125B CN 201911214745 A CN201911214745 A CN 201911214745A CN 110969125 B CN110969125 B CN 110969125B
- Authority
- CN
- China
- Prior art keywords
- spectrum
- spectral
- target
- detection
- divergence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
- G06F2218/16—Classification; Matching by matching signal segments
Abstract
The invention discloses a detection spectrum optimization method for a target identification task, which comprises the following steps: the method comprises the following steps: primarily selecting a target detection spectrum set based on an apparent contrast model and a comprehensive signal-to-noise ratio model; step two: in the initially selected target detection spectrum set, calculating the relative difference of radiation intensity between different targets on each spectrum based on a spectrum relative distance model, and accumulating and sequencing the relative difference values of radiation of all targets on each spectrum; step three: and in the initially selected target detection spectrum set, calculating the sum of the spectral information divergence of each target under different spectrum combination conditions based on a spectral information divergence model, and taking the spectral information divergence and the maximum spectrum combination as a preferred spectrum set. The method realizes the optimization of the detection spectrum section through spectrum section initial selection and spectrum section optimization confirmation, is suitable for optimizing the detection spectrum section facing to the target identification task, can improve the detectability and the identification performance of the system to the target, and supports the improvement of the comprehensive efficiency of the system.
Description
Technical Field
The invention belongs to the technical field of target detection and identification, and relates to a detection spectrum band optimization method for a target identification task.
Background
The remote detection of the aircraft target needs to obtain all characteristics of the target, including position, speed, model and the like, in a short time so as to realize the rapid tracking and identification of the target. Due to the fact that the detection distance is long, the target is represented as a point target on the image, information such as the shape, the size and the texture of the target cannot be utilized, and the target signal is weak and is often submerged by complex background clutter and system noise. Other characteristics of the target need to be utilized to complete the identification of the target.
The spectral characteristics of different targets are different, the target model can be judged through spectral matching, and the spectrum selection is the key of target identification. The target has larger difference with the background and noise in certain spectral bands, and if the spectral bands are selected for detection, the detectability of the target can be improved. The spectral characteristics of different targets in different spectral bands are different, and the spectral band with the large spectral characteristic difference among different targets is selected to detect the target, so that the accuracy of subsequent target identification can be improved.
The existing target detection spectral band selection method is based on a contrast model and a signal-to-noise ratio model. The contrast model takes into account the effects of the target, background radiation characteristics, and atmospheric transmission characteristics on target detection. The signal-to-noise model also takes into account the influence of the imaging characteristics of the detection system. The existing detection spectrum selection method only aims at improving the target detection efficiency, and does not consider the reliability of target identification in the detection spectrum.
Disclosure of Invention
Aiming at the problem that the target identification is not carried out in the selected spectrum section in the traditional target detection spectrum section selection method, the invention provides a detection spectrum section optimization method facing a target identification task. The method optimizes the detection spectrum by maximizing the separability of the signals and the categories, realizes the spectrum optimization method meeting the requirements of the target identification task, can ensure the efficient detection of the target in the optimized spectrum, better distinguishes different targets, and greatly improves the accuracy of target identification.
The purpose of the invention is realized by the following technical scheme:
a detection spectrum segment optimization method for an object recognition task comprises the following steps:
the method comprises the following steps: primarily selecting a target detection spectrum set based on an apparent contrast model and a comprehensive signal-to-noise ratio model;
step two: in the initially selected target detection spectrum set, calculating the relative difference of radiation intensity between different targets on each spectrum based on a spectrum relative distance model, and accumulating and sequencing the relative difference values of radiation of all targets on each spectrum;
step three: and in the primarily selected target detection spectrum set, calculating the sum of the spectral information divergence of each target under the condition of different spectrum combinations based on a spectral information divergence model, and taking the spectral information divergence and the maximum spectrum combination as a preferred spectrum set.
Compared with the prior art, the invention has the following advantages:
(1) The invention optimizes the detection spectrum by utilizing the signal and category separability to the maximum extent, can optimize the detection spectrum meeting the requirement of a target identification task, realizes high-efficiency detection of the target, greatly improves the identifiability of the system to the target, and has important significance for improving the information processing performance of the system.
(2) Aiming at the limitation that the traditional spectrum selection method does not consider the subsequent target identification in the detection spectrum, the invention adopts the spectrum initial selection and the spectrum optimization confirmation process to complete the optimization of the detection spectrum, and compared with the traditional method, the optimized spectrum is beneficial to improving the accuracy of the subsequent target identification.
Drawings
FIG. 1 is a preferred flow chart of a detection spectrum segment for a target identification task according to the present invention;
FIG. 2 is a schematic diagram of the preferred identification of the spectral range in the present invention;
FIG. 3 is an infrared image of a part of a spectrum band in the example;
FIG. 4 shows the calculation results of the apparent contrast of all spectral images in the example;
FIG. 5 is a calculation result of the integrated SNR in the example;
FIG. 6 is an infrared image of each spectral band within the preferred target detection spectral band in an embodiment;
FIG. 7 shows spectral characteristics of three targets in a preferred set of target detection bands in an embodiment.
Detailed Description
The technical solution of the present invention is further described below with reference to the accompanying drawings, but not limited thereto, and any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention shall be covered by the protection scope of the present invention.
The invention provides a detection spectrum optimization method for a target identification task, which comprises the following specific implementation steps as shown in figure 1:
the method comprises the following steps: and primarily selecting a target detection spectrum set based on the apparent contrast model and the comprehensive signal-to-noise ratio model.
The premise of target identification is that a target can be detected, and in the target detection process, background clutter and system noise need to be suppressed as much as possible to reduce interference of non-target signals on target signals. The apparent contrast model has fewer consideration factors and simple calculation. The comprehensive signal-to-noise ratio model analyzes the influence of all factors on target detection from the angle of the whole imaging link, and the calculated amount is large. By combining the apparent contrast model and the comprehensive signal-to-noise ratio model, the calculation complexity can be reduced on the premise that the initially selected target detection spectrum set has higher reliability. The specific steps of the primary selection of the target detection spectrum section are as follows:
(1) And calculating the apparent contrast of the target radiation relative to the background radiation at the entrance pupil of the camera based on the target and background apparent contrast model, and preselecting a detection spectral range according to the calculation result of the apparent contrast.
The apparent contrast is defined as:
in the formula: c is the apparent contrast; i is t Is the target radiation intensity; tau. a Is the atmospheric transmittance between the object and the entrance pupil of the detection system; l is a radical of an alcohol B The atmospheric background radiation brightness at the entrance pupil of the detection system is obtained; a. The t Is the projected area of the target along the line of sight direction.
(2) And based on the comprehensive signal-to-noise ratio model, calculating comprehensive signal-to-noise ratios with different central wavelengths and different spectrum widths in a preselected spectrum range, and optimizing the preselected target detection spectrum range according to the calculation results of the comprehensive signal-to-noise ratios in the different spectrum ranges to obtain a result of the initially selected target detection spectrum.
The integrated signal-to-noise ratio is defined as:
in the formula: SET is the system noise equivalent target and SET is the target radiation intensity at the entrance pupil of the detection system when SSNR equals 1.
The SET is defined as:
SET＝(NET 2 +CET 2 ) 1/2 (3)；
in the formula: NET is a noise equivalent target and is defined as the target radiation intensity at the entrance pupil when the signal-to-noise ratio is 1; CET is a clutter equivalent target defined as the target radiation intensity at the entrance pupil with a signal-to-clutter ratio of 1.
Step two: and in the initially selected target detection spectrum set, calculating the relative difference of the radiation intensity between different targets on each spectrum section based on a spectrum relative distance model, and accumulating and sequencing the relative difference values of the radiation of all the targets on each spectrum section.
If the spectral relative distance sum between different targets on a certain spectrum section is larger, the radiation intensity difference between different targets on the spectrum section is larger, and the spectrum section is used as a target detection spectrum section, so that the accuracy of subsequent target identification is improved. The steps of accumulating and sequencing the relative difference values of the radiation of all targets on each spectral band in the initially selected target detection spectral band set are as follows:
(1) And calculating the spectral relative distance between different targets on each spectral band in the initially selected target detection spectral band set.
The spectral relative distance of target 1 from target 2 is defined as:
in the formula: i is an absolute value operation, I 1 Is the radiation intensity of the target 1, I 2 Is the radiation intensity of the target 2.
(2) The relative distances of the spectra between different targets in each spectral band are summed.
(3) And sequencing all the spectral bands in the spectral band set from large to small according to the accumulation result of the relative distances of the spectrums, wherein the sequencing result provides support for the optimal spectral band of the subsequent iteration.
Step three: and in the initially selected target detection spectrum set, calculating the sum of the spectral information divergence of each target under different spectrum combination conditions based on a spectral information divergence model, and taking the spectral information divergence and the maximum spectrum combination as a preferred spectrum set.
The spectral information divergence can physically reflect the difference between the spectral characteristics of the two targets, and the maximum spectral characteristic difference between different targets in the optimal spectral band set can be ensured through the maximum spectral information divergence principle. The preferred method for confirming the detection spectrum is shown in fig. 2, and comprises the following steps:
(1) Initializing a preferred spectral set phi by using the spectral relative distance in the initially selected target detection spectral set and the maximum spectral band 1 。
(2) Calculate phi 1 Divergence of spectral information between different targets within the spectrum.
Defining the divergence of the spectral information as:
in the formula: sigma is a summation operation, lg is a logarithm operation, and an N-dimensional spectral vector P = (P) 1 ,P 2 ,...,P N ) And Q = (Q) 1 ,Q 2 ,...,Q N ) Respectively is p = (p) 1 ,p 2 ,…,p N ) And q = (q) 1 ,q 2 ,…,q N )，
(3) To phi 1 And accumulating the spectral information divergence between different targets in the optimal spectrum set to obtain the spectral information divergence sum between different targets in the optimal spectrum set.
(4) Selecting spectral relative distance and suboptimal spectral range from other spectral ranges remaining in the primary spectral range set 1 Set of constituent spectral bands phi 2 。
(5) Calculate phi 2 The spectral information divergence between different targets in the spectrum set is accumulated, and the larger the spectral information divergence sum in the spectrum set is, the larger the spectral difference between different targets in the spectrum set is.
(6) Repeating the steps (4) - (5), and gradually increasing the number of the spectrum segments in the preferred spectrum segment set until phi x The divergence sum of the spectral information between different targets is not increased any more, and the final target detection spectrum set phi is obtained x-1 And x-1 is the number of spectral bands in the set of spectral bands.
Example (b):
the invention adopts simulation hyperspectral data, the spectral range is 2-14 μm, the image background is typical cirrus cloud, the target comprises a target 1, a target 2 and a target 3, and the targets are positioned at the same position in the image, and the infrared image of part of spectral range is shown in figure 3. As can be seen from fig. 3, the difference between the target and the background in the upper two images is large, and the difference between the target and the background in the lower two images is small, and is almost submerged in the background clutter and noise. The invention aims to select the spectral bands similar to the two images above for target detection, and the target detection spectral bands are optimized based on the principle of maximum class separability, so that the efficiency of target detection and subsequent target identification is improved.
The invention firstly preselects a target detection spectral band by adopting an apparent contrast model, namely calculating the apparent contrast of a target on each spectral band image of 2-14 mu m relative to a background, wherein the apparent contrast calculation results of all the spectral band images are shown in figure 4, curves have obvious peak values in the ranges of 2.5-3 mu m, 3.5-4 mu m, 4-4.6 mu m and 5-8 mu m, and the spectral band ranges are preselected for subsequent comprehensive signal-to-noise ratio calculation, so that the calculation amount is obviously reduced compared with the comprehensive signal-to-noise ratio calculation in the full spectral band range of 2-14 mu m.
Next, the integrated signal-to-noise ratio at different center wavelengths and at different spectral widths is calculated within the preselected set of target detection spectral bands. The interval between two adjacent central wavelengths is 5nm, for each central wavelength, the comprehensive signal-to-noise ratio within the spectrum width range of 5-1000 nm is respectively calculated, the width change step of the spectrum is 5nm, the calculation result of the comprehensive signal-to-noise ratio is shown in figure 5, as can be seen in the figure, the comprehensive signal-to-noise ratios within the spectrum bands of 2.72-2.82 μm, 3.86-3.96 μm, 4.18-4.28 μm, 4.35-4.45 μm, 5.87-5.97 μm and 6.4-6.6 μm are higher, and the spectrum bands are used as the initial target detection spectrum bands.
And respectively calculating the spectral relative distance between different targets on each initially selected target detection spectrum section, and accumulating and summing. The relative radiation difference between different targets is the largest in the spectral relative distance and the largest spectral band, i.e. the distinguishable performance of different targets is the highest. The relative distance of the spectrum and the smaller spectrum band are low in efficiency of classifying different targets.
And next, in the primarily selected target detection spectrum set, iteratively calculating the spectral information divergence of different targets under different spectrum combinations, and taking the spectral information divergence and the maximum spectrum combination as an optimal target detection spectrum set, wherein the optimal target detection spectrum set comprises four spectrum sections: 2.72-2.82 μm, 3.86-3.96 μm, 4.18-4.28 μm and 4.35-4.45 μm, the infrared image of each spectral band is shown in fig. 6, the target can be clearly seen in the preferred four spectral band images, the spectral characteristics of the three targets in the preferred target detection spectral band set are shown in fig. 7, and the spectral characteristic difference among the three targets is large.
The following data analysis and evaluation were performed for the present invention:
the invention selects spectral angles to describe the spectral feature difference between different targets in the preferred target detection spectrum set, and the spectral angles are defined as follows:
in the formula: t and R are two spectral vectors in an N-dimensional space, and T = [ T = [ [ T ] 1 ,t 2 ,...,t N ]，R＝[r 1 ,r 2 ,...,r N ]。
Calculating the spectral angles of different targets in the optimal target detection spectral band set, wherein the spectral angle between the target 1 and the target 2 is 65.2 degrees, the spectral angle between the target 1 and the target 3 is 65.3 degrees, and the spectral angle between the target 2 and the target 3 is 17.8 degrees, because the flight conditions, the structures and the fuel components of the target 2 and the target 3 are similar, the spectral angles of the target 1 and the target 3 are smaller, the high physical similarity determines that the discriminative power of the target 1, the target 2 and the target 3 is not high, but the discriminative power of the target 1, the target 2 and the target 3 is high.
Claims (8)
1. A detection spectrum segment optimization method for an object recognition task is characterized by comprising the following steps:
the method comprises the following steps: primarily selecting a target detection spectrum set based on an apparent contrast model and a comprehensive signal-to-noise ratio model;
step two: in the initially selected target detection spectrum set, calculating the relative difference of radiation intensity between different targets on each spectrum based on a spectrum relative distance model, and accumulating and sequencing the relative difference values of radiation of all targets on each spectrum according to the following steps:
(1) Calculating the spectral relative distance between different targets on each spectral band in the initially selected target detection spectral band set;
(2) Accumulating the relative spectral distances between different targets on each spectral band;
(3) Sequencing all the spectral bands in the spectral band set from large to small according to the accumulation result of the relative distances of the spectrums, wherein the sequencing result provides support for the optimal spectral band of the subsequent iteration;
step three: and in the primarily selected target detection spectrum set, calculating the sum of the spectral information divergence of each target under the condition of different spectrum combinations based on a spectral information divergence model, and taking the spectral information divergence and the maximum spectrum combination as a preferred spectrum set.
2. The object-oriented identification task detection spectrum optimization method according to claim 1, wherein in the first step, the specific steps of initially selecting the object detection spectrum are as follows:
(1) Calculating the apparent contrast of target radiation relative to background radiation at the entrance pupil of the camera based on the target and background apparent contrast model, and preselecting a detection spectral range according to the calculation result of the apparent contrast;
(2) And based on the comprehensive signal-to-noise ratio model, calculating comprehensive signal-to-noise ratios with different central wavelengths and different spectrum widths in a preselected spectrum range, and optimizing the preselected target detection spectrum range according to the calculation results of the comprehensive signal-to-noise ratios in the different spectrum ranges to obtain a result of the initially selected target detection spectrum.
3. The object-recognition-task-oriented detection spectral range optimization method according to claim 2, wherein the apparent contrast is defined as:
c is apparent contrast; I.C. A t Is the target radiation intensity; tau is a Is the atmospheric transmittance between the object and the entrance pupil of the detection system; l is B The ambient background radiance at the entrance pupil of the detection system; a. The t Is the projected area of the target along the line of sight direction.
4. The object-oriented recognition task detection spectral segment optimization method according to claim 3, wherein the integrated signal-to-noise ratio is defined as:
in the formula: SET is the system noise equivalent target, and is the target radiation intensity at the entrance pupil of the detection system when SSNR equals 1.
5. The object-oriented recognition task detection spectrum preference method as claimed in claim 4, wherein the SET is defined as:
SET＝(NET 2 +CET 2 ) 1/2 ；
in the formula: NET is a noise equivalent target and is defined as the target radiation intensity at the entrance pupil when the signal-to-noise ratio is 1; CET is a clutter equivalent target defined as the target radiation intensity at the entrance pupil with a signal-to-clutter ratio of 1.
6. The object-recognition-task-oriented detection spectrum optimization method according to claim 1, wherein the spectral relative distance is defined as:
in the formula: i is an absolute value operation, I 1 Is the radiation intensity of the target 1, I 2 Is the radiation intensity of the object 2.
7. The object-oriented recognition task detection spectrum optimization method according to claim 1, wherein in the third step, the specific steps of preferentially confirming the detection spectrum are as follows:
(1) Initializing optimal spectrum set phi by using the relative spectral distance in the initially selected target detection spectrum set and the maximum spectrum 1 ；
(2) Calculate phi 1 The divergence of spectral information between different targets;
(3) To phi 1 Accumulating the divergence degrees of the spectral information among different targets in the optimal spectral band set to obtain the sum of the divergence degrees of the spectral information among different targets in the optimal spectral band set;
(4) Selecting spectral relative distance and suboptimal spectral range from other spectral ranges remaining in the primary spectral range set 1 Set of constituent spectral bands phi 2 ；
(5) Calculate phi 2 The spectral information divergence between different targets in the system is accumulated;
(6) Repeating the steps (4) - (5), and gradually increasing the number of the spectrum segments in the preferred spectrum segment set until phi x The divergence sum of the spectral information between different targets is not increased any more, and the final target detection spectrum set phi is obtained x-1 And x-1 is the number of the spectrum segments in the spectrum segment set.
8. The object-recognition-task-oriented detection spectrum optimization method according to claim 7, wherein the spectral information divergence is as follows:
in the formula: sigma is a summation operation, lg is a logarithm operation, and an N-dimensional spectral vector P = (P) 1 ,P 2 ,...,P N ) And Q = (Q) 1 ,Q 2 ,...,Q N ) Respectively, is p = (p) 1 ,p 2 ,…,p N ) And q = (q) 1 ,q 2 ,…,q N )，
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911214745.7A CN110969125B (en) | 2019-12-02 | 2019-12-02 | Detection spectrum band optimization method for target identification task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911214745.7A CN110969125B (en) | 2019-12-02 | 2019-12-02 | Detection spectrum band optimization method for target identification task |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110969125A CN110969125A (en) | 2020-04-07 |
| CN110969125B true CN110969125B (en) | 2022-10-25 |
Family
ID=70032643
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911214745.7A Active CN110969125B (en) | 2019-12-02 | 2019-12-02 | Detection spectrum band optimization method for target identification task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110969125B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102243763A (en) * | 2011-05-23 | 2011-11-16 | 华中科技大学 | Infrared imaging spectrum optimization and selection method under high-speed conditions |
| CN104268582A (en) * | 2014-08-26 | 2015-01-07 | 中国科学院遥感与数字地球研究所 | Band selection method and device of hyperspectral images |
| CN109697431A (en) * | 2018-12-29 | 2019-04-30 | 哈尔滨工业大学 | A kind of detection method of small target based on high spectrum image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102004058728A1 (en) * | 2004-12-06 | 2006-06-08 | Leica Microsystems Cms Gmbh | Method for selecting a wavelength and microscope |
-
2019
- 2019-12-02 CN CN201911214745.7A patent/CN110969125B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102243763A (en) * | 2011-05-23 | 2011-11-16 | 华中科技大学 | Infrared imaging spectrum optimization and selection method under high-speed conditions |
| CN104268582A (en) * | 2014-08-26 | 2015-01-07 | 中国科学院遥感与数字地球研究所 | Band selection method and device of hyperspectral images |
| CN109697431A (en) * | 2018-12-29 | 2019-04-30 | 哈尔滨工业大学 | A kind of detection method of small target based on high spectrum image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110969125A (en) | 2020-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107862705B (en) | Unmanned aerial vehicle small target detection method based on motion characteristics and deep learning characteristics | |
| CN113449680B (en) | Knowledge distillation-based multimode small target detection method | |
| CN109146889B (en) | Farmland boundary extraction method based on high-resolution remote sensing image | |
| CN108389188B (en) | Sparse hyperspectral abnormal target detection method | |
| CN109711418B (en) | Contour corner detection method for object plane image | |
| CN107622502B (en) | Path extraction and identification method of visual guidance system under complex illumination condition | |
| US9390348B2 (en) | Method for categorizing objects in image | |
| CN110889324A (en) | Thermal infrared image target identification method based on YOLO V3 terminal-oriented guidance | |
| CN108596213A (en) | A kind of Classification of hyperspectral remote sensing image method and system based on convolutional neural networks | |
| CN111241931A (en) | Aerial unmanned aerial vehicle target identification and tracking method based on YOLOv3 | |
| CN111489309B (en) | Sparse unmixing pretreatment device and method | |
| CN103150577B (en) | Method for selecting high spectrum remotely sensed image self-adaptive wave band based on particle swarm optimization | |
| CN103226196B (en) | Radar target recognition method based on sparse feature | |
| CN109583469B (en) | K-means hyperspectral image band clustering method based on mutual information | |
| CN109697431B (en) | Hyperspectral image-based weak and small target detection method | |
| CN111783555A (en) | Deep learning satellite data cloud detection algorithm using unified samples | |
| CN117409339A (en) | Unmanned aerial vehicle crop state visual identification method for air-ground coordination | |
| CN115292334A (en) | Intelligent planting method and system based on vision, electronic equipment and storage medium | |
| CN110969125B (en) | Detection spectrum band optimization method for target identification task | |
| CN115343674A (en) | Visible light-based outdoor high-precision positioning method for unmanned aerial vehicle | |
| CN111626224B (en) | Near infrared spectrum and SSA optimization-based ELM (enzyme-linked immunosorbent assay) quick coal gangue identification method | |
| Ashton et al. | A novel method for illumination suppression in hyperspectral images | |
| CN113327271A (en) | Decision-level target tracking method and system based on double-optical twin network and storage medium | |
| CN116188585A (en) | Mountain area photovoltaic target positioning method based on unmanned aerial vehicle photogrammetry | |
| CN112363171B (en) | Autonomous tracking method for solving passive warning picture target track crossing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |