CN108332721A - The parallel sky three of aviation image and recursion fusion method - Google Patents
The parallel sky three of aviation image and recursion fusion method Download PDFInfo
- Publication number
- CN108332721A CN108332721A CN201810172145.8A CN201810172145A CN108332721A CN 108332721 A CN108332721 A CN 108332721A CN 201810172145 A CN201810172145 A CN 201810172145A CN 108332721 A CN108332721 A CN 108332721A
- Authority
- CN
- China
- Prior art keywords
- areas
- image
- sky
- recursion
- parallel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/30—Interpretation of pictures by triangulation
- G01C11/34—Aerial triangulation
Abstract
The invention discloses a kind of parallel skies three of aviation image and recursion fusion method, the areas Xian Dui great Ce are divided into multiple areas Zi Ce using KD tree algorithms by image GPS plane coordinates, the image quantity in area is surveyed in control per height, then area is surveyed using existing SFM algorithms antithetical phrase in a manner of multi-host parallel and builds free net and GPS supported Bundle Block Adjustments respectively, it finally uses Recursive Algorithm antithetical phrase to survey empty three results in area gradually to be merged, obtains entire three result of sky for surveying area.The advantage of the invention is that：By the way of carrying out piecemeal SFM to the areas great Ce, solving the problems, such as that big data quantity SFM errors are big, speed is slow can not even succeed, and use recursion fusion method, so as to get three result of sky in every image only have unique one group of result of calculation, solve survey area's edge fit problem of misalignment.
Description
Technical field
The present invention relates to Surveying and mapping technical field more particularly to a kind of parallel sky three of aviation image and the recursion sides of fusion
Method.
Background technology
Aerophotogrammetry is the important technical in surveying production field, and principle is by being carried on aviation aircraft
Camera photograph to ground region, then photogrammetric aerial triangulation (referred to as sky three) is used to carry out aerial images
Geometric manipulations obtain the internal and external orientation and Pass point of image.
Traditional aeroplane photography is generally shot using a camera vertically downward, and course line comparison rule (such as course line
Between be mutually parallel or vertically), sky three can more easily be carried out using traditional three method of sky and calculated.And with unmanned plane
And the rise of oblique photograph technology, it is general to go back while making in addition to camera lens vertically downward at this stage when carrying out aeroplane photography
With towards sideways camera lens so that visual angle, different scale are very big between image, are along sense when in addition often using unmanned plane
The place flight of interest, the line of flight are very random, and the above reason is resulted in can not be obtained correctly using empty three algorithms of tradition
Result of calculation.Processing for this kind of data, using currently a popular computer vision field " from exercise recovery three dimensional field
Scape " (Structure from Motion, SFM) technology can obtain ideal handling result.But the shortcomings that SFM technologies, is
When data volume is very big (such as more than 1000 images), there are calculating speeds, and image position that is very slow, calculating acquisition is believed
There is the problems such as prodigious drift and poor computational accuracy in breath.At present when carrying out aeroplane photography, often acquired data volume ten
It is point huge, generally can be more than 10,000, some projects even up to 100,000 images handle to sky three and cause prodigious choose
War.
The method of existing processing big data quantity is piecemeal processing to be carried out to big data, then to each piece of difference at present
Carry out sky three.The edge fit problem in piecemeal intersection is thus caused, if having a part of identical shadow between adjacent piecemeal
As (being known as superimposed image), after two piecemeals individually carry out sky three, the internal and external orientation of these images is in two piecemeals
Just there is different calculated values, if carrying out subsequent mapping, modeling etc. with these empty three results, it may appear that go out in boundary model
The case where now misplacing cannot be satisfied actual demand.
Invention content
The object of the present invention is to provide a kind of parallel skies three of aviation image and recursion fusion method, solve big data quantity boat
Three problem of sky of empty image.
The purpose of the present invention is what is be achieved through the following technical solutions：
A kind of parallel sky three of aviation image and recursion fusion method, including：
Aviation image is divided into multiple areas Zi Ce by GPS plane coordinates based on KD trees；
By the way of multiple stage computers parallel processing, independent empty three are carried out to all areas Zi Ce of division and is calculated,
After obtaining empty three results, the sky three under object coordinates system is obtained using GPS supported Bundle Block Adjustments and is become as a result, each height is made to survey area
Change to unified coordinate system；
According to the KD tree index informations retained when dividing the areas Zi Ce, area is surveyed using recursive mode antithetical phrase and is merged two-by-two
Bundle adjustment, each fusion results are by the fusion for next time, until being fused to root node.
As seen from the above technical solution provided by the invention, on the one hand, the son of division is surveyed by the way of piecemeal
Area carries out parallel sky three, improves empty three efficiency and precision；On the other hand, it is only calculated in one group for every image in fusion
Elements of exterior orientation eliminates the piecemeal edge fit problem of misalignment in conventional method, therefore the present invention has in field of aerial photography measurement
Wide application prospect.
Description of the drawings
In order to illustrate the technical solution of the embodiments of the present invention more clearly, required use in being described below to embodiment
Attached drawing be briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill in field, without creative efforts, other are can also be obtained according to these attached drawings
Attached drawing.
Fig. 1 is a kind of flow chart of aviation image provided in an embodiment of the present invention parallel sky three and recursion fusion method；
Fig. 2 is showing based on the areas the KD Shu Ance minimum outsourcing rectangle principal direction division areas Zi Ce provided in an embodiment of the present invention
It is intended to；
Fig. 3 is that empty three results of recursion provided in an embodiment of the present invention merge schematic diagram.
Specific implementation mode
With reference to the attached drawing in the embodiment of the present invention, technical solution in the embodiment of the present invention carries out clear, complete
Ground describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Based on this
The embodiment of invention, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, belongs to protection scope of the present invention.
A kind of parallel sky three of aviation image of offer of the embodiment of the present invention and recursion fusion method, as described in Figure 1, this method
Mainly include the following steps：
Aviation image is divided into multiple areas Zi Ce by step 1 based on KD trees by GPS plane coordinates.
In the embodiment of the present invention, the image number that each areas Zi Ce include is no more than setting value K, K=k*N；Wherein, N tables
Show number of cameras；K is a constant, such as can be set as 70.
Due to KD trees divide the areas Zi Ce when be by be parallel to change in coordinate axis direction divide, for the survey area of inclined direction, press
The areas reference axis division Hou Zice shape is very irregular, is unfavorable for carrying out SFM skies three.In the embodiment of the present invention, first extraction is entire
Survey the minimum outsourcing rectangle in area, if minimum outsourcing rectangle and reference axis there are certain angle of inclination (shown on the left of Fig. 2),
The GPS plane coordinates of image is rotated according to the principal direction of minimum outsourcing rectangle, make it is postrotational survey area principal direction with
Reference axis is parallel, then divides the areas Zi Ce with KD trees, to obtain the areas Zi Ce (as shown in the right side of fig 2) of regular shape.The present invention
In embodiment, using KD trees without directly using grid at equal intervals, allow in the areas image track Dian Ce and non-homogeneous point
Cloth, some places are more intensive than other places image, and uniform piecemeal result can be obtained using KD trees.
Assuming that the angle of minimum outsourcing rectangle principal direction and X-axis is φ, image plane coordinate is carried out using following formula
Rotation：
Wherein, (x ', y ') is postrotational plane coordinates, and (x, y) is the plane coordinates before rotation.
In the embodiment of the present invention, using KD tree algorithms structure KD trees index, when the image number that a certain node is included is few
When setting value K, do not continue to divide as leaf node, it is final so that therefore all leaf nodes image number for being included is equal
Less than or equal to setting value K；Leaf node herein is the areas Zi Ce.
The son divided through the above way surveys section and non-overlapping image, merges, is needed to each for the ease of the areas Zi Ce
The outsourcing rectangle in the areas Zi Ce carries out a degree of expansion, and each areas Zi Ce is made to include other images of certain amount, referred to as sub
Area's expansion is surveyed, so that there is certain overlapping between the areas Zi Ce.Illustratively, the mode of cycle may be used, each antithetical phrase is surveyed
The scope expansion 5% of area's outsourcing rectangle surveys area's internal image number until surveying the external image number that area is included and reaching
10%.
Step 2, by the way of multiple stage computers parallel processing, independent empty three meter is carried out to all areas Zi Ce of division
It calculates, after obtaining empty three results, the sky three under object coordinates system is obtained as a result, making each height using GPS supported Bundle Block Adjustments
It surveys area and transforms to unified coordinate system.
It will be understood by those skilled in the art that sky three, which calculates, may be used existing SFM algorithms to realize.
Step 3, according to the KD tree index informations that retain when dividing the areas Zi Ce, area's progress two is surveyed using recursive mode antithetical phrase
Two fusion bundle adjustments, each fusion results are by the fusion for next time, until being fused to root node.
Recursion fusion is carried out by the way of subsequently traversing, and from root node, for any node, first judges that it is
It is no to have the child node not merged, if any then first merging to left child node, then merged to right child node；Such as
No, then node itself is merged；Until being fused to root node.Wherein, leaf node is initialized as having merged
State.
KD trees as illustrated one four layers in Fig. 3, a total of 7 leaf nodes, that is, have 7 areas Ge Zice, the areas Zi Ce
In fusion sequence such as figure shown in the arrow with serial number.
It is to do whole flux of light method to the image data in the areas Liang Gezice to put down when being merged to the areas Liang Gezice
Difference, steps are as follows：First, the superimposed image and overlapping pass point, wherein superimposed image searched out in the areas Liang Gezice passes through
It searches for the identical image of image name in the areas Liang Gezice to obtain, overlapping pass point has identical picture point by searching in the areas Liang Zice
Pass point obtain；Then, to superimposed image and overlapping pass point, the side for taking number average value in the areas Liang Gezice is used
Formula calculates corresponding new initial parameter, and each superimposed image is made there was only unique one group of parameter with each Chong Die pass point
Value；Finally, bundle adjustment is carried out, empty three results are optimized.
After independently carrying out sky three due to every height survey area, the parameter value of sub- survey section superimposed image and Chong Die pass point is always
There are a degree of differences, only to the approximation of its legitimate reading by the way of being averaged, if when merging adjustment
Initial value error is too big, can cause bundle adjustment that can not restrain.Therefore, it needs to use in the areas Zi Ce fusion process and only melt every time
The mode in the areas He Lianggece keeps the error in initial value small as possible, can just obtain satisfactory fusion results.
In the said program of the embodiment of the present invention, using KD trees to press the postrotational survey area of minimum outsourcing rectangle principal direction into
Row divide, make division the areas Zi Ce image number uniformly, the areas Zi Ce shape it is regular, be conducive to SFM resolving；Using melting for recursion
Conjunction mode, every time only fusion Liang Gece areas, it is possible to reduce the error of unknown number before fusion, it is ensured that the convergence of bundle adjustment and
Last fusion accuracy.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment can
By software realization, the mode of necessary general hardware platform can also be added to realize by software.Based on this understanding,
The technical solution of above-described embodiment can be expressed in the form of software products, the software product can be stored in one it is non-easily
In the property lost storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.), including some instructions are with so that a computer is set
Standby (can be personal computer, server or the network equipment etc.) executes the method described in each embodiment of the present invention.
The foregoing is only a preferred embodiment of the present invention, but scope of protection of the present invention is not limited thereto,
Any one skilled in the art is in the technical scope of present disclosure, the change or replacement that can be readily occurred in,
It should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with the protection model of claims
Subject to enclosing.
Claims (6)
1. a kind of parallel sky three of aviation image and recursion fusion method, which is characterized in that including：
Aviation image is divided into multiple areas Zi Ce by GPS plane coordinates based on KD trees；
By the way of multiple stage computers parallel processing, independent empty three are carried out to all areas Zi Ce of division and is calculated, is being obtained
After empty three results, the sky three under object coordinates system is obtained using GPS supported Bundle Block Adjustments and is transformed to as a result, each height is made to survey area
Unified coordinate system；
According to the KD tree index informations retained when dividing the areas Zi Ce, area is surveyed using recursive mode antithetical phrase and is merged light beam two-by-two
Method adjustment, each fusion results are by the fusion for next time, until being fused to root node.
2. a kind of parallel sky three of aviation image according to claim 1 and recursion fusion method, which is characterized in that described
The step of aviation image is divided into multiple areas Zi Ce by GPS plane coordinates based on KD trees is as follows：
The entire minimum outsourcing rectangle for surveying area of extraction, if there are certain angle of inclination, roots with reference axis for minimum outsourcing rectangle
The GPS plane coordinates of image is rotated according to the principal direction of minimum outsourcing rectangle, makes the postrotational principal direction and seat for surveying area
Parameter is parallel, then divides the areas Zi Ce with KD trees, to obtain the areas Zi Ce of regular shape；
Assuming that the angle of minimum outsourcing rectangle principal direction and X-axis is φ, image plane coordinate is rotated using following formula：
Wherein, (x ', y ') is postrotational plane coordinates, and (x, y) is the plane coordinates before rotation.
3. a kind of parallel sky three of aviation image according to claim 1 or 2 and recursion fusion method, which is characterized in that
The image number that each areas Zi Ce include is no more than setting value K；
Using KD tree algorithms structure KD trees index, when the image number that a certain node is included is less than setting value K, as leaf
Node does not continue to divide, it is final so that therefore all leaf nodes image number for being included be respectively less than and be equal to setting value K；Herein
Leaf node be the areas Zi Ce；
Wherein, setting value K is the product of number of cameras N and constant k.
4. a kind of parallel sky three of aviation image according to claim 1 or 2 and recursion fusion method, which is characterized in that
This method further includes：A degree of expansion is carried out to the outsourcing rectangle in each areas Zi Ce, it includes a fixed number to make each areas Zi Ce
Other images of purpose, so that there is certain overlapping between the areas Zi Ce.
5. a kind of parallel sky three of aviation image according to claim 1 and recursion fusion method, which is characterized in that recurrence
Formula fusion is carried out by the way of subsequently traversing, and from root node, for any node, is first judged whether it has and is not melted
The child node of conjunction, if any then first merging to left child node, then merged to right child node；If it has not, then to node
Itself is merged；Until being fused to root node.
6. a kind of parallel sky three of aviation image and recursion fusion method according to claim 1 or 5, which is characterized in that
It is that whole bundle adjustment is done to the image data in the areas Liang Gezice, step is such as when being merged to the areas Liang Gezice
Under：
First, the superimposed image and overlapping pass point in the areas Liang Gezice are searched out, wherein superimposed image is by searching for two
The identical image of image name obtains in the areas Zi Ce, and overlapping pass point is by searching for the pass point with identical picture point in the areas Liang Zice
It obtains；
Then, superimposed image and overlapping pass point are calculated and is corresponded to using the mode for taking number average value in the areas Liang Gezice
New initial parameter, so that each superimposed image and each Chong Die pass point there was only unique one group of parameter value；
Finally, bundle adjustment is carried out, empty three results are optimized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810172145.8A CN108332721B (en) | 2018-03-01 | 2018-03-01 | Aviation image parallel air-space three and recursive fusion method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810172145.8A CN108332721B (en) | 2018-03-01 | 2018-03-01 | Aviation image parallel air-space three and recursive fusion method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108332721A true CN108332721A (en) | 2018-07-27 |
| CN108332721B CN108332721B (en) | 2020-12-11 |
Family
ID=62930464
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810172145.8A Expired - Fee Related CN108332721B (en) | 2018-03-01 | 2018-03-01 | Aviation image parallel air-space three and recursive fusion method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108332721B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111536871A (en) * | 2020-05-07 | 2020-08-14 | 武汉大势智慧科技有限公司 | Accurate calculation method for volume variation of multi-temporal photogrammetric data |
| CN113405533A (en) * | 2021-05-27 | 2021-09-17 | 武汉天远视科技有限责任公司 | Aerial triangulation operation method for aerial images |
| CN114279412A (en) * | 2021-11-26 | 2022-04-05 | 武汉大势智慧科技有限公司 | Multi-block space-three adjustment merging method based on aerial oblique photography image |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102519436A (en) * | 2011-12-28 | 2012-06-27 | 武汉大学 | Chang'e-1 (CE-1) stereo camera and laser altimeter data combined adjustment method |
| CN102706326A (en) * | 2012-04-27 | 2012-10-03 | 北京市测绘设计研究院 | Processing method of light beam method aerial triangulation file data |
| CN104299228A (en) * | 2014-09-23 | 2015-01-21 | 中国人民解放军信息工程大学 | Remote-sensing image dense matching method based on accurate point location prediction model |
| CN104897176A (en) * | 2015-06-29 | 2015-09-09 | 北京建筑大学 | Multicore parallel photogrammetry block adjustment method |
| CN105004321A (en) * | 2015-07-17 | 2015-10-28 | 湖北省电力勘测设计院 | Unmanned plane GPS-supported bundle djustment method in consideration of non-synchronous exposal |
| JP2017026552A (en) * | 2015-07-27 | 2017-02-02 | 株式会社パスコ | Three-dimensional measurement device, three-dimensional measurement method and program |
| CN107504959A (en) * | 2017-08-22 | 2017-12-22 | 北京中测智绘科技有限公司 | Utilize the method for oblique aerial radiographic measurement house wall base profile |
-
2018
- 2018-03-01 CN CN201810172145.8A patent/CN108332721B/en not_active Expired - Fee Related
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102519436A (en) * | 2011-12-28 | 2012-06-27 | 武汉大学 | Chang'e-1 (CE-1) stereo camera and laser altimeter data combined adjustment method |
| CN102706326A (en) * | 2012-04-27 | 2012-10-03 | 北京市测绘设计研究院 | Processing method of light beam method aerial triangulation file data |
| CN104299228A (en) * | 2014-09-23 | 2015-01-21 | 中国人民解放军信息工程大学 | Remote-sensing image dense matching method based on accurate point location prediction model |
| CN104897176A (en) * | 2015-06-29 | 2015-09-09 | 北京建筑大学 | Multicore parallel photogrammetry block adjustment method |
| CN105004321A (en) * | 2015-07-17 | 2015-10-28 | 湖北省电力勘测设计院 | Unmanned plane GPS-supported bundle djustment method in consideration of non-synchronous exposal |
| JP2017026552A (en) * | 2015-07-27 | 2017-02-02 | 株式会社パスコ | Three-dimensional measurement device, three-dimensional measurement method and program |
| CN107504959A (en) * | 2017-08-22 | 2017-12-22 | 北京中测智绘科技有限公司 | Utilize the method for oblique aerial radiographic measurement house wall base profile |
Non-Patent Citations (7)
| Title |
|---|
| XIONG XIAODONG: "3D Building Change Detection by Combining LiDAR Point Clouds and Aerial Imagery", 《GEOMATICS AND INFORMATION SCIENCE OF WUHAN UNIVERSITY》 * |
| ZHU JUNFENG: "ANALYSIS AND APPLICATION OF LIDAR WAVEFORM DATA USING A PROGRESSIVE WAVEFORM DECOMPOSITION METHOD", 《ISPRS WORKSHOP LASER SCANNING 2011》 * |
| 吴子敬: "ＳＦＭ的倾斜航空影像空中三角测量", 《测绘科学》 * |
| 张永军、熊小东: "机载激光雷达点云与定位定姿系统数据辅助的航空影像自动匹配方法", 《测绘学报》 * |
| 熊小东: "基于多种特征的机载激光点云与航空影像配准方法研究", 《中国博士学位论文全文数据库》 * |
| 郑茂腾、朱俊峰: "一种快速有效的大数据区域网平差方法", 《测绘学报》 * |
| 陈茂霖: "一种基于线性KD树的点云数据组织方法", 《测绘通报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111536871A (en) * | 2020-05-07 | 2020-08-14 | 武汉大势智慧科技有限公司 | Accurate calculation method for volume variation of multi-temporal photogrammetric data |
| CN113405533A (en) * | 2021-05-27 | 2021-09-17 | 武汉天远视科技有限责任公司 | Aerial triangulation operation method for aerial images |
| CN113405533B (en) * | 2021-05-27 | 2022-04-12 | 武汉天远视科技有限责任公司 | Aerial triangulation operation method for aerial images |
| CN114279412A (en) * | 2021-11-26 | 2022-04-05 | 武汉大势智慧科技有限公司 | Multi-block space-three adjustment merging method based on aerial oblique photography image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108332721B (en) | 2020-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11686849B2 (en) | Augmented three dimensional point collection of vertical structures | |
| US9269188B2 (en) | Densifying and colorizing point cloud representation of physical surface using image data | |
| CN104966281B (en) | The IMU/GNSS guiding matching process of multi-view images | |
| CN108332721A (en) | The parallel sky three of aviation image and recursion fusion method | |
| CN108286965A (en) | Unmanned plane based on subtle three-dimensional landform gets higher course line method, terminal and system | |
| WO2021035618A1 (en) | Point cloud segmentation method and system, and movable platform | |
| CN108876828A (en) | A kind of unmanned plane image batch processing three-dimensional rebuilding method | |
| CN105761257A (en) | Elimination method for gross error in unmanned aerial vehicle image matching on cross air strip and device thereof | |
| CN108399631A (en) | A kind of inclination image of scale invariability regards dense Stereo Matching method more | |
| CN1825203A (en) | Airborne inclined camera photographing device | |
| CN113177974A (en) | Point cloud registration method and device, electronic equipment and storage medium | |
| CN117315146A (en) | Reconstruction method and storage method of three-dimensional model based on trans-scale multi-source data | |
| Zhou et al. | Application of UAV oblique photography in real scene 3d modeling | |
| CN108050995B (en) | Oblique photography non-image control point aerial photography measurement area merging method based on DEM | |
| CN115100296A (en) | Photovoltaic module fault positioning method, device, equipment and storage medium | |
| CN113790711B (en) | Unmanned aerial vehicle low-altitude flight pose uncontrolled multi-view measurement method and storage medium | |
| CN115797256A (en) | Unmanned aerial vehicle-based tunnel rock mass structural plane information processing method and device | |
| CN112989079B (en) | Novel image data retrieval method and system | |
| CN114119892B (en) | Three-dimensional digital road network construction method based on BIM and GIS technologies | |
| CN117033675B (en) | Safe space calculation generation method and device based on city live-action model | |
| CN114608540A (en) | Measurement network type determining method of digital photogrammetric system | |
| Liu et al. | Research on the Conservation of Historical Buildings Based on Digital 3D Reconstruction | |
| CN115457193A (en) | Three-dimensional model construction method based on air-ground union, terminal and storage medium | |
| CN117330031A (en) | Unmanned aerial vehicle inclined image real-time positioning method integrating airborne LiDAR point cloud | |
| CN114821365A (en) | Unmanned aerial vehicle photogrammetry overlapping degree calculation method and system considering surface relief |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20201211 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |