CN105005964B - Geographic scenes panorama sketch rapid generation based on video sequence image - Google Patents
Geographic scenes panorama sketch rapid generation based on video sequence image Download PDFInfo
- Publication number
- CN105005964B CN105005964B CN201510386060.6A CN201510386060A CN105005964B CN 105005964 B CN105005964 B CN 105005964B CN 201510386060 A CN201510386060 A CN 201510386060A CN 105005964 B CN105005964 B CN 105005964B
- Authority
- CN
- China
- Prior art keywords
- subgraph
- image
- matching
- video sequence
- panorama sketch
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 claims abstract description 31
- 239000011159 matrix material Substances 0.000 claims abstract description 25
- 238000001514 detection method Methods 0.000 claims abstract description 7
- 238000001914 filtration Methods 0.000 claims abstract description 4
- 238000005192 partition Methods 0.000 claims abstract description 3
- 239000000284 extract Substances 0.000 claims abstract 3
- 230000009466 transformation Effects 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 238000005452 bending Methods 0.000 abstract 1
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 230000004927 fusion Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 241000255749 Coccinellidae Species 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000013011 mating Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G06T3/12—
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of geographic scenes panorama sketch rapid generation based on video sequence image, step are：1) grid partition is carried out to video frame and obtains subimage block, and establish the one-to-one relationship between each subimage block of successive video frames；2) critical point detection and filtering are carried out to each subgraph with multi-threaded parallel strategy, constructs point set to be matched；Matching search is carried out between each association subgraph, extracts the matching double points between each subgraph；3) merge the matching double points between each subgraph, and thus calculate the geometry splicing parameter of each video frame；4) video sequence is spliced according to splicing parameter, obtains a width panorama sketch；5) it constructs global spin matrix and the bending panorama sketch that shake generates is shot with modifying factor.The beneficial effects of the invention are as follows：Video sequence suitable for shooting common camera to geographical scene rotation is spliced into the panoramic picture of no deformation, and uses multi-threading parallel process, and splicing speed is fast.
Description
Technical field
The present invention relates to the fields such as geographical information visualization, Digital Image Processing, computer vision more particularly to a kind of bases
In the method that video sequence image quickly generates geographic scenes panorama sketch.
Background technique
Video sequence due to have the characteristics that time-space attribute, contain much information, high resolution, expression it is intuitive, fastly short-term training
For generalization, socialization geography information source and modeling analysis and expression means.Panoramic picture makes people's quick obtaining all
The location information enclosed, therefore it will be spliced into the bigger panoramic picture of field range around 360 ° of video sequences of shooting, it can not only
The integrality for expressing geographic scenes in all directions, makes people that the entire content of video sequence be understood more intuitively, can be reduced simultaneously
The space-time redundancy of video sequence.
Some commercial companies have developed Related product, such as the Google Street View that Google company develops, lead to
It crosses using special device, and the mm professional camera special demarcated shoots the multi-angle video in street, and is spliced into panorama
Image；The Ladybug multi-angle video filming apparatus of Grey Point company exploitation, is equally fixed and has been demarcated using position
Mm professional camera special, the SDK of the mating offer of company, which can splice filmed video sequence, generates panoramic picture.But these business
The price for the professional video panorama that company is developed is too high, and being unfavorable for ordinary user will be around 360 ° of video sequence of shooting
Column are spliced into panoramic picture.
Summary of the invention
The invention proposes a kind of geographic scenes panorama sketch rapid generation based on video sequence image, this method are adopted
Grid piecemeal is carried out to video sequence with multi-threaded parallel mode, establishes the local searching strategy of key point under piecemeal to control
With region of search, in the matching process, matching search efficiency is improved by reducing the number of key point, to realize video sequence
The rapid registerings of column and panoramic picture it is seamless spliced, reduce the time of video sequence splicing, improve splicing efficiency, especially
Being spliced into panoramic picture to high-resolution video sequence has good effect, and the video sequence for adapting to shoot in ordinary user is spelled
It is connected into panoramic picture.
To achieve the goals above, the present invention adopts the following technical scheme that：
Geographic scenes panorama sketch rapid generation based on video sequence image, includes the following steps：
1) grid partition is carried out into different subgraphs to video frame images, and establishes each subgraph between successive video frames
The one-to-one relationship of block；
2) critical point detection is carried out to each subgraph using piecemeal multi-threaded parallel strategy, and the key point of detection is carried out
Filtering, retains a small amount of key point as point to be matched；Matching search is established between each association subgraph, is extracted between each subgraph
Matching double points；
3) matching double points between each subgraph are merged, calculates each video frame using the matching point set after merging
Geometry splices parameter；
4) splice parameter using the geometry of each video frame, each frame image of video sequence is spliced, it is complete to obtain a width
Scape image；
5) it is the shake for avoiding video camera from generating in shooting process, constructs global spin matrix to correct curved panorama
Figure, makes the panorama sketch finally exported more meet the expression of objective world.
Detailed process is as follows for the step 2)：
(2-1) carries out critical point detection to each subgraph using piecemeal multi-threaded parallel mode；
(2-2) calculates its local message entropy to all key points, and carries out entropy filtering, retains the biggish preceding n of entropy
A key point is used for characteristic matching；
(2-3) is extracted between each association subgraph and is effectively matched a little pair, to ensure the matched accuracy of sub-block, using symmetrical
Matching process match between subgraph；
(2-4) can have Mismatching point in matching set, finally use RANSAC since noise and mobile target exist
(Random Sample Consensus, random sampling consistency) method rejects Mismatching point；
(2-5) establishes the matching corresponding relationship of subgraph pair to the filtered key point of rejecting between each association subgraph,
Extraction is effectively matched a little pair.
Detailed process is as follows for the step 3)：
(3-1) merges the matching double points between each subgraph of extraction, as the matching pair between two images；
(3-2) according to matching pair, homography matrix is converted between calculating image, estimates the rotation transformation of camera parameters and image
With translation.
Detailed process is as follows for the step 4)：
(4-1) utilizes the geometry between the obtained image of step 3) to splice parameter, carries out image to all video frame images
Registration；
(4-2) in the image projection to cylindrical surface after registration, will eliminate the tone difference between image by image co-registration,
Finally export revised panoramic picture.
Compared with the prior art, method of the present invention has following features：
1, piecemeal is carried out to video frame images, and establishes the one-to-one relationship between subgraph, strict control correspondence image
Between matching region of search, and accelerate using image block multi-threaded parallel strategy to match search speed；
2, by constructing global spin matrix, wavy panoramic picture is corrected, this method is suitble to the view of ordinary user's shooting
Frequency is spliced into panoramic picture.
Therefore, the present invention adapts to carry out the spelling of rotary taking video sequence to geographic scenes in the common hand-held camera of use
It is connected into panoramic picture, splicing speed is fast, is handled the shake occurred in shooting process, and spliced panorama sketch is avoided to produce
Change shape, is a kind of general panorama mosaic method for being adapted to arbitrary geographic scene.
Detailed description of the invention
Fig. 1 is the flow chart of the method for the present invention；
Fig. 2 is the image block and multi-threaded parallel matching strategy of the embodiment of the present invention；
Fig. 3 is the wavy panoramic picture of the embodiment of the present invention；
Fig. 4 is the cylindrical surface projecting of the embodiment of the present invention；
Fig. 5 is the revised panoramic picture of the embodiment of the present invention.
Specific embodiment
It is described in further detail with reference to the accompanying drawings and examples.
Such as
Shown in Fig. 1, the geographic scenes panorama sketch rapid generation based on video sequence image, this method includes following four
A part：
The image of the video sequence of step 1 pair input carries out grid piecemeal, after establishing the corresponding relationship between subgraph, uses
Multi-threaded parallel optimisation strategy carries out feature extracting and matching to subgraph；
Step 2 merges subgraph matching set；
Step 3 is according to matching characteristic point to calculating image mosaic geometric parameter；
The fusion of step 4 video sequence image；
The specific implementation step of the present embodiment is as follows：
The image of the video sequence of step 1 pair input carries out grid piecemeal, carries out after establishing the corresponding relationship between subgraph
The feature extracting and matching of subgraph；
(1-1) reads two continuous frames image A and B in video sequence frame by frame, and the grid piecemeal of M × N is carried out to A and B.
(1-2) carries out key point inspection using ORB algorithm to each subgraph using multi-threaded parallel strategy (as shown in Figure 2)
It surveys；
(1-3) calculates the arrangement of its local message entropy descending to all key points, and the biggish key point of n entropy is selected to use
In Image Feature Matching.Calculation method is as follows：
(1) interest region is set for each key point, be usually arranged as centered on key point 3 × 3 or 5 × 5 is adjacent
Domain sets 3 × 3 for contiguous range to consider calculating speed.
(2) local entropy of key point in 3 × 3 neighborhoods is calculated.If the gamma function of image block is f (x, y), and f (x, y) >=
0, then have：
Wherein：pijIt is pixel in image block i, the probability of the position j.
(4) 2~3 are repeated until the local entropy of each key point has been calculated.
(1-4) carries out characteristic matching to subgraph using symmetrical matching process.
(1-5) can have Mismatching point in matching set since noise and mobile target exist, according to polar curve principle, two
A corresponding key point is distributed on respective polar curve, and expression formula is as follows：
Wherein, u1And u2For two image Corresponding matching point coordinates；F12For basic matrix；According to formula (3), using the side RANSAC
Method rejects Mismatching point.
Step 2 repeats the process of step 1, until each pair of subgraph is to having handled, and by the institute in set of sub-images
There is matching set of the Match merging as whole image pair.
Step 3 is according to the images match point pair of extraction, and homography matrix converts between calculating image, estimates camera parameters and figure
The rotation transformation and translation of picture.
(1) image homography matrix converts
Two images n (>4) to match pointIt can be calculated under the meaning for differing an invariant and singly answer square
Battle array, algorithm are as follows：
It enablesIts vector form：H=(h01,h02,h03,h11,h12,h13,h21,h22,h23)T.It is right
In matching double points m=(x, y, 1)T, m'=(x', y', 1)T, from following two linear equations about h can be obtained：
(x, y, 1,0,0,0, x'x, x'y) h=x'
(4)
(0,0,0, x, y, 1, y'x, y'y) h=y'
Homography matrix H can be solved by choosing 4 pairs or more of match point in this way.
(2) focal length of camera is estimated
The homography matrix H in video sequence between consecutive image pair is calculated firstij, defined according to plane homography, H
=K (R+tnT)K-1, have：
(6) formula is rewritten as：
Wherein,Utilize spin matrix R10Orthogonality,
It is available：
It can be calculated by (8) and (9) two formulas：
And
Or：And h00h10≠-h01h11 (11)
f1Equally find out by a similar method, due to it is assumed herein that video camera focal length remains unchanged in shooting process,
Then the final estimation of focal length of camera f can pass through f0And f1Geometric average calculate,In panoramic image sequence,
Respective focal length estimation is obtained from different homography matrixs, final f should take the geometric average of all focal lengths.Last basis is asked
Internal reference matrix K out seeks the rotational transformation matrix R between two images.
(3) global spin matrix calculates
Spin matrix between the image calculated according to homography conversion between image is relative rotation matrices, is caused using above-mentioned
The image of method splicing is at wavy or skewed, as shown in Figure 3.In order to make the image finally spliced be " straight ", need
Establish global rotating coordinate system.
By 3d space point xiThe point x being mapped in two dimensional imageik, use following expression：
xik~KkRkxi (12)
Consider to may move and inclined camera in the shooting process of image, but can generally keep camera horizontal edge (x
Axis) it is parallel to ground level (vertical with y-axis direction), therefore it is y-axis that the world coordinate system of selection, which is vertically corresponding, level is x
Axis, third axis are z-axis along optical axis direction.
In order to establish global rotating coordinate system, matrix R need to be constructedg, so that RgThe right side multiplies RkIt can guarantee global y-axis j=later
(0,1,0) perpendicular to image x-axis i=(1,0,0), image z-axis is expressed as k=(0,0,1).
iTRkRgJ=0 (13)
Expression formula (13) is equivalent to require matrix RkThe first row rk0=iTRkWith matrix RgSecondary series rg1=RgJ is mutual
Vertically.The constraint of all images constitutes a set, and is expressed as least square problem.
In order to determine global spin matrix R completelyg, it is also necessary to an additional constraint condition, i.e., single spin matrix
Z-axis average valueR is met with the z-axis of global spin matrixg2=Rgk。
In the following, complete spin matrix can be calculated by 3 steps：
1.
2.rg0=N ((∑ Rk2)×rg1)
3.rg2=rg0×rg1
Wherein, N ((∑ Rk2)×rg1) be vector normalization indicate.
Pass through global spin matrix RglobalThe local spin matrix of transformation every image of amendmentWherein i representative image
Number：
(4) cylinder environment map constructs
The present invention uses the projecting method based on cylindrical coordinates, projects image on cylindrical surface.
Point on cylinder is determined by angle, θ and height h parametrization, as shown in figure 4, corresponding relationship is as follows：
(sinθ,h,cosθ)∝(x,y,f) (16)
According to (16) formula corresponding relationship, the coordinate form that cylindrical surface is mapped as from the plane of delineation can be calculated：
Wherein, (x, y) is flat image coordinate, and (x ', y ') is cylindrical coordinates, and s is the radius of cylinder, and general s=f is with most
The deformation extent of smallization picture centre.
Since cylinder is developable surface, image, which is translated and rotated under cylindrical coordinates, can keep shape invariance.Inverse mapping is public
Formula is expressed as follows：
Step 4 video sequence image fusion, using what is generated after laplacian pyramid fusion method processing image mosaic
Tone difference phenomenon.It is as shown in Figure 5 by fused panoramic picture.
Claims (3)
1. the geographic scenes panorama sketch rapid generation based on video sequence image, which is characterized in that include the following steps：
1) grid partition is carried out into different subgraphs to video frame images, and establishes each subimage block between successive video frames
One-to-one relationship；
2) critical point detection is carried out to each subgraph using piecemeal multi-threaded parallel strategy, and the key point of detection was carried out
Filter, retains a small amount of key point as point to be matched；Matching search is established between each association subgraph, extracts between each subgraph
With point pair；Detailed process is as follows：
(2-1) carries out critical point detection to each subgraph using piecemeal multi-threaded parallel mode；
(2-2) calculates its local message entropy to all key points, and carries out entropy filtering, and it is preceding to retain entropy size ranking
N key point is used for characteristic matching；
(2-3) is extracted between each association subgraph and is effectively matched a little pair, to ensure the matched accuracy of sub-block, using symmetrically matching
Method match between subgraph；
(2-4) can have Mismatching point in matching set, finally be picked using RANSAC method since noise and mobile target exist
Except Mismatching point；
(2-5) establishes the matching corresponding relationship of subgraph pair to the filtered key point of rejecting, extracts between each association subgraph
It is effectively matched a little pair；
3) matching double points between each subgraph are merged, the geometry of each video frame is calculated using the matching point set after merging
Splice parameter；
4) splice parameter using the geometry of each video frame, each frame image of video sequence is spliced, a width panorama sketch is obtained
Picture；
5) it is the shake that avoids video camera from generating in shooting process, constructs global spin matrix to correct curved panorama sketch,
The panorama sketch finally exported is set more to meet the expression of objective world.
2. the geographic scenes panorama sketch rapid generation according to claim 1 based on video sequence image, feature
It is, detailed process is as follows for the step 3)：
(3-1) merges the matching double points between each subgraph of extraction, as the matching pair between two images；
(3-2) according to matching pair, homography matrix is converted between calculating image, estimates the rotation transformation of camera parameters and image and flat
It moves.
3. the geographic scenes panorama sketch rapid generation according to claim 1 based on video sequence image, feature
It is, detailed process is as follows for the step 4)：
(4-1) utilizes the geometry between the obtained image of step 3) to splice parameter, carries out image registration to all video frame images；
(4-2) in the image projection to cylindrical surface after registration, will eliminate the tone difference between image by image co-registration, finally
Export revised panoramic picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510386060.6A CN105005964B (en) | 2015-06-30 | 2015-06-30 | Geographic scenes panorama sketch rapid generation based on video sequence image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510386060.6A CN105005964B (en) | 2015-06-30 | 2015-06-30 | Geographic scenes panorama sketch rapid generation based on video sequence image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105005964A CN105005964A (en) | 2015-10-28 |
| CN105005964B true CN105005964B (en) | 2018-11-16 |
Family
ID=54378623
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510386060.6A Expired - Fee Related CN105005964B (en) | 2015-06-30 | 2015-06-30 | Geographic scenes panorama sketch rapid generation based on video sequence image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105005964B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10282814B2 (en) * | 2016-01-07 | 2019-05-07 | Mediatek Inc. | Method and apparatus of image formation and compression of cubic images for 360 degree panorama display |
| KR20170124814A (en) | 2016-05-03 | 2017-11-13 | 삼성전자주식회사 | Image display apparatus and operating method for the same |
| CN107767336A (en) * | 2016-08-19 | 2018-03-06 | 中国移动通信有限公司研究院 | A kind of method and device of image procossing |
| CN106651999B (en) * | 2016-12-27 | 2020-12-04 | 北京儒博科技有限公司 | Frame animation loading acceleration method and device |
| CN108668138B (en) * | 2017-03-28 | 2021-01-29 | 华为技术有限公司 | Video downloading method and user terminal |
| CN110049226B (en) * | 2018-01-16 | 2021-01-29 | 飞狐信息技术（天津）有限公司 | Method and system for shooting panoramic picture material |
| CN110149542B (en) * | 2018-02-13 | 2021-12-03 | 华为技术有限公司 | Transmission control method |
| CN109688329B (en) * | 2018-12-24 | 2020-12-11 | 天津天地伟业信息系统集成有限公司 | Anti-shake method for high-precision panoramic video |
| CN111586414B (en) * | 2020-04-07 | 2022-04-15 | 南京师范大学 | SVC and DASH-based 360-degree video stream scheduling method |
| CN111526302A (en) * | 2020-04-28 | 2020-08-11 | 飞友科技有限公司 | Stackable panoramic video real-time splicing method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6504569B1 (en) * | 1998-04-22 | 2003-01-07 | Grass Valley (U.S.), Inc. | 2-D extended image generation from 3-D data extracted from a video sequence |
| CN101877140A (en) * | 2009-12-18 | 2010-11-03 | 北京邮电大学 | Panorama-based panoramic virtual tour method |
| CN101882308A (en) * | 2010-07-02 | 2010-11-10 | 上海交通大学 | Method for improving accuracy and stability of image mosaic |
| CN102426705A (en) * | 2011-09-30 | 2012-04-25 | 北京航空航天大学 | Behavior splicing method of video scene |
| WO2013079660A1 (en) * | 2011-11-30 | 2013-06-06 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Disparity map generation including reliability estimation |
-
2015
- 2015-06-30 CN CN201510386060.6A patent/CN105005964B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6504569B1 (en) * | 1998-04-22 | 2003-01-07 | Grass Valley (U.S.), Inc. | 2-D extended image generation from 3-D data extracted from a video sequence |
| CN101877140A (en) * | 2009-12-18 | 2010-11-03 | 北京邮电大学 | Panorama-based panoramic virtual tour method |
| CN101882308A (en) * | 2010-07-02 | 2010-11-10 | 上海交通大学 | Method for improving accuracy and stability of image mosaic |
| CN102426705A (en) * | 2011-09-30 | 2012-04-25 | 北京航空航天大学 | Behavior splicing method of video scene |
| WO2013079660A1 (en) * | 2011-11-30 | 2013-06-06 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Disparity map generation including reliability estimation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105005964A (en) | 2015-10-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105005964B (en) | Geographic scenes panorama sketch rapid generation based on video sequence image | |
| JP7328366B2 (en) | Information processing method, positioning method and device, electronic device and storage medium | |
| CN110349251B (en) | Three-dimensional reconstruction method and device based on binocular camera | |
| CN105245841B (en) | A kind of panoramic video monitoring system based on CUDA | |
| JP6563609B2 (en) | Efficient canvas view generation from intermediate views | |
| CN109242961B (en) | Face modeling method and device, electronic equipment and computer readable medium | |
| KR101121034B1 (en) | System and method for obtaining camera parameters from multiple images and computer program products thereof | |
| WO2020206903A1 (en) | Image matching method and device, and computer readable storage medium | |
| CN107767339B (en) | Binocular stereo image splicing method | |
| US8660309B2 (en) | Image processing apparatus, image processing method, image processing program and recording medium | |
| CN112085659B (en) | Panorama splicing and fusing method and system based on dome camera and storage medium | |
| CN107665483B (en) | Calibration-free convenient monocular head fisheye image distortion correction method | |
| JP2007048290A (en) | Method for generating a composite image from a set of images | |
| CN106462943A (en) | Aligning panoramic imagery and aerial imagery | |
| CN103345736A (en) | Virtual viewpoint rendering method | |
| Mistry et al. | Image stitching using Harris feature detection | |
| JP5035195B2 (en) | Image generating apparatus and program | |
| JP6174104B2 (en) | Method, apparatus and system for generating indoor 2D plan view | |
| CN109242898B (en) | Three-dimensional modeling method and system based on image sequence | |
| Bastanlar et al. | Multi-view structure-from-motion for hybrid camera scenarios | |
| CN108109148A (en) | Image solid distribution method, mobile terminal | |
| JP2016114445A (en) | Three-dimensional position calculation device, program for the same, and cg composition apparatus | |
| Fu et al. | Image Stitching Techniques Applied to Plane or 3D Models: A Review | |
| CN113112532B (en) | Real-time registration method for multi-TOF camera system | |
| JP2010079506A (en) | Image generating apparatus, method, communication system, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20181116 Termination date: 20210630 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |