CN103049749A - Method for re-recognizing human body under grid shielding - Google Patents
Method for re-recognizing human body under grid shielding Download PDFInfo
- Publication number
- CN103049749A CN103049749A CN2012105929180A CN201210592918A CN103049749A CN 103049749 A CN103049749 A CN 103049749A CN 2012105929180 A CN2012105929180 A CN 2012105929180A CN 201210592918 A CN201210592918 A CN 201210592918A CN 103049749 A CN103049749 A CN 103049749A
- Authority
- CN
- China
- Prior art keywords
- human body
- image
- vector
- body image
- zone
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a method for re-recognizing a human body under grid shielding. The method comprises the steps of: detecting a human body image in a video image; dividing the human body image into a plurality of areas; removing areas with grid barriers in the plurality of areas obtained after division; determining a feature vector of each area, and matching the plurality of feature vectors with a plurality of reference vectors in a pre-acquired database; and taking the successfully matched human body image in the database as a recognition result. According to the steps, the shielding areas formed by the barriers are eliminated in a detection process, the human body image can be accurately determined in the database, and a label or an ID (Identifier) of the determined human body image is taken as the detected human body image. Therefore, the movement range of each human body image in a video can be mastered.
Description
Technical field
The present invention relates to field of video monitoring, in particular to the again recognition methods of a kind of human body under grid blocks.
Background technology
At present to the human body recognition technology in the video image, in identifying, because environment on every side causes blocking human body, cause affecting recognition result, for example, there are the shelters such as fence in the security protection zone of shooting, forms blocking such as lattice-shaped at the human body image of taking.Existing video identification technology, can only identify the human body image in the video, can not confirm the individuality of human body image, under the above-mentioned environment, the image of taking can cause again the grid shelter to human body image, cause different recognition results, thereby cause to distinguish the motion track of everyone volume image, can not determine the identity of the human body image in the current video.
Summary of the invention
The present invention aims to provide the again recognition methods of a kind of human body under grid blocks, the problem that must not confirm the individuality of human body image with solution.
In an embodiment of the present invention, provide the again recognition methods of a kind of human body under grid blocks, having comprised: detect the human body image in the video image; Described human body image is divided into a plurality of zones; In a plurality of zones after will cutting apart, remove the zone at grid barrier place; Determine the proper vector in each described zone, a plurality of proper vectors and a plurality of reference vector in the database that gathers are in advance mated; With the human body image that the match is successful in the described database as recognition result.
By above-mentioned step, can in database, determine human body image, with the human body image identity the determined identity as the human body image that detects.Thereby can in video, grasp people corresponding to everyone volume image scope of activities.Owing to having eliminated the occlusion area that barrier forms in the testing process, having increased the accuracy rate of identification.
Description of drawings
Accompanying drawing described herein is used to provide a further understanding of the present invention, consists of the application's a part, and illustrative examples of the present invention and explanation thereof are used for explaining the present invention, do not consist of improper restriction of the present invention.In the accompanying drawings:
Fig. 1 shows the process flow diagram of embodiment;
Embodiment
Below with reference to the accompanying drawings and in conjunction with the embodiments, describe the present invention in detail.Referring to Fig. 1, the step of embodiment comprises:
S11: detect the human body image in the video image;
S12: described human body image is divided into a plurality of zones;
S13: in a plurality of zones after will cutting apart, remove the zone at grid barrier place;
S14: determine the proper vector in each described zone, a plurality of proper vectors and a plurality of reference vector in the database that gathers are in advance mated;
S15: with the human body image that the match is successful in the described database as recognition result.
By above-mentioned step, can in database, determine human body image, with the human body image identity the determined identity as the human body image that detects.Thereby can in video, grasp people corresponding to everyone volume image and scope of activities.Owing to having eliminated the occlusion area that barrier forms in the testing process, having increased the accuracy rate of identification.
Preferably, among the embodiment, the step of human body image comprises: use the Gaussian Background modeling to detect the moving region in video.In order to eliminate noise, use corrosion and expansion algorithm that the foreground picture that detects is filtered.The foreground picture region is drawn a circle to approve out, as the scope of human detection.
In the moving region that detects, use based on histograms of oriented gradients (HOG) with the object detecting method of the support vector machine (latent SVM) of implicit parameter, on different scale, the human body image in the video is detected.
Preferably, among the embodiment, during to Image Segmentation Using, can adopt watershed algorithm to Image Segmentation Using.In image, choose gray-scale value and be the point of local minimum as the seed of watershed algorithm, the half-tone information of image is used watershed algorithm, be different zones with picture segmentation.
The formula of the gray scale of calculating pixel point is as follows: Y=0.2999R+0.5870G+0.1140B
The watershed algorithm split image: watershed algorithm is the half-tone information according to image, image is carried out a kind of method of Region Segmentation.At first all pixels in the image being sorted from small to large according to gray-scale value, is that the point of local minimum is as Seed Points with gray-scale value.Make up the zone in each Seed Points position.Process one by one each pixel according to gray-scale value order from small to large afterwards, among the zone that processed pixel adding is adjacent with it.After all pixels all are added into the zone, just obtain the carve information of image.The zone of cutting apart is generally the upper part of the body image of human body image, lower part of the body image and head, even foot etc. can also be arranged.
Adopt the watershed algorithm specific implementation as follows:
M1M2 ... the set of the coordinate of local minizing point presentation video g(x, y .MR).R is positive integer.
The set of point in the catchment basin that C (Mi) expression and local minimum Mi interrelate.
T[n]=(s, t) | g (s, t)＜n} represents to be positioned at plane g(x, y)=set of the point of n below.S, t are coordinate points.
Cn (Mi)=C (Mi) ∩ T[n] represent that n stage catchment basin there is not the set of part by water logging.Mi=M1~MR
Q represents T[n] in the continuously set of component.Each continuous component q ∈ there are three kinds of possibilities
(a) q ∩ C[n-1] be empty
(b) q ∩ C[n-1] comprise C[n-1] in a connected component.
(c) q ∩ C[n-1] comprise C[n-1] more than a connected component.
When running into new minimum value, eligible (a) incorporates q into c[n-1], consist of c[n];
When Q was positioned at the catchment basin of some local minimum formation, eligible (b) incorporated q into c[n-1] formation c[n], when running into all or part of catchment basin of separation, eligible (c) sets up dam at q.Dam is the separatrix, edge of the image of two different colours.
End condition is n=max+1.The color interval of max pixel, for example: in gray scale, 255 for the highest.
Preferably, the image after cutting apart is eliminated over-segmentation: obtain calculating each regional average gray after the image segmentation information, the average gray in adjacent zone is compared, when difference is no more than threshold value 5, two zones are merged into one.
Preferably, among the embodiment, determine that the process of proper vector comprises:
Be the HSV form with the image transitions that detects, and extract the color distribution histogram.
Conversion from the RGB color space to the hsv color space, computing formula is as follows:
v＝max
Max=max (r, g, b) wherein, min=min (r, g, b).Such as, be the pixel of (0.1,0.2,0.5) for the RGB color value, the value in the hsv color space is (225,0.8,0.5).
Calculate color histogram:, its color is added up for each pixel in the image.For example, the v component is black less than threshold value 1, and the v component is white greater than threshold value 2 and s component less than threshold value 3, the v component between threshold value 1 and threshold value 2 and the v component be grey less than threshold value 3, other colors are colour.
For colour, evenly be divided into 6 kinds of colors according to h component from 0 to 360, namely [0,60), [60,120), [120,180), [180,240), [240,300), [300,360).
Color to each pixel is added up, and calculates every kind of color proportion in each zone of human body image, stores into successively among the array x, uses as the Characteristic of Image vector.
For example, in an image-region 10 pixels are arranged.Wherein black color dots and white point respectively have 3, other 4 points belong to color [60,120), this regional characteristic of correspondence vector is (0.3,0.3,0,0,0.4,0,0,0,0) so.
Preferably, among the embodiment, the reference vector in the described database is determined by following steps:
Gather in advance several video images of everyone volume image;
By described several video images, determine a plurality of zones of this human body image and the stack features vector corresponding with each zone, as the reference vector of this zone correspondence.
For grid form to the blocking of human body image, the feature of grid can be set in program in advance, the barrier for grid shape is characterized by porous.Therefore, only need to cut apart the zone that numbers of hole among the good zone surpasses threshold value and exclude, remaining zone is human region.For example: outdoor grid fence, the fence in roadside etc.
This process can be by the method realization of cluster, for example: use K-means (K average) scheduling algorithm.
When using the K-means training, the color histogram in everyone the volume image zone that obtains in the testing process is carried out cluster as proper vector, obtain the cluster centre of proper vector, and the area sample that comprises of each cluster centre.
The K mean algorithm need to be inputted a parameter k, and several proper vectors.These proper vectors can be divided into k class and the sample that comprises in each class by the calculating of K mean algorithm.Like this, just the sample of input can be divided into the k class, each class represents a human body image.
The regional characteristic of correspondence vector that cluster centre is obtained each class stores in the database.
Above-mentioned matching process comprises:
Each proper vector corresponding to each described zone of computing respectively with described database in the distance of reference vector of regional of everyone volume image;
To a plurality of distance-taxis that each proper vector obtains, determine two minimums apart from d1 and d2; Wherein, d1＜d2;
If described 1.5d1＜d2 determines that then this proper vector and the reference vector that is used for the described d1 of computing are complementary.
Determine the human body image corresponding with the region of the nearest reference vector of each described proper vector, and the summation of the number of times that mated of the reference vector of the regional of everyone volume image of statistics correspondence;
Find out label or the ID of the human body image of the unique and the highest value of the number of times summation that is determined, as the described human body image that the match is successful.
Wherein, be used for calculating the reference vector of minimum Euclidean distance as the highest reference vector of distance.The formula of Euclidean distance is as follows:
Wherein d is the distance of proper vector and reference vector, and x is the Characteristic of Image vector, the reference vector that X obtains for training, and the figure place of i representation feature vector or reference vector, N is the dimension of proper vector or reference vector.
Suppose that human body image is divided into some zones, wherein i zone is identified as block p
i, come from s in the database
iIndividual human body image.To s
iCarry out statistics with histogram, and whole human body image is classified as the corresponding model of component maximum in the histogram.
For example: if one have 5 human body image samples, a plurality of reference vector of a plurality of zone institute difference correspondence that each sample comprises by a human body image.
The human body image that detects is divided into 3 zones, totally 3 proper vectors; Comprise 5 human body image samples in the database, each sample comprises 3 zones, then has 15 zones, the corresponding reference vector in each zone.Each proper vector that calculating detects and the distance of 15 reference vector obtain 5 groups of data.
Comprise 15 distances in every group of data, find two minimum distances, d1 and d2, and satisfy 1.5d1＜d2, then think to match reference vector.
Add up the number of times that each reference vector of everyone volume image is mated.For example: detect certain regional proper vector and be (1,0,0,0,0,0,0,0,0), two nearest reference vector are respectively (0.8,0,0,0,0,0,0,0,0.2) and (0.5,0.5,0,0,0,0,0,0,0) with it.Then can calculate d
1≈ 0.283, d
2≈ 0.707, and 1.5d
1＜d
2Determine that this proper vector and the reference vector that is used for the described d1 of computing are complementary.The reference vector that is used for the described d1 of computing is the human body image of sample 1, and then the human body image of sample 1 is the successful human body image of identification.
Regional is identified as respectively the zone in following each sample; As: sample 1, sample 1, sample 2, then statistic histogram is (2,1), and sample 1 is high and the most unique sample, and the human body image that is detected finally is identified as sample 1 corresponding human body image again.
In addition, in order to realize exact matching, the human body image that identifies and the image of sample are extracted the ORB unique point, use the hamming distance that unique point is mated, and use RANSAC algorithm eliminating error coupling.Determine whether that according to final matching result the match is successful.
Obviously, those skilled in the art should be understood that, above-mentioned each module of the present invention or each step can realize with general calculation element, they can concentrate on the single calculation element, perhaps be distributed on the network that a plurality of calculation elements form, alternatively, they can be realized with the executable program code of calculation element, thereby, they can be stored in the memory storage and be carried out by calculation element, perhaps they are made into respectively each integrated circuit modules, perhaps a plurality of modules in them or step are made into the single integrated circuit module and realize.Like this, the present invention is not restricted to any specific hardware and software combination.
The above is the preferred embodiments of the present invention only, is not limited to the present invention, and for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any modification of doing, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (9)
1. an again recognition methods of the human body under grid blocks is characterized in that, comprising:
Detect the human body image in the video image;
Described human body image is divided into a plurality of zones;
In a plurality of zones after will cutting apart, remove the zone at grid barrier place;
Determine the proper vector in each described zone, a plurality of proper vectors and a plurality of reference vector in the database that gathers are in advance mated;
With the human body image that the match is successful in the described database as recognition result.
2. method according to claim 1 is characterized in that, described cutting procedure comprises:
Local minimum adopts watershed algorithm to be divided into a plurality of zones as seed in the selection human body image.
3. method according to claim 2 is characterized in that, also comprises:
The color gray scale of adjacent area relatively when difference during less than threshold value, merges described adjacent area.
4. method according to claim 2 is characterized in that, the process of described definite proper vector comprises:
Be the HSV form with the image transitions in described zone;
Add up the interior versicolor pixel quantity in zone of described HSV form;
Determine the stack features vector corresponding with this zone according to described versicolor pixel quantity.
5. method according to claim 4 is characterized in that, the reference vector in the described database is determined by following steps:
Gather in advance several video images of everyone volume image;
By described several video images, determine a plurality of zones of everyone volume image and the stack features vector corresponding with each zone, as the reference vector of this zone correspondence;
Described matching process comprises:
Each proper vector corresponding to each described zone of computing respectively with described database in the distance of reference vector of regional of everyone volume image;
To a plurality of distance-taxis that each proper vector obtains, determine two minimums apart from d1 and d2; Wherein, d1＜d2;
If described 1.5d1＜d2 determines that then this proper vector and the reference vector that is used for the described d1 of computing are complementary.
6. method according to claim 5 is characterized in that,
Determine the human body image corresponding with the region of the nearest reference vector of each described proper vector, and the summation of the number of times that mated of the reference vector of the regional of everyone volume image of statistics correspondence;
Find out label or the ID of the human body image of the unique and the highest value of the number of times summation that is determined, as the described human body image that the match is successful.
7. method according to claim 5 is characterized in that, also comprises: adopt the described distance of following Euclidean distance formula operation;
Wherein d is the distance of proper vector and reference vector, and x is the Characteristic of Image vector, the reference vector that X obtains for training, and the figure place of i representation feature vector or reference vector, N is the dimension of proper vector or reference vector.
8. method according to claim 6 is characterized in that, also comprises:
The match is successful if do not have, and then the proper vector with the regional of described detected human body image joins described database as new reference vector.
9. method according to claim 1 is characterized in that, also comprises:
Current frame image and before video image in, adopt and minimum to state this human body image that detects with color receptacle frame residence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210592918.0A CN103049749B (en) | 2012-12-30 | 2012-12-30 | The recognition methods again of human body under grid blocks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210592918.0A CN103049749B (en) | 2012-12-30 | 2012-12-30 | The recognition methods again of human body under grid blocks |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103049749A true CN103049749A (en) | 2013-04-17 |
| CN103049749B CN103049749B (en) | 2016-06-29 |
Family
ID=48062381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210592918.0A Expired - Fee Related CN103049749B (en) | 2012-12-30 | 2012-12-30 | The recognition methods again of human body under grid blocks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103049749B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103729462B (en) * | 2014-01-13 | 2016-09-14 | 武汉大学 | A kind of pedestrian retrieval method blocked based on rarefaction representation process |
| WO2018058531A1 (en) * | 2016-09-30 | 2018-04-05 | 富士通株式会社 | Target tracking method and device, and image processing apparatus |
| CN108154171A (en) * | 2017-12-20 | 2018-06-12 | 北京奇艺世纪科技有限公司 | A kind of character recognition method, device and electronic equipment |
| CN111077990A (en) * | 2019-06-03 | 2020-04-28 | 广东小天才科技有限公司 | Method for determining content to be read and learning equipment |
| CN111339996A (en) * | 2020-03-20 | 2020-06-26 | 北京百度网讯科技有限公司 | Method, device and equipment for detecting static obstacle and storage medium |
| CN111832361A (en) * | 2019-04-19 | 2020-10-27 | 杭州海康威视数字技术股份有限公司 | Pedestrian re-identification method and device and computer equipment |
| CN114549921A (en) * | 2021-12-30 | 2022-05-27 | 浙江大华技术股份有限公司 | Object recognition method, electronic device, and computer-readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070122009A1 (en) * | 2005-11-26 | 2007-05-31 | Hyung Keun Jee | Face recognition method and apparatus |
| CN101311947A (en) * | 2008-06-12 | 2008-11-26 | 浙江大学 | Real time intelligent control method based on natural video frequency |
| CN101325690A (en) * | 2007-06-12 | 2008-12-17 | 上海正电科技发展有限公司 | Method and system for detecting human flow analysis and crowd accumulation process of monitoring video flow |
| CN101329725A (en) * | 2008-07-30 | 2008-12-24 | 电子科技大学 | Method for dividing fingerprint image based on gradient projection and morphology |
| KR100904916B1 (en) * | 2008-06-19 | 2009-07-01 | 주식회사 다우엑실리콘 | System and method for recognition of face |
| CN101577052A (en) * | 2009-05-14 | 2009-11-11 | 中国科学技术大学 | Device and method for detecting vehicles by overlooking |
| CN101710422A (en) * | 2009-12-11 | 2010-05-19 | 西安电子科技大学 | Image segmentation method based on overall manifold prototype clustering algorithm and watershed algorithm |
-
2012
- 2012-12-30 CN CN201210592918.0A patent/CN103049749B/en not_active Expired - Fee Related
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070122009A1 (en) * | 2005-11-26 | 2007-05-31 | Hyung Keun Jee | Face recognition method and apparatus |
| CN101325690A (en) * | 2007-06-12 | 2008-12-17 | 上海正电科技发展有限公司 | Method and system for detecting human flow analysis and crowd accumulation process of monitoring video flow |
| CN101311947A (en) * | 2008-06-12 | 2008-11-26 | 浙江大学 | Real time intelligent control method based on natural video frequency |
| KR100904916B1 (en) * | 2008-06-19 | 2009-07-01 | 주식회사 다우엑실리콘 | System and method for recognition of face |
| CN101329725A (en) * | 2008-07-30 | 2008-12-24 | 电子科技大学 | Method for dividing fingerprint image based on gradient projection and morphology |
| CN101577052A (en) * | 2009-05-14 | 2009-11-11 | 中国科学技术大学 | Device and method for detecting vehicles by overlooking |
| CN101710422A (en) * | 2009-12-11 | 2010-05-19 | 西安电子科技大学 | Image segmentation method based on overall manifold prototype clustering algorithm and watershed algorithm |
Non-Patent Citations (2)
| Title |
|---|
| 任磊等: "基于改进分水岭的棉花图像分割方法", 《计算机工程与应用》, vol. 48, no. 34, 1 December 2012 (2012-12-01), pages 207 - 211 * |
| 罗佳: "基于SIFT的人脸识别研究", 《中国优秀硕士学位论文全文数据库信息科技辑（月刊）》, no. 05, 15 May 2011 (2011-05-15), pages 24 - 40 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103729462B (en) * | 2014-01-13 | 2016-09-14 | 武汉大学 | A kind of pedestrian retrieval method blocked based on rarefaction representation process |
| WO2018058531A1 (en) * | 2016-09-30 | 2018-04-05 | 富士通株式会社 | Target tracking method and device, and image processing apparatus |
| CN109478328A (en) * | 2016-09-30 | 2019-03-15 | 富士通株式会社 | Method for tracking target, device and image processing equipment |
| CN108154171A (en) * | 2017-12-20 | 2018-06-12 | 北京奇艺世纪科技有限公司 | A kind of character recognition method, device and electronic equipment |
| CN108154171B (en) * | 2017-12-20 | 2021-04-23 | 北京奇艺世纪科技有限公司 | Figure identification method and device and electronic equipment |
| CN111832361A (en) * | 2019-04-19 | 2020-10-27 | 杭州海康威视数字技术股份有限公司 | Pedestrian re-identification method and device and computer equipment |
| CN111832361B (en) * | 2019-04-19 | 2023-08-29 | 杭州海康威视数字技术股份有限公司 | Pedestrian re-identification method and device and computer equipment |
| CN111077990A (en) * | 2019-06-03 | 2020-04-28 | 广东小天才科技有限公司 | Method for determining content to be read and learning equipment |
| CN111077990B (en) * | 2019-06-03 | 2024-03-19 | 广东小天才科技有限公司 | Method for determining content to be read on spot and learning equipment |
| CN111339996A (en) * | 2020-03-20 | 2020-06-26 | 北京百度网讯科技有限公司 | Method, device and equipment for detecting static obstacle and storage medium |
| CN111339996B (en) * | 2020-03-20 | 2023-05-09 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for detecting static obstacle |
| CN114549921A (en) * | 2021-12-30 | 2022-05-27 | 浙江大华技术股份有限公司 | Object recognition method, electronic device, and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103049749B (en) | 2016-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103049749A (en) | Method for re-recognizing human body under grid shielding | |
| Ramírez-Alonso et al. | Auto-adaptive parallel SOM architecture with a modular analysis for dynamic object segmentation in videos | |
| CN104268583B (en) | Pedestrian re-recognition method and system based on color area features | |
| CN103065126A (en) | Re-identification method of different scenes on human body images | |
| CN101470809B (en) | Moving object detection method based on expansion mixed gauss model | |
| CN105184818B (en) | A kind of video monitoring anomaly detection method and its detecting system | |
| CN103093274B (en) | Method based on the people counting of video | |
| CN103246896B (en) | A kind of real-time detection and tracking method of robustness vehicle | |
| Alonso et al. | Robust vehicle detection through multidimensional classification for on board video based systems | |
| CN101777124A (en) | Method for extracting video text message and device thereof | |
| CN103473539A (en) | Gait recognition method and device | |
| CN102063613A (en) | People counting method and device based on head recognition | |
| CN102867188A (en) | Method for detecting seat state in meeting place based on cascade structure | |
| CN104978567A (en) | Vehicle detection method based on scenario classification | |
| CN109446989A (en) | Crowd massing detection method, device and storage medium | |
| CN104268520A (en) | Human motion recognition method based on depth movement trail | |
| CN103714181A (en) | Stratification specific figure search method | |
| Patel et al. | Automatic licenses plate recognition | |
| CN103390151A (en) | Face detection method and device | |
| Du et al. | Abnormal event detection in crowded scenes based on structural multi-scale motion interrelated patterns | |
| CN104966305A (en) | Foreground detection method based on motion vector division | |
| Al-Hmouz et al. | License plate localization using a statistical analysis of Discrete Fourier Transform signal | |
| CN105069816A (en) | Method and system for counting inflow and outflow people | |
| Mammeri et al. | North-American speed limit sign detection and recognition for smart cars | |
| Saidane et al. | Robust binarization for video text recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160629 Termination date: 20191230 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |