CN102800086B - Offshore scene significance detection method - Google Patents

Offshore scene significance detection method Download PDF

Info

Publication number
CN102800086B
CN102800086B CN201210207271.5A CN201210207271A CN102800086B CN 102800086 B CN102800086 B CN 102800086B CN 201210207271 A CN201210207271 A CN 201210207271A CN 102800086 B CN102800086 B CN 102800086B
Authority
CN
China
Prior art keywords
significantly
brightness
frame
offshore
offshore scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201210207271.5A
Other languages
Chinese (zh)
Other versions
CN102800086A (en
Inventor
任蕾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Maritime University
Original Assignee
Shanghai Maritime University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Maritime University filed Critical Shanghai Maritime University
Priority to CN201210207271.5A priority Critical patent/CN102800086B/en
Publication of CN102800086A publication Critical patent/CN102800086A/en
Application granted granted Critical
Publication of CN102800086B publication Critical patent/CN102800086B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Image Analysis (AREA)

Abstract

The invention discloses an offshore scene significance detection method. The method comprises the following steps of: 1, extracting an offshore scene image sequence; 2, transferring each frame image to the CIELab colour space, and extracting the characteristic pattern of luminance and colour passages; 3, using the absolute value of the difference between the extracted characteristics and the global mid value as the global significance map; 4, using the absolute value of the difference between the characteristics and the local mean value filtration as the local significance map; 5, combining the global significance map and the local significance map of the characteristics to obtain an overall significance map; 6, linearly combining the significance maps of the colour passages of frame images, and respectively combining the combined significance maps with the luminance significance maps to form an overall significance map; 7, accumulating by using each frame detection result as the centre and modifying the significance map of the current frame; and 8, converting the overall significance map into a binarization image to obtain an offshore scene significance target area. By adopting the method, the significance area in the offshore scene can be extracted rapidly, and the interference of sea noise wave can be favorably inhibited. The method is simple in implementation, and is suitable for real-time application.

Description

A kind of offshore scene significance detection method
Technical field
The present invention relates to the detection technique of a kind of machine vision and image processing field, be specifically related to the offshore scene significance detection method that a kind of utilization utilizes image procossing and machine vision technique.
Background technology
At present, domestic and international most visual attention computation model directly carries out conspicuousness detection in the spatial domain of image, extracts and significantly schemes.These class methods utilize the thinking of image spectrum relatively, without the need to carrying out the orthogonal transformation such as Fourier transform or discrete cosine transform to image.
Based on the conspicuousness detection method of spatial domain, its core how to define conspicuousness.Each class methods of current proposition mainly utilize the feature difference tolerance conspicuousness between spatial domain pixel, and common feature comprises brightness, color, direction, texture etc.Meanwhile, information theory, graph theory and bayesian theory etc. are introduced in the calculating of conspicuousness by a lot of scholar, in natural scene conspicuousness detects, obtain better effects.In order to solve the not high problem of frequency domain saliency detection method remarkable figure resolution, Achanta etc. propose frequency tuning conspicuousness detection method (the Frequency tuned method realized in former figure size, FrT), define each characteristic mean in image CIELab color space and be significance to the difference value after its gaussian filtering.The method is simple and easy to realize, and can extract more complete well-marked target, its internal consistency is good.
The scholar of Some Domestic, to based on ship detection problem in the offshore scene of visible images, applies vision noticing mechanism and has carried out preliminary discussion.Leaf is intelligent waits people to propose based on HIS(Hue, Saturation, Intensity) the ship detecting visual attention model in space, namely in HSI color space, adopt multiple dimensioned Difference Calculation to obtain each component characterization figure to three components, and then linear fusion acquisition significantly figure is carried out to it.The people such as Wu Qiying introduce vision noticing mechanism in moving target Real-Time Monitoring and tracker at sea, propose a kind of linear low-pass wave method based on the little template of inverted triangle of iteration, realize the smoothing denoising on coarse resolution image fast, highlight target with this.Wu Qiying etc. also propose the movement overseas target method for quick based on Detection Method in Optical Image Sequences, utilize visual attention model first segmented sense region-of-interest (ROI in still image, region of interest), so only in the region of interest application enhancements time differencing method detect moving target.
But there is limitation in these methods proposed.First, notice that these methods are all the detections for well-marked target in natural scene or land scene, therefore, target mostly is general objective; Meanwhile, because algorithm is more complicated, often needed to carry out down-sampling to original image before carrying out conspicuousness calculating.For large-sized well-marked target, it is too many that down-sampling can not cause target information to be lost, and therefore conspicuousness testing result is more satisfactory.But due to the singularity of offshore scene, namely naval target mostly is Small object, especially point target, and be scattered in offshore scene, therefore directly apply existing spatial domain conspicuousness detection method, its result is undesirable, especially not good enough to the Detection results of Small object, analyzing its essential reason, is because image down sampling causes Small object information dropout too much.Frequency tuning conspicuousness detection method realizes simple, but owing to only considering global contrast in this model as conspicuousness, directly applied to offshore scene, due to the feature of a large amount of sea clutter, all far away higher than characteristic mean, namely its global contrast and target are very close, cause outside testing result target is highlighted, further comprises a large amount of clutters.In addition, existing offshore scene visual attention model, has used for reference the visual attention computation model of Itti, has realized relative complex.
In sum, existing related work mainly solves the conspicuousness test problems in land scene or natural scene.Owing to there is a large amount of sea clutter in offshore scene and naval target mostly is Small object, existing methods effect is undesirable.For the defect of prior art, special proposition is a kind of utilizes new offshore scene significance detection method, to solve above-mentioned problem.
Summary of the invention
The invention provides a kind of offshore scene significance detection method, utilize the feature of offshore scene image self, realize extracting marking area at CIELab color space.
For achieving the above object, the invention provides a kind of offshore scene significance detection method, be characterized in, the method includes the steps of:
Step 1, extraction offshore scene image sequence;
Every two field picture of step 2, offshore scene image to CIELab color space, and extracts its brightness and two Color Channels as essential characteristic by RGB color space conversion, obtains the characteristic pattern of brightness and two Color Channels;
The absolute value of the brightness of all two field pictures of step 3, offshore scene image and two color characteristics median difference overall with it respectively is significantly schemed as the overall situation;
If of input image sequence frame , L ifor brightness, a iand b ibe two color characteristics, this is years old frame is any frame in offshore scene image;
Respectively to brightness and two color characteristics, calculate its overall intermediate value,
(1)
(2)
(3)
Wherein L imfor the overall intermediate value of brightness, a imand b imit is the overall intermediate value of two colors;
Then, the overall situation calculating each feature is significantly schemed:
(4)
(5)
(6)
Wherein, represent absolute value, for the overall situation of brightness is significantly schemed, with the overall situation being respectively two color characteristics is significantly schemed;
Brightness and two color characteristics of all two field pictures of step 4, offshore scene image are significantly schemed as local with the absolute value of its local mean value filtering difference respectively:
(7)
(8)
(9)
Wherein, be local mean value template, namely , symbol representation space territory convolution algorithm, for the local of brightness is significantly schemed, with be that the local of two color characteristics is significantly schemed;
The brightness of all two field pictures of step 5, offshore scene image and the overall situation of two color characteristics is significantly schemed and local significantly figure merge respectively, obtain total significantly figure of these three features:
(10)
(11)
(12)
Wherein, for total significantly figure of brightness, with be total significantly figure of two color characteristics;
The remarkable figure of two Color Channels of all two field pictures of step 6, offshore scene image carries out linear combining respectively, then is fused to the remarkable figure of its brightness respectively and always significantly schemes;
In every frame, the Color Channel that the remarkable figure linear combining of two Color Channels obtains significantly figure is:
(13)
Wherein, for Color Channel is significantly schemed;
Color Channel is significantly schemed significantly scheme to merge with brightness, obtains total significantly figure of this frame offshore scene:
(14)
Wherein, be of input image sequence frame total significantly figure of offshore scene;
Step 7, to every frame testing result, utilize centered by it, with the time window of regular length, by this time altogether the corresponding significantly figure of n frame accumulate, the remarkable figure of present frame is revised:
(15)
Wherein the length of time window, , be normalization sign of operation;
Step 8, threshold value according to setting, be converted to binary image by total significantly figure, obtain offshore scene well-marked target region.
In above-mentioned step 7, n desirable 5 or 7 or 9 or 11.
Threshold value described in above-mentioned step 8 is normalized threshold, and its span is 0.2 to 0.5.
A kind of offshore scene significance detection method of the present invention compared to the prior art, its advantage is, use of the present invention can marking area in rapid extraction offshore scene, be conducive to target detection in offshore scene, inhibit the interference of sea clutter preferably, situation about can be flooded by wave some frame Small Target and when occurring disturb compared with strong sea clutter, ensures the detection of marine remarkable Small object.Method of the present invention realizes simple, is applicable to application in real time, and can provide the supplementary means of machine vision for all kinds of maritime affairs monitor staff.
Accompanying drawing explanation
Fig. 1 is the method flow diagram of a kind of offshore scene significance detection method of the present invention.
Embodiment
Below in conjunction with accompanying drawing, further illustrate specific embodiments of the invention.
The invention discloses a kind of offshore scene significance detection method, utilize the space domain characteristic of offshore scene image, in the conspicuousness detection method that CIELab color space realizes.The method utilizes the overall situation and the local conspicuousness of offshore scene, and extraction brightness of image and Color Channel are significantly schemed and merge it, with outstanding target area respectively.Meanwhile, in order to strengthen the marking area of offshore scene, remove sea clutter further, the remarkable figure between multiframe is accumulated.
The present invention is the space domain characteristic utilizing offshore scene image, in the conspicuousness detection method that CIELab color space realizes.Utilize the fusion of offshore scene brightness of image and the Color Channel overall situation and the remarkable figure in local in the method, obtain marking area.Meanwhile, in order to better remove sea clutter, have employed simple interframe conspicuousness accumulation method.
The present invention mainly have employed when implementing: the conspicuousness computing method of every frame offshore scene image, and the remarkable figure accumulation method of interframe.
The present invention can be applicable in perils of the sea search and rescue, maritime affairs patrol, based on fields such as the ship collision prevention of video, anti-pirate monitoring, Zhi Ban lookouts, is combined simultaneously, can be maritime bridge etc. and provide comprehensive visual information with infrared, remote sensing, radar imaging technology.
As shown in Figure 1, offshore scene significance detection method of the present invention comprises following steps:
Step 1, utilize video capture device (such as video camera etc.), obtain original marine image sequence, extract offshore scene image sequence.
Every two field picture of step 2, offshore scene image to CIELab color space, and extracts its luminance channel L and two Color Channel a and b as essential characteristic by RGB color space conversion, obtains the characteristic pattern of brightness and two Color Channels.
This is because between passage each in RGB color space, correlativity is higher, therefore adopts CIELab color space, extracts brightness and the color characteristic of visual scene.
Wherein above-mentioned CIELab color space only has two color space a or b.
The absolute value of the brightness of all two field pictures of step 3, offshore scene image and two color characteristics median difference overall with it respectively is significantly schemed as the overall situation.
Calculation process for a wherein frame, if of input image sequence frame , its brightness and two color characteristics are respectively: , L ifor brightness, a iand b ibe two color characteristics.
Respectively to brightness and two color characteristics, calculate its overall intermediate value,
(1)
(2)
(3)
Wherein L imfor the overall intermediate value of brightness, a imand b imit is the overall intermediate value of two colors.
And the overall situation utilizing following formula to calculate each feature is significantly schemed, and is respectively:
(4)
(5)
(6)
Wherein, represent absolute value, for the overall situation of brightness is significantly schemed, with the overall situation being respectively two color characteristics is significantly schemed.
Brightness and two color characteristics of all two field pictures of step 4, offshore scene image are significantly schemed as local with the absolute value of its local mean value filtering difference respectively.
With frame is example, utilizes the absolute value of the filtering of each feature local mean value and each feature difference significantly to scheme as its local, is respectively above-mentioned three passages:
(7)
(8)
(9)
Wherein, be local mean value template, namely .Symbol representation space territory convolution algorithm. for the local of brightness is significantly schemed, with be that the local of two color characteristics is significantly schemed.
The brightness of all two field pictures of step 5, offshore scene image and the overall situation of two color characteristics is significantly schemed and local significantly figure merge respectively, obtain total significantly figure of these three features.
With frame is example, and total significantly figure of brightness and two color characteristics is shown below:
(10)
(11)
(12)
Wherein, for total significantly figure of brightness, with be total significantly figure of two color characteristics.
The remarkable figure of two Color Channels of all two field pictures of step 6, offshore scene image carries out linear combining respectively, then is fused to the remarkable figure of its brightness respectively and always significantly schemes.
With frame is example, and the Color Channel that the remarkable figure linear combining of two Color Channels obtains significantly figure is:
(13)
Wherein, for Color Channel is significantly schemed.
The result of above formula (13) and brightness are significantly schemed to merge, the total significantly figure finally obtaining this frame offshore scene is:
(14)
Wherein, be of input image sequence frame total significantly figure of offshore scene.
Step 7, to every frame testing result, utilize centered by it, the testing result of each 3 frames in front and back significantly schemes accumulation; To 3 frames before image sequence and rear 3 two field pictures, then the remarkable figure of continuous 7 frame comprising this frame is utilized to accumulate.
The present invention utilizes the time window of regular length, and accumulated by corresponding for all frames in this time significantly figure, revise the remarkable figure of present frame, object strengthens target, suppresses sea clutter.
Namely revised frame significantly figure is:
(15)
Wherein be the length of time window, the value of n generally gets odd number, and the value of such as n desirable 5 or 7 or 9 or 11, this n can not be excessive or too small, and frame per second when itself and video acquisition has relation.In the present embodiment get 7, , be normalization sign of operation, object is the gray-scale value of remarkable figure to unite, so that conspicuousness accumulation below.
Especially, for 3 frames before image sequence and last 3 frames, then do with the remarkable figure of continuous 7 frame comprising this frame and accumulate.
Step 8, threshold value according to setting, be converted to binary image by total significantly figure, obtain offshore scene well-marked target region.Wherein above-mentioned default threshold value is normalized threshold, and its span is the numerical value between 0.2 to 0.5, and this threshold value is an empirical value.
Although content of the present invention has done detailed introduction by above preferred embodiment, will be appreciated that above-mentioned description should not be considered to limitation of the present invention.After those skilled in the art have read foregoing, for multiple amendment of the present invention and substitute will be all apparent.Therefore, protection scope of the present invention should be limited to the appended claims.

Claims (3)

1. an offshore scene significance detection method, is characterized in that, the method includes the steps of:
Step 1, extraction offshore scene image sequence;
Every two field picture of step 2, offshore scene image to CIELab color space, and extracts its brightness and two Color Channels as essential characteristic by RGB color space conversion, obtains the characteristic pattern of brightness and two Color Channels;
The absolute value of the brightness of all two field pictures of step 3, offshore scene image and two color characteristics median difference overall with it respectively is significantly schemed as the overall situation;
If of input image sequence frame , L ifor brightness, a iand b ibe two color characteristics, this is years old frame is any frame in offshore scene image;
Respectively to brightness and two color characteristics, calculate its overall intermediate value,
(1)
(2)
(3)
Wherein L imfor the overall intermediate value of brightness, a imand b imit is the overall intermediate value of two colors;
Then, the overall situation calculating each feature is significantly schemed:
(4)
(5)
(6)
Wherein, || || represent absolute value, for the overall situation of brightness is significantly schemed, with the overall situation being respectively two color characteristics is significantly schemed;
Brightness and two color characteristics of all two field pictures of step 4, offshore scene image are significantly schemed as local with the absolute value of its local mean value filtering difference respectively:
(7)
(8)
(9)
Wherein, be local mean value template, namely , symbol representation space territory convolution algorithm, for the local of brightness is significantly schemed, with be that the local of two color characteristics is significantly schemed;
The brightness of all two field pictures of step 5, offshore scene image and the overall situation of two color characteristics is significantly schemed and local significantly figure merge respectively, obtain total significantly figure of these three features:
(10)
(11)
(12)
Wherein, for total significantly figure of brightness, with be total significantly figure of two color characteristics;
The remarkable figure of two Color Channels of all two field pictures of step 6, offshore scene image carries out linear combining respectively, then is fused to the remarkable figure of its brightness respectively and always significantly schemes;
In every frame, the Color Channel that the remarkable figure linear combining of two Color Channels obtains significantly figure is:
(13)
Wherein, for Color Channel is significantly schemed;
Color Channel is significantly schemed significantly scheme to merge with brightness, obtains total significantly figure of this frame offshore scene:
(14)
Wherein, be of input image sequence frame total significantly figure of offshore scene;
Step 7, to every frame testing result, utilize centered by it, with the time window of regular length, by this time altogether the corresponding significantly figure of n frame accumulate, the remarkable figure of present frame is revised:
(15)
Wherein the length of time window, , be normalization sign of operation;
Step 8, threshold value according to setting, be converted to binary image by total significantly figure, obtain offshore scene well-marked target region.
2. a kind of offshore scene significance detection method as claimed in claim 1, is characterized in that, in described step 7, and n desirable 5 or 7 or 9 or 11.
3. a kind of offshore scene significance detection method as claimed in claim 1, is characterized in that, the threshold value described in described step 8 is normalized threshold, and its span is 0.2 to 0.5.
CN201210207271.5A 2012-06-21 2012-06-21 Offshore scene significance detection method Expired - Fee Related CN102800086B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201210207271.5A CN102800086B (en) 2012-06-21 2012-06-21 Offshore scene significance detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201210207271.5A CN102800086B (en) 2012-06-21 2012-06-21 Offshore scene significance detection method

Publications (2)

Publication Number Publication Date
CN102800086A CN102800086A (en) 2012-11-28
CN102800086B true CN102800086B (en) 2015-02-04

Family

ID=47199184

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201210207271.5A Expired - Fee Related CN102800086B (en) 2012-06-21 2012-06-21 Offshore scene significance detection method

Country Status (1)

Country Link
CN (1) CN102800086B (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103020965B (en) * 2012-11-29 2016-12-21 奇瑞汽车股份有限公司 A kind of foreground segmentation method based on significance detection
CN103268586B (en) * 2013-04-27 2015-11-18 电子科技大学 A kind of window fusion method based on diffusion theory
CN103337075B (en) * 2013-06-20 2016-04-27 浙江大学 A kind of image significance computing method based on isophote
CN103413320B (en) * 2013-08-30 2016-12-28 上海海事大学 A kind of port contaminant saliency detection method
CN103413127B (en) * 2013-09-10 2016-06-08 上海海事大学 Marine target significance detection method based on spectrum singular value decomposition
CN104050674B (en) * 2014-06-27 2017-01-25 中国科学院自动化研究所 Salient region detection method and device
CN106991682B (en) * 2016-01-21 2019-12-20 深圳力维智联技术有限公司 Automatic port cargo ship extraction method and device
CN105893957B (en) * 2016-03-30 2019-03-22 上海交通大学 View-based access control model lake surface ship detection recognition and tracking method
CN107169516B (en) * 2017-05-11 2020-10-23 上海海事大学 Marine small target significance detection method based on K-L transformation
CN107886533B (en) * 2017-10-26 2021-05-04 深圳大学 Method, device and equipment for detecting visual saliency of three-dimensional image and storage medium
CN107862262A (en) * 2017-10-27 2018-03-30 中国航空无线电电子研究所 A kind of quick visible images Ship Detection suitable for high altitude surveillance
CN108009984B (en) * 2017-11-21 2020-07-07 中国地质大学(武汉) Water environment monitoring-oriented water surface saliency area detection method
CN107967474A (en) * 2017-11-24 2018-04-27 上海海事大学 A kind of sea-surface target conspicuousness detection method based on convolutional neural networks
CN108229342B (en) * 2017-12-18 2021-10-26 西南技术物理研究所 Automatic sea surface ship target detection method
CN108334865B (en) * 2018-03-12 2019-03-12 贵州迦太利华信息科技有限公司 Big data analysis device based on image
CN109188421B (en) * 2018-07-25 2023-07-04 江苏科技大学 Maritime search and rescue system and method for unmanned search and rescue boat

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101976439A (en) * 2010-11-02 2011-02-16 上海海事大学 Visual attention model with combination of motion information in visual system of maritime search and rescue machine
CN102087744A (en) * 2010-02-25 2011-06-08 上海海事大学 Structure tensor method for quick detection of small video target under dynamic ocean background
CN102156881A (en) * 2011-04-13 2011-08-17 上海海事大学 Method for detecting salvage target based on multi-scale image phase information

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102087744A (en) * 2010-02-25 2011-06-08 上海海事大学 Structure tensor method for quick detection of small video target under dynamic ocean background
CN101976439A (en) * 2010-11-02 2011-02-16 上海海事大学 Visual attention model with combination of motion information in visual system of maritime search and rescue machine
CN102156881A (en) * 2011-04-13 2011-08-17 上海海事大学 Method for detecting salvage target based on multi-scale image phase information

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
TARGET DETECTION IN MARITIME SEARCH AND RESCUE USING SVD AND FREQUENCY DOMAIN CHARACTERISTICS;Lei Ren;《Proceedings of the 2011 International Conference on Machine Learning and Cybernetics》;20110713;556-560 *
应用改进频率调谐的海上小目标检测方法;任蕾;《中国图象图形学报》;20120331;第17卷(第3期);565-569 *

Also Published As

Publication number Publication date
CN102800086A (en) 2012-11-28

Similar Documents

Publication Publication Date Title
CN102800086B (en) Offshore scene significance detection method
CN101673396B (en) Image fusion method based on dynamic object detection
CN106846289B (en) A kind of infrared light intensity and polarization image fusion method
Guo et al. Lightweight deep network-enabled real-time low-visibility enhancement for promoting vessel detection in maritime video surveillance
Jia et al. A two-step approach to see-through bad weather for surveillance video quality enhancement
CN103413127B (en) Marine target significance detection method based on spectrum singular value decomposition
Zhang et al. A robust haze-removal scheme in polarimetric dehazing imaging based on automatic identification of sky region
CA2780595A1 (en) Method and multi-scale attention system for spatiotemporal change determination and object detection
Zhang et al. Single image dehazing based on fast wavelet transform with weighted image fusion
Feng et al. Infrared target detection and location for visual surveillance using fusion scheme of visible and infrared images
Liu et al. Haze removal for a single inland waterway image using sky segmentation and dark channel prior
Patel et al. A review on infrared and visible image fusion techniques
CN115830064B (en) Weak and small target tracking method and device based on infrared pulse signals
CN116109682A (en) Image registration method based on image diffusion characteristics
Si-ming et al. Moving shadow detection based on Susan algorithm
Li et al. Infrared small target detection based on 1-D difference of guided filtering
Zhang et al. Multisensor Infrared and Visible Image Fusion via Double Joint Edge Preservation Filter and Nonglobally Saliency Gradient Operator
Babu et al. Efficient brightness adaptive deep-sea image stitching using biorthogonal multi-wavelet transform and harris algorithm
Baraiya et al. An introduction of image fusion techniques
Ren et al. Target detection of maritime search and rescue: Saliency accumulation method
CN109685825A (en) Local auto-adaptive feature extracting method, system and storage medium for thermal infrared target tracking
Chen et al. Research on contour extraction and edge detection of sugarcane image based on MATLAB
Yang et al. Parts shape recognition based on improved Harris corner detection algorithm
Qi et al. A new defogging method with nested windows
Zhu et al. An adaptive shadow elimination algorithm using shadow position and edges attributes

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20150204

Termination date: 20180621

CF01 Termination of patent right due to non-payment of annual fee