CN100393486C - Method and apparatus for quick tracing based on object surface color - Google Patents
Method and apparatus for quick tracing based on object surface color Download PDFInfo
- Publication number
- CN100393486C CN100393486C CNB2004100688713A CN200410068871A CN100393486C CN 100393486 C CN100393486 C CN 100393486C CN B2004100688713 A CNB2004100688713 A CN B2004100688713A CN 200410068871 A CN200410068871 A CN 200410068871A CN 100393486 C CN100393486 C CN 100393486C
- Authority
- CN
- China
- Prior art keywords
- image
- objects
- video camera
- color
- tracking method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000033001 locomotion Effects 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 13
- 230000000007 visual effect Effects 0.000 claims abstract description 8
- 239000003086 colorant Substances 0.000 claims abstract 2
- 239000004744 fabric Substances 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000012876 topography Methods 0.000 claims description 3
- 238000003672 processing method Methods 0.000 abstract description 4
- 238000001514 detection method Methods 0.000 abstract 1
- 238000002054 transplantation Methods 0.000 abstract 1
- 210000000707 wrist Anatomy 0.000 description 6
- 238000010586 diagram Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000009017 pursuit movement Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The present invention relates to a quick tracing method based on object surface colors and a device. An image acquisition card is arranged in a computer, and images of moving objects are acquired to be sent to the computer through a video camera and the image acquisition card. Then, specific image processing algorithm is adopted, needed objects are selected according to the characters of color blocks of object surfaces, and the centroid positions of images of the objects are given. The difference between the centroid position of the images of the object and a position of a given image point is used as feedback controlling quantity; the motions of robots are controlled to drive the video camera to move; the quick tracing to the objects is realized. The present invention has the advantages of brief image processing method, high speed, strong adaptability and strong transplantation, and can independently become a unit, a learning method based on color information is adopted in image processing, the present invention has better adaptability to the change of the objects and ambient light, and the objects can be kept in the visual field of the video camera all the time. The present invention is suitable for the fields of intelligent surveillance, automatic detection to manufactured products, visual control to production lines, etc.
Description
Technical field
The invention belongs to the vision tracking technique field among the robot field, specifically be used to obtain the surface image of moving object, according to the color characteristic of image, the method and apparatus that selects certain objects and realize following the tracks of fast.
Background technology
At present, in rapid movement object tracking research based on vision, need tracked object to have the obvious color feature, so, method (its typical structure is seen " Hu Ying, Zhao Shuying, Xu Xinhe; colour code design and identification algorithm research; Chinese image graphics journal, the 7th volume (A version), 12 phases; in December, 2002,1291 ~ 1295 pages ") all adopted at object exterior pasting solid color colour code.The method of pasting colour code has certain limitation, can not be applicable to occasions such as intelligent monitoring, the tracking of streamline part.
Summary of the invention
The purpose of this invention is to provide a kind of fast tracking method based on color of object surface, applicable to vision follow the tracks of simply, image processing method fast and effectively.
Another object of the present invention provides a kind of device of realizing based on the fast tracking method of color of object surface.
To achieve these goals, technical scheme of the present invention provides a kind of fast tracking method based on color of object surface, and it carries out image recognition according to following steps in the object of which movement process:
The first step: at first the object that needs are discerned and followed the tracks of is separated from background, gather image then in real time, to each width of cloth image, through with the threshold ratio of the HSV that calculates, the pixel area that will meet color gamut remains, remainder is rejected as a setting, and subject image is split:
F(x，y)＝1(t1＜＝F(x，y)＜＝t2)
F (x, y)=0 (other);
Second step: the object binaryzation that will cut apart generates the black white image of binaryzation, and to this bianry image filtering processing, obtains level and smooth
Black and whiteImage;
The 3rd step: with Canny operator sharpen edges, and with expansion algorithm to reach the effect of removal aperture;
The 4th step: use the edge extracting algorithm, obtain the profile of object;
The 5th step: based on the image recognition of shape facility,, reject the pixel region that does not meet geometrical model, find the barycenter of tracked subject image according to the geometrical model of tracked object;
The 6th step: after determining the barycenter of subject image, with the difference of the position of subject image barycenter and given image point locations as the FEEDBACK CONTROL amount, the control robot drives camera motion, and the image of object is remained in the visual field of video camera, follows the tracks of this moving object.
Described fast tracking method, its described the 5th step: the image recognition based on shape facility, be to use form parameter, form parameter F has described regional compactedness to a certain extent, and it is to calculate according to the area A in the girth B in zone and zone:
F＝B*B/(4*PI*A)
Wherein, form parameter is got minimum of a value 1 to border circular areas, and when the zone when being other shape F always greater than 1.
Described fast tracking method, it also is included in before the motion tracking, learns, and it adopts the method for on-line study: a) before carrying out tracing task, through image collection card, obtain a digitized RGB coloured image; B) user uses the mouse selection to need the rectangular area of the object of tracking; C) operating system with the topography that chooses on computers with the storage of BMP document form, the foundation of discerning needed threshold value and the image of real-time collection being cut apart as the back; D) this partial color image is converted into the HSV model, two components of its H, S are done histogram respectively, obtain H, the S threshold value of selection area.
Described fast tracking method, its described H, S threshold value, in realtime graphic identification subsequently, this threshold value can not change as the standard of object segmentation, learns again up to the user.
Described fast tracking method, it is under the situation of video camera not being demarcated, and the error of utilizing set point in the image and object mass center is the control feedback quantity, realizes that vision is followed the tracks of fast and accurately.
Described fast tracking method, it is a foundation with color of object surface information, recognition object carries out vision and follows the tracks of.
A kind of device of realizing based on the fast tracking method of color of object surface, comprise robot, robot control system, vision processing system is formed, its robot control system is made up of main control computer and robot controller, vision processing system is by video camera, image pick-up card and pattern process computer are formed, wherein, video camera is installed in the robot end, the map interlinking of video camera output is as capture card, IMAQ is placed in the pattern process computer, robot is electrically connected with robot controller, and robot controller is electrically connected with master computer respectively with pattern process computer.
Described device, the artificial robot with five degrees of freedom of its described machine, be made up of the Cartesian robot of a Three Degree Of Freedom and the rotation wrist of a two-freedom, the rotation wrist is installed in the vertical the tip of the axis of Cartesian robot, is connected with video camera on this rotation wrist; Robot is by a main control computer and robot controller control.
Described device, its described image pick-up card and pattern process computer are to select the pci bus image collection card for use, image card are installed in the general purpose PC of dominant frequency for 〉=2.8G the composing images treatment system.
Outstanding feature of the present invention is that video camera does not need to demarcate, and does not need to paste colour code, can follow the tracks of fast the object of the multiple color of surface coverage.
Image processing method of the present invention is succinct, and speed is fast, and is effective, independently becomes a unit, and adaptability is strong, and transplantability is strong.Adopted learning method during image is handled based on colouring information, to object change, the change of ambient light has excellent adaptability.Under the situation of video camera not being demarcated, use video camera and image pick-up card, obtain the image of moving object, adopt special image processing algorithm, the color characteristic of body surface and the area of color block are learnt, the result of study is as in the moving body track process, recognition object and define the standard of object.After obtaining the image of tracked object, calculate image centroid position c (u
c, v
c), with subject image centroid position c (u
c, v
c) with image in set point s (u
s, v
s) between pixel difference e, as the visual feedback amount, the control robot drives camera motion, utilizes the rotary joint pursuit movement object of robot, reflection rapidly, tracking velocity is fast, the image that can remain object is in the visual field of video camera.The visual processing method that the present invention sets forth, insensitive to the variation of ambient light, and be applicable to the moving body track that surperficial multiple color covers.
The present invention is suitable for fields such as intelligent monitoring, industrial products detect automatically, the control of streamline vision.
Description of drawings
Fig. 1 is the schematic diagram of the present invention's realization based on the device of the fast tracking method of color of object surface;
Fig. 2 the present invention is based on the fast tracking method of color of object surface to moving image processing procedure schematic diagram.
The specific embodiment
A kind of device of realizing based on the fast tracking method of color of object surface comprises robot, robot control system, and vision processing system is formed, and the principle of single unit system as shown in Figure 1.The robot control system device is made up of main control computer and robot controller, and vision processing system is by video camera, image pick-up card, and pattern process computer is formed.Wherein, the present invention is installed in the robot end with video camera, and the map interlinking of video camera output is as capture card, and IMAQ is placed in the pattern process computer, robot is electrically connected with robot controller, and robot controller is electrically connected with master computer respectively with pattern process computer.
In image processing algorithm, adopt the method for on-line study, before carrying out tracing task,, obtain a digitized RGB coloured image through image collection card.The user uses the mouse selection to need the rectangular area of the object of tracking.System stores the topography that chooses on computers with the BMP document form, discerns threshold value that needs and the foundation that the image of real-time collection is cut apart as the back.This partial color image is converted into the HSV model, two components of its H, S are done histogram respectively, obtain H, the S threshold value of selection area.In realtime graphic identification subsequently, this threshold value can not change as the standard of object segmentation, learns again up to the user.
The benefit of this learning process is under the situation of following the tracks of the object variation, need not program inside is made any change, each when the condition variation, such as light generation significant change, under the situation that tracked object changes, as long as before tracking, take a width of cloth photo current, choose tracked object just to finish the process of study with mouse.
When following the tracks of beginning, program at first reads the picture BMP of the district portion file of object, this BMP file is generated the HSV histogram and the threshold value of following the tracks of object, image card is gathered image in real time in the mode of concurrent working, every width of cloth image all compares with this threshold value, reject background, cut apart object, find the image border and the central point of object.Do not change at tracked object, light does not have under the situation of strong variations yet, does not need to relearn, and finishes until tracing process.Complete processing procedure as shown in Figure 2.
When moving body track, robot drives camera motion, makes moving object be in video camera all the time within sweep of the eye, and in this process, the step of object image identification is as follows:
The first step: at first the object that needs are discerned and followed the tracks of is separated from background.Background is the set of actionless pixel on the image, and it does not belong to any object that moves before video camera.Gather image then in real time, to each width of cloth image, through with the threshold ratio of the HSV that calculated just now, will meet
ColorPixel area in the scope remains, and remainder is rejected as a setting, object is got image split.
F(x，y)＝1(t1＜＝F(x，y)＜＝t2)
F (x, y)=0 (other)
Second step: the object binaryzation that will cut apart generates the black white image of binaryzation, and to this bianry image filtering processing, obtains level and smooth
Black and whiteImage;
The 3rd step: with Canny operator sharpen edges, and with expansion algorithm to reach the effect of removal aperture;
The 4th step: use the edge extracting algorithm, obtain the profile of object;
The 5th step:,, find the central point of object according to the geometrical model of object based on the image recognition of shape facility.Use form parameter, form parameter F has described regional compactedness to a certain extent, and it is to calculate according to the area A in the girth B in zone and zone:
F＝B*B/(4*PI*A)
Wherein, form parameter is got minimum of a value 1 to border circular areas, and when the zone was other shape, F was always greater than 1.For example in the identification spherical body, at first consider and remove by the area threshold noise that area is too small.Consider that then F near 1 zone, can distinguish circle and other shape of rule in the picture.Shi Bie Else Rule shape such as square, can obtain foursquare F value by foursquare feature and approach 4/PI (=1.3) if desired.
The 6th step: after determining the central point of object, the control robot drives camera motion, and the image of object is remained in the visual field of video camera, follows the tracks of this moving object.
Provide an example of the present invention below.In the example, video camera is installed in the robot end of a five degree of freedom, robot is made up of the Cartesian robot of a Three Degree Of Freedom and the rotation wrist of a two-freedom, the rotation wrist is installed in the vertical the tip of the axis of Cartesian robot, and robot is by a main control computer and controller control.An industry standard colour TV camera is fixed on the rotation wrist, selects OK series pci bus image collection card for use, image card is installed in the general purpose PC that dominant frequency is 2.8G the composing images treatment system.The operation principle of whole device as shown in Figure 1.
The application example system under the natural lighting irradiation, follows the tracks of a remote control car.The telecar surface is yellowish green alternate color, and the front and back vehicle window is a black, uses the learning method of describing among the present invention, before motion tracking, learns, and obtains H, the S threshold value of telecar surface color.The control moving of car that uses a teleswitch according to flow process shown in Figure 2, adopts the first step to the image-recognizing method in six steps, has realized the motion tracking of remote operated vehicle.
As seen, the method and apparatus among the present invention can not have under the demarcation situation at video camera, does not need to paste colour code, to the surface color complex objects, realizes quick vision tracking.
Claims (6)
1. fast tracking method based on color of object surface is characterized in that: in the object of which movement process, carry out image recognition according to following steps:
The first step: at first the object that needs are discerned and followed the tracks of is separated from background, gather image then in real time, to each width of cloth image, through with the threshold ratio of the HSV that calculates, the pixel area that will meet color gamut remains, remainder is rejected as a setting, and subject image is split, and this step can be partitioned into the close object of a plurality of colors:
F(x，y)＝1(t1＜＝F(x，y)＜＝t2)
F (x, y)=0 (other);
Second step: the subject image binaryzation that will cut apart generates the black white image of binaryzation, and to this bianry image filtering processing, obtains level and smooth black white image;
The 3rd step: with Canny operator sharpen edges, and with expansion algorithm to reach the effect of removal aperture;
The 4th step: use the edge extracting algorithm, obtain the profile of object;
The 5th step: based on the image recognition of shape facility,, reject the pixel region that does not meet geometrical model, find the barycenter of tracked subject image according to the geometrical model of tracked object;
The 6th step: after determining the barycenter of subject image, with the difference of the position of subject image barycenter and given image point locations as the FEEDBACK CONTROL amount, the control robot drives camera motion, and the image of object is remained in the visual field of video camera, follows the tracks of this moving object.
2. fast tracking method as claimed in claim 1, it is characterized in that: described the 5th step: based on the image recognition of shape facility, be to use form parameter, form parameter F has described regional compactedness to a certain extent, and it is to calculate according to the area A in the girth B in zone and zone:
F＝B*B/(4*PI*A)
Wherein, form parameter is got minimum of a value 1 to border circular areas, and when the zone when being other shape F always greater than 1.
3. fast tracking method as claimed in claim 1, it is characterized in that: also be included in before the motion tracking, learn, it adopts the method for on-line study: a) before carrying out tracing task, through image collection card, obtain a digitized RGB coloured image; B) user uses the mouse selection to need the rectangular area of the object of tracking; C) operating system with the topography that chooses on computers with the storage of BMP document form, the foundation of discerning needed threshold value and the image of real-time collection being cut apart as the back; D) this partial color image is converted into the HSV model, two components of its H, S are done histogram respectively, obtain H, the S threshold value of selection area.
4. fast tracking method as claimed in claim 3 is characterized in that: described H, S threshold value, in realtime graphic identification subsequently, this threshold value can not change as the standard of object segmentation, learns again up to the user.
5. fast tracking method as claimed in claim 1 is characterized in that: under the situation of video camera not being demarcated, the error of utilizing set point in the image and object mass center is the control feedback quantity, realizes that vision is followed the tracks of fast and accurately.
6. fast tracking method as claimed in claim 1 is characterized in that: with color of object surface information is foundation, and recognition object carries out vision and follows the tracks of.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100688713A CN100393486C (en) | 2004-07-13 | 2004-07-13 | Method and apparatus for quick tracing based on object surface color |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100688713A CN100393486C (en) | 2004-07-13 | 2004-07-13 | Method and apparatus for quick tracing based on object surface color |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1721144A CN1721144A (en) | 2006-01-18 |
| CN100393486C true CN100393486C (en) | 2008-06-11 |
Family
ID=35911929
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2004100688713A Expired - Fee Related CN100393486C (en) | 2004-07-13 | 2004-07-13 | Method and apparatus for quick tracing based on object surface color |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100393486C (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101030244B (en) * | 2006-03-03 | 2010-08-18 | 中国科学院自动化研究所 | Automatic identity discriminating method based on human-body physiological image sequencing estimating characteristic |
| CN101453660B (en) * | 2007-12-07 | 2011-06-08 | 华为技术有限公司 | Video object tracking method and apparatus |
| CN101685309B (en) * | 2008-09-24 | 2011-06-08 | 中国科学院自动化研究所 | Method for controlling multi-robot coordinated formation |
| CN101587591B (en) * | 2009-05-27 | 2010-12-08 | 北京航空航天大学 | Visual accurate tracking technique based on double parameter thresholds dividing |
| CN101783964A (en) * | 2010-03-18 | 2010-07-21 | 上海乐毅信息科技有限公司 | Auxiliary driving system for achromate or tritanope based on image identification technology |
| CN101913147B (en) * | 2010-07-12 | 2011-08-17 | 中国科学院长春光学精密机械与物理研究所 | High-precision fully-automatic large transfer system |
| CN101964114B (en) * | 2010-09-16 | 2013-02-27 | 浙江吉利汽车研究院有限公司 | Auxiliary traffic light recognition system for anerythrochloropsia drivers |
| CN102096927A (en) * | 2011-01-26 | 2011-06-15 | 北京林业大学 | Target tracking method of independent forestry robot |
| CN102431034B (en) * | 2011-09-05 | 2013-11-20 | 天津理工大学 | Color recognition-based robot tracking method |
| CN102917171B (en) * | 2012-10-22 | 2015-11-18 | 中国南方电网有限责任公司超高压输电公司广州局 | Based on the small target auto-orientation method of pixel |
| CN103056864A (en) * | 2013-01-24 | 2013-04-24 | 上海理工大学 | Device and method for detecting position and angle of wheeled motion robot in real time |
| CN103177259B (en) * | 2013-04-11 | 2016-05-18 | 中国科学院深圳先进技术研究院 | Color lump recognition methods |
| CN104281832A (en) * | 2013-07-04 | 2015-01-14 | 上海高威科电气技术有限公司 | Visual identity industrial robot |
| CN103895023B (en) * | 2014-04-04 | 2015-08-19 | 中国民航大学 | A kind of tracking measurement method of the mechanical arm tail end tracing measurement system based on coding azimuth device |
| AU2015357284B2 (en) * | 2014-12-05 | 2019-11-21 | Ars S.R.L. | Device for orienting parts, particularly for gripping by robots, automation means and the like |
| CN106934813A (en) * | 2015-12-31 | 2017-07-07 | 沈阳高精数控智能技术股份有限公司 | A kind of industrial robot workpiece grabbing implementation method of view-based access control model positioning |
| CN107305378A (en) * | 2016-04-20 | 2017-10-31 | 上海慧流云计算科技有限公司 | A kind of method that image procossing follows the trail of the robot of object and follows the trail of object |
| CN106096599B (en) * | 2016-04-28 | 2019-03-26 | 浙江工业大学 | A kind of inside truck positioning method based on painting color lump |
| CN107403437A (en) * | 2016-05-19 | 2017-11-28 | 上海慧流云计算科技有限公司 | The method, apparatus and robot of robotic tracking's object |
| CN106863332B (en) * | 2017-04-27 | 2023-07-25 | 广东工业大学 | Robot vision positioning method and system |
| CN107564037A (en) * | 2017-08-07 | 2018-01-09 | 华南理工大学 | A kind of multirobot detection and tracking based on local feature |
| CN108032313B (en) * | 2018-01-04 | 2019-05-03 | 北京理工大学 | The manipulator of intelligent terminal touch screen game is automatically performed based on bionic principle |
| CN110666801A (en) * | 2018-11-07 | 2020-01-10 | 宁波赛朗科技有限公司 | Grabbing industrial robot for matching and positioning complex workpieces |
| CN113103256A (en) * | 2021-04-22 | 2021-07-13 | 达斯琪(重庆)数字科技有限公司 | Service robot vision system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2715931Y (en) * | 2004-07-13 | 2005-08-10 | 中国科学院自动化研究所 | Apparatus for quick tracing based on object surface color |
-
2004
- 2004-07-13 CN CNB2004100688713A patent/CN100393486C/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2715931Y (en) * | 2004-07-13 | 2005-08-10 | 中国科学院自动化研究所 | Apparatus for quick tracing based on object surface color |
Non-Patent Citations (3)
| Title |
|---|
| 微尺寸视觉精密检测系统设计. 廖强，米林，周忆，徐宗俊.重庆大学学报(自然科学版)，第25卷第12期. 2002 * |
| 机器视觉中的摄像机定标方法综述. 吴文琪，孙增圻.计算机应用研究，第2期. 2004 * |
| 色标设计与辨识算法研究. 胡英，赵姝颖，徐心和.中国图象图形学报，第7卷第12期. 2002 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1721144A (en) | 2006-01-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100393486C (en) | Method and apparatus for quick tracing based on object surface color | |
| Malima et al. | A fast algorithm for vision-based hand gesture recognition for robot control | |
| CN104217428B (en) | A kind of fusion feature matching and the video monitoring multi-object tracking method of data correlation | |
| CN100544446C (en) | The real time movement detection method that is used for video monitoring | |
| CN107038424A (en) | A kind of gesture identification method | |
| CN103325126A (en) | Video target tracking method under circumstance of scale change and shielding | |
| CN105046206B (en) | Based on the pedestrian detection method and device for moving prior information in video | |
| CN106056053A (en) | Human posture recognition method based on skeleton feature point extraction | |
| CN103226701B (en) | A kind of video semantic event modeling method | |
| CN106981075A (en) | The skeleton point parameter acquisition devices of apery motion mimicry and its recognition methods | |
| CN104036523A (en) | Improved mean shift target tracking method based on surf features | |
| CN106326860A (en) | Gesture recognition method based on vision | |
| CN108734172A (en) | Target identification method, system based on linear edge feature | |
| CN108453739B (en) | Stereoscopic vision positioning mechanical arm grabbing system and method based on automatic shape fitting | |
| CN104239886A (en) | Image analysis based lawn and background boundary extraction method | |
| CN109117838A (en) | Object detection method and device applied to unmanned boat sensory perceptual system | |
| CN104299246A (en) | Production line object part motion detection and tracking method based on videos | |
| CN2715931Y (en) | Apparatus for quick tracing based on object surface color | |
| CN115816460A (en) | Manipulator grabbing method based on deep learning target detection and image segmentation | |
| CN108089695A (en) | A kind of method and apparatus for controlling movable equipment | |
| Foresti | Object detection and tracking in time-varying and badly illuminated outdoor environments | |
| CN109079777A (en) | A kind of mechanical arm hand eye coordination operating system | |
| CN108509025A (en) | A kind of crane intelligent Lift-on/Lift-off System based on limb action identification | |
| CN115657531A (en) | System and method for determining bonsai grabbing pose and parking robot | |
| Ruchanurucks et al. | Humanoid robot painter: Visual perception and high-level planning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20080611 Termination date: 20190713 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |