CN105165255B - A kind of intelligent cotton picker system based on computer vision technique - Google Patents
A kind of intelligent cotton picker system based on computer vision technique Download PDFInfo
- Publication number
- CN105165255B CN105165255B CN201510556986.5A CN201510556986A CN105165255B CN 105165255 B CN105165255 B CN 105165255B CN 201510556986 A CN201510556986 A CN 201510556986A CN 105165255 B CN105165255 B CN 105165255B
- Authority
- CN
- China
- Prior art keywords
- cotton
- module
- target
- original image
- control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 229920000742 Cotton Polymers 0.000 title claims abstract description 189
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 22
- 230000009471 action Effects 0.000 claims abstract description 13
- 238000000605 extraction Methods 0.000 claims abstract description 7
- 241000219146 Gossypium Species 0.000 claims description 174
- 238000003306 harvesting Methods 0.000 claims description 9
- 238000004891 communication Methods 0.000 claims description 3
- 125000004122 cyclic group Chemical group 0.000 claims description 3
- 241000241602 Gossypianthus Species 0.000 claims 1
- 230000006399 behavior Effects 0.000 claims 1
- 239000012535 impurity Substances 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000004753 textile Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000465 moulding Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000009941 weaving Methods 0.000 description 1
Landscapes
- Manipulator (AREA)
Abstract
A kind of intelligent cotton picker system based on computer vision technique, including：Image capture module, data memory module, image processing module, control module and performing module；Image capture module controls the original image of binocular camera BR collection cotton using camera master control borad；Data memory module is stored by stack-incoming operation to original image, and raw image data is supplied into image processing module by Pop operations；Image processing module obtains cotton spatial coordinated information collection in original image by carrying out information extraction to original image, and the locus coordinate for concentrating selected distance binocular camera nearest from spatial coordinated information is used as target cotton position；Control module reaches target cotton position according to target cotton position by controlling performing module；Performing module is used to perform cotton picking action, and the cotton plucked is put into cotton storage tank.Present invention reduces the complexity of control program, cotton picking efficiency is improved.
Description
Technical field
The present invention relates to intelligent cotton picking technical field, and in particular to a kind of intelligent cotton picker based on computer vision technique
System and method.
Background technology
Cotton is one of major economic crops of China, is widely used in industries such as China's weaving, medical treatment；Traditional is artificial
Cotton picking mode seriously governs cotton picking efficiency and harvesting cost, but to realize that complete cotton picking mechanization is also very difficult, and one
Aspect textile industry requires very high to acquisition quality, and another aspect cotton variety and cropping system also constrain grinding for universal machine
Study carefully；Traditional cotton picker has backpack cotton picker, Handheld cotton picker and large-scale self propelled cotton picker etc., and backpack cotton picker is adopted
Pluck quality general, labor intensity is very big, and the cotton plucked in addition has cotton leaf impurity；Handheld electric cotton picker harvesting quality is high,
But labor intensity is big；Large-scale self propelled cotton picker picking efficiency is high, but plucks of poor quality, on the one hand meeting in the cotton of harvesting
There are many cotton leaf impurity, the fibre length of another aspect cotton is unable to reach the requirement of textile industry；Therefore, it is traditional artificial to adopt
Cotton mode seriously governs cotton picking efficiency and harvesting cost；And although mechanization improve cotton picking efficiency, but pluck quality one
As, have the shortcomings that cotton fiber length is not enough and cotton picking cleannes are inadequate；The present situation received for Cotton in China, in order to find
The balance of cotton picking efficiency and cotton picking quality, the development of intelligent cotton picker is extremely urgent.
Such as Chinese invention patent CN101152716A, the patent No.：CN200610141979.X, discloses a kind of electromechanical
The cotton picking robot of change, is a kind of supermatic cotton field harvest machinery.The cotton picking robot is suspended on tractor, cotton picking
Robot is taken pictures acquisition view data by camera in cotton field under computer control, and view data is obtained after treatment
Space coordinate of the cotton in cotton field, computer drives mechanical arm action in place according to this coordinate parameters output control signal, from
And multiple mechanical arms are realized in cotton field while cotton picking.This cotton picking robot, one-time investment is small, and institute's cotton picking flower doping is few,
It is environmentally safe, it is adaptable to size cotton field, it is applicable in cotton picking early stage, mid-term and later stage, greatly reduce the labor of cotton grower
Fatigue resistance.Its weak point is that operating efficiency is relatively low, and the setting in control system is relatively complicated, inconvenient operation.
The content of the invention
The technical problems to be solved by the invention are to improve a kind of intelligent cotton picker system based on computer vision technique
System and its method, to can quickly and accurately detect cotton position and reliable, efficient, high-quality cotton picking, so as to improve
On the basis of the low harvesting cost with reduction of cotton picking efficiency, harvesting quality is improved so that cotton fiber length and the arrival of cotton picking cleannes
It is required that.
The technical problems to be solved by the invention are realized using following technical scheme：
A kind of intelligent cotton picker system based on computer vision technique, including：Image capture module, data storage mould
Block, image processing module, control module and performing module；
Described image acquisition module controls the original of binocular camera BR collection cotton using camera master control borad
Image；
The data memory module is stored by stack-incoming operation to the original image, and by Pop operations by institute
State raw image data and be supplied to described image processing module；
Described image processing module carries out the extraction of depth information by computer vision technique to the original image, obtains
The spatial coordinated information collection of all cottons in the original image is obtained, and is concentrated from the spatial coordinated information described in selected distance
The nearest locus coordinate of binocular camera is used as target cotton position；
The control module is the integrated monolithic machine of 32, and single-chip microcomputer and binocular camera shooting are connected by camera master control borad
Head；
The control module is reached according to the target cotton position by controlling performing module described in driving stepper motor
The target cotton position, wherein control module include feedback element, and feedback element obtains the reality that the performing module is reached
Border position, and be compared with the target cotton position, the difference of acquisition feeds back to the control module；
The difference that the control module is obtained according to the feedback element controls the performing module to eventually arrive at the mesh
Mark cotton position；
The performing module is a six degree of freedom manipulator, for performing the cotton picking action command that control module is sent, and
The cotton plucked is put into cotton storage tank.
A kind of method of the intelligent cotton picker system based on computer vision technique, applied on the cotton picker with cotton storage tank
, carry out as follows：
Central point between step 1, two using in binocular camera cameras is origin o, using the earth as reference planes,
X-axis is in direction using cotton picker advance, using the normal perpendicular to the reference planes as y-axis, and z is obtained according to right-hand rule
Axle, so as to set up o-xyz coordinate systems；Then position coordinates of the cotton storage tank in the o-xyz coordinate systems is initial coordinate
(x0, y0, z0)；
Step 2, definition cyclic variable are i, and initialize i=1；
Step 3, described image acquisition module utilize camera master control borad control binocular camera BR collection cotton
I-th original image；
Step 4, the data memory module store i-th original image by press process, pass through processing of popping
I-th original image is provided for described image processing module；
Step 4, described image processing module carry out cotton by computer vision technique to i-th original image
The extraction of depth information, obtains the spatial coordinated information collection of all cottons in i-th original image；
Step 5, described image processing module are concentrated from the spatial coordinated information and chosen and the binocular camera distance
Nearest spatial coordinated information is as target cotton position coordinates and is supplied to the control module；
Step 6, the control module read the target cotton position coordinates by serial communication, and calculate stepper motor
The rotation number of turns so that it is described to control performing module described in driving stepper motor to be moved to from the initial coordinate (x0, y0, z0)
Target cotton position coordinates；
Step 7, the feedback element obtain the actual position coordinate of the performing module, and with the target cotton position
Coordinate carries out asking poor calculating, and the difference of acquisition feeds back to the control module；
Step 8, the control module judge whether the difference is 0, if 0, then control the performing module to be adopted
Cotton operates；Otherwise, the performing module is controlled to be moved to the target cotton position from actual position coordinate according to the difference
Coordinate, and return to step 7 performs；
Step 9, the performing module are the manipulator of a six degree of freedom, and after cotton picking action is performed, from the target
Cotton position coordinates resets to the initial coordinate (x0, y0, z0) place, so that the cotton plucked is put into cotton storage tank；
Step 10, i+1 value is assigned to i, and feedback step 3 is sequentially performed.
The beneficial effects of the invention are as follows：
1. the image that binocular camera is similar to a pair of cameras of eyes to gather using relative position, to obtain cotton
Depth information provides more accurately raw information；Single-chip microcomputer controls binocular camera to shoot by camera master control borad, with electricity consumption
Brain control is compared, and greatlys save the volume of financial cost and cotton picking equipment；
2. being that one kind of biological vision is simulated using computer and relevant device by computer vision technique, pass through place
Manage the image of binocular camera collection to obtain the depth information of cotton, so as to obtain the sky of all cottons in the original image
Between position coordinate set, instead of it is artificial find target cotton, and provide the exact space position coordinate of cotton；
3. BR gathers image, target cotton positional information is provided in real time, it is ensured that the accuracy of target cotton position,
Accurate data is provided for the six degree of freedom manipulator arrival target cotton position of control and precise control in real time；
4. a more than cotton, the selected distance institute from the locus coordinate set may be included in piece image data
The nearest locus coordinate of binocular camera is stated as target cotton position, selects cotton target closest in image excellent
First handle, can guarantee that higher cotton picking efficiency；
5. driving six degree of freedom manipulator to reach target cotton position by control module, then cotton picking action is performed, instead of
The action of artificial cotton picking, so as to greatly reduce hand labor intensity；
6. feedback element compares the actual position coordinate of six degree of freedom manipulator with target cotton position, and difference is anti-
Feed control system, so as to ensure that manipulator reaches the accuracy of target cotton position；
7. the cotton picking action of six degree of freedom manipulator is relatively softr, it can prevent that cotton fiber length is destroyed, and ensure
Cotton picking cleannes, it is cotton storage tank position that the cotton after harvesting, which returns to initial coordinate, unclamps six degree of freedom manipulator, is completed
Cotton picking；
8. the setting of initial coordinate so that first motion reduces control to initial position after the completion of each cotton picking action
The complexity of program, improves cotton picking efficiency；
9. camera master control borad control camera BR gathers the original image of cotton, it can not influence to extract cotton
On the premise of spatial position precision, the space of data memory module is saved, and improves the speed of image procossing, it is ensured that in real time
The possibility and cotton picking efficiency of cotton picking.
Brief description of the drawings
Fig. 1 is present system schematic diagram；
Fig. 2 is the inventive method flow chart.
Embodiment
In order that technological means, creation characteristic, reached purpose and effect that the present invention is realized are easy to understand, below
With reference to being specifically illustrating, the present invention is expanded on further.
As shown in figure 1, a kind of intelligent cotton picker system based on computer vision technique, including：Image capture module, number
According to memory module, image processing module, control module and performing module；
Image capture module controls the original image of binocular camera BR collection cotton using camera master control borad,
And the original image collected is sent to data memory module；Binocular camera is similar to taking the photograph for a pair for eyes using relative position
The image gathered as head, more accurately raw information is provided to obtain cotton depth information；
Data memory module is stored by stack-incoming operation to original image, and by Pop operations by original image number
According to image processing module is supplied to, using multi-channel data acquisition board, transmission and the storage speed of data can be accelerated；
Image processing module carries out the extraction of depth information by computer vision technique to original image, obtains original graph
The spatial coordinated information collection of all cottons as in, and concentrate the nearest space of selected distance binocular camera from spatial coordinated information
Position coordinates instead of artificial searching target cotton, and provide the exact space position of cotton and sit as target cotton position
Mark, improves the efficiency of cotton picking；
Control module is the integrated monolithic machine of 32, and single-chip microcomputer and binocular camera are connected by camera master control borad；Control
Molding root tuber reaches target cotton position, wherein controlling according to target cotton position by controlling driving stepper motor performing module
Module includes feedback element, and feedback element obtains the physical location that performing module is reached, and is compared with target cotton position
Compared with, the difference of acquisition feeds back to control module, to control in real time, accurate control machinery hand reach target cotton position；
The difference control performing module that control module is obtained according to feedback element eventually arrives at target cotton position；
Performing module is a six degree of freedom manipulator, for performing cotton picking action, and the cotton plucked is transported into storage
Hopper mouthful, unclamps color cotton pawl and make it that cotton falls into cotton storage tank, completes a cotton picking action, and manipulator closes up cotton picking by three pawls,
So cotton fiber length will not be destroyed during cotton picking, the impurity such as cotton leaf are there will not be, so cleannes are very high.
As shown in Fig. 2 said system is applied on the cotton picker with cotton storage tank, and cotton picking control is carried out as follows
System operation：
Central point between step 1, two using in binocular camera cameras is origin o, using the earth as reference planes, with
The direction that cotton picker advances is x-axis, using the normal perpendicular to reference planes as y-axis, z-axis is obtained according to right-hand rule, so as to build
Vertical o-xyz coordinate systems；Then position coordinates of the cotton storage tank in o-xyz coordinate systems is initial coordinate (x0, y0, z0)；
Step 2, definition cyclic variable are i, and initialize i=1；
Step 3, image capture module control the i-th of binocular camera BR collection cotton using camera master control borad
Open original image；
Step 4, data memory module store i-th original image by press process, are processed as by popping at image
Manage module and i-th original image is provided；
Step 4, image processing module carry out the depth information of cotton by computer vision technique to i-th original image
Extraction, so as to obtain the three-dimensional information of cotton, that is, obtain the spatial coordinated information collection of all cottons in i-th original image；
Step 5, due to might have multiple cottons in piece image, so image processing module is from spatial coordinated information collection
It is middle selection with the closest spatial coordinated information of binocular camera is as target cotton position coordinates and is supplied to control module；
Step 6, control module read target cotton position coordinates by serial communication, and calculate the turning collar of stepper motor
Number, so as to control driving stepper motor performing module to be moved to target cotton position coordinates from initial coordinate (x0, y0, z0)；
Step 7, feedback element obtain the actual position coordinate of performing module, and ask poor with the progress of target cotton position coordinates
Calculate, the difference of acquisition feeds back to control module；
Step 8, control module judge whether difference is 0, if 0, then control performing module to carry out cotton picking operation；Otherwise,
Performing module is controlled to be moved to target cotton position coordinates from actual position coordinate according to difference, and return to step 7 is performed；
Step 9, performing module are the manipulator of a six degree of freedom, and after cotton picking action is performed, from target cotton position
Coordinate resets to initial coordinate (x0, y0, z0) place, so that the cotton plucked is put into cotton storage tank；
Step 10, i+1 value is assigned to i, and feedback step 3 is sequentially performed.
The general principle and principal character and advantages of the present invention of the present invention has been shown and described above.The technology of the industry
Personnel are it should be appreciated that the present invention is not limited to the above embodiments, and the simply explanation described in above-described embodiment and specification is originally
The principle of invention, without departing from the spirit and scope of the present invention, various changes and modifications of the present invention are possible, these changes
Change and improvement all fall within the protetion scope of the claimed invention.The claimed scope of the invention by appending claims and
Its equivalent thereof.
Claims (1)
1. a kind of intelligent cotton picker system based on computer vision technique, it is characterised in that including：Image capture module, data
Memory module, image processing module, control module and performing module；
Described image acquisition module controls the original image of binocular camera BR collection cotton using camera master control borad；
The data memory module is stored by stack-incoming operation to the original image, and by Pop operations by the original
Beginning view data is supplied to described image processing module；
Described image processing module carries out the extraction of depth information by computer vision technique to the original image, obtains institute
The spatial coordinated information collection of all cottons in original image is stated, and binocular described in selected distance is concentrated from the spatial coordinated information
The nearest locus coordinate of camera is used as target cotton position；
The control module is the integrated monolithic machine of 32, and single-chip microcomputer and binocular camera are connected by camera master control borad；
The control module is described by controlling performing module described in driving stepper motor to reach according to the target cotton position
Target cotton position, wherein control module include feedback element, and feedback element obtains the actual bit that the performing module is reached
Put, and be compared with the target cotton position, the difference of acquisition feeds back to the control module；
The difference that the control module is obtained according to the feedback element controls the performing module to eventually arrive at the target cotton
Flower position；
The performing module is a six degree of freedom manipulator, for performing the cotton picking action command that control module is sent, and by institute
The cotton of harvesting is put into cotton storage tank；
The method of work of said system is, applied on the cotton picker with cotton storage tank, to carry out as follows：
Central point between step 1, two using in binocular camera cameras is origin o, using the earth as reference planes, with described
The direction that cotton picker advances is x-axis, using the normal perpendicular to the reference planes as y-axis, and z-axis is obtained according to right-hand rule, from
And set up o-xyz coordinate systems；Then position coordinates of the cotton storage tank in the o-xyz coordinate systems be initial coordinate (x0, y0,
z0)；
Step 2, definition cyclic variable are i, and initialize i=1；
Step 3, described image acquisition module control the i-th of binocular camera BR collection cotton using camera master control borad
Open original image；
Step 4, the data memory module store i-th original image by stacking processing, and institute is processed as by popping
State image processing module and i-th original image is provided；
Step 4, described image processing module carry out the depth of cotton by computer vision technique to i-th original image
The extraction of information, obtains the spatial coordinated information collection of all cottons in i-th original image；
Step 5, described image processing module are concentrated from the spatial coordinated information chooses closest with the binocular camera
Spatial coordinated information is as target cotton position coordinates and is supplied to the control module；
Step 6, the control module read the target cotton position coordinates by serial communication, and calculate turning for stepper motor
Moving-coil number, so as to control performing module described in driving stepper motor to be moved to the target from the initial coordinate (x0, y0, z0)
Cotton position coordinates；
Step 7, the feedback element obtain the actual position coordinate of the performing module, and with the target cotton position coordinates
Poor calculating is asked in progress, and the difference of acquisition feeds back to the control module；
Step 8, the control module judge whether the difference is 0, if 0, then control the performing module to carry out cotton picking behaviour
Make；Otherwise, the performing module is controlled to be moved to the target cotton position coordinates from actual position coordinate according to the difference,
And return to step 7 is performed；
Step 9, the performing module are the manipulator of a six degree of freedom, and after cotton picking action is performed, from the target cotton
Position coordinates resets to the initial coordinate (x0, y0, z0) place, so that the cotton plucked is put into cotton storage tank；
Step 10, i+1 value is assigned to i, and feedback step 3 is sequentially performed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510556986.5A CN105165255B (en) | 2015-09-02 | 2015-09-02 | A kind of intelligent cotton picker system based on computer vision technique |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510556986.5A CN105165255B (en) | 2015-09-02 | 2015-09-02 | A kind of intelligent cotton picker system based on computer vision technique |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105165255A CN105165255A (en) | 2015-12-23 |
| CN105165255B true CN105165255B (en) | 2017-07-14 |
Family
ID=54888148
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510556986.5A Expired - Fee Related CN105165255B (en) | 2015-09-02 | 2015-09-02 | A kind of intelligent cotton picker system based on computer vision technique |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105165255B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105940864B (en) * | 2016-06-30 | 2018-01-05 | 江西农业大学 | A kind of autonomous camellia oleifera fruit harvester of rubber roller type |
| CN107527367A (en) * | 2017-10-19 | 2017-12-29 | 新疆秦域工业设备制造安装有限公司 | A kind of cotton identification and localization method based on binocular camera |
| CN109197160B (en) * | 2018-09-12 | 2020-05-15 | 华南农业大学 | Guava picking robot and implementation method thereof |
| CN110402683B (en) * | 2019-07-30 | 2022-05-10 | 盐城正邦环保科技有限公司 | Intelligent cotton picker and control method thereof |
| CN113330915B (en) * | 2021-05-26 | 2022-08-30 | 华南农业大学 | Self-adaptive cotton harvesting method based on binocular vision recognition and intelligent mechanical harvesting device |
| CN114885661B (en) * | 2022-05-09 | 2023-06-13 | 五邑大学 | Chinese yam harvesting device, method and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101152716A (en) * | 2006-09-30 | 2008-04-02 | 托乎达洪·玉米西 | Cotton picking robot |
| US7677169B2 (en) * | 2007-09-06 | 2010-03-16 | Cnh America Llc | Apparatus and method using imaging in building a cotton module |
| CN102640622B (en) * | 2012-04-16 | 2014-01-15 | 中国农业大学 | Cotton picker navigation information image detection method and system |
-
2015
- 2015-09-02 CN CN201510556986.5A patent/CN105165255B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN105165255A (en) | 2015-12-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105165255B (en) | A kind of intelligent cotton picker system based on computer vision technique | |
| Hayashi et al. | Robotic harvesting system for eggplants | |
| CN103529855B (en) | A kind of object recognition and detection device of rotary adjustable binocular vision and the application at agriculture fruit picking mechanical thereof | |
| CN105123127A (en) | Wolfberry picking robot and control method thereof | |
| CN110089350A (en) | A kind of Mushroom Picking Robot system and picking method | |
| CN109392462A (en) | Pick the robot and method of day lily | |
| CN111165176A (en) | Tea artificial intelligence picking robot | |
| CN105259832A (en) | Intelligent cotton picker control system based on multi-view stereo vision | |
| CN109699300A (en) | A kind of control system and its control method of intelligence citrus picking machine | |
| CN108508845B (en) | A kind of complex-curved quick numerical control engraving and milling method based on geometric self-adaptation | |
| CN103198477A (en) | Apple fruitlet bagging robot visual positioning method | |
| CN110293559A (en) | A kind of installation method of automatic identification positioning alignment | |
| CN205030148U (en) | Machine people is plucked to matrimony vine | |
| CN108811766A (en) | A kind of man-machine interactive fruits and vegetables of greenhouse harvesting robot system and its collecting method | |
| CN109272553A (en) | Localization method, controller and the ablation device extractd for the cotton top heart | |
| CN109247153A (en) | A kind of fertile mandarin orange branch pruning intelligent robot based on Internet of Things | |
| CN103213128A (en) | Seedling grafting robot control system | |
| CN109605375A (en) | A kind of control method of intelligence Tomato-harvesting robot | |
| CN109197160A (en) | A kind of guava picking robot and its implementation | |
| CN115316129A (en) | Self-adaptive bionic picking device based on binocular vision recognition and cluster fruit picking method | |
| CN111684924A (en) | Strawberry picking robot based on raspberry group | |
| CN109087343A (en) | A kind of generation method and system of workpiece grabbing template | |
| CN110063157A (en) | A kind of trimming of Cold region apple jujube tree, girdling robot | |
| CN106846462A (en) | Insect identifying device and method based on three-dimensional simulation | |
| CN114830915A (en) | Litchi vision picking robot based on laser radar navigation and implementation method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170714 Termination date: 20190902 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |