CN103185949B - Auto-focusing imaging modules and Atomatic focusing method - Google Patents
Auto-focusing imaging modules and Atomatic focusing method Download PDFInfo
- Publication number
- CN103185949B CN103185949B CN201110444036.5A CN201110444036A CN103185949B CN 103185949 B CN103185949 B CN 103185949B CN 201110444036 A CN201110444036 A CN 201110444036A CN 103185949 B CN103185949 B CN 103185949B
- Authority
- CN
- China
- Prior art keywords
- image
- sensed
- computing module
- object distance
- focusing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Studio Devices (AREA)
- Automatic Focus Adjustment (AREA)
Abstract
A kind of auto-focusing imaging modules, it includes a sampling image lens, an image sensor, a color-separated unit, a processor, a graphics processing unit and a driver.This sampling image lens is for catching the image of object, and by the image focusing that captured in this image sensor surface.This image sensor is for sensing the image that this sampling image lens is caught.This color-separated unit represents with Red Green Blue respectively for the image sensed by this image sensor.This processor carries out MTF computing for the image that sensed this image sensor, determines current shooting pattern according to operation result, and according to determined by current shooting model selection control this graphics processing unit or this driver.The invention still further relates to a kind of Atomatic focusing method.
Description
Technical field
The present invention relates to a kind of imaging modules, particularly relate to a kind of auto-focusing with automatic focusing function
Imaging modules and a kind of Atomatic focusing method.
Background technology
Along with the progress of science and technology, the camera lens module of most electronic equipments possesses automatic focusing function.Camera lens mould
The Autofocus Technology of group is divided into mechanical type Autofocus Technology and digital Autofocus Technology.
In mechanical type Autofocus Technology uses frame for movement moving lens, eyeglass is focused.Due to machinery
Formula focusing technology to use the frame for movement of complexity, causes autofocus lens cost increase and volume bigger.
The image that image sensor is sensed by digital Autofocus Technology by software emulation, calculating
Process so that because of on image sensor pixel because of image out of focus and fuzzy become clear.Such as,
Extended depth-of-field (Extend Depth of Field:EDoF) technology, its utilize light three primary colors (red,
Green, blue) each own optimal MTF curve when different distance, object, can when different distance
Algorithm Digital Simulation is utilized to go out other two primary colors with the optimal primary colors of distance instantly, to reach the clear of full width
Image number.So, the defect of digital focusing focusing is for low coverage imaging capability not enough, in general,
If object distance is within 40cm, then the focus effects of Digital Autofocus Technology tends not to satisfactory.
Summary of the invention
In view of this, it is necessary to a kind of auto-focusing imaging modules avoiding the problems referred to above is provided, also must
A kind of Atomatic focusing method is provided.
A kind of auto-focusing imaging modules, it includes a sampling image lens, one and described sampling image lens light
Image sensor that axle aligns, color-separated unit being connected with described image sensor, one
The processor being connected with described color-separated unit, the image being connected with described processor respectively
Processing unit and a driver.Described sampling image lens is for catching the image of object, and will be caught
The image focusing arrived is in described image sensor surface.Described image sensor is used for sensing described image capture lens
The image that head is caught.Described color-separated unit is for the image sensed by described image sensor
Represent with Red Green Blue respectively.Described processor is for being sensed described image sensor
Image carry out MTF computing, determine current shooting pattern according to operation result, and according to determined by when
Front screening-mode selects to control described graphics processing unit or described driver.Described graphics processing unit
Image for being sensed described image sensor by image procossing mode carries out fuzzy revising out of focus.
Described driver is used for driving described sampling image lens to focus.
A kind of Atomatic focusing method, it comprises the steps:
The image sensed by image sensor carries out color-separated, this image be expressed as red, green,
Primary colors image；
The image-region that pixel cell each to described image sensor is sensed carries out MTF computing,
The mtf value that the image-region that sensed to each pixel cell is corresponding；
The mtf value of the image sensed according to described each pixel cell determines described each pixel cell
The object distance of the image sensed；
According to the object distance of the image that described each pixel cell is sensed, determine current shooting pattern；
If current shooting pattern is nearly burnt pattern, then carry out the following step:
According to the object distance of the image that described each pixel cell is sensed, determine optimal focusing position；
The focusing drive volume of sampling image lens is determined according to described optimal focusing position；
Drive sampling image lens to optimal focusing position according to described focusing drive volume；
If current shooting pattern is remote burnt pattern, then carry out the following step:
The mtf value that the image-region that sensed according to each pixel cell is corresponding, determines respective pixel list
The fuzzy quantity of the image that unit is sensed；
According to the fuzzy quantity of the image that described each pixel cell is sensed, determine respective pixel unit institute
The fuzzy revising amount of the image sensed；
According to described fuzzy revising amount, the image being sensed each pixel cell carries out fuzzy revising.
For prior art, described auto-focusing imaging modules and and Atomatic focusing method utilize software
The mode of computer sim-ulation determines the object distance of subject, and determines current shooting pattern according to the situation of object distance,
Select computed in software mode according to screening-mode or drive the mode of sampling image lens to focus, Ke Yida
Become the nearlyest Jiao or far beyond under screening-mode, image clearly of focusing can be obtained.
Accompanying drawing explanation
Fig. 1 is the schematic diagram of the auto-focusing imaging modules of embodiment of the present invention.
Fig. 2 is the flow chart of the Atomatic focusing method of embodiment of the present invention.
Main element symbol description
Auto-focusing imaging modules 100
Sampling image lens 10
Eyeglass 11
Image sensor 20
Color-separated unit 30
Processor 40
MTF computing module 41
Object distance computing module 42
Object distance judge module 43
Fuzzy quantity computing module 44
Fuzzy revising amount computing module 45
Focusing position computing module 46
Drive volume computing module 47
Graphics processing unit 50
Driver 60
Following detailed description of the invention will further illustrate the present invention in conjunction with above-mentioned accompanying drawing.
Detailed description of the invention
Below in conjunction with accompanying drawing, the present invention is made one specifically to introduce.
Refer to Fig. 1, show the schematic diagram of the auto-focusing imaging modules 100 of embodiment of the present invention,
Described auto-focusing imaging modules 100 includes a sampling image lens 20, color of 10, image sensor
Color separative element 40, graphics processing unit 50 of 30, processor and a driver 60.Institute
The center stating image sensor 20 is directed at the optical axis of described sampling image lens 10, described color-separated unit 30 with
Described image sensor 20 is connected, and described processor 40 is connected with described color-separated unit 30, described figure
As processing unit 50 and described driver 60 are all connected with described processor 40, described driver 60 also with
Described sampling image lens 10 is connected.
Described sampling image lens 10 is for catching the image of object, and is projected to described by the image focusing of seizure
The sensing region of image sensor 20.Include that at least one has positive light coke in described sampling image lens 10
Eyeglass 11, described eyeglass 11 is aspherical lens.
Described image sensor 20 is for sensing the image that described sampling image lens 10 is captured.Described image
Sensor 20 includes multiple pixel cell (not shown), and the plurality of pixel cell is that array shape is distributed in
Effective sensing region of described image sensor 20.Wherein, each pixel cell all include three primary colors (red,
Green, blue) pixel.It is preferred that described image sensor 20 at least includes 2048 × 1536 pixel cells.
In present embodiment, described image sensor 20 can be that Charged-coupled Device (CCD) passes
Sensor or Complementary Metal Oxide Semiconductor (CMOS) sensor.
Described color-separated unit 30 is for being separated into use by the image that described image sensor 20 is sensed
The image that three primary colors represents.
Described processor 40 include a modulation transfer function (MTF) (Modulation Transfer Function:
MTF) 42, object distance judge module of 41, object distance computing module of computing module 43, obscures
Amount computing module 45, focusing position computing module 46 and of 44, fuzzy revising amount computing module
Individual drive volume computing module 47.Described MTF computing module 41 is connected with described color-separated unit 30, institute
Stating object distance computing module 42 to be connected with described MTF computing module 41, described object distance judge module 43 is with described
Object distance computing module 42 is connected, described focusing position computing module 46 and described fuzzy quantity computing module 44
All it is connected with described object distance judge module 43, described drive volume computing module 47 and described focusing position computing
Module 46 and described driver 60 are connected；Described fuzzy revising amount computing module 45 and described fuzzy revising
Amount computing module 45 and described graphics processing unit 50 are connected.
Described MTF computing module 41 is for being sensed each pixel cell on described image sensor 20
Image-region carry out MTF computing, obtain the mtf value of corresponding region.In present embodiment, described MTF
Computing module 41 carries out mtf value computing respectively to the three-primary-color image that each pixel cell is corresponding.
Described object distance computing module 42, for the operation result according to described MTF computing module, determines each
The object distance of the image that pixel cell is sensed.
Described object distance judge module 43, for the operation result according to described object distance computing module 42, determines and works as
Front screening-mode.Specifically, the computing of described object distance computing module is tied by described object distance judge module 43
Fruit makees comprehensive computing, and the result of this comprehensive computing is compared with a default standard value, according to than
Relatively result determines current shooting pattern.In present embodiment, described comprehensive computing is to described object distance computing
The object distance of the sensed image of each pixel cell obtained by module 42 is sampled, and according to sampling
Data operation obtains the object distance token state of the distance for characterizing current shooting main target thing.Described default
Standard value be used for distinguishing current shooting pattern for nearly burnt pattern or remote burnt pattern, in present embodiment,
Described standard value is 40cm, if described object distance token state is more than 40cm, then current shooting pattern is remote burnt
Pattern, if described object distance token state is less than (being equal to) 40cm, then the closely burnt pattern of current shooting pattern.
Described fuzzy quantity computing module 44, for the operation result according to described MTF computing module 41, determines
Mtf value obtained by this each pixel cell computing and the homologue difference away from internal standard mtf value, and root
The fuzzy quantity of image that each pixel cell sensed is determined according to this difference.Described standard mtf value is every
One pixel cell is at the homologue mtf value away from interior sensed picture rich in detail region, therefore, described
The mtf value of each pixel cell that MTF computing module 41 computing obtains and corresponding standard mtf value
Between difference can characterize the fuzzy quantity of the image that each pixel cell is sensed.In present embodiment,
Described fuzzy quantity computing module 44 carries out fuzzy quantity computing respectively to the three-primary-color image of each pixel cell.
Described fuzzy quantity computing module 44 determines according to screening-mode determined by described object distance judge module 43
Whether its function no opens.In present embodiment, when described object distance judge module 43 judges current shooting mould
When formula is remote burnt pattern, described fuzzy quantity computing module 44 function is opened, when described object distance judge module 43
When judging current shooting pattern as nearly burnt pattern, described fuzzy quantity computing module 44 function is closed.
Described fuzzy revising amount computing module 45 is for according to the mould obtained by described fuzzy quantity computing module 44
Paste amount, determines that the image being sensed each pixel cell carries out the correction of fuzzy revising.This enforcement
In mode, described fuzzy revising amount computing module 45 carries out three primary colors respectively to the image of each pixel cell
Fuzzy revising amount computing.
Described focusing position computing module 46 is for the operation result according to described object distance computing module 42, really
The optimal focusing position of fixed described sampling image lens 10.Described focusing position computing module 46 is according to described object distance
Screening-mode determined by judge module 43 and determine whether whether its function opens.In present embodiment,
When described object distance judge module 43 judges current shooting pattern as nearly burnt pattern, described focusing position computing
Module 46 function is opened, when described object distance judge module 43 judges current shooting pattern as remote burnt pattern,
Described focusing position computing module 46 function is closed.
Described drive volume computing module 47 is for according to the sampling image lens obtained by described object distance computing module 42
The optimal focusing position of 10, determines the focusing drive volume of described sampling image lens 10.
Described graphics processing unit 50 is for according to the correction obtained by described fuzzy revising amount computing module 45
Amount, the image being sensed each pixel cell carries out fuzzy revising, to obtain picture rich in detail.This reality
Executing in mode, described graphics processing unit 50 carries out trichromatic correction to the image of each pixel cell.
Described driver 60 is for driving according to the focusing drive volume obtained by described drive volume computing module 47
Described sampling image lens 10 is to optimal focusing position.In present embodiment, described driver 60 is piezoelectric type horse
Reaching, certainly, described driver 60 can also be the other type of driving assemblies such as voice coil motor.
Referring to Fig. 2, the Atomatic focusing method of embodiment of the present invention comprises the steps:
The image sensed by image sensor carries out color-separated, this image be expressed as red, green,
Primary colors image；
The image-region that pixel cell each to described image sensor is sensed carries out MTF computing,
The mtf value that the image-region that sensed to each pixel cell is corresponding；
The mtf value of the image sensed according to described each pixel cell determines described each pixel cell
The object distance of the image sensed；
According to the object distance of the image that described each pixel cell is sensed, determine current shooting pattern；
If current shooting pattern is nearly burnt pattern, then carry out the following step:
According to the object distance of the image that described each pixel cell is sensed, determine optimal focusing position；
The focusing drive volume of sampling image lens is determined according to described optimal focusing position；
Drive sampling image lens to optimal focusing position according to described focusing drive volume；
If current shooting pattern is remote burnt pattern, then carry out the following step:
The mtf value that the image-region that sensed according to each pixel cell is corresponding, determines respective pixel list
The fuzzy quantity of the image that unit is sensed；
According to the fuzzy quantity of the image that described each pixel cell is sensed, determine respective pixel unit institute
The fuzzy revising amount of the image sensed；
According to described fuzzy revising amount, the image being sensed each pixel cell carries out fuzzy revising.
The mode that described auto-focusing imaging modules and Atomatic focusing method utilize computed in software to emulate determines
The object distance of subject, and determine current shooting pattern according to the situation of object distance, select according to screening-mode
The mode of computed in software mode or driving sampling image lens is focused, and can reach the nearest burnt or remote
Under burnt screening-mode, image clearly of focusing can be obtained.
It addition, those skilled in the art also can do other change in spirit of the present invention, certainly, these depend on
The change done according to present invention spirit, within all should being included in scope of the present invention.
Claims (9)
1. an auto-focusing imaging modules, it includes a sampling image lens, one and described image capture lens
The image sensor that head optical axis aligns, a color-separated list being connected with described image sensor
Unit, processor being connected with described color-separated unit, it is connected with described processor respectively
A graphics processing unit and a driver；Described sampling image lens is for catching the figure of object
Picture, and by the image focusing that captured in described image sensor surface；Described image sensor is used
In sensing the image that described sampling image lens is caught；Described color-separated unit is for passing described image
The image that sensor is sensed represents with Red Green Blue respectively；Described processor is for institute
State the image that image sensor sensed and carry out MTF computing, determine current shooting according to operation result
Pattern, and according to determined by current shooting model selection control described graphics processing unit or described
Driver；Described graphics processing unit is for sensing described image sensor by image procossing mode
To image carry out fuzzy revising out of focus；It is right that described driver is used for driving described sampling image lens to carry out
Burnt；Described processor includes that a MTF computing module, object distance computing module, an object distance are sentenced
Disconnected module, fuzzy quantity computing module, fuzzy revising amount computing module, a focusing position
Computing module and one by one drive volume computing module；Described MTF computing module is for described shadow
The image-region sensed as pixel cell each on sensor carries out MTF computing, obtains corresponding district
The mtf value in territory；Described object distance computing module is used for the operation result according to described MTF computing module,
Determine the object distance of the image that each pixel cell sensed；Described object distance judge module is for according to institute
State the operation result of object distance computing module, determine current screening-mode；Described fuzzy quantity computing module
For the operation result according to described MTF computing module, obtained by determining this each pixel cell computing
Mtf value and the homologue difference away from internal standard mtf value, and determine each pixel list according to this difference
The fuzzy quantity of the image that unit is sensed；Described fuzzy revising amount computing module is for according to described fuzzy
Fuzzy quantity obtained by amount computing module, determines that the image being sensed each pixel cell carries out mould
Stick with paste the correction revised；Described focusing position computing module is for the fortune according to described object distance computing module
Calculate result, determine the optimal focusing position of described sampling image lens；Described drive volume computing module is used for root
According to the optimal focusing position of the sampling image lens obtained by described object distance computing module, determine described image capture lens
The focusing drive volume of head.
2. auto-focusing imaging modules as claimed in claim 1, it is characterised in that: described image capture lens
Head includes that at least one has the aspherical lens of positive light coke.
3. auto-focusing imaging modules as claimed in claim 1, it is characterised in that: described MTF transports
Calculate module and the three-primary-color image that each pixel cell is corresponding is carried out mtf value computing respectively.
4. auto-focusing imaging modules as claimed in claim 1, it is characterised in that: described object distance is sentenced
The operation result of described object distance computing module is made comprehensive computing by disconnected module, and by the knot of this comprehensive computing
Fruit compares with a standard value preset, and determines current shooting pattern according to comparative result.
5. auto-focusing imaging modules as claimed in claim 4, it is characterised in that: described comprehensive fortune
Calculate as the object distance of the sensed image of each pixel cell obtained by described object distance computing module is entered
Row sampling, and obtain the distance for characterizing current shooting main target thing according to the data operation of sampling
Object distance token state.
6. auto-focusing imaging modules as claimed in claim 5, it is characterised in that: described default
Standard value is 40cm, if described object distance token state is more than 40cm, the most described object distance judge module is sentenced
Disconnected current shooting pattern is remote burnt pattern, if described object distance token state is less than or equal to 40cm, then
Described object distance judge module judges that current shooting pattern is as nearly burnt pattern.
7. auto-focusing imaging modules as claimed in claim 6, it is characterised in that: current shooting mould
When formula is remote burnt pattern, described fuzzy quantity computing module function is opened, and current shooting pattern is nearly burnt mould
During formula, described focusing position computing module function is opened.
8. auto-focusing imaging modules as claimed in claim 1, it is characterised in that: described driver
For piezoelectric type motor.
9. an Atomatic focusing method, it comprises the steps:
The image sensed by image sensor carries out color-separated, this image be expressed as red,
Green, primary colors image；
The image-region that pixel cell each to described image sensor is sensed carries out MTF computing,
Obtain the mtf value that image-region that each pixel cell sensed is corresponding；
The mtf value of the image sensed according to described each pixel cell determines described each pixel
The object distance of the image that unit is sensed；
According to the object distance of the image that described each pixel cell is sensed, determine current shooting pattern；
If current shooting pattern is nearly burnt pattern, then carry out the following step:
According to the object distance of the image that described each pixel cell is sensed, determine optimal focusing position；
The focusing drive volume of sampling image lens is determined according to described optimal focusing position；
Drive sampling image lens to optimal focusing position according to described focusing drive volume；
If current shooting pattern is remote burnt pattern, then carry out the following step:
The mtf value that the image-region that sensed according to each pixel cell is corresponding, determines respective pixel
The fuzzy quantity of the image that unit is sensed；
According to the fuzzy quantity of the image that described each pixel cell is sensed, determine respective pixel unit
The fuzzy revising amount of the image sensed；
According to described fuzzy revising amount, the image being sensed each pixel cell carries out fuzzy repairing
Just.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110444036.5A CN103185949B (en) | 2011-12-27 | 2011-12-27 | Auto-focusing imaging modules and Atomatic focusing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110444036.5A CN103185949B (en) | 2011-12-27 | 2011-12-27 | Auto-focusing imaging modules and Atomatic focusing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103185949A CN103185949A (en) | 2013-07-03 |
| CN103185949B true CN103185949B (en) | 2016-09-14 |
Family
ID=48677206
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110444036.5A Expired - Fee Related CN103185949B (en) | 2011-12-27 | 2011-12-27 | Auto-focusing imaging modules and Atomatic focusing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103185949B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108012145A (en) * | 2017-12-14 | 2018-05-08 | 信利光电股份有限公司 | Solution method, system and the equipment of foreign matter are arranged at camera module motor bottom |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1464323A (en) * | 2002-06-13 | 2003-12-31 | 北京中星微电子有限公司 | Process for automatic focusing |
| CN101110906A (en) * | 2006-07-18 | 2008-01-23 | 三星电子株式会社 | Apparatus and method for selecting a shooting mode |
| CN101685194A (en) * | 2008-09-26 | 2010-03-31 | 鸿富锦精密工业（深圳）有限公司 | Automatic focusing device and automatic focusing method |
| CN101950063A (en) * | 2009-07-10 | 2011-01-19 | 佛山普立华科技有限公司 | Automatic focusing system and automatic focusing method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI323125B (en) * | 2004-04-16 | 2010-04-01 | Hon Hai Prec Ind Co Ltd | An method for auto-adjusting the focus of the digital camera |
| JP4649128B2 (en) * | 2004-06-15 | 2011-03-09 | キヤノン株式会社 | Automatic focus adjustment apparatus and imaging apparatus having the automatic focus apparatus |
| JP4859625B2 (en) * | 2006-10-27 | 2012-01-25 | Ｈｏｙａ株式会社 | Camera with image stabilization device |
-
2011
- 2011-12-27 CN CN201110444036.5A patent/CN103185949B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1464323A (en) * | 2002-06-13 | 2003-12-31 | 北京中星微电子有限公司 | Process for automatic focusing |
| CN101110906A (en) * | 2006-07-18 | 2008-01-23 | 三星电子株式会社 | Apparatus and method for selecting a shooting mode |
| CN101685194A (en) * | 2008-09-26 | 2010-03-31 | 鸿富锦精密工业（深圳）有限公司 | Automatic focusing device and automatic focusing method |
| CN101950063A (en) * | 2009-07-10 | 2011-01-19 | 佛山普立华科技有限公司 | Automatic focusing system and automatic focusing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103185949A (en) | 2013-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103293823B (en) | Auto-focusing imaging modules | |
| CN104519328B (en) | Image processing equipment, image capture device and image processing method | |
| EP2833623A1 (en) | Image sensor, imaging method, and imaging device | |
| CN108028895A (en) | The calibration of defective image sensor element | |
| CN107534733A (en) | Camera device, the image processing method of the camera device and program | |
| US9066000B2 (en) | 3D auto-focusing camera module and 3D imaging method using same | |
| CN105635565A (en) | Shooting method and equipment | |
| CN104184935B (en) | Image capture devices and method | |
| CN107409177A (en) | Electronic equipment and computer program product | |
| CN103139492B (en) | Camera head, image capture method and the system of supervision | |
| CN102972020B (en) | Camera model and the method driving camera model | |
| CN107566741A (en) | Focusing method, device, computer-readable recording medium and computer equipment | |
| US8786959B2 (en) | Auto-focusing camera module and auto-focusing method using same | |
| WO2007043325A1 (en) | Image processing system and image processing program | |
| JP2022106735A (en) | Image pick-up device and imaging apparatus | |
| KR100651809B1 (en) | Camera and display method of luminance distribution | |
| US9922410B2 (en) | Image processing apparatus and image processing method | |
| CN103185949B (en) | Auto-focusing imaging modules and Atomatic focusing method | |
| CN107534732A (en) | Image processing apparatus, camera device, image processing method and image processing program | |
| JP2007158941A (en) | Object area detection apparatus and method, imaging device and program | |
| CN103188499B (en) | 3D imaging modules and 3D formation method | |
| CN110677576A (en) | Focusing system of camera module | |
| JP4718113B2 (en) | White balance adjustment method and image processing apparatus | |
| CN103188424B (en) | 3D imaging modules and 3D imaging methods | |
| US20230217126A1 (en) | Imaging apparatus, imaging method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160914 Termination date: 20171227 |